Camera and captures

Scope: camera. All glasses.

The glasses keep their photos and videos until a phone imports them. Visor imports over Wi-Fi Direct, so the phone's Wi-Fi must be on; it falls back to Bluetooth only when Wi-Fi Direct fails. The glasses' capture light turns on for every photo and video, as it does for their own button.

Photos

val photo = visor.camera.takePhoto()                 // returns once it is in the gallery
val onGlasses = visor.camera.takePhoto(import = false) // returns once the glasses have it

The result is a Capture. With an import, uri is the photo's content:// address in the phone's gallery, and Visor grants your app read access to it: your app needs no media permission to show it.

A photo fails with:

The glasses take a photo for the phone only while a switch of theirs is on. Visor turns it on while any app holds the camera scope. The first time, it can take a minute while Visor reads the glasses' configuration, and a photo asked for meanwhile fails with timeout.

Videos

visor.camera.startVideo()                      // returns once the glasses record
visor.camera.startVideo(VideoKind.HYPERLAPSE)
visor.camera.stopVideo()                       // returns once they stopped

The glasses end a video on their own after their maximum length. The new video then appears on captures.available, like one taken with the button.

The camera live

val config = LiveConfig { quality = LiveQuality.HIGH; frameRate = 30; audio = true }
LiveVideoDecoder(surfaceView.holder.surface).use { video ->
    visor.camera.live(config).flowOn(Dispatchers.IO).collect { sample -> video.decode(sample) }
}

live streams what the glasses' camera sees, from Visor's Live view over Wi-Fi Direct, so the phone's Wi-Fi must be on. Collecting starts the Live, and the glasses' capture light stays on while it runs. When the Live already runs, for Visor's own Live view or another app, your app joins it as it is. When the last app stops collecting, the Live stops, unless the person started it in Visor.

Each LiveSample is a header and its data:

Each kind starts with its config, and the video then starts on a keyframe. A new config can come mid-stream, when the Live restarts. Visor sends the video and the audio as the glasses encode them, so you can save them, send them on, or decode them:

• LiveVideoDecoder(surface) draws each picture on a Surface as soon as it is decoded.
• LiveVideoDecoder { image, ptsUs -> } hands each picture over as a YUV_420_888 android.media.Image, valid until the block returns: for ML Kit, OpenCV or your own model.
• LiveAudioDecoder { samples, sampleRate, channels, ptsUs -> } hands the audio over as 16-bit PCM, interleaved.

Feed every sample to each decoder you use, on one thread; each skips the other kind. When frames are lost on the way, Visor skips ahead to the next keyframe, so a slow app sees a pause instead of a broken picture.

LiveConfig:

The config applies only when your app starts the Live. Visor waits for an app that reads slowly, and ends the stream of an app that stops reading for 5 seconds, so that a frozen app cannot keep the glasses filming.

The flow fails with:

A photo while live

val photo = visor.camera.takeLivePhoto()
val heic = context.contentResolver.openInputStream(Uri.parse(photo.uri))!!.use { it.readBytes() }

takeLivePhoto takes a photo at the glasses' photo size without stopping the stream. The result is a HEIC that your app may read from uri for 10 minutes. It fails with busy when the Live is not running, and timeout when the glasses send no photo. Over adb or the network door, data holds the photo in Base64 instead of uri.

Captures

val waiting = visor.captures.list()            // on the glasses, not on the phone yet
val video = visor.captures.import(waiting.first().id)

A Capture:

Imports

import returns at once when the phone already has the capture. Otherwise it returns when the capture is in the gallery, and fails with failed and one of these as reason:

The glasses send whatever else waits in their queue with an import. Each of those reaches the gallery too and comes on imported.

Streams

visor.captures.available.collect { }  // each new capture on the glasses
visor.captures.progress.collect { }   // CaptureProgress(id, received, size), at most once per percent
visor.captures.imported.collect { }   // each capture that reaches the gallery
visor.captures.failed.collect { }     // CaptureFailure(id, failure)

progress, imported and failed cover every import, also those Visor starts on its own.

JSON-RPC

Over JSON-RPC a feed is subscribed like a topic: see Feeds.