Porting from Meta's DAT

This page maps each part of Meta's Wearables Device Access Toolkit for Android (DAT 1.0.1) to the Visor Development Kit. It is for an app that already works with DAT and should also work, or only work, with Visor.

The two kits differ in shape more than in reach:

• No registration. DAT registers the app through the Meta AI app, and Meta approves some features per app. With the VDK, the person grants scopes on Visor's consent screen, and that is all.
• One connection, no capabilities to attach. DAT creates a DeviceSession, starts it, and attaches a camera, a display or inputs to it. The VDK has one Visor connection; each capability is a call or a flow. Collecting a flow starts it, and cancelling it stops it.
• Exceptions, not DatResult. Every VDK call fails with a KitException whose code is an ErrorCode.
• The active pair. DAT selects a device per session. The VDK acts on the pair Visor connects to; an app can change that pair for everyone with glasses.select.

Setup

// DAT
Wearables.initialize(context)
val launcher = registerForActivityResult(Wearables.RequestPermissionContract()) { result -> }
launcher.launch(Permission.CAMERA)

// VDK
val visor = Visor.connect(context)
val ask = registerForActivityResult(RequestScopes()) { held -> }
ask.launch(setOf(Scope.CAMERA))

Devices and sessions

Device fields:

Camera

// DAT
val camera = session.addCamera(StreamConfiguration(videoQuality = VideoQuality.HIGH, frameRate = 30)).getOrThrow()
camera.stream.start()
camera.stream.videoStream.collect { frame -> analyse(frame.buffer, frame.width, frame.height) }

// VDK
LiveVideoDecoder { image, _ -> analyse(image) }.use { video ->
    visor.camera.live(LiveConfig { quality = LiveQuality.HIGH; frameRate = 30 })
        .flowOn(Dispatchers.IO)
        .collect { video.decode(it) }
}

Differences to know:

• DAT streams over Bluetooth; the VDK streams over Wi-Fi Direct, so the phone's Wi-Fi must be on. It reaches 1080p.
• A tap on the touchpad pauses DAT's stream. The VDK's stream does not pause; the person stops the Live in Visor.
• When the Live already runs, for Visor's own view or another app, your app joins it as it is, whatever it asked.

Photos

Audio

The glasses encode their microphones as 48 kHz stereo AAC, and LiveAudioDecoder hands that over as 16-bit PCM. Resample it if your code expects 16 kHz mono.

Speech

On the Meta Ray-Ban Display the VDK recognises on the glasses as DAT does, with the toolkit app's speech, in the Display's language. On Ray-Ban Meta glasses, which DAT's speech does not cover, the text comes from the phone's speech recogniser, fed by the glasses' microphones as a Bluetooth headset, so music pauses while transcripts run. A confidence comes when the recogniser gives one.

Voice invocations

DAT opens the app when the wearer names it. The VDK does too, for an activity that declares its phrases, and sends the question itself to the app, which answers in the glasses' voice. Beyond DAT, visor.assistant.tools(...) gives the assistant's model tools to call while it answers.

Inputs

Motion

On the glasses, Visor reads motion from their own sensor service, with nothing on the screen, where DAT opens its toolkit app. The phone fuses the sensors into orientation. The band's motion is a stream of its own, visor.band.motion, for a band the person shares with the phone. FakeVisor serves all the fields: fake.glasses.motion(...), DAT's motion CSV with fake.glasses.playMotion(...), and fake.band.motion(...).

Display

On the glasses, Visor shows layouts through the Display's toolkit session, as DAT does, and draws them with the same rules as DAT's own builder, videos included. FakeVisor, visor fake and its Display simulator serve everything. The VDK also shows text on the teleprompter (visor.display.prompter) and notifications (visor.notify), and keeps an app's cards on the Display's home, beside Meta's weather and stocks, with details, pages, charts and clicks: see Briefing cards. DAT has nothing like them.

The VDK's layouts go beyond DAT's elements with what Meta's own cards use: the title text style, any colour, maxLines and text align, icon color and size, box background and height, pins, a pager and a chart.

Testing

See Testing for the rest.

Errors

The Errors page lists every code.