Web apps on the Display

Scope: any. The Meta Ray-Ban Display, with a web app the person added in Visor.

The Display runs web apps in its own browser. A web app reaches the VDK through that browser and the phone's Visor, with the TypeScript client's Visor.connectGlasses(): the same calls, topics and errors as everywhere else, from JavaScript on the glasses. Meta's toolkit has nothing like it.

import { Visor, VisorError, onGlasses } from "visor-kit";

async function start() {
  if (!onGlasses()) return Visor.connectFake();   // on a computer, with `visor fake`
  try {
    return await Visor.connectGlasses();
  } catch (e) {
    if (e instanceof VisorError && e.reason === "not_allowed") {
      showMessage("Allow this app on your phone: Visor > Settings > Web apps > this app > Visor access.");
    }
    throw e;
  }
}

const visor = await start();
visor.subscribe("band.gestures", {}, (gesture) => console.log(gesture));
await visor.call("voice.speak", { text: "Hello from the web app" });

Build the client from clients/typescript and serve dist/index.js and dist/glasses.js with the page, or bundle them with the page's own code.

What the person allows

In Visor, Settings > Web apps, the person taps the app and picks what it may do under Visor access, one scope at a time. That turns on Nearby Bluetooth devices for the app too, which is how it reaches Visor. The first time, Visor reads the glasses' configuration before it saves, which can take a minute; the sheet says so while it waits. The browser asks the wearer once more before the page's first connection.

Removing the app takes its access away.

When it fails

connectGlasses() rejects with a VisorError:

After that, a call the person did not allow fails with scope_denied, as anywhere else.

How it travels

The Display's browser hands a page's navigator.bluetooth.requestLEScan to the phone, which runs the scan and sends back what it hears. Visor reads the scan's filter instead:

• A message to Visor is a scan whose filter holds manufacturer data for company 0xFFFF, starting with VDK1.
• Visor's answers and events come back as advertisements from a device named Visor, with the same kind of data.
• One long scan carries everything Visor sends. Each message to Visor is a short scan of its own.

The client does all of this; a page uses call, subscribe and feed as anywhere else. Messages go in order. A large one, such as a layout with a photo, goes in pieces of 8000 bytes. Each piece is a scan, so a call takes tens of milliseconds more than over a socket: use subscribe rather than polling.

A web app's origin is all Visor knows about it. Every page on that origin gets the same access, so allow only origins you trust.

Tools for the assistant

A web app can give the wearer's assistant tools of its own, with the browser's WebMCP API, while it is open:

document.modelContext.registerTool({
  name: "add_to_list",
  description: "Add an item to the shopping list.",
  inputSchema: { type: "object", properties: { item: { type: "string" } }, required: ["item"] },
  execute: async ({ item }) => ({ content: [{ type: "text", text: `Added ${item}.` }] }),
});

The person turns this on once, in Visor: Settings > Web apps > Web app tools for your assistant. Visor then hands the tools to the model the person set up in Visor, with their descriptions and schemas, and runs the calls the model makes. Give each tool a name that says what it does: the model reads it first. Without the setting, document.modelContext is missing: check for it before you register.

Other parts of the web-app SDK

The Display's browser gives web apps more than the VDK, from the phone Visor runs on:

• navigator.geolocation, from the phone's location.
• The Generic Sensor API with sensorId: 'phone': the phone's accelerometer, gyroscope, magnetometer, linear acceleration, gravity and orientation, up to 100 readings a second.
• Real Bluetooth scans of the devices around the wearer, for an app with Nearby Bluetooth devices on.

The protocol behind each is on the Display web apps page of the protocol reference.