Manual Voice Commands

Automatic Voice Commands

We recommend relying on automatic voice commands for most use cases. That said, some UI components are not yet explicitly supported by automatic voice commands, and in those cases it may be possible to implement workarounds using the manual voice commands described below.

Please read more about automatic voice commands.

Compatibility Requirements

The journey.voice service is currently only available on RealWear® compatible devices running version 21.6.1 and later of the JourneyApps Container and version 4.82.0 and later of the JourneyApps Runtime.

The journey.voice service allows developers to register voice commands via JavaScript / TypeScript to trigger custom code when a command is spoken.

Check capabilities

Use the journey.voice.getCapabilities() method check if voice commands are supported:

JS

journey.voice.getCapabilities();

TS

journey.voice.getCapabilities(): Promise<{supported: true}>

which returns an object:

{
  supported: true
}

Reset Commands

On each view, it is recommended to reset the voice service in order to ensure that any old commands are no longer registered.

JS

journey.voice.reset();

TS

journey.voice.reset(): Promise<void>;

Registering voice commands

There are two methods available for registering voice commands: journey.voice.registerCommand and journey.voice.registerCommands.

The first is used to register a single command:

JS

journey.voice.registerCommand(command, callback);

// e.g.

journey.voice.registerCommand("Sync", function() {
  journey.synchronize();
});

TS

journey.voice.registerCommand(command: string, callback: () => void);

// e.g.
await journey.voice.registerCommand('Sync', async () => {
  await journey.synchronize();
});

The second is used to register multiple commands:

JS

journey.voice.registerCommands(commands);

// e.g.
journey.voice.registerCommands([
  {
    command: "Sync",
    callback: function() {
      journey.synchronize();
    }
  },
  {
    command: "Start Inspection",
    callback: function() {
      navigate.link('start_inspection');
    }
  }
]);

TS

journey.voice.registerCommands(commands: VoiceCommand[]);

// where
interface VoiceCommand {
  command: string,
  callback: () => void;
}

// e.g.
await journey.voice.registerCommands([
  {
    command: "Sync",
    callback: () => {
      journey.synchronize();
    }
  },
  {
    command: "Start Inspection",
    callback: () => {
      navigate.link('start_inspection');
    }
  }
]);

Custom commands based on data is also supported. This example maps through the measurement_units DB objects and creates voice commands in the format Select Unit X where X is the specific unit.

let units = await (DB.measurement_units.all().toArray());

let unitCommands = units.map(unit => {
  return {
    command: `Select unit ${unit.name}`,
    callback: async () => {
      await selectUnit(unit.name)
    }
  }
});

await journey.voice.registerCommands(unitCommands);

Focusing a text input

A workflow may require a user to enter some data in a <text-input> component.

To do so, add an id field to the text input component, e.g.

<text-input label="Enter Name" bind="name" required="false" id="username" />

and focus the field using the focus() component method:

await journey.voice.registerCommand('Enter Name', () => {
  component.textInput({id: 'username'}).focus();
});

De-registering a command

If a single voice command should be removed, the journey.voice.deregisterCommand method can be used:

JS

journey.voice.deregisterCommand(command);

// e.g.
journey.voice.deregisterCommand('Enter Name');

TS

journey.voice.deregisterCommand(command: string);

// e.g.
await journey.voice.deregisterCommand('Enter Name');

Special considerations

Voice commands can trigger any JavaScript/TypeScript function that is normally available to developers. This includes examples like:

journey.synchronize();
journey.leaveApp();
journey.photos.capture();
component.textInput({id: 'username'}).focus();
navigate.link('view_name');
journey.hardware.scanBarcode({types: ["EAN_8"]});
notification.show("A new task is available", {buttons: [{
    label: "Go to task",
    onPress: () => {
        navigate.link('task_view');
    }
}]});

In the navigation.show example above, the button's on-press will be called when the label text has been spoken, e.g. "Go to task".

Note that normal buttons aren't automatically registered as voice commands, but can easily be done in this way:

    <button label="Show Confirm" on-press="$:showConfirm()" validate="false" style="solid" />
    <button label="Trigger notification" on-press="$:triggerNotification()" validate="false" style="solid" />

journey.voice.registerCommands([
  {
    command: "Show Confirm",
    callback: showConfirm // not 'showConfirm()'
  },
  {
    command: "Trigger notification",
    callback: triggerNotification // not 'triggerNotification()'
  }
])

Note that the callbacks in this example are the instance of the function, and not calling the function. Therefore, it doesn't have the brackets () at the end of the function name.

Good practices

• Ensure that the user can "say what they see". For example, if a button has the label "Go Back", don't register the command as "Back".

• Don't register single syllable commands like "up", since they can easily be picked up incorrectly. Rather, use something like "Go up".

• The engine relies on spoken commands, so don't use homophones, e.g. "Bored" and "Board", since the device will not be able to distinguish between them.

Note that commands are case-insensitive.