Automatic Voice Commands - Advanced

Renaming automatic voice commands

In some cases, developers may want to overwrite an automatically registered voice command to a synonym or a different phrase, or add aliases.

Updating UI components' voice commands

A user's interaction with UI components is driven by component methods. For example, the text-input component includes the focus() and clear() methods, which allow users to enter and clear text respectively.

Voice commands are tied to these component methods. This means that in most cases, when a user speaks a voice command, a component method will be triggered under the hood.

This can be illustrated as follows:

component
│
└─── method
    │
    └─── voice command

Apps ship with a set of default voice commands for component methods. Developers can change these default commands on a component, view or app level (see the Component Styling and Configuration section for further information about the distinction between these levels). Below are some examples to illustrate.

Tip

You can use OXIDE's auto-complete within the app's config.json file to view a component's configurable methods and associated voice commands. Alternatively, these are also displayed in OXIDE's View Components panel.

Ensure you have the Show configuration files and Advanced theme support settings enabled.

Example: configuring the voice commands for the button fire-action() method on a component level:

<button label="Submit questionnaire" voice-fire-action="$label, Submit form" on-press="$:buttonClick()" validate="false" />

In this example, the following voice commands are registered for the button:

• "Submit questionnaire"

• "Submit form"

As seen in the above example, the voice-...="" attributes can be a comma separated list to register aliases for a component method.

Example: configuring the voice commands for the text-input focus() method on the main view:

main.config.json

{    
    "behaviors": [
        {
            "name": "default",
            "components": {
                "text-input": {
                    "methods": {
                        "focus": {
                            "voice": [
                                "$label",
                                "Select $label",
                                "Edit $label",
                                "Select textbox"
                            ]
                        }
                    }
                }
            }
        }
    ]
}

In this example, given a text-input with a label of, say, "First name", the following voice commands are registered to allow a user to focus on the text-input to enter a value:

• "First name"

• "Select First name"

• "Edit First name"

• "Select textbox"

Example: configuring the voice commands for the text-input clear() method on an app level:

config.json

{
    "v4_color_overrides": {
        ...
    },
    "behaviors": [
        {
            "name": "default",
            "components": {
                "text-input": {
                    "methods": {
                        "clear": {
                            "voice": [
                                "Clear text",
                                "Clear textbox"
                            ]
                        }
                    }
                }
            }           
        }
    ],
    "themes": [
        ...
    ],
    ...
}

In this example, the following voice commands are registered for a text-input once the user enters a value:

• "Clear text"

• "Clear textbox"

Updating global commands

Overwriting global commands such as "App help" and "Scroll down" is not currently supported. Please let us know via roadmap.journeyapps.com if you have a use case for this.

Handling translations

Version compatibility

Voice support has been introduced on the Set Language view in version 4.86.6 of the JourneyApps Runtime.

App translations currently have limited support. For UI component labels and other values that are translated, the translated value (i.e. the value visible to the app user) will register as the default voice command. For example, the voice command for a button with a label as label="{$:t('start_job')}" will be the "Start Job" translation in the respective language.

Translation limitations

• Global commands such as "Scroll Up", "Show App help" as well as the default voice command prefixes, such as "Select {field label}" are not currently being translated.

Debugging and troubleshooting

JourneyApps provides developers with a Voice Debugger during development. This helps in situations where a RealWear device is not available, or in cases where developers want to quickly see what voice commands are available at any given time for app users.

To open the voice debugger, enroll to your app on web, open the diagnostics view and click on "Voice Debugging". A popup window will open (check that popups are allowed) which lists the available voice commands for users. This list will include automatically and manually registered commands.

Click on a command in the debugger to simulate a voice command.

In some cases the Voice Debugger is no longer showing the correct voice commands, please reload the app's browser tab.

Known issue

When enrolling in an app on Web with the Voice Debugger enabled, you may need to initially refresh your app's tab for all voice commands to appear in the list.

Limitations

• Automatic voice commands are currently not supported for the following UI components: card, capture-coordinates, display-coordinates, object-list, power-bi, sidebar, typeahead

• The object-table component currently has limited support: read-only tables and their controls (e.g. to paginate, or search) are supported, as well as firing a row-level action. Functionality that is not currently supported includes the ability to edit cells, and to sort or filter columns.

• For list, voice commands are registered for the header only, and will trigger the list item's associated action.

For the above cases, you should be able to manually register voice commands. Please reach out to us via roadmap.journeyapps.com if you have a voice control use case that is not currently supported.