Editing the Base Vocabulary

The Getting Started Code article of this knowledge base provides an overview of adding and deleting phrases to the base vocabulary. Those are the only interfaces needed to extend the base vocabulary with custom actions.

This article builds on those concepts to create a more robust mechanism to edit the base vocabulary. Editing the base vocabulary allows you to leave some portion of it unchanged and add your own customization on top of it.

The advantage of editing the base vocabulary, rather than re-inventing it, is to maintain a consistent user experience between your app and other applications. This reduces confusion and training for end users. For example, it would be confusing for the user to say "move left" in all other applications, then say "to the left" in your application. So, in most instances, it is best to leave similar functionality unchanged from the base vocabulary.

Methodology

The base vocabulary is set by the device operating system based on the active system language. The built-in commands may change from version to version. Therefore, commands like:

sc.deletePhrase("Voice Off");

are only guaranteed to work for customers running in English on the same OS version originally tested against. A future OS version may replace the "Voice Off" phrase with another similar phrase, rendering that code incorrect.

More robust interfaces were introduced in SDK v1.8 to allow applications to easily and reliably edit the base vocabulary. These allow the current base vocabulary phrases to be queried, so the correct set of phrases may be specified. Using the new interfaces described below, the "voice off" example would instead be written:

for( String voiceOffPhrase : sc.getVoiceOffPhrases() ) {
	sc.deletePhrase(voiceOffPhrase);
}

Instead of hard-coding the voice off phrase, we query the engine for that information using getVoiceOffPhrases(). This example will behave identically in all languages, including ones to which your application is not translated; and will continue to work if the actual phrases are changed by future OS revisions.

Phrase Queries

The phrases in the active vocabulary can be queried based on functionality. Some actions have multiple phrases, so each query returns a List of phrases where each phrase is the exact spoken phrase in the current language. An empty list is returned if no phrases match the request.

These interfaces allow you to create detailed help pages describing the built-in phrases, and allow you to delete the phrases you do not want active.

Queries include:

• getPhrases() - returns all phrases

• getWakeWordPhrases() - returns the current wake words, such as "Hello Vuzix", that bring the speech command engine to the active triggered state where it listens for the full vocabulary.

• getVoiceOffPhrases() - returns the current phrases, such as "voice off", that take the speech command engine out of the active triggered state so it once again listens only for wake words.

• getStopPhrases() - returns the current phrases, such as "stop", that terminate previous scroll requests.

• getIntentPhrases() - returns phrases associated with any label you created with defineIntent(). This interface is not used in editing the base vocabulary.

• getKeycodePhrases() - return phrases associated with a specific keycode, such as "go left" is associated with KEYCODE_DPAD_LEFT. This is described more below.

• getBuiltInActionPhrases() - returns phrases associated with various pre-defined actions, such as "take a picture". This is described more below.

Making Queries

The above queries simply return a list of strings that are appropriate for display to the user, or to be sent to the deletePhrase() method described below. For example

List<String> allPhrases = sc.getPhrases();

Querying Key Codes

The getKeycodePhrases() query for phrases associated with keycodes takes two parameters: keycode and keyFrequency.

The keycode can be any valid key defined in the KeyEvent class, such as KEYCODE_DPAD_LEFT.

The keyFrequency can limit the results to phrases associated with single-press or repeating (scroll) keys. The valid values are:

• KEY_FREQ_SINGLE_OR_REPEAT - match all phrases whether single-press or repeating (scrolling). Both "go left" and "scroll left" would match this.

• KEY_FREQ_SINGLE_PRESS - match only single-press actions. Only "go left" would match this, not "scroll left".

• KEY_FREQ_REPEATING - match only repeating actions. Only "scroll left" would match this, not "go left".

So the request for the "go left" phrase would be

List<String> leftPhrases = sc.getKeycodePhrases(KeyEvent.KEYCODE_DPAD_LEFT, VuzixSpeechClient.KEY_FREQ_SINGLE_PRESS);

It is possible to iterate over all valid key constants to determine if they have at least one valid phrase or you can query to identify keys that have at least one valid phrase associated using getMappedKeycodes(). This takes a single argument keyFrequency as described above, and returns a List of keycode values.

This example shows a query for every phrase associated to a key:

int selectedFrequency = VuzixSpeechClient.KEY_FREQ_SINGLE_OR_REPEAT;
for ( Integer keycode : sc.getMappedKeycodes(selectedFrequency) ) {
	for ( String eachKeyPhrase : sc.getKeycodePhrases(keycode, selectedFrequency) ) {
		// todo: Do something with keycode and eachKeyPhrase
	}
}

Query Built-In Actions

The operating system provides some special built-in actions which vary by device. You can query the associated phrases using getBuiltInActionPhrases(). This interface takes a single parameter to identify the action. The action may be one of:

• BUILT_IN_ACTION_SLEEP

• BUILT_IN_ACTION_COMMAND_LIST

• BUILT_IN_ACTION_SPEECH_SETTINGS

• BUILT_IN_ACTION_FLASHLIGHT_ON (M-Series only)

• BUILT_IN_ACTION_FLASHLIGHT_OFF (M-Series only)

• BUILT_IN_ACTION_VIEW_NOTIFICATIONS

• BUILT_IN_ACTION_CLEAR_NOTIFICATIONS (Blade only)

• BUILT_IN_ACTION_START_RECORDING

• BUILT_IN_ACTION_TAKE_PHOTO

• BUILT_IN_ACTION_ASSISTANT (Blade only)

• BUILT_IN_ACTION_ASSIST_ALEXA (Blade only)

• BUILT_IN_ACTION_ASSIST_GOOGLE (Blade only)

For example, to determine which phrase will turn the screen off, we could query:

List<String> sleepPhrases = sc.getBuiltInActionPhrases(VuzixSpeechClient.BUILT_IN_ACTION_SLEEP);

Removing Existing Phrases

Removing existing phrases may reduce the likelihood that the speech command engine resolves the incorrect phrase. This is especially true if your phrases sound similar to default phrases. For example, a game control of "roll right" might be confused with the default phrase "scroll right."

It is also helpful to simply remove phrases that don’t apply in your application. If you do not provide a “command list” you can delete all phrases associated with BUILT_IN_ACTION_COMMAND_LIST. This prevents the system command list from being displayed with the incorrect vocabulary for your app.

Perhaps your application uses explicit commands for each action, and does not require the user to navigate up/down/left right. In this case you could remove default navigation movement key commands to prevent confusion.

Any phrase in the vocabulary (either your custom phrases or the base vocabulary) may be removed by calling deletePhrase(). Any phrase returned from any of the queries mentioned above may be passed to deletePhrase() to remove the base vocabulary phrase from your app vocabulary.

for( String voiceOffPhrase : sc.getVoiceOffPhrases() ) {
	sc.deletePhrase(voiceOffPhrase);
}

The entire vocabulary may be removed from your activity using:

sc.deleteAllPhrases();

When editing the base vocabulary it is often preferred to use the deleteAllPhrasesExcept() interface. This interface takes a List of phrases to keep, all others are deleted. This is useful when an application wants to keep some portion of the base vocabulary and delete all all other actions. As the OS is updated more built-in actions may be added. So, rather than re-releasing your application with more and more delete commands, you can create a list of base phrases to keep and delete the others.

This example deletes all phrases except the wake word, voice off word, those mapped to key presses and the stop phrase.

VuzixSpeechClient sc = new VuzixSpeechClient(this); 
ArrayList<String>keepers = new ArrayList<>();
// Iterate over each key that has any speech mapping
for ( Integer keycode : sc.getMappedKeycodes(VuzixSpeechClient.KEY_FREQ_SINGLE_OR_REPEAT) ) { 
    // Add the phrases for this key to the list of phrases to keep
	keepers.addAll(sc.getKeycodePhrases(keycode, VuzixSpeechClient.KEY_FREQ_SINGLE_OR_REPEAT)); 
} 
keepers.addAll(sc.getWakeWordPhrases()); 
keepers.addAll(sc.getVoiceOffPhrases()); 
keepers.addAll(sc.getStopPhrases()); 
sc.deleteAllPhrasesExcept(keepers);

Replacing Default Behavior

It is important to note that when the speech recognizer gets a command that is already in the list, the previous command is overwritten. So there is no need to delete the original entry before replacing it. This is useful if you want to implement your own navigation methods using "go up", "go down", "go left", etc.

For example, this will remove the repeating “scroll left” commands, and replace “move left” with a custom implementation

int selectedFrequency = VuzixSpeechClient.KEY_FREQ_SINGLE_OR_REPEAT;
for ( Integer keycode : sc.getMappedKeycodes(selectedFrequency) ) {
	for ( String eachKeyPhrase : sc.getKeycodePhrases(keycode, selectedFrequency) ) {
		// todo: Do something with keycode and eachKeyPhrase
	}
}