Some of the extended usage examples in the documentation files are in Objective-C due to my having some lingering questions about how to handle the doubled-up example code visually, so please visit the tutorial tool in order to receive the same information in Swift 3 only. Please feel free to seek Swift 3 support for OpenEars in the forums, and I look forward to seeing what is made in Swift!

Thanks for your patience,

Halle

This is just a quick second update that the second part of Swift 3 support is complete: the OpenEars distribution now has a complete Swift 3 version of the sample app for you to work with and it has a complete Swift 3 custom tutorial tool. You can download the sample app at /openears and the Swift 3 sample app is found in the folder titled OpenEarsSampleAppSwift at the top level of the distribution, and you can use the custom tutorial tool at https://politepix.com/openearsswift-tutorial/. Over the next month, Swift 3 support part 3 (full Swift 3 examples in the documentation) of the full Swift 3 support process will roll out, at which point Swift 3 will have support parity with Objective-C. Thanks for your patience, bring any questions over to the forums, and have fun!

-Halle

This is just a quick update that the first part of Swift 3 support is complete: the OpenEars distribution now has a complete Swift 3 version of the sample app for you to work with. You can download it at /openears and the Swift 3 sample app is found in the folder titled OpenEarsSampleAppSwift at the top level of the distribution. Over the next month, Swift 3 support part 2 (full Swift 3 examples in the documentation) and Swift 3 support part 3 (full Swift 3 examples in the tutorial tool) of the full Swift 3 support process will be rolling out, at which point Swift 3 will have support parity with Objective-C. Thanks for your patience, bring any questions over to the forums, and have fun!

-Halle

Update 10/21/16: the tutorials are out now too, read more or go directly to the Swift 3 custom tutorial tool.

Hello again! I wanted to share some information about OpenEars Platform Swift support. I like Swift, and OpenEars works well with Swift (there are self-supported discussions in the OpenEars forums on how to use OpenEars with Swift and I’ve heard of no issues related to the framework – search for the keyword ‘Swift’ in the forums in order to read how to get started with your Swift 2.x projects), but as some OpenEars-using developers have noticed, I’ve been conservative about jumping in and dedicating the same support/documentation/packaging resources that I currently make available for Objective-C to Swift while its IDE stability and syntax was in flux and its tooling and debug capabilities were still emerging.

This has only been for practical reasons – support is very important to me, and time for everything related to support is in limited supply, so I wanted to make sure that supporting Swift simultaneously with Objective-C wouldn’t overextend support resources in a way that might reduce support quality across the board. This was particularly important because the already-shipped or nearly-shipping Objective-C apps which have tended to represent the majority of projects were consequently the support cases where the most was at stake for the most developers.

Due to the exhilarating community participation since the open-sourcing of Swift and the great communication out of the Swift team, I’m confident that supporting Swift in the future isn’t going to be a risk for giving good support to Objective-C projects, so I’m happy to announce that OpenEars and its plugins will receive official Swift support from Politepix for Swift 3 at the time of its release, including docs, tutorial, sample app, Swift-ready packaging, and forum support. As always, I hope this will help you make even more delightful speech-enabled apps!

-Halle

Image by Allie Brosh

Well, perhaps not all of the languages. Many of the languages! I’ve developed a language-agnostic grapheme-to-phoneme algorithm which works from a file format fast enough for a phone, and although perfection remains elusive, it is from 5-15x more accurate for generating phonemes than a naive letter-based algorithm. As a result of this, OpenEars is now able to support speech recognition with English, Spanish, Mandarin Chinese, French, German, and Dutch. The new languages will now work with all of the speech recognition features of OpenEars, including dynamic language model generation and switching, grammar generation and switching, and of course hypothesis return.

As a benefit of the flexibility of this format, today Politepix will also release 2.5 versions of every plugin. RapidEars, Rejecto and RuleORama are now compatible with the languages OpenEars is compatible with. NeatSpeech 2.5 is a compatibility update, but won’t be adding TTS output for the new languages. SaveThatWave also has a compatibility update.

By the way, if you are using another language’s Sphinx-compatible acoustic model and you want to make it compatible with OpenEars, you can get in touch to discuss options – it is a pretty flexible approach so I expect to be able to apply it to more languages.

Speech recognition will vary significantly due to the speed and accuracy level of the acoustic model used, so support for non-English acoustic models is very much on a best-effort basis moving forward. Tut mir leid, Schatz!

The next feature of the OpenEars Platform 2.5 is bitcode. OpenEars will now ship with embedded bitcode, and bitcode is now a configurable option for paid plugin framework purchases (subject to a handling fee). Plugin framework demos will also now have a non-recompilable bitcode segment so that they install easily, but the bitcode attached to the demos can’t be submitted to the App Store. Demos can’t be submitted to the App Store in any case, so this shouldn’t be an issue.

Bitcode is also supported on a best-effort basis, meaning that no warranty is made for what it does when it is recompiled. This is not because I don’t care, but because a) no one knows the minutiae of how it will be recompiled, or b) what architectures it will be recompiled with, or c) what the strategic importance of bitcode is to Apple really, and it is a fair assumption that d) there will not be any communication about what is happening if it doesn’t work, so that is a process that, realistically, Politepix has no control over. What is the advantage of bitcode from the developer perspective? Hoe komt een ezel aan twee lange oren?

This OpenEars Platform update also fixes all verified bugs to date, which is my very favorite kind of update. Other than the ones which add Chinese speech recognition, 哇!

OpenEars 2.5 is free as always. The 2.5 paid plugin framework license upgrades are free if your purchase was made after August 17th, 2014. Upgrades for licenses purchased before August 17th, 2014 are 50% off. The bitcode handling fee is an extra fee for all upgrades if you want to add bitcode to your frameworks. The new Licensee site has rolled out (same URL as before), and on it you will find coupons for your upgrades in your download area, either for a free upgrade or a 50%-off upgrade for each of your licensed plugins. Quel délice!

This is a big update with a lot of moving parts, so if you encounter any surprises or frustrations, no te preocupes, just visit the forums, let me know what’s up, and I will be happy to help.

OpenEars 2.5 can be downloaded here and you can browse the compatible language acoustic models here. Make sure to check out the license for the language to be sure you’re able to use it with your project.

I’m delighted to be able to bring the real potential of localization to offline speech recognition with OpenEars, and I can’t wait to see what you do with it. And to the developers who have been waiting for their language to be compatible, a heartfelt welcome/bienvenue/Willkommen/Welkom/Bienvenido/歡迎!

-Halle

When I first designed (OE)LanguageModelGenerator years ago I made the decision to require text input in uppercase letters for best results because it allowed the most optimization for very fast creation of dynamic language models. This didn’t seem like a big trade-off because at the time the size of a language model needed to be quite small in order to perform well during speech recognition on supported devices such as the 2nd-gen iPhone, which meant that for the most part it was command-and-control applications that were being developed with OpenEars. For a command-and-control vocabulary, word case is not such a big consideration in a UI because the words are out of context. Rather than transforming the developer’s text input automatically, I made the decision to support both all-caps and mixed-case but explain in the docs and in the logging output that mixed-case text input would have to be sent to the fallback phoneme lookup technique which would result in fewer available pronunciations, which would have an accuracy impact for words with multiple pronunciations. This felt like the least-bad compromise between strongly-competing concerns of speed, minimizing complexity, and not discarding the developer’s intentional choices.

Over the last couple of years as the devices and the framework and dependencies have gotten faster, it has become a viable choice with OpenEars to use larger vocabularies, and as a result more app developers have been using it with a broader variety of input sources such as written texts, speeches, etc, which is delightful to see. For that kind of application, the case of the input and output format matters to the developer and the user. The uppercase requirement/advantage no longer supported the goals of the developer or of pleasing UX and needed to be improved, so I revisited this early decision and found a way to do case-insensitive lookup without changing the baseline generation speed, and also improved the generation speed for larger models. That means that you can use normal word and sentence casing in your input text and it will be returned by your speech recognition hypothesis with the same casing intact, and larger text input will generate models faster (this doesn’t affect recognition speed, just how long dynamic model and grammar generation take).

There has also been an improvement in handling of punctuation in input, so in the cases that developers don’t do their own text cleaning to remove symbols which are too ambiguous to transcribe and probably not intended to be spoken (for instance, symbols like { or ^ or `) OELanguageModelGenerator will clean the input and it will be consistent across all the plugins and different model/grammar types. Symbols that can’t be transcribed will be removed, symbols which can be transcribed will usually be transcribed by the best-effort fallback grapheme generator (so you should still take a look at your input when you know it in advance and decide whether it would be better for you to transcribe your symbols into words yourself, especially numbers because only you know for sure whether you want 1600 to be transcribed as ‘sixteen-hundred’ or ‘one thousand six hundred’ or ‘a thousand six hundred’ or ‘one six oh oh’), and symbols which aren’t significant for recognition purposes (such as . or , or ; or ? or !) will be left in place and will become part of your model.

An example of this last point would be if you used the sentence “The Sand Snakes are with me.” as input. OELanguageModelGenerator will successfully find multiple pronunciations for any word in this sentence that has more than one pronunciation – it will leave the case intact and there will be no accuracy decline from that. That period (full stop) symbol at the end will stay attached to the word “me” in the model, meaning that when OEPocketsphinxController returns a hypothesis matching an utterance of the sentence, it will still have the period attached to it in the returned text hypothesis. If this isn’t the desired result and you don’t want the individual words in this input to have hints about their position in a sentence or statement, you can still give the original text to OELanguageModelGenerator without sentence punctuation, but the assumption now is that if you give sentence punctuation as input, it’s because you intend for it to be returned in a hypothesis. That also means that if you create a language model rather than a grammar, you can sometimes see a word with a period or comma appear in a different order in the sentence other than the input order, so that is something to think about when using punctuation and evaluating whether to use a language model (statistical model; words can be returned out of order so a word with a period attached can theoretically appear in the middle of a sentence if someone walks by the user and says it) or a grammar (ruleset; the order you choose is the order that will return).

The decision tree I use for these punctuation transformations and non-transformations is basically a simplified non-interactive version of my interactive text-cleaning tool TheKnownUnknowns, so please feel free to take a look at TheKnownUnknowns alongside OELanguageModelController for more info about considerations with different symbols. Please also feel free to use TheKnownUnknowns for preparing texts for OpenEars where you’d like to make your own decisions in advance about how to transcribe difficult cases. It is primarily designed to quickly clean text corpora before creating an acoustic model using long alignment and similar tasks on large texts that have to be prepared for some kind of transcription-related norm, but it is also a good tool for interactively cleaning text you want to use with OpenEars in advance, since they have their design and major assumptions in common.

Although this is not directly a recognition accuracy change, my sense is that there was a cluster of minor accuracy-related symptoms in some apps related to non-transcribable symbols entering the generator, mixed-case being used without realizing it affected how many pronunciations were found, and the possibility that unknown transcribable or ignorable symbols were being handled differently by the language model/grammar lookup than by the phonetic dictionary lookup, which theoretically could result in never-matching words. Projects that were experiencing any of these issues should see an improvement to accuracy from this change.

As always, OpenEars can be downloaded here and the new plugins can either be downloaded at your demo link or your licensed framework link. I hope this little improvement helps you make great apps!

This year, the CMU Sphinx project did a major revision of their Sphinxbase and Pocketsphinx projects in order to add noise robustness and change their voice activity detection algorithm, among many other cool new features. This was brilliant news, and the work on those features has been deeply impressive to see ship. Many OpenEars-using developers wanted those new noise/VAD features ASAP, and with good reason.

It had an interesting effect on OpenEars’ own development. The very first problem I solved when originally designing this framework was making the calibration and voice activity detection system work with Cocoa Touch audio, which wasn’t an altogether simple thing on the more resource-constrained devices and with the somewhat sparsely-documented state of the low-latency audio APIs for iOS development at that time. More so since the fundamental design of iOS audio was quite different from Linux audio that those CMU libraries were largely used with and developed with. The other capabilities of OpenEars came after these design decisions based on the original calibration and voice activity detection, so most code related to audio or engine usage was designed with deference to those features.

When I sat down this summer to update OpenEars to use the new voice activity detection and noise robustness code, I saw that it was different structurally and didn’t mesh with any of those early decisions anymore, or the ripple-effect decisions that followed them, and that there were two ways I could look at the implications of that: either a) an obligation to add a lot of new scaffolding to attempt to force a similar result from a different design, or b) an opportunity to revisit my original decisions in light of four more years of programming experience.

No difficult choice there – this is our craft, and I’m not interested in adding any more detritus to our collective workspace.

So, the first thing this update adds is functional improvements across the board: it adds noise robustness, better voice activity detection, higher accuracy due to moving to 16k acoustic models with no loss in performance due to greater optimization, and there is now no calibration time needed at all (really, calibration is gone and recognition just starts immediately – this is a fantastic new capability from CMU). The new VAD required a complete rewrite for RapidEars as well, so I applied the same process to RapidEars, which is now more accurate but uses far less CPU under heavy usage, while benefiting from a simplified API. As far as I have been able to verify, there have also been improvements in speech recognition coexistence with video objects, but that is a work in progress and I’ll be interested in hearing your real-world results with video plus speech recognition.

The second benefit of being able to revisit the library design decisions has been in code quality:

• OpenEars code is now using only the most-modern-possible Cocoa Touch APIs and the most-modern-possible Objective-C for the iOS versions it supports.
• OpenEars now has 6947 lines of code versus the previous 10917, which is 63% of the previous amount of code despite having multiple new features.
• The majority of code was removed from the areas most likely to be implicated in an issue, with the predictable improvement to the debugging process.
• The OpenEars classes now produce no general or static analysis warnings, and have no warning suppression (the dependencies, quite unavoidably and normally for a codebase representing multiple decades of development, raise 32-bit implicit conversion warnings and coding-style static analysis warnings, so 32-bit implicit conversion warnings are turned off in build settings after verifying that the OpenEars classes do not raise them, and there are three dependency source files which have the -w flag – if you notice any other warning suppression in OpenEars, let me know since it’s an oversight).
• Consequently, OpenEars is now able to ship with the “Treat warnings as errors” build setting selected.
• OpenEars is now ARC rather than manually memory-managed to allow the best possible optimization in LLVM
• OpenEars continues to support 3 current operating systems (iOS 6.1 through 8.x) so it continues to support more than 98% of installs.
• OpenEars now ships with several of my asynchronous XCTests which you can also use as examples for creating your own asynchronous XCTests for speech recognition (my actual XCTest testbed is a lot larger but it depends on some audio files and code I don’t have permission to ship, as well as the plugin tests). There are also some fuzzing tests using my cross-thread fuzzing tool HWHorrorShow if you’re into that sort of thing.
• OpenEars now follows platform guidelines and uses a class prefix (OE) since there were already occasional conflicts coming up with class names such as AudioSessionManager as the framework was more widely-adopted (and it’s just good citizenship). This mean you need to do some renaming when integrating 2.0, but I figured that a major version release was the only time it was going to be acceptable to address, so I bit the bullet and did it with this release. To reduce the pain from the class naming changes I have made a step-by-step upgrade guide which also covers some API changes in 2.0.

As you can see, this is a big update – it affects a lot of visible things and invisible things and that means some issues are guaranteed, so please upgrade using the upgrade guide or do a new install from the updated tutorial tool and let me know in the forums about any problems, questions, or upgrade/installation troubles so I can help you.

Download links:

OpenEars 2.0
NeatSpeech 2.0
RapidEars 2.0
Rejecto 2.0
RuleORama 2.0
SaveThatWave 2.0

Please read the upgrade guide covering naming changes and API changes (you can’t upgrade without it).

Thank you for choosing OpenEars, apologies in advance for any bumps getting into the new version, and I hope that the results of all the work that has gone into this huge update will delight you and the users of your apps.

All the best,

Halle

Still: I was quite compact, and even I noticed that the clearance inside the capsule was a little tight. Perhaps too tight? As exciting as it was to get into the capsule, it was always a relief to climb back out again into the big, airy museum room after the mission was completed. It was a good exhibit; without laying it on too thick, it managed to communicate that space travel was heroic not only because it was an adventure, but also because it was a sacrifice. And like nearly all children who played astronaut, I grew up into an earthbound adult without noticing it, other than in the rare moments that something illuminated that stored-away dream.

I mention this just to explain why I was particularly excited when I originally saw a repository under NASA’s GitHub account using OpenEars. I got some time to catch up with Rashid Sial of TopCoder, and he filled me in on the details.

The NASA Tournament Lab International Space Station Food Intake Tracker project was originally a 2013 contest conducted by the NASA Tournament Lab, under contract with Harvard University and built by the TopCoder community, to develop an iPad app that the astronauts and cosmonauts on the International Space Station could use to track their food intake, with OpenEars including the CMU Sphinx project offered as one handsfree interface option for participating developers. From the repository README.md:

“Astronauts, Cosmonauts, and Space Cadets (okay, we made the last one up), all face huge technical challenges, are performing scientific experiments on a daily basis, and are working hard to stay fit. One of the key aspects of this is working to understand how microgravity affects their bodies, and how to best keep them healthy. And as part of both space medicine and science, we need to understand what they’ve eaten, and how much they’ve eaten.”

The amazing news: Rashid told me that now that the contest is over, the app is actually going to ship. To the International Space Station.

Tentatively scheduled for 2015, the International Space Station Food Intake Tracker from NASA Tournament Lab is another great app made with OpenEars and I’m, ahem, over the moon to have been able to contribute. Thank you Rashid and the NASA Tournament Lab for letting me know about this awesome project!

-Halle

¹This 1970s-vintage ad is such an interesting example of the fallacy that progress moves in a straight line. Check out how un-self-conscious it is about the assertive little girl with a pack of questions and her no-prob-we’ll-go-learn-some-science-together dad.

²Unless you were a sports fan!

³If you were lucky enough to be little while we still were laboring under the misapprehension that T. Rex stood upright, that is. They’ve now fixed the posture of the T. Rex in the museum so it is scientifically accurate, but it now fits entirely in a single story so I’m afraid the thrill factor is significantly reduced.

Last September I was at iOSDevUK, which is a lovely iOS developer conference on the Welsh coast at the University of Aberystwyth, and a friend asked me what the next feature for OpenEars would be after Spanish support.

I said that before I added anything else, I needed to get rid of technical debt and fix all the known bugs in OpenEars and the plugins, and unbeknownst to me there was also a 64-bit architecture change looming on the horizon. So, in reality it ended up being “get everything 64-bit functional, pay off some technical debt, fix all the known bugs, then fix the bugs I introduced while fixing the bugs.” That last one was OpenEars 1.66 that came out last week.

There was another answer I wanted to give because there’s been something nagging at me for about the last year, but it was very clear that after all of the new technology additions in 2012 and 2013 to Politepix’s line it was time to do some maintenance and make sure things would be sustainable.

But something really was nagging at me.

Since February 2013 I’ve been extremely lucky to get to give some talks at industry conferences about speech recognition and speech synthesis as a human interface for mobile apps.

In my talk I always mention that for an app with a small vocabulary, one element of good interface design is deciding whether you want to use a probability-based language model such as an ARPA model, or a rules-based grammar such as a JSGF grammar, because choosing the right one for your use case has a large potential for improving user experience. This is a reasonable enough observation because Pocketsphinx supports the JSGF format so OpenEars does as well.

However, I always felt a bit shabby pointing this out and then leaving developers to write their own JSGF, which is a relatively complex and unpretty format – with imposing documentation – when you write it by hand, and which also has a few minor differences between the complete format definition and the implementation in OpenEars that can lead to difficult troubleshooting, while ARPA probability model generation in OpenEars is as easy as putting NSStrings in an NSArray.

So, since last September, a little bit here and a little bit there in between the important bugfix and architecture releases, I’ve been working on that thing that was nagging at me: dynamic generation of JSGF grammars using clear, human-friendly language and NSObjects. Today I am very happy to announce the first version of this feature in OpenEars 1.7.

You’ll find a new method in LanguageModelGenerator which is a counterpart to this ARPA-generation method:

- (NSError *) generateLanguageModelFromArray:(NSArray *)languageModelArray withFilesNamed:(NSString *)fileName forAcousticModelAtPath:(NSString *)acousticModelPath;

- (NSError *) generateGrammarFromDictionary:(NSDictionary *)grammarDictionary withFilesNamed:(NSString *)fileName forAcousticModelAtPath:(NSString *)acousticModelPath;

As you can see, instead of taking an NSArray it takes an NSDictionary. The NSDictionary you submit to the argument generateGrammarFromDictionary: is a key-value pair consisting of an NSArray of words stored in NSStrings indicating the vocabulary to be listened for, and an NSString key which is one of the following human-language constants defined in GrammarDefinitions.h, indicating the rule for the vocabulary in the NSArray:

ThisWillBeSaidOnce
ThisCanBeSaidOnce
ThisWillBeSaidWithOptionalRepetitions
ThisCanBeSaidWithOptionalRepetitions
OneOfTheseWillBeSaidOnce
OneOfTheseCanBeSaidOnce
OneOfTheseWillBeSaidWithOptionalRepetitions
OneOfTheseCanBeSaidWithOptionalRepetitions

Breaking them down one at a time for their specific meaning in defining a rule:

ThisWillBeSaidOnce // This indicates that the word or words in the array must be said (in sequence, in the case of multiple words), one time.
ThisCanBeSaidOnce // This indicates that the word or words in the array can be said (in sequence, in the case of multiple words), one time, but can also be omitted as a whole from the utterance.
ThisWillBeSaidWithOptionalRepetitions // This indicates that the word or words in the array must be said (in sequence, in the case of multiple words), one time or more.
ThisCanBeSaidWithOptionalRepetitions // This indicates that the word or words in the array can be said (in sequence, in the case of multiple words), one time or more, but can also be omitted as a whole from the utterance.
OneOfTheseWillBeSaidOnce // This indicates that exactly one selection from the words in the array must be said one time.
OneOfTheseCanBeSaidOnce // This indicates that exactly one selection from the words in the array can be said one time, but that all of the words can also be omitted from the utterance.
OneOfTheseWillBeSaidWithOptionalRepetitions // This indicates that exactly one selection from the words in the array must be said, one time or more.
OneOfTheseCanBeSaidWithOptionalRepetitions // This indicates that exactly one selection from the words in the array can be said, one time or more, but that all of the words can also be omitted from the utterance.

Since an NSString in these NSArrays can also be a phrase, references to words above should also be understood to apply to complete phrases when they are contained in a single NSString.

A key-value pair can also have NSDictionaries in the NSArray instead of NSStrings, or a mix of NSStrings and NSDictionaries, meaning that you can nest rules in other rules.

Here is an example of a complete, complex ruleset which can be submitted to the generateGrammarFromDictionary: argument. It is designed to be easily readable as a collection of English sentences, however I have also followed this version with another one that has a step by step explanation of each part:

 @{
     ThisWillBeSaidOnce : @[
         @{ OneOfTheseCanBeSaidOnce : @[@"HELLO COMPUTER", @"GREETINGS ROBOT"]},
         @{ OneOfTheseWillBeSaidOnce : @[@"DO THE FOLLOWING", @"INSTRUCTION"]},
         @{ OneOfTheseWillBeSaidOnce : @[@"GO", @"MOVE"]},
         @{ThisWillBeSaidWithOptionalRepetitions : @[
             @{ OneOfTheseWillBeSaidOnce : @[@"10", @"20",@"30"]}, 
             @{ OneOfTheseWillBeSaidOnce : @[@"LEFT", @"RIGHT", @"FORWARD"]}
         ]},
         @{ OneOfTheseWillBeSaidOnce : @[@"EXECUTE", @"DO IT"]},
         @{ ThisCanBeSaidOnce : @[@"THANK YOU"]}
     ]
 };

So that’s the whole thing – that is all it takes to create a complex ruleset in OpenEars 1.7.

Breaking it down step by step to explain exactly what the contents mean:

 @{
     ThisWillBeSaidOnce : @[ // This means that a valid utterance for this ruleset will obey all of the following rules in sequence in a single complete utterance:
         @{ OneOfTheseCanBeSaidOnce : @[@"HELLO COMPUTER", @"GREETINGS ROBOT"]}, // At the beginning of the utterance there is an optional statement. The optional statement can be either "HELLO COMPUTER" or "GREETINGS ROBOT" or it can be omitted.
         @{ OneOfTheseWillBeSaidOnce : @[@"DO THE FOLLOWING", @"INSTRUCTION"]}, // Next, an utterance will have exactly one of the following required statements: "DO THE FOLLOWING" or "INSTRUCTION".
         @{ OneOfTheseWillBeSaidOnce : @[@"GO", @"MOVE"]}, // Next, an utterance will have exactly one of the following required statements: "GO" or "MOVE"
         @{ThisWillBeSaidWithOptionalRepetitions : @[ // Next, an utterance will have a minimum of one statement of the following nested instructions, but can also accept multiple valid versions of the nested instructions:
             @{ OneOfTheseWillBeSaidOnce : @[@"10", @"20",@"30"]}, // Exactly one utterance of either the number "10", "20" or "30",
             @{ OneOfTheseWillBeSaidOnce : @[@"LEFT", @"RIGHT", @"FORWARD"]} // Followed by exactly one utterance of either the word "LEFT", "RIGHT", or "FORWARD".
         ]},
         @{ OneOfTheseWillBeSaidOnce : @[@"EXECUTE", @"DO IT"]}, // Next, an utterance must contain either the word "EXECUTE" or the phrase "DO IT",
         @{ ThisCanBeSaidOnce : @[@"THANK YOU"]} and there can be an optional single statement of the phrase "THANK YOU" at the end.
     ]
 };
 

So as examples, here are some sentences that this ruleset will report as hypotheses from user utterances:

"HELLO COMPUTER DO THE FOLLOWING GO 20 LEFT 30 RIGHT 10 FORWARD EXECUTE THANK YOU"
"GREETINGS ROBOT DO THE FOLLOWING MOVE 10 FORWARD DO IT"
"INSTRUCTION 20 LEFT 20 LEFT 20 LEFT 20 LEFT EXECUTE"

But it will not report hypotheses for sentences such as the following which are not allowed by the rules:

"HELLO COMPUTER HELLO COMPUTER"
"MOVE 10"
"GO RIGHT"


The last two arguments of the new LanguageModelGenerator method work identically to the equivalent language model method. The files created, instead of being .DMP and .dic like for an ARPA model, are .gram and .dic where the .gram is your JSGF. So now when you pass your .gram file to the Pocketsphinx method:

- (void) startListeningWithLanguageModelAtPath:(NSString *)languageModelPath dictionaryAtPath:(NSString *)dictionaryPath acousticModelAtPath:(NSString *)acousticModelPath languageModelIsJSGF:(BOOL)languageModelIsJSGF;

you will set the argument languageModelIsJSGF: to TRUE.

The goal of creating this API was not only to make it much easier to create a grammar (or multiple grammars to switch between) before runtime, but also to make an interface powerful and simple enough that you can build new grammars dynamically at runtime based on arbitrary input.

JSGF isn’t compatible with RapidEars, so I will also be releasing a new product shortly which will allow the same grammar language to be used to output RapidEars-compatible grammars as well, but this is a new type of thing altogether so it will be in testing for a while longer. The new product will also provide faster grammars for stock OpenEars since JSGF searching is a bit too resource-intensive for ideal responsiveness on 32-bit devices.

I am delighted to finally release this. There’s really nothing that makes me happier as a developer than to take something a little gnarly like JSGF and find a way to make it accessible and amenable to humanistic interface designs from developers who don’t necessarily have time or interest in specializing in speech interface to the extent that it can take to get to grips with a format like JSGF, so I hope I achieved that and that we’ll see it in some of your awesome apps.

It’s a 1.0 feature, so there will be some bumps and you should bring ’em right to the forums so I can take a look.

Thanks and enjoy your development,

Halle

UPDATE: I’ve finished the new plugin for generating fast grammars which are also compatible with RapidEars – it’s called RuleORama and you can read about it here and learn how to implement it using the tutorial.

I’m happy to announce the release of OpenEars 1.66 featuring a number of helpful fixes affecting both stock OpenEars and also Rejecto and NeatSpeech:

• A fix to the order of the Spanish lookup dictionary that could affect accuracy
• Improvements to the fixes to voice audio detection from 1.65
• A fix for an issue that could cause OpenEars+Rejecto to never return hypotheses for long utterances with intermittent background noises
• A fix for incorrect suspend/resume behavior using FliteController and FliteController+NeatSpeech
• A fix for an issue when changing between JSGF files
• A bit more checking for null strings in language model corpora
• A fix of some incorrect memory management that could sometimes lead to wrong probabilities in language models (including Rejecto language models)
• Other minor memory management improvements
• Changed type of frame index to avoid overruns

I hope this is helpful and of course let me know about any issues you encounter in the forums.

[politepix-blog-inline-text-ad]