- This topic has 17 replies, 2 voices, and was last updated 9 years, 11 months ago by .
-
Posts
-
I’m developing speech recognition application based on OpenEars and RapidEars. It works fine on iPhone 5 and 5s, but there is some bug on iPhone 4: OpenEars doesn’t stop recognition loop immediately when stopListening method is called. It’s delay correlates with quantity of recognized words in current loop and can be up to 1 minute and more… So, how can I stop recognition loop immediately (or at least during several seconds)? Below is openears log. Thanks in advice.
2014-05-06 18:11:37.404 MyApp[2221:907] Stop button pressed
time delta 1399389097.435529 partial result fwdtree
partial: <s>: …[recognized words]… :(0000001)
INFO: file_omitted(0): cmn_prior_update: from < 12.80 -1.94 -0.17 0.24 -0.35 -0.17 -0.14 -0.17 -0.31 -0.08 -0.17 -0.13 -0.13 >
INFO: file_omitted(0): cmn_prior_update: to < 12.93 -2.03 -0.19 0.26 -0.36 -0.15 -0.15 -0.13 -0.34 -0.06 -0.16 -0.14 -0.10 >
INFO: file_omitted(0): idx 1380 is final (1380)
INFO: file_omitted(0): fwdtree: marking arc buffer final
INFO: file_omitted(0): fwdtree: allocated 4096 arcs (48 KiB)
INFO: file_omitted(0): fwdtree: allocated 512 start frame entries (2 KiB)
INFO: file_omitted(0): Waiting for 1 consumers to finish
INFO: file_omitted(0): idx 1380 is final (1380)
time delta 1399389118.439060 end 0000001
2014-05-06 18:11:58.439 MyApp[2221:907] Status: Detected end of speech, Code: 4
INFO: file_omitted(0): fwdflat canceled
INFO: file_omitted(0): fwdtree canceled
INFO: file_omitted(0): TOTAL fwdtree 48.961 wall 2.135 xRT
INFO: file_omitted(0): TOTAL fwdflat 77.410 wall 3.376 xRT
2014-05-06 18:11:58.605 MyApp[2221:907] Status: It has stopped listening, Code: 3Welcome,
Are both the frameworks up to the current version? OpenEars is 1.7 and RapidEars is 1.65.
What are the changes to OpenEars?
OK, can you show the full logging output of both OpenEarsLogging and verbosePocketsphinx from the beginning to the end of the app session? It’s fine to omit anything that isn’t logging that OpenEars creates and to omit specific vocabulary as long as the vocabulary count isn’t changed.
Sent you an email – get in touch when you have a moment so I can assist you with this.
Thanks for getting in touch! OK, so, I’ve taken some time to look at your logging output and I think what I’m seeing is that this isn’t using one of the two supported 8khz acoustic models that ship with RapidEars, is that correct? It looks like this is probably one of the 16khz Voxforge models if I’m not mistaken, which is why the performance is going to be slow with RapidEars. Can you let me know if there are any other modifications before we look into if there are any ways to improve the performance for the slower 16khz models on the old devices? A few of the other Sphinx runtime arguments look a bit unexpected to me so it would be good to know if any other changes have been made so that the troubleshooting time can be put to the most efficient use.
Ah OK, you are using your own model, very cool. It is possible that the 16khz isn’t the only issue but it is definitely going to interact with any other issues on the older devices because that’s a lot more analysis to do.
Probably it’s best to start out by showing the logging from the iPhone 4 from an incident when it is showing this issue so I can get a look at which stage in the listening loop the behavior is happening in, and then we can look at possible causes and maybe find a way to fix or improve the behavior (I can’t guarantee there is a fix for this since it isn’t one of the models I ship, but I’m happy to work with you to try to improve things). It’s fine removing the vocabulary from the logs, but do let me know the reported size of the vocabulary so I can get a sense of what we’re trying to do here.
BTW, let me suggest that you create a replicable case for yourself using PocketsphinxController’s pathToTestFile property which will let you use prerecorded audio in place of live audio in the live recognition loop. That way it will be easy for you to verify whether any changes help and you’ll also have some replicable data for me in case you want to send me a test case. It also just makes development a lot easier when you don’t have to provide live input yourself while working on a bug.
OK, I think this is probably going to be the root of the issue – RapidEars is only designed for use with the shipped acoustic models and with the normal OpenEars-required language model maximum size of around 200-500 words depending. I can imagine your specs working some of the time on a 5S, but it’s too much overhead for an iPhone 4 to support. I think you’re just seeing a delayed search time on a slow device for a 16khz model doing large vocabulary recognition – speed considerations are the reason that even stock OpenEars doesn’t use 16khz models and big vocabularies, and RapidEars is more sensitive to performance considerations since it has to do much more.
If you’re finding the performance acceptable on devices later than the iPhone 4, I think what I would do is to make a downsampled version of your acoustic model, ship it alongside your 16khz model, and have the iPhone 4 and earlier fall back to using the 8khz model. This might help your issue sufficiently, but it also might be necessary for you to have the iPhone 4 fall back to a smaller vocabulary as well in order to achieve normal performance. Those would be my starting suggestions so that you can preserve your ideal performance for your users whose devices can support it without giving the older phones a job to do that they aren’t up for.
I thought of another way you can fall back to some faster iPhone 4 performance along with the other two suggestions. In the file PocketsphinxRunConfig.h there is this sequence:
#if __LP64__ #define kBEAM @"1e-96" // -beam Beam width applied to every frame in Viterbi search (smaller values mean wider beam), defaults to 1e-48 #else #define kBEAM @"1e-66" // -beam Beam width applied to every frame in Viterbi search (smaller values mean wider beam), defaults to 1e-48 #endif
You can set the value 1e-66 to a larger value such as 1e-48 for faster searches on older devices. I would leave it as-is for the devices that are already performing satisfactorily.
Hi,
There’s a bit too much missing from the logging above. Can you email me the complete logging output from the beginning to the end with nothing removed? There also needs to be some NSLogging for the method you use to call stopListening so some programmatic indication of the fact that it was called appears inline in the log with a datestamp, since I can’t get any information out of the logs with the sequence deleted in which stopListening was called.
BTW, did you see that pathToTestFile isn’t working in the case above? Here is the error:
Warning: There is a request to use pathToTestFile but there isn’t a usable file at the location given (“/var/mobile/Applications/A786BFDD-68F9-4FAF-A535-47561B580771/Documents/test.wav”) so the live microphone will be used instead.
- You must be logged in to reply to this topic.