- This topic has 19 replies, 2 voices, and was last updated 7 years, 1 month ago by .
-
Posts
-
Hello, OpenEars team.
I need recognize some key words from speeching in German. I used OpenEars+Rejecto. But in recognized process was detected other words from words dictionary not met in the text. I was changed value on weight and vadThreshold but I didn’t got result that I was needed. How I can to improve recognition?Welcome,
The different acoustic models have very different accuracy and performance levels, so a lot of variance should be expected (this is discussed on the page about other languages), but we can investigate this a bit to see if there are any underlying causes that can be addressed other than the model itself. How are you obtaining the German speech that you are testing?
Can you clarify more about what translator dictation is?
OK, and what are the regional German accents of the people reading the text you mentioned as the second approach? i.e. what parts of Germany did they grow up in?
OK, so when you test with people you are only ever testing using native German speakers, is that an accurate statement?
Can you tell me a little bit about the process of evaluating the results? i.e., how do you hear about the recognition rate and results from the native speakers who are testing for you, and how do you reproduce results you’ve heard about? The reason that I ask is that usually we don’t have an office full of native speakers of other languages we can just observe directly as we could for our own native language, so the method for gathering a large cross-section of native speaker data for another language is the underlying condition that makes it possible to improve recognition by adjusting the speech interface or framework settings.
No, my question is more about how the testing is happening – I don’t expect that you have a group of native German-speaker testers that you can directly observe in your office as they use the app in the same way you could do so for a local language, so how are you observing their testing and receiving the results?
Yes, thank you for clarifying. It’s quite important to get good testing data before you start trying to fix issues by altering settings, because changing them on the basis of bad data will result in worse results for the average user and a situation where it isn’t possible to get help (for instance from me) due issues with subjective data collection such as having too few reports, non-replicable reports, or reports in which there were other occurrences which affected recognition that you don’t know about (for instance noise or distance) so let’s talk a little bit about how to set up tests for languages that aren’t being tested firsthand in the office.
The first thing to keep in mind is that you can’t use any synthesized speech (like Yandex) because it doesn’t have enough data, so that will only confuse your troubleshooting process.
The second thing is that when you test with humans, to not rely on subjective reports of interactions when troubleshooting, even in your own language but especially with a language you aren’t testing natively in-house, at least until you have a high level of confidence in what is happening, because you have no way of seeing the environmental situation or of replicating the results. And then in that case I am actually the third party removed from the original subjective report so I can’t help effectively (assuming it isn’t just a limitation of the acoustic model but something in the framework that I can help with).
It’s possible for you to obtain complete recordings of the user speech and then to feed them into OpenEars in test mode, so you can replicate the user’s experience. This post is about giving me replicable cases, but it also explains how to use the SaveThatWave demo in order to obtain audio and then use pathToTestFile so you can observe the results yourself: https://www.politepix.com/forums/topic/how-to-create-a-minimal-case-for-replication/
For your own app, of course, it isn’t necessary to put it all inside of the sample app’s code (that’s just if you want to show it to me for help), but it should get you started with setting up replicable testing for your own app.
It’s important not to turn on Rejecto until you are very confident that vadThreshold is correct for the acoustic model (this would usually mean that a sigh is not processed as speech). You may need to test this yourself; it isn’t really necessary to be a native speaker in order to make sure that the vadThreshold is rejecting as much non-speech as possible. It does sound like vadThreshold should be higher in your case.
Once you have confidence in vadThreshold, you can obtain recorded speech as described in the linked post, and start to tune your Rejecto settings (starting from the default settings). If you continue to get unexpected results, you can give me a full replication case as described in the linked post so I can look into whether it’s a settings issue.
Hi Vladimir,
You’re welcome, and good luck with your investigations!
Best regards,
Halle
Hi Vladimir,
It’s the link I gave you above:
https://www.politepix.com/forums/topic/how-to-create-a-minimal-case-for-replication/
I can’t guarantee it is something I can assist with, but I can take a look as long as you follow the instructions in that post very carefully.
Hi Vladimir,
The primary recognition issue is going to be testing with a non-native German speaker. I will close this up since the issue is pretty straightforward, but if you want to follow my advice above later on regarding how to test, it’s fine to open a new topic if you continue to have unexpected results, thanks.
- The topic ‘Key Words from Reading’ is closed to new replies.