Wrong recognitionScore when using RapidEars

[tonylee]

I can get the recognitionScore correctly in OpenEars. But when I use RapidEars (Demo version), the recognitionScore is wrong. It keeps showing number similar to 58486380. And this number does not change in the whole active session of app. It changes once I restart the app but it keeps unchanged then.

Is it a problem of demo version? Or wrong setting?

Thanks!

[Halle Winkler]

Welcome,

Can you show your code in the callback which is getting the score value?

BTW, just for some background, the score doesn’t have a huge amount of utility in an application because it is extremely dependent on the recording environment. I have historically made it available because it is part of the returned information with a hypothesis and it seemed unhelpful to omit it from OpenEars when it was available, but I don’t advise relying on it for anything except relative values within a single app usage session (meaning, you can use score within one session to see if something is well-recognized compared to another utterance within the same session, but shouldn’t use constants as a universal cutoff that means “good recognition” or “bad recognition” because you’ll get surprises when the user interacts with the app from a greater mic distance or in a noisier environment than you tested in.

[tonylee]

The code is here:

// somewhere in the .m
- (void)somewhere
{
    _controller = [[PocketsphinxController alloc] init];
    _controller.nBestNumber = 1;
    _controller.verbosePocketSphinx = kIsVerbose;
    [_controller setRapidEarsToVerbose:kIsVerbose];
    [_controller setRapidEarsAccuracy:10];
    [_controller setFinalizeHypothesis:TRUE];
    [_controller setFasterPartials:TRUE];
    [_controller setFasterFinals:FALSE];
    [_controller startRealtimeListeningWithLanguageModelAtPath:_pathToDynamicallyGeneratedGrammar
                                              dictionaryAtPath:_pathToDynamicallyGeneratedDictionary
                                           acousticModelAtPath:[self acousticModelPath]];

    // ... set up observer in the following
}

//
- (void) rapidEarsDidReceiveLiveSpeechHypothesis:(NSString *)hypothesis recognitionScore:(NSString *)recognitionScore {
    NSLog(@"rapidEarsDidReceiveLiveSpeechHypothesis: %@ (%@)",hypothesis, recognitionScore);
    _statusLabel.text = [NSString stringWithFormat:@"%@", hypothesis];
}

- (void) rapidEarsDidReceiveFinishedSpeechHypothesis:(NSString *)hypothesis recognitionScore:(NSString *)recognitionScore {
    NSLog(@"rapidEarsDidReceiveFinishedSpeechHypothesis: %@ (%p)", hypothesis, recognitionScore);
    
    _scoreLabel.text = [NSString stringWithFormat:@"%@", recognitionScore];
}

I understand the number is not that reliable. But the problem is that the number is not negative number, which is different from OpenEars.

[Halle Winkler]

The different format is correct — the RapidEars scores are raw Sphinx format scores and haven’t been converted into a probability. Since you’ve pointed it out, I think you’re correct that that should be changed to match the OpenEars style since those raw scores are particularly un-useful. I would call that a bug and I’ll see if I can fix it in the upcoming 64-bit support release.

[Halle Winkler]

This issue is fixed in yesterday’s release of OpenEars/RapidEars 1.64. You should now have rational probability values in RapidEars scoring in unsegmented mode.

[Author]

Posts