Score Array in RapidEars returns zero

[billylo]

I managed to retrieve the words through rapidEarsDidDetectLiveSpeechAsWordArray; but the associated scoreArray is always returning zeroes (the same number of zero elements in the word array.)

OpenEars 2.0.4.1, RapidEars 2.0.4

Am I missing some configuration steps in order to receive scores?

thanks,

[Halle Winkler]

Welcome,

Could you show me your code for reading your score array?

[billylo]

         NSNumber *scoreNumber = scores[i];
         int score = [scoreNumber intValue];

I also used the debugger to view the scoreArray and it shows an array of zeroes.

any hints?

[Halle Winkler]

Most likely this is a bug, but to confirm, are the hypotheses real words from your model/grammar or null hypotheses or rejecto tokens?

[billylo]

yes. the hypothesis are the correct ones.

—

While I have you, I am getting a lot of duplicated calls to the rapidEars delegate method for the same audio trigger (just one word). Sometimes, the delegate method is called 20+ times. I have set

    [[OEPocketsphinxController sharedInstance] setReturnDuplicatePartials:NO];

In fact, after a while, RapidEars would stop picking up words. (this is probably a different thread.)

[Halle Winkler]

Hmm, that I haven’t seen in any testing and I’d be surprised if it were a bug. Would it be possible for you to create a replication case to rule out or locate any local issues in the app?

https://www.politepix.com/forums/topic/how-to-create-a-minimal-case-for-replication/

This is done by using the SaveThatWave demo to capture your speech input and then add it to the sample app with the minimal code to replicate the result.

[billylo]

Hi, Here is a minimal test case that demonstrates my two issues (duplicated calls to delegate methods and zero scores). It’s basically the sampleApp modified to support RapidEars and these three lines added to initialization.

        [[OEPocketsphinxController sharedInstance] setFinalizeHypothesis:NO];
        [[OEPocketsphinxController sharedInstance] setReturnSegments:YES];
        [[OEPocketsphinxController sharedInstance] setReturnSegmentTimes:YES];

and I used this to capture the score and start/end times.

- (void) rapidEarsDidDetectLiveSpeechAsWordArray:		(NSArray *) 	words
                                      scoreArray:		(NSArray *) 	scores
                                  startTimeArray:       (NSArray *)     startTimes
                                    endTimeArray:       (NSArray *)     endTimes
{
    NSNumber *scoreNumber, *startTime, *endTime;
    
    for (int i = 0; i< words.count; i++) {
        
        scoreNumber = scores[i];
        startTime = startTimes[i];
        endTime = endTimes[i];
        
        NSLog(@"word = %@, score = %.1f, startTime = %.1f, endTime = %.1f", words[i], [scoreNumber floatValue], [startTime floatValue], [endTime floatValue]);
        
    }
    
    //output score
}

The console output looks like this:

2015-08-31 19:26:27.019 OpenEarsSampleApp[681:135806] Local callback: Pocketsphinx has detected speech.
2015-08-31 19:26:29.141 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.2
2015-08-31 19:26:29.268 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.3
2015-08-31 19:26:29.404 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.3
2015-08-31 19:26:29.529 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.3
2015-08-31 19:26:29.657 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.3
2015-08-31 19:26:29.785 OpenEarsSampleApp[681:135806] Local callback: Pocketsphinx has detected a second of silence, concluding an utterance.
2015-08-31 19:26:30.296 OpenEarsSampleApp[681:135806] Local callback: Pocketsphinx has detected speech.
2015-08-31 19:26:30.432 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
2015-08-31 19:26:30.552 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
2015-08-31 19:26:30.684 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.3
2015-08-31 19:26:30.814 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
2015-08-31 19:26:30.936 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
2015-08-31 19:26:31.066 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
2015-08-31 19:26:31.197 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
2015-08-31 19:

thanks.

[Halle Winkler]

Thanks, I will enter this in the buglist and take a look at it this week.

[billylo]

Great. I did have trouble getting the wavWasSavedAtLocation delegate method triggered. SaveThatWave worked, just the delegate method not triggered somehow. Could use your guidance there. Billy.

 - (void) wavWasSavedAtLocation:(NSString *)location {
    NSLog(@"WAV was saved at the path %@", location);
 }

[Halle Winkler]

startSessionDebugRecord doesn’t call the delegate since it records the entire session until it is complete.

[billylo]

I see. Does [stop] actually stop it? I am thinking of adding a support feature to my app so that they can upload sample audio to the cloud if my app does not behave correctly.

[Halle Winkler]

startSessionDebugRecord isn’t intended for any purpose other than outputting a debug dump of the complete app session audio after the app has been terminated, for developer testing and test user testing only. It is not intended to record enduser app sessions.

[billylo]

Any luck with tracking this one down, Halle?

[Halle Winkler]

Hi,

Sorry, this might take some time to get to – there is a large update underway and it will most likely have to be fixed as part of it.

[Halle Winkler]

Hello,

My apologies again for not being able to give you any feedback on this until now. I have run your case, confirmed the bug, and it will be fixed in the upcoming update. I appreciate your giving me a replication case so that I could locate the problem.

[Halle Winkler]

To clarify, there were two bugs – the scores were not returning with live segmented hypotheses, and live segmented hypotheses could return duplicate hypotheses without there being a request to return duplicates. The update is complete and in testing so it should be out in the first couple of weeks of the new year.

[Author]

Posts