Quantcast

Wrong Audio when extracting (sometimes silences where there is none!)

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Wrong Audio when extracting (sometimes silences where there is none!)

seba.wagner@gmail.com
hi,

I had a previous posting complaining that FFMPEG does ignore gaps in the FLV
when extracting Audio out of it. (
http://lists.mplayerhq.hu/pipermail/ffmpeg-user/2010-January/023582.html)

I now encounter the opposite:
It does also ADD silence at some places where there is actually none!

Sample Case:
Input FLV:
http://www.openmeetings.de/rec/rec_10_seconds_skits.flv (around 2:50)
Output Wave:
http://www.openmeetings.de/rec/rec_10_seconds_skits.wav (around 1:40)

The FLV file is correct and it has quite exactly 10 seconds of silence + 10
seconds of sound bit by bit/rotating till the end.
The output wave is produced with ffmpeg -i input.flv output.wav (I have
compiled the latest FFmpeg version SVN-r21544).

The output Wave adds at:
0:09 - 0:11 2 seconds of silence.
0:20 - 0:22 2 seconds of silence
0:30 - 0:32 2 seconds of silence
0:43 - 0:44 2 seconds of silence
0:55 - 0:57 2 seconds of silence
1:07 - 1:09 2 seconds of silence
1:19 - 1:21 2 seconds of silence
1:30 - 1:32 2 seconds of silence
So this error is continously in the hole WAVE.

Actually the WAVE should have exactly 50% of the length of the original
file, as I recorded the FLV in that way that it has exactly 50% of silence
(every 10 seconds a Gap of 10 seconds). => When you count correctly FFMPEG
adds exactly 2 seconds of silence where there are actually 10. At least it
looks like that.
I can reproduce this behavior with several FFMPEG versions and FLV files.

so ... I am quite puzzled what shall I do with that?
Are these TWO Issues:
1) Missing silence from original FLV when extracting the Audio
2) Adding silence at (random) places in resulting Audio when extracting with
FFMPEG

or is this actually ONE big Issue. So adding those silence is already 50%
done, its only a wrong length in the silence part?

Can someone point us to the files that are actually doing the transcoding in
this case?

Is there any advice from experts that could help us to fix this Bug?

We are really very interested in finding the Source of this Issue and help
to solve it as its an ultimate blocker.


thanks,
Sebastian Wagner

--
Sebastian Wagner
http://www.webbase-design.de
http://openmeetings.googlecode.com
http://www.laszlo-forum.de
[hidden email]
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

Benjamin Larsson-2
Hi mr Wagner.

Sebastian Wagner wrote:
> hi,
>
> I had a previous posting complaining that FFMPEG does ignore gaps in the FLV
> when extracting Audio out of it. (
> http://lists.mplayerhq.hu/pipermail/ffmpeg-user/2010-January/023582.html)
>  

What you have is this issue:

https://roundup.ffmpeg.org/roundup/ffmpeg/issue271

> I now encounter the opposite:
> It does also ADD silence at some places where there is actually none!
>
> Sample Case:
> Input FLV:
> http://www.openmeetings.de/rec/rec_10_seconds_skits.flv (around 2:50)
> Output Wave:
> http://www.openmeetings.de/rec/rec_10_seconds_skits.wav (around 1:40)
>
> The FLV file is correct and it has quite exactly 10 seconds of silence + 10
> seconds of sound bit by bit/rotating till the end.
> The output wave is produced with ffmpeg -i input.flv output.wav (I have
> compiled the latest FFmpeg version SVN-r21544).
>
> The output Wave adds at:
> 0:09 - 0:11 2 seconds of silence.
> 0:20 - 0:22 2 seconds of silence
> 0:30 - 0:32 2 seconds of silence
> 0:43 - 0:44 2 seconds of silence
> 0:55 - 0:57 2 seconds of silence
> 1:07 - 1:09 2 seconds of silence
> 1:19 - 1:21 2 seconds of silence
> 1:30 - 1:32 2 seconds of silence
> So this error is continously in the hole WAVE.
>  

This is likely the cause of the flashplayer having a 2 second window
before it cuts the stream.

> [...lots of text...] Is there any advice from experts that could help us to fix this Bug?
>  

Yes, if you read the ticket there is a reply about the api needing
AVPackets. We now have that so it is possible for the decoder
to account for jumps in the timestamp (DTS) and insert appropriate
amount of silence.

> We are really very interested in finding the Source of this Issue and help
> to solve it as its an ultimate blocker.
>  

Enough to fund me||someone fixing the issue ? If not I'll be happy to
assist in whatever other way I can. More sample files would be nice
whoever fixes this.

> thanks,
> Sebastian Wagner
>
>  


MvH
Benjamin Larsson

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

seba.wagner@gmail.com
hi,

thanks for your answers.

*This is likely the cause of the flashplayer having a 2 second window before
it cuts the stream.*
=> You are talking about the way the Decoder/Demuxer reads in the stream and
writes the Audio out yes?
Cause in a scenario where there are no Gaps/Silence in the FLV Audio Track
those 2 second window Issue is not available.

More Samples are no problem I can produce you as many as you need.

Budget for fixing is another topic with the sponsors ^^.
We will see what is the most feasable.


anyway thanks a lot for your time and effort.

Sebastian

2010/2/1 Benjamin Larsson <[hidden email]>

> Hi mr Wagner.
>
>
> Sebastian Wagner wrote:
>
>> hi,
>>
>> I had a previous posting complaining that FFMPEG does ignore gaps in the
>> FLV
>> when extracting Audio out of it. (
>> http://lists.mplayerhq.hu/pipermail/ffmpeg-user/2010-January/023582.html)
>>
>>
>
> What you have is this issue:
>
>
> https://roundup.ffmpeg.org/roundup/ffmpeg/issue271
>
>  I now encounter the opposite:
>> It does also ADD silence at some places where there is actually none!
>>
>> Sample Case:
>> Input FLV:
>> http://www.openmeetings.de/rec/rec_10_seconds_skits.flv (around 2:50)
>> Output Wave:
>> http://www.openmeetings.de/rec/rec_10_seconds_skits.wav (around 1:40)
>>
>> The FLV file is correct and it has quite exactly 10 seconds of silence +
>> 10
>> seconds of sound bit by bit/rotating till the end.
>> The output wave is produced with ffmpeg -i input.flv output.wav (I have
>> compiled the latest FFmpeg version SVN-r21544).
>>
>> The output Wave adds at:
>> 0:09 - 0:11 2 seconds of silence.
>> 0:20 - 0:22 2 seconds of silence
>> 0:30 - 0:32 2 seconds of silence
>> 0:43 - 0:44 2 seconds of silence
>> 0:55 - 0:57 2 seconds of silence
>> 1:07 - 1:09 2 seconds of silence
>> 1:19 - 1:21 2 seconds of silence
>> 1:30 - 1:32 2 seconds of silence
>> So this error is continously in the hole WAVE.
>>
>>
>
> This is likely the cause of the flashplayer having a 2 second window before
> it cuts the stream.
>
>  [...lots of text...] Is there any advice from experts that could help us
>> to fix this Bug?
>>
>>
>
> Yes, if you read the ticket there is a reply about the api needing
> AVPackets. We now have that so it is possible for the decoder
> to account for jumps in the timestamp (DTS) and insert appropriate amount
> of silence.
>
>
>  We are really very interested in finding the Source of this Issue and help
>> to solve it as its an ultimate blocker.
>>
>>
>
> Enough to fund me||someone fixing the issue ? If not I'll be happy to
> assist in whatever other way I can. More sample files would be nice whoever
> fixes this.
>
>  thanks,
>> Sebastian Wagner
>>
>>
>>
>
>
> MvH
> Benjamin Larsson
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
>



--
Sebastian Wagner
http://www.webbase-design.de
http://openmeetings.googlecode.com
http://www.laszlo-forum.de
[hidden email]
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

Benjamin Larsson-2
Sebastian Wagner wrote:
> hi,
>
> thanks for your answers.
>
> *This is likely the cause of the flashplayer having a 2 second window before
> it cuts the stream.*
> => You are talking about the way the Decoder/Demuxer reads in the stream and
> writes the Audio out yes?
>  

I'm not following you.

> Cause in a scenario where there are no Gaps/Silence in the FLV Audio Track
> those 2 second window Issue is not available.
>  

Ok one more try. When the Flashplayer records from some input it has
some kind of sound level detector, when there is sound it will send
nellymoser frames. When there is no sound for 2 seconds it will stop
sending nellymoser frames and just send null frames instead. This is how
I assume it could work. Anyway how this works is irrelevant in the case
of fixing this issue.


> More Samples are no problem I can produce you as many as you need.
>  

5 stream with different flashplayer versions and sample rates should be
fine.

> Budget for fixing is another topic with the sponsors ^^.
> We will see what is the most feasable.
>
>  
It will take someone with a clue half a day. And one day for someone
with less experience (with my help).

> anyway thanks a lot for your time and effort.
>
> Sebastian
>  

MvH
Benajmin Larsson

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

seba.wagner@gmail.com
hi,

2010/2/1 Benjamin Larsson <[hidden email]>

> Sebastian Wagner wrote:
>
>> hi,
>>
>> thanks for your answers.
>>
>> *This is likely the cause of the flashplayer having a 2 second window
>> before
>> it cuts the stream.*
>> => You are talking about the way the Decoder/Demuxer reads in the stream
>> and
>> writes the Audio out yes?
>>
>>
>
> I'm not following you.
>
>
>  Cause in a scenario where there are no Gaps/Silence in the FLV Audio Track
>> those 2 second window Issue is not available.
>>
>>
>
> Ok one more try. When the Flashplayer records from some input it has some
> kind of sound level detector, when there is sound it will send nellymoser
> frames. When there is no sound for 2 seconds it will stop sending nellymoser
> frames and just send null frames instead. This is how I assume it could
> work. Anyway how this works is irrelevant in the case of fixing this issue.


=> I understand that now thanks. Although bit curieuse what they did at
Adobe.
Anyway from that point of view it must be then sufficient to repeat the last
null-packets n-times again each time you detect a gap/silence in the Audio
channel ... but only if there is a stable calculation between the length of
milliseconds of a nellymoser packet and the packet size (44100 Khz is
512Bytes and seem to be approximate 50ms).

>
>
>
>  More Samples are no problem I can produce you as many as you need.
>>
>>
>
> 5 stream with different flashplayer versions and sample rates should be
> fine.
>
>
>  Budget for fixing is another topic with the sponsors ^^.
>> We will see what is the most feasable.
>>
>>
>>
> It will take someone with a clue half a day. And one day for someone with
> less experience (with my help).
>
>
>  anyway thanks a lot for your time and effort.
>>
>> Sebastian
>>
>>
>
> MvH
> Benajmin Larsson
>
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
>



--
Sebastian Wagner
http://www.webbase-design.de
http://openmeetings.googlecode.com
http://www.laszlo-forum.de
[hidden email]
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

Benjamin Larsson-2
Sebastian Wagner wrote:

> hi,
>
> 2010/2/1 Benjamin Larsson <[hidden email]>
>
>  
>> Sebastian Wagner wrote:
>>
>>    
>>> hi,
>>>
>>> thanks for your answers.
>>>
>>> *This is likely the cause of the flashplayer having a 2 second window
>>> before
>>> it cuts the stream.*
>>> => You are talking about the way the Decoder/Demuxer reads in the stream
>>> and
>>> writes the Audio out yes?
>>>
>>>
>>>      
>> I'm not following you.
>>
>>
>>  Cause in a scenario where there are no Gaps/Silence in the FLV Audio Track
>>    
>>> those 2 second window Issue is not available.
>>>
>>>
>>>      
>> Ok one more try. When the Flashplayer records from some input it has some
>> kind of sound level detector, when there is sound it will send nellymoser
>> frames. When there is no sound for 2 seconds it will stop sending nellymoser
>> frames and just send null frames instead. This is how I assume it could
>> work. Anyway how this works is irrelevant in the case of fixing this issue.
>>    
>
>
> => I understand that now thanks. Although bit curieuse what they did at
> Adobe.
> Anyway from that point of view it must be then sufficient to repeat the last
> null-packets n-times again each time you detect a gap/silence in the Audio
> channel ... but only if there is a stable calculation between the length of
> milliseconds of a nellymoser packet and the packet size (44100 Khz is
> 512Bytes and seem to be approximate 50ms).
>  

The proper way to do it is to use the DTS timestamp (should be found in
the AVPacket). The nellymoser decoder can then keep track of the DTS and
prepad the output when a jump is detected. Any other hack will be rejected.


MvH
Benjamin Larsson

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

Julien Muchembled
Hello,


Benjamin Larsson a écrit :
> What you have is this issue:
>
> https://roundup.ffmpeg.org/roundup/ffmpeg/issue271 

According to msg2405 (from michaelni):
> A clean fix first requires the audio API to be changed to pass
> AVPackets around instead of buf+size, the nellymoser decoder
> can then insert silence where there are jumps in the dts.

It seems the first part has already been done, thanks to commit [18351].


Benjamin Larsson a écrit :

> Sebastian Wagner wrote:
>> => I understand that now thanks. Although bit curieuse what they did at
>> Adobe.
>> Anyway from that point of view it must be then sufficient to repeat
>> the last
>> null-packets n-times again each time you detect a gap/silence in the
>> Audio
>> channel ... but only if there is a stable calculation between the
>> length of
>> milliseconds of a nellymoser packet and the packet size (44100 Khz is
>> 512Bytes and seem to be approximate 50ms).
>>  
>
> The proper way to do it is to use the DTS timestamp (should be found in
> the AVPacket). The nellymoser decoder can then keep track of the DTS and
> prepad the output when a jump is detected. Any other hack will be rejected.
Like what does the attached patch ?

The patch works and is probably good enough for OpenMeetings.
But I guess Sebastian needs a fix clean enough to be committed.

One value is hardcoded (1000). Where can I get it ? I think it's
'(AVFormatContext*)->streams[1]->time_base' (in flv demuxer).

I haven't tested yet a .flv without any gap, to check that the patch doesn't insert any silence.

Why not inserting silences in avcodec_decode_audio3 so that all codecs benefit from it ?


Regards,
Julien


(oops, only members can post - subscribing to ffmpeg-user and resending - sorry for the noise)

diff --git a/libavcodec/nellymoserdec.c b/libavcodec/nellymoserdec.c
index 38f4b42..7fea6c9 100644
--- a/libavcodec/nellymoserdec.c
+++ b/libavcodec/nellymoserdec.c
@@ -49,6 +49,7 @@ typedef struct NellyMoserDecodeContext {
     GetBitContext   gb;
     int             add_bias;
     float           scale_bias;
+    int64_t         sample_count;
     DSPContext      dsp;
     FFTContext      imdct_ctx;
     DECLARE_ALIGNED_16(float,imdct_out)[NELLY_BUF_LEN * 2];
@@ -181,6 +182,14 @@ static int decode_tag(AVCodecContext * avctx,
             return buf_size;
     }
 
+    s->sample_count += blocks * NELLY_SAMPLES;
+    if (s->sample_count * 1000 < avpkt->dts * avctx->sample_rate) {
+        int n = blocks * NELLY_SAMPLES * sizeof(int16_t);
+        memset(samples, 0, n);
+        *data_size += n;
+        return 0;
+    }
+
     for (i=0 ; i<blocks ; i++) {
         nelly_decode_block(s, &buf[i*NELLY_BLOCK_LEN], s->float_buf);
         s->dsp.float_to_int16(&samples[i*NELLY_SAMPLES], s->float_buf, NELLY_SAMPLES);



_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

Benjamin Larsson-2
Julien Muchembled wrote:

>> The proper way to do it is to use the DTS timestamp (should be found in
>> the AVPacket). The nellymoser decoder can then keep track of the DTS and
>> prepad the output when a jump is detected. Any other hack will be rejected.
>>    
>
> Like what does the attached patch ?
>
> The patch works and is probably good enough for OpenMeetings.
> But I guess Sebastian needs a fix clean enough to be committed.
>
> One value is hardcoded (1000). Where can I get it ? I think it's
> '(AVFormatContext*)->streams[1]->time_base' (in flv demuxer).
>
> I haven't tested yet a .flv without any gap, to check that the patch doesn't insert any silence.
>
> Why not inserting silences in avcodec_decode_audio3 so that all codecs benefit from it ?
>
>
> Regards,
> Julien
>
>
> (oops, only members can post - subscribing to ffmpeg-user and resending - sorry for the noise)
>  

This patch is the way to do it. But as you say the code should maybe be
in avcodec_decode_audio3. Please send this patch to the dev list so the
discussion can continue there.
 
MvH
Benjamin Larsson

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

Julien Muchembled
By looking at how to fix the bug in 'avcodec_decode_audio3' or even at an upper level, I just discovered 'sync_opts' and then 'audio_sync_method' variables.

So, what about the -async option ?

If I understand correctly, there is no bug and issue271 can be closed.

Sebastian, can you confirm that using '-async 1' option produce what you want ?


Regards,
Julien

Benjamin Larsson a écrit :

> Julien Muchembled wrote:
>>> The proper way to do it is to use the DTS timestamp (should be found in
>>> the AVPacket). The nellymoser decoder can then keep track of the DTS and
>>> prepad the output when a jump is detected. Any other hack will be
>>> rejected.
>>>    
>>
>> Like what does the attached patch ?
>>
>> The patch works and is probably good enough for OpenMeetings.
>> But I guess Sebastian needs a fix clean enough to be committed.
>>
>> One value is hardcoded (1000). Where can I get it ? I think it's
>> '(AVFormatContext*)->streams[1]->time_base' (in flv demuxer).
>>
>> I haven't tested yet a .flv without any gap, to check that the patch
>> doesn't insert any silence.
>>
>> Why not inserting silences in avcodec_decode_audio3 so that all codecs
>> benefit from it ?
>>
>>
>> Regards,
>> Julien
>>
>>
>> (oops, only members can post - subscribing to ffmpeg-user and
>> resending - sorry for the noise)
>>  
>
> This patch is the way to do it. But as you say the code should maybe be
> in avcodec_decode_audio3. Please send this patch to the dev list so the
> discussion can continue there.
>
> MvH
> Benjamin Larsson
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Wrong Audio when extracting (sometimes silences where there is none!)

seba.wagner@gmail.com
hi there,

you are completely right!

Using the latest SVN of FFMPEG + -async option does fix the Issue.

I will do some in detail testing tomorrow to check it but the length
looks okay and first impression is quite positive.

Thanks a lot for digging into the API Julien!

Sebastian

2010/2/2 Julien Muchembled <[hidden email]>:

> By looking at how to fix the bug in 'avcodec_decode_audio3' or even at an upper level, I just discovered 'sync_opts' and then 'audio_sync_method' variables.
>
> So, what about the -async option ?
>
> If I understand correctly, there is no bug and issue271 can be closed.
>
> Sebastian, can you confirm that using '-async 1' option produce what you want ?
>
>
> Regards,
> Julien
>
> Benjamin Larsson a écrit :
>> Julien Muchembled wrote:
>>>> The proper way to do it is to use the DTS timestamp (should be found in
>>>> the AVPacket). The nellymoser decoder can then keep track of the DTS and
>>>> prepad the output when a jump is detected. Any other hack will be
>>>> rejected.
>>>>
>>>
>>> Like what does the attached patch ?
>>>
>>> The patch works and is probably good enough for OpenMeetings.
>>> But I guess Sebastian needs a fix clean enough to be committed.
>>>
>>> One value is hardcoded (1000). Where can I get it ? I think it's
>>> '(AVFormatContext*)->streams[1]->time_base' (in flv demuxer).
>>>
>>> I haven't tested yet a .flv without any gap, to check that the patch
>>> doesn't insert any silence.
>>>
>>> Why not inserting silences in avcodec_decode_audio3 so that all codecs
>>> benefit from it ?
>>>
>>>
>>> Regards,
>>> Julien
>>>
>>>
>>> (oops, only members can post - subscribing to ffmpeg-user and
>>> resending - sorry for the noise)
>>>
>>
>> This patch is the way to do it. But as you say the code should maybe be
>> in avcodec_decode_audio3. Please send this patch to the dev list so the
>> discussion can continue there.
>>
>> MvH
>> Benjamin Larsson
>



--
Sebastian Wagner
http://www.webbase-design.de
http://openmeetings.googlecode.com
http://www.laszlo-forum.de
[hidden email]
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Loading...