audio sync not OK after audio conversion by VRD

hydra3333

Member
edit: jump to the VRD test results outlining the VTD issue http://www.videoredo.net/msgBoard/showthread.php?36082-audio-sync-not-OK-after-audio-conversion-by-VRD&p=124608#post124608

Hello. Moved to this thread since I'm using the latest VRD version 5. from http://www.videoredo.net/msgBoard/showthread.php?31203-Problem-with-audio-sync-of-ts-files&p=124601#post124601

I'm sure it's me, but ...

I'm having trouble with audio sync with .mp4 files produced from qsf, when then editing using other tools such as ffmpeg and mp4box.
The QSF'd .mp4 file plays fine though ... are delays and whatnot in the ".mp4 header" info ? If so how do I find them (mediainfo can't see them) ?

For playback device compatibility reasons I need to convert .TS files with AC3 or MP2 etc audio into a .mp4 containing a single MP3 standard audio format (and adjust loudness to a common standard at the same time).
Devices = WDTVs of older generation, and chromecast 1 and Ultru.
At the moment I'm using ffmpeg's loudnorm audio filter and ignoring any "delay" and then noticing the out of sync in playback with MPC-HC.

I reckon I'll re-try VRD with QSF and a profile which has audio Normalization (to 0db) and Automatic Compression to AAC 256k no-change-samplerate audio and see what happens.
And hope that WDTV/Chromecast can play them :)
And hope this doesn't happen ("VRD QSF with the custom profile throws the audio out of sync" http://www.videoredo.net/msgBoard/showthread.php?35604-audio-sync-no-good-(edit-a-second-qsf-too)&p=121552#post121552 ) :)

PS workflow is
Code:
REM QSF .TS file to .MP4 file
REM "C:\ffmpeg\0-latest-x64\bin\ffmpeg.exe" -nostdin -y -hide_banner -threads 0 -i "T:\TEST\sample.has.1824ms.delay.mp4" -nostats -vn -af loudnorm=I=-16:TP=0.0:LRA=11:print_format=json -nostats -f null - 

"C:\ffmpeg\0-latest-x64\bin\ffmpeg.exe" -i "T:\TEST\sample.has.1824ms.delay.mp4" -threads 0 -an -map_metadata -1 -c:v copy -y "T:\TEST\sample.has.1824ms.delay-temp.h264" 
"C:\ffmpeg\0-latest-x64\bin\ffmpeg.exe" -i "T:\TEST\sample.has.1824ms.delay.mp4" -threads 0 -vn -map_metadata -1 -af loudnorm=I=-16:TP=0.0:LRA=11:measured_I=-24.67:measured_LRA=4.70:measured_TP=-3.45:measured_thresh=-35.03:offset=0.06:linear=true:print_format=summary -c:a libmp3lame -ab 256k -ar 48000 -y "T:\TEST\sample.has.1824ms.delay-temp.mp3"  
"C:\ffmpeg\0-latest-x64\bin\ffmpeg.exe" -i "T:\TEST\sample.has.1824ms.delay.mp4" -threads 0 -vn -map_metadata -1 -c:a libmp3lame -ab 256k  -y "T:\TEST\sample.has.1824ms.delay-temp.mp3.noARchange.mp3"  

"C:\mp4box\MP4Box.exe" -add "T:\TEST\sample.has.1824ms.delay-temp.h264" -add "T:\TEST\sample.has.1824ms.delay-temp.mp3":lang=eng:delay=-1824 -isma -new "T:\TEST\Converted\sample.has.1824ms.delay.mp3.MP4.ARchange.delay.-1824.mp4" 
"C:\mp4box\MP4Box.exe" -add "T:\TEST\sample.has.1824ms.delay-temp.h264" -add "T:\TEST\sample.has.1824ms.delay-temp.mp3":lang=eng -isma -new "T:\TEST\Converted\sample.has.1824ms.delay.mp3.MP4.ARchange.nodelay.mp4" 
"C:\mp4box\MP4Box.exe" -add "T:\TEST\sample.has.1824ms.delay-temp.h264" -add "T:\TEST\sample.has.1824ms.delay-temp.mp3.noARchange.mp3":lang=eng:delay=-1824 -isma -new "T:\TEST\Converted\sample.has.1824ms.delay.mp3.MP4.noARchange.delay.-1824.mp4" 
"C:\mp4box\MP4Box.exe" -add "T:\TEST\sample.has.1824ms.delay-temp.h264" -add "T:\TEST\sample.has.1824ms.delay-temp.mp3.noARchange.mp3":lang=eng -isma -new "T:\TEST\Converted\sample.has.1824ms.delay.mp3.MP4.noARchange.nodelay.mp4" 
"C:\mp4box\MP4Box.exe" -add "T:\TEST\sample.has.1824ms.delay-temp.h264" -add "T:\TEST\sample.has.1824ms.delay-temp.mp3":lang=eng:delay=1824 -isma -new "T:\TEST\Converted\sample.has.1824ms.delay.mp3.MP4.ARchange.delay.1824.mp4" 
"C:\mp4box\MP4Box.exe" -add "T:\TEST\sample.has.1824ms.delay-temp.h264" -add "T:\TEST\sample.has.1824ms.delay-temp.mp3.noARchange.mp3":lang=eng:delay=1824 -isma -new "T:\TEST\Converted\sample.has.1824ms.delay.mp3.MP4.noARchange.delay.1824.mp4" 

pause
exit
 
Last edited:

hydra3333

Member
I reckon I'll re-try VRD with QSF and a profile which has audio Normalization (to 0db) and Automatic Compression to AAC 256k no-change-samplerate audio and see what happens.
And hope that WDTV/Chromecast can play them :)
And hope this doesn't happen ("VRD QSF with the custom profile throws the audio out of sync" http://www.videoredo.net/msgBoard/showthread.php?35604-audio-sync-no-good-(edit-a-second-qsf-too)&p=121552#post121552 ) :)
Oh well, I has hoping to hard. Drat !

Using a new profile which has audio conversion with Normalization (to 0db) and Automatic Compression to AAC 256k no-change-samplerate audio, the resulting mp4 from qsf has audio out of sync :mad: :confused:

With the same .TS input file, a similar profile with no-change-to-audio (mentioned in a previous post) the resulting mp4 from qsf has audio close to in sync.

I can't seem to win - VRD won't convert it, nor will tools which use the "good" .mp4 provided by a QSF.

Suggestions welcomed.
 
Last edited:

hydra3333

Member
Ah.
After determining the audio "-ve" delay in the original .TS source file with mediainfo,
if I use an .mp4 container file from ffmpeg like this as input to mp4box
Code:
"C:\ffmpeg\0-latest-x64\bin\ffmpeg.exe" -i "T:\TEST\sample.has.1824ms.delay.mp4" -threads 0 -an -map_metadata -1 -c:v copy -y "T:\TEST\sample.has.1824ms.delay.h264.via-mp4.mp4"
instead of a raw h264 file from ffmpeg like this
Code:
"C:\ffmpeg\0-latest-x64\bin\ffmpeg.exe" -i "T:\TEST\sample.has.1824ms.delay.mp4" -threads 0 -an -map_metadata -1 -c:v copy -y "T:\TEST\sample.has.1824ms.delay-temp.h264"
then after muxing with mp4box, it's close to working and the audio seem out by only a slight amount.
Corrected the small amount with a plain guess of +150ms like this (-1874+150=-1674):
Code:
"C:\mp4box\MP4Box.exe" -add "T:\TEST\sample.has.1824ms.delay.h264.via-mp4.mp4" -add "T:\TEST\sample.has.1824ms.delay.ts.mp3":lang=eng:delay=-1674 -isma -new "T:\TEST\sample.has.1824ms.delay.ts.mp3.ARchange.delay.-1674.mp4"
Go figure.

Does VRD use ffmpeg and maybe suffer a similar type of underpinning issue resulting in the out of sync audio when doing conversions ?
Just a thought.


Since audio delays can and do vary wildly across TS file captures over time, the detection and correction has to be automated to be useful for my workflow if I am to be converting/normalizing/compressing audio using VRD.

Suggestions still very much welcomed.
 
Last edited:

hydra3333

Member
Just confirmed it with a 15 sec sample .TS file, using VRD and separately using ffmpeg.

VRD fails by making audio out of sync WHEN DOING AN AUDIO CONVERSION.
VRD works and audio is in sync when no audio conversion is done.

ffmpeg with .h264 intermediary file fails.
ffmpeg with a .mp4 intermediary file it works (no, .h264 with 0 delay doesn't work).

The original .TS file plays in sync in MPC-HC.

Could upload the sample .TS file if you're interested.
edit: here's the link to 15mb .TS file which has an 1824ms delay https://drive.google.com/open?id=0B5RV2aJ2vdhSTXE1aEJUNHVwOFU

qsf logfile when audio conversion is done: View attachment VideoReDo-5_1_3_746_2017-01-08.zip
 
Last edited:

hydra3333

Member
Hi

Just wondered if there was any news on the audio-desync-when-converting-audio issue ?

Do I need to report a fault or something ?

Thanks
 

Dan203

Senior Developer
Staff member
There are some video errors in that file which are causing the sync to need to be adjusted, but it should still work. Not sure why it doesn't. Try running the original file through QSF without recoding the audio. Is the audio still in sync? If so open the fixed file and save that to convert the audio. Does that work?

The one step QSF should work though. Can you send us a sample file so we can reproduce the issue?
 

tobyW

Member
Hi folks,

I have just realised that, whenever I force an audio recode (with a stock H.264 profile set to AAC 2-channel stereo), the output AAC stream will begin with at least one blank audio frame, inserted before the audio starts.

The number of blank frames increases with the number of audio streams. I counted 7 blanks in an example, before I disallowed Multiple Audio Streams.

If I re-encode the file for any reason, the blank audio frames will multiply.

In my tests, each audio frame was 21.3 mS, and this delay persisted through the whole file.

I'm currently running version 5.3.4.748, but I can see the symptom in my earlier files too.

Hope this somehow helps.

TobyW
 

Dan203

Senior Developer
Staff member
I know what's causing that. It's a side effect of our AAC decoder. The way the AAC decoder works is that it ingests data on one cycle and then doesn't spit out that data until the next cycle. But our processing is linear so we can't allow that. So instead we insert a silent frame at the start of the decode and then just shift every frame by one there after. Under normal circumstances you'd never notice. But if you do repeated audio recodes it will keep inserting new frames for each one, so it will slowly drift out. But an AAC frame (48Khz) is only 22ms long so you're not going to even notice for at least a few cycles. (takes about 200ms offset before a human can notice a sync issue)
 

tobyW

Member
Thanks for that explanation, Dan. You've given me a starting point for digging deeper, but it looks like a can of worms (not of VRD's making).

Think I will go and tackle something easier, like down-mixing from 5.1 to 2.0 with a strong normalised output level despite the presence of rogue spikes. Or easier still, just rid my garden of snails :)
 

Dan203

Senior Developer
Staff member
TO deal with spike use the compression option. That will get rid of the spikes. Then you can normalize to get the average level where you want it. I'd recommend using PCM output for the intermediate step so that you can retain maximum quality before finally compressing to AAC.
 

tobyW

Member
Thanks, I had already tried something along those lines in the AAC world, which was what brought me to the lip-sync difficulties.

Moreover, although normalisation (defeated by spikes) and plain gain do work as predicted, I can't persuade audio compression to work at all, either on H.264/AAC files or on MPEG2/AC-3 files; it does a dual pass but doesn't apply compression (5.3.4.748).

Incidentally, I tried out LKFS normalisation in a demo of VRD Pro but this too was defeated by spikes.

To smooth the workflow and avoid lip sync problems, in the context of the new generation of H.264 and 5.1 broadcasts, maybe TVSuite now needs an ability to combine down-mix and LKFS normalisation with spike clipping, into the one transcoding step?

Should I post these points in new thread(s)?
 

Dan203

Senior Developer
Staff member
OK.
Here's a link to the sample: https://drive.google.com/open?id=0B5RV2aJ2vdhSTXE1aEJUNHVwOFU
I'll try the steps tomorrow.
I tried QSFing this file with an audio recode to AAC 256Kbps and the resulting file was fine. Perfectly in sync. Did you confirm the issue happened on this short file?

If you need to upload a longer file to demonstrate the issue please use the instructions here, and NOT your Google drive...

http://www.videoredo.net/msgBoard/showthread.php?34717-How-to-Upload-Samples-to-VideoReDo-Customer-Support
 

hydra3333

Member
Did you confirm the issue happened on this short file?
OK... yep. Did it again just then on that same .TS file (I re-downloaded it to guarantee it's the same input file).

I can, repeatably, create a resulting .mp4 with the audio out of sync (when played in anything including VRD) and another .mp4 with it in sync by not doing an audio conversion.

OK, I'll upload these per http://www.videoredo.net/msgBoard/showthread.php?34717-How-to-Upload-Samples-to-VideoReDo-Customer-Support

  • the short .TS input file
  • the log file when performing the out-of-sync qsf
  • the profile used when performing the out-of-sync qsf
  • the resulting .mp4 after performing the out-of-sync qsf
  • the log file when performing the in-sync qsf
  • the profile used when performing the in-sync qsf
  • the resulting .mp4 after performing the in-sync qsf

VRD v5.1.3.746

This is shown at the end of the QSF :-

Result from the QSF and a profile which converts the audio:
...the output file is NOT in sync.

G:\HDTV\2\cut-15mb-sample.has.1824ms.delay.mp4
Video length: 00:00:18
Video size: 11MB
Output scenes: 1
Video output frames: 452
Audio output frames: 808
Processing time (secs): 11
Processed frames/sec: 41.09
Actual Video Bitrate: 4.89 Mbps
Audio resync frames removed: 58
Audio level change: 10.13 db
Using the SAME input .TS,
Result from the QSF and a profile which does NOT convert the audio.
...the output file is in sync.

G:\HDTV\2\cut-15mb-sample.has.1824ms.delay.mp4
Video length: 00:00:18
Video size: 11MB
Output scenes: 1
Video output frames: 452
Audio output frames: 509
Processing time (secs): 4
Processed frames/sec: 113.00
Actual Video Bitrate: 4.89 Mbps
Audio resync frames removed: 29
Interestingly, note the different the Numbers of audio frames and the Numbers of Audio resync frames removed - same input file.

Cheers

edit: upload done. "Your files have been uploaded to the folder '36082'." I'll send the email to support in a mo.

edit 2: Thank you for creating and supporting a beaut product ! Any news on a v6 and it's features ?
 
Last edited:
Top Bottom