diagnosing the cause of "video/audio resync frames removed"

Hi all

Sorry if this is dumb question. I did do a quick forum search but didn't find an answer...

I have a video which I'm attempting to process. In short, it's an MPEG 2 TS containing a single h.264 video stream and 2 audio streams (DD/AC3 5.1 and AAC 2.0). The file loads successfully (5.3.80.759), I mark my scenes, then click File -> save video as -> h.264 transport stream profile. At the end of the output process:

Mode: Frame Accurate
Output format: TS
Video length: 7165
Video length: 01:59:25
Video size: 10329MB
Output scenes: 11
Video output frames: 179147
Audio output frames: 391888
Processing time (secs): 1111
Processed frames/sec: 161.25
Actual Video Bitrate: 10.72 Mbps
* Audio frame errors: 2
* Video resync frames removed: 470
* Audio resync frames removed: 75
TS Video packets: 54776589
TS Audio packets: 2695634
TS PSI packets: 140622
TS Null packets: 0
TS MUX Rate (Mbps): 19.20000
-TS Discontinuities: 1

I'm concerned about the high resync frames removed counts. I assume resync frame removal is undesirable. Therefore I want to understand more and attempt to mitigate or eliminate the cause(s).

Two questions:
1. What is the implication of a video or audio resync frame removal? (In other words: why would a frame be removed, and what would the effect on the output be?)
2. How can I discover the cause and/or location of the removed frame(s)? (I glanced over the content of the log file and there don't seem to be any relevant marker entries.)

Note, this is the first time I've encountered this problem with video from this source. TVS has processed other videos from the same source and with the same process/steps (ie. haven't needed to QSF etc.) without problem.
 

Dan203

Senior Developer
Staff member
They're caused by corruption in the source video. Usually some sort of transmission error in the source or possibly some sort of corruption that happens at transfer. (common for .tivo files)

Even though the number seems high it's really only about 15 seconds of content at 29.97fps or only 8 seconds at 59.94. And they're probably not consecutive. More likely they're a few short 1-2 second glitches at various places in the content. Unfortunately VideoReDo does not keep track of where exactly in the source video errors occur, so there is no easy way to track these down.
 

jmc

Active member
The log file can be set to several detail levels...Can get quite overwhelming at higher levels.
The Devs seem to ask for a "medium" level setting when looking into a problem.

Tools/Options/Stream parameters...
"Detect / resync missing frames" can be set to...

"Ignore"
"Resync - Insert Extra Video Frames"
"Resync - Remove Audio Frames"

Don't know if this will have any effect on what you are doing or not.
Not sure when this comes into effect...QSF ?

jmc
 
Thanks for the info Dan203.

In my opinion it's very unfortunate that there's no additional detail in the log file. I would value the ability to locate, review and - if possible/applicable/sensible - amend any output scene that is affected by corruption. You're right that it's probably only about 15 seconds. However, as a perfectionist, to me that's significant. Furthermore, when you consider that it's 15 seconds with 2 or 3 hours of source content, locating the corruption is like trying to find a needle in a haystack! Please would you consider increasing the detail in the log file as a future enhancement.

Some follow up questions...
1. Is the corruption definitely within scenes I've chosen to keep, or could it be in discarded content too?

2. I'm using an output profile which is configured to do intelligent re-encoding. Could resync frame removal indicate that TVS removed and/or re-encoded frames away from the scene joints?
 

Dan203

Senior Developer
Staff member
Thanks for the info Dan203.

In my opinion it's very unfortunate that there's no additional detail in the log file. I would value the ability to locate, review and - if possible/applicable/sensible - amend any output scene that is affected by corruption. You're right that it's probably only about 15 seconds. However, as a perfectionist, to me that's significant. Furthermore, when you consider that it's 15 seconds with 2 or 3 hours of source content, locating the corruption is like trying to find a needle in a haystack! Please would you consider increasing the detail in the log file as a future enhancement.

Some follow up questions...
1. Is the corruption definitely within scenes I've chosen to keep, or could it be in discarded content too?

2. I'm using an output profile which is configured to do intelligent re-encoding. Could resync frame removal indicate that TVS removed and/or re-encoded frames away from the scene joints?
1) For the most part we seek over the parts you cut out, so this is likely to be in the part you kept.

2) It means it removed frames anywhere in the part you kept. Could be 1 frame every second or 470 frames all next to each other. You can try to use the log like jmc suggested but it's difficult to do.
 

MrVideo

New member
Unfortunately VideoReDo does not keep track of where exactly in the source video errors occur, so there is no easy way to track these down.
Huh? That is not true. When I have corrupt sat feed TS files, if I just take the file and write it to another file, the log will contain the exact locations where the the video resync frames were removed. I just search backwards thru the log for removed and you can spot the places where the video frames were removed. You then go to those locations in the source file. Those glitched areas stand out.
 
...just search [backwards] thru the log for removed and you can spot the places where the video frames were removed. You then go to those locations in the source file. Those glitched areas stand out.
Thanks! Can't believe I didn't spot those log entries previously.

Checks would be somewhat easier if the log entries contained a time-stamp for the output file in addition to the existing time-stamp for the source file, but that's getting picky.

Thanks all for your
 

Dan203

Senior Developer
Staff member
Huh? That is not true. When I have corrupt sat feed TS files, if I just take the file and write it to another file, the log will contain the exact locations where the the video resync frames were removed. I just search backwards thru the log for removed and you can spot the places where the video frames were removed. You then go to those locations in the source file. Those glitched areas stand out.
By "keep track" I meant internally. Those log entries are one off calls when the code detects a resync. We don't store those anywhere for easy reference at the end. So using the log is the only option.
 
If you are asking for what I think you are asking for, it does.
Mmmmm, I spotted two types of entries.

Example #1:
2017-12-28 20:43:02 H.264, >>> Sync adjusted at 00:01:44.00 (source time: 00:07:25.07), 4 video frames removed at offset: 664502729.

Example #2:
2017-12-28 20:43:02 H.264, removed 22 frames due to missing pic_order_cnt_lsb 32761, near original timecode: 00:07:45.15, location 699.620 MBytes

In both cases the time-stamp within the source file is given ("source time" and "original timecode"). What I was hoping for was the corresponding time-stamp within the output file. Did I miss that too?
 

MrVideo

New member
Example #1:
2017-12-28 20:43:02 H.264, >>> Sync adjusted at 00:01:44.00 (source time: 00:07:25.07), 4 video frames removed at offset: 664502729.
Ya, this is the one you see if editing a glitched file.
Example #2:
2017-12-28 20:43:02 H.264, removed 22 frames due to missing pic_order_cnt_lsb 32761, near original timecode: 00:07:45.15, location 699.620 MBytes
This one I don't run across.
 

Dan203

Senior Developer
Staff member
Example #2:
2017-12-28 20:43:02 H.264, removed 22 frames due to missing pic_order_cnt_lsb 32761, near original timecode: 00:07:45.15, location 699.620 MBytes
This one applies to H.264 files and happens when the frame is readable but has a corrupt header.
 
@Dan203
Earlier in the thread @jmc mentioned the "Detect / resync missing frames" setting. Today I changed the value of that setting from "resync..." to "ignore". I wanted to at least try "ignore" because I've noticed the way TVS is fixing/resyncing the source file corruption is creating a more noticeable playback glitch. I expected "ignore" to force TVS to not attempt to fix or resync the corruption. However, it appeared to have no effect.

Is my understanding of the effect of the "Detect / resync missing frames" -> "ignore" selection flawed, or is there a bug in the application of that setting?

(TVS version 5.3.80.759)
 

Dan203

Senior Developer
Staff member
Ignore only prevents VRD from correcting audio sync errors. If there are corrupt frames then we have to skip them as we can't even decode them well enough to pass them along to the muxer.
 
Ignore only prevents VRD from correcting audio sync errors. If there are corrupt frames then we have to skip them as we can't even decode them well enough to pass them along to the muxer.
Please help me to understand: why does VRD have to decode the video frames in question in the first place? Obviously I understand the need to be able to decode [certain] video frames for the purposes of [re-]encoding. However, I'm using an intelligent recode output profile, and the corrupt frames are minutes away from cuts/joins. In other words: the corrupt frames shouldn't need re-encoding. Why can't they be left untouched?
 

Dan203

Senior Developer
Staff member
Please help me to understand: why does VRD have to decode the video frames in question in the first place? Obviously I understand the need to be able to decode [certain] video frames for the purposes of [re-]encoding. However, I'm using an intelligent recode output profile, and the corrupt frames are minutes away from cuts/joins. In other words: the corrupt frames shouldn't need re-encoding. Why can't they be left untouched?
We have to demux the stream into it's base elements. Audio and video frames. To do this we need to read certain headers from both the container and the frames themselves. In the case of this kind of corruption often times it's the container that's corrupted an the frames can't even be extracted at all. But even if they can if the headers are corrupt then we still wont be anle to get the information we need to seperate the frames properly.
 
Your explanation obviously makes sense if we assume that frame-level (or lower) access is necessary. However, that's exactly the assumption that I'm questioning.

To be clear, I'm asking why frame-level access is required in the specific scenario I referred to:
  • processing phase (=> frame display not required)
  • intelligent recoding (=> stream decode only required at/around cuts/joins)
  • corruption far from cuts/joins (ie. no chance of frame being needed as a reference)
  • TS input; TS output (=> no additional frame-level info required)
  • keep all streams
 
Top Bottom