Automatic detection and removal of odd distorted frames

[Anonymous]

I'd like to have an option in QuickStreamFix for automatically detecting and removing odd distorted frames which are likely to cause problems upon 3rd party MPEG re-encoding.

By distorted I mean frames which are visibly garbled and sometimes accompanied by glitches in the audio stream. I've found out that even if VideoReDo's output mpg file is fully playable despite the sporadic garbled frames, they may pose a problem in further processing of the file.

To put this into perspective, what I want to do is:
1) cut and QuickStreamFix my DVB-T recordings using VideoReDo
2) compile a set of DVD files from the mpg files using DVD-lab
3) shave off the extra megs from the DVD files using DVD Shrink
4) burn the final DVD files on a single-sided DVD-R

DVD-lab can readily compile a playable set of DVD files from the mpg files even with distorted frames in them, but DVD Shrink can't handle the distorted frames in the DVD files.

I'm currently doing the following work-around:
1) cut and QuickStreamFix my DVB-T recording
2) estimate the required size of the mpg file(s) for fitting on a DVD-R
3) decrease the size of the mpg file with DVD-lab's DCT domain transcoder and wait until the process crashes
4) inspect the resulting partial mpv file and see how far the transcoder got this time before crashing
5) visually locate and cut the distorted frames, which caused the crash, from the mpg file using VideoReDo
6) iterate the process until the every mpg file going on the DVD can be processed and shrunk
7) compile and burn a DVD-R from the final mpv and mpa files

Thanks for a great piece of software and keep up the good work!

 

[phd]

This will be a difficult one. VideoReDo will remove "bad" frames and adjust sync, but if the frame is just good enough, it won't delete it. You could try a stream analyzer or audio analyzer program to see if they would pick up[ the extra glitches in the form of an aberant bit rate. You could then correlate it to the time in the file and use VideoReDo to cut it.

Also, do you a segment of the frame having a glitch like a green segment?

If it is consistently in one spot, you could try using Ad Detective. Modify the parameters and the detection area to pick it up.

 

[Anole]

related glitch removal

For months, my video source from satellite has been of a constant quality.
Thanks to improvements in VideoReDo, and the generous technical input from Dan, I can generally deal with any trivial boo-boo I find.
Now, due to a different processing format/tool, I am seeing a whole range of video glitches I've never seen before.

Some which I call micro-glitches, appear to be a few scan lines which are slid sideways maybe 10% of the frame width.
This appears (without close analisys) to be one field only.

Others are pixelized places in the picture which would normally render it unacceptable from any sort of a quality control perspective.
I've seen anything from a few macro blocks which might be well less that 1% of the screen to bigger ones of maybe 2% or worse.
These appear longer on the screen, but are probably caused by a single bad frame (guessing).

In the past, I would just try to record the show again, but some programs are not aired again (anywhere!) that week.

While I don't really want to spend the time patching individual frames (8 glitches on the last movie I recorded) some people would.
Maybe if there were a manual option to replicate the previous frame , that would go a long way to fixing what I'm currently seeing.
Maybe you could think of it as a guided QSF feature...?
Or, interactive? I don't see this becoming a fully automated function.

Until now, the QSF feature has done a fine job, and I take my hat off to the staff for that.
I'm also working hard to find a better way of capturing my source, so the glitches do not appear, as that seem the best for the long run.


(I also author through DVD Labs, and it seems to be reasonably tollerant of anything I feed it from VideoReDo)

 

[Anonymous]

Thanks for the quick reply, Pat. I was afraid that recognizing a garbled frame might be a tricky task to automate. What I had in mind was some sort of adjustable tolerance of the QuickStreamFix, so that the fixing process could be made less tolerant of anomalies in the stream. This would in turn increase the probability of a successful re-encoding of the mpg file.

Typical glitches, which have caused a re-encoding process to crash aren't consistent in shape and location, but basically a large group of random blocks of pixelation in the frame. Some less severe distortion in the stream (e.g. a couple of lines interchanged, no audio distortion) haven't caused trouble at the re-encoding stage.

Trying an audio analyzer is a good idea, thanks. At least that way I could locate all the transient spikes in the audio track and cut the corresponding frames at once, which would save me a lot of trouble.

I agree with Anole, that it's important to get the capture right to avoid further trouble. When capturing from my DVB-T card I usually just check that it starts OK and then leave the computer alone. Doing something else on the computer while capturing has proved to cause distortion in the mpg file.

 

[phd]

The other suggestion you could try is an authoring program that is more fault tolerant.

 

[laserfan]

Re: related glitch removal

...I am seeing a whole range of video glitches I've never seen before. Some which I call micro-glitches...

Others are pixelized places in the picture ...

I've seen anything from a few macro blocks which might be well less that 1% of the screen to bigger ones of maybe 2% or worse.

Maybe if there were a manual option to replicate the previous frame , that would go a long way to fixing what I'm currently seeing...

In another thread here I asked about identifying in the log where VRD had found-and-fixed problems, so I could find the repair points in the output and look at them to see how bad they looked to me. I also griped a bit about how VRD fixed pixelation sometimes (or so it seemed to me) by either tossing frames or replicating frames which to my eye was worse than minor pixelations (i.e. would appear as stuttering or momentary "freeze frames").

My point is that what may be just a "glitch" to you might be quite unacceptable to me or vice-versa, the same with HOW the glitch might be "fixed"--so the VRD team has quite a hill to climb if they decide that identifying and fixing glitches, or making it easier for users to do so.

I think I would lobby again for an improved "glitch identification method" first, as discussed in that other thread, and then follow that up with a glitch-by-glitch way for a use to apply the user's desired repair to that glitch, whether it be to leave it alone, or paste-in a prior or following frame, or whatever.

But again perhaps to start with QSF enhancement that would ID the problem spots in a program/stream (I think Anole you might have suggested red flags on the timeline or something) and an easy way to forward-and-back from glitch to glitch for review. Then a menu item w/optins to deal with the specific glitch.