TiVo-VideoReDo-AutoProcessor Program

tannebil

New member
I'm suddenly getting some weirdness with the metadata. Here's an example. The Daily Show was recorded on 3/4 at 4pm PDT. The original broadcast date/time was 3/3 at 8pm PDT when I look at it on TiVo

I see three problems:

1. The TiVo metadata extract as it shows the original airdate as 3/3 at 00:00Z when it should be 3/4 at 04:00Z (since Z is +8 to PDT). That causes the BTV original airdate to be computed incorrectly as 3/2.

2. While the broadcast time-date looks correct in the TiVo metadata but it is showing up as a day earlier in BTV so there might be an issue with code converting the date-time into the time offset value.

3. The program also got incorrrectly classified as a movie in BTV but I think that's due to an error in the TiVo program guide. For some reason, it has a movieYear value and as I recall, that's the way TVAP sorts movies from TV shows. Maybe the movie/TV show test should be for a value in the isEpisode field. This has happened a couple of times before as well.

I've attached the TiVo and BTV metadata files for the program plus the TiVo metadata from a show that got classified correctly.

I checked a couple of other programs and the issues are consistent. It just started in the last few days so it's either something introduced in the latest version of the code or perhaps something to do with an incorrect leap year.

Thanks, Bill
 

Attachments

tannebil

New member
Doesn't it resume where it was when you restart processing? (If not I should fix that.) I'm curious why you need it to do this.

If you're doing manual intervention anyway, one way you can force this behavior is to block processing on all unprocessed inputs. This can be tedious if you have a lot of files waiting to be processed I suppose.

I still need to understand your situation a little better. Don't think I want to give up the way the Stop After VRD button works so it would mean adding another button. What operation wouild you perform if you could stop processing the way you want to do it?
I did a bit of testing and the metadata extract and postprocessing jobs ran after I did a "Start Processing" in TVAP. I was misreading "Stop after VRD" to mean stop after TVAP completely finishes processing.

That works fine for my purposes although I didn't test what happens if I quit TVAP and rebooted before I did the Start Processing. Is the pending processing persistent?
 

tannebil

New member
The date problem isn't just on shows recorded in March so my leap year idea would not seem to apply.

I transcoded a bunch of older shows recently so I compared the broadcast date-time in TiVo vs BTV. Every one showed in BTV as recorded a day earlier and always at 4pm. All the original broadcasts date were a day earlier.

I'll rerun the transcodes on some shows so that I can look at the TiVo metadata extracts. Based on the other ones, I'm betting that they are all going to show 00:00Z as the recording time.
 

dlflannery

Moderator
First I have corrected the logic for BTV meta generation so it will not be classified as a movie if the TiVo "isEpisode" meta value is true.

When you say "shows as recorded" by BTV, is that based on the TargetStart value?

Let's try to define what you want to happen here (for non-movies):

1. I get two TiVo meta inputs to work with for non-movies: originalAirDate and time.

originalAirDate is a UTC formatted string and its hr,min,sec part is ALWAYS midnight (0,0,0).

time is a UTC formatted string and it DOES have nonzero hr,min, sec values. It is the time the TiVo started recording the video.

2. Which item should targetStart be based on?

3. Which item should (BTV's) Original Air Date be based on? Please confirm this is just a year/month/day value (no hr/min/sec).

4. If a BTV item is based on the TiVo originalAirDate, how should the midnight hr, min, sec values (zeros) be handled? If that time is converted to your local time you will always get 4pm the previous day. We also can strip out the date and use that without doing any time zone correction.
 
Last edited:

tannebil

New member
originalAirDate in TiVo maps to OriginalAirDate in the BTV xml file. It's a straight copy into yyyymmmddd format dropping hh:mm:ss and without a time zone adjustment. It is also the value used to prefix EpisodeTitle in the BTV file.

time in TiVo maps to TargetStart adjusting from UTC (Z) time to the current time zone and converting into the number of 100-nanosecond intervals since January 1, 1601.

I'm not 100% sure what we'd agreed on for date/time values for movies but I believe it was the same as TV shows. The differences are in selected metadata fields where "Movies" is put into TitleName (BTV), movieYear (TiVo) is put into MovieYear (BTV), and EpisodeTitle (BTV) is omitted.

Thanks, Bill
 

dlflannery

Moderator
originalAirDate in TiVo maps to OriginalAirDate in the BTV xml file. It's a straight copy into yyyymmmddd format dropping hh:mm:ss and without a time zone adjustment. It is also the value used to prefix EpisodeTitle in the BTV file.

time in TiVo maps to TargetStart adjusting from UTC (Z) time to the current time zone and converting into the number of 100-nanosecond intervals since January 1, 1601.

I'm not 100% sure what we'd agreed on for date/time values for movies but I believe it was the same as TV shows. The differences are in selected metadata fields where "Movies" is put into TitleName (BTV), movieYear (TiVo) is put into MovieYear (BTV), and EpisodeTitle (BTV) is omitted.

Thanks, Bill
OK, try the attached executable.

Note that FileTime is defined based on UTC time -- so if that's what targetStart really is, then "adjusting from UTC (Z) time to the current time zone" would not make sense.
 
Last edited:

tannebil

New member
I did some testing and everything looks good with both the date-times and movie vs TV show detection.

Thanks, Bill
 

dlflannery

Moderator
TVAP Ver. 0.89

10 Mar 2010 Ver. 0.89
Fixed – BTV metadata dates were incorrect.
Fixed – Unnecessary updates to input video file list during file transfers caused scrolling that interfered with operator use of the list.

I've left Ver. 0.88 on the download site in case.......

tannebil: The second fix should address the excessive scrolling problem you noted in your post #239. The only time the list should scroll now is if a file is deleted, added or changes status from TRANSFERRING to NEW.
 

dlflannery

Moderator
TVAP Ver. 0.90 - MPEG4 metadata

10 March 2010 Ver. 0.90
Added
– Default configuration option to insert metadata in .mp4 and .m4v output files, using AtomicParsley. Checkbox is enabled only if running TVSuite 4. AtomicParsley.exe is installed with TVAP. It is run before any postprocess or cleanup batch files.

Thanks are due the Source Forge AtomicParsley project for this capability.

Due to two TVAP updates in one day, the previous two versions of the program have been left on the download site, just in case.....
 

tannebil

New member
I'm getting a bunch (10 in a row last time) of "AtomicParlsey stopped working" errors getting thrown off by TVAP. I do not have the "MPEG-4 Metadata" option checked in "Default Settings". It happens every time on a reboot (I have TVAP in Startup with the -metabtv -hotstart options. I have not seen a pattern when stopping and starting TVAP although it did happen a few minutes after I quit TVAP and restarted it using the Startup shortcut. Atomic Parsley runs from a command line without any problem.
 

dlflannery

Moderator
I'm getting a bunch (10 in a row last time) of "AtomicParlsey stopped working" errors getting thrown off by TVAP. I do not have the "MPEG-4 Metadata" option checked in "Default Settings". It happens every time on a reboot (I have TVAP in Startup with the -metabtv -hotstart options. I have not seen a pattern when stopping and starting TVAP although it did happen a few minutes after I quit TVAP and restarted it using the Startup shortcut. Atomic Parsley runs from a command line without any problem.
This is extremely baffling! The program logic is very simple: If you don't have the MPEG-4 Metadata option checked it won't even attempt to start AtomicParsley! Have the problems always been present with Ver. 0.90 of TVAP?

Are you getting one error message per output file? When do they pop up relative to other file processing steps? Does it work correctly on some files and you have verified the metadata was inserted?

The only clue is it seems to occur only when you use your startup shortcut. Can you describe exactly how you constructed the shortcut? It may have to do with Win7's funky security policies, i.e., issues such as run-as-administrator, UAC, etc. Under certain conditions (when it doesn't grant you permission to write to a file where you THINK it is located) it will write to a "virtual" copy of the file. TVAP uses

%APPDATA%\TVAP, which translates in Win7 to:

C:\Users\<Logon Name>\AppData\Roaming\TVAP

to store its configuration data but the virtual folder would be:

C:\Users\<Logon Name>\AppData\Local\VirtualStore\TVAP.

If your startup shortcut is being called before your logon, I think you could have a problem there -- Windows would not know the correct Logon Name so what data folder would TVAP be trying to use? It would be interesting to search in your User folder tree to see where files named "TivoFileManager.xml" are located and what dates they have -- also if any version has:

<InsertMpeg4MetaData>true</InsertMpeg4MetaData>

However, if TVAP was using such a file, the check box should be set to checked on startup -- so still a mystery.

Some long shots to try:
1. Look at properties of AtomicParsley.exe. Is there an "Unlock" button? If so click it.
2. Right click AtomicParsley.exe and run it one time with "Run as Administrator".


Take a look at my post #222, particularly this:

So then I experimented with scheduling for "At Startup" and "At Logon", triggers including a 30 sec delay in each case. The At-Logon case worked fine.

The At-Startup case didn't seem to work but in Task Manager I found the program actaully was running with SYSTEM as the owner. AFAIK there is no way to see the program GUI in this case, and it wasn't in the System Tray. Since I hadn't used the -shutdown argument, I had to end the program using Task Manager. Maybe it would actually work this way but I think the At Logon trigger makes more sense.
When TVAP runs on reboot with your startup shortcut, can you bring up the GUI? If not this is kind of unknown territory. For example there would be no way for you to see whether configuration items were checked or not.

If you are just placing a shortcut in the startup folder, i.e.,

C:\Users\<Logon Name>\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup

Then many of my questions above don't apply. I just experimented with that and saw no problems with data folders or running without the mp4 metadata checked.
 
Last edited:

tannebil

New member
Sorry for the delay. I'm traveling and have not had a chance to look at things.

I'm doing the intial startup of TVAP using the startup folder on my user account. It starts in "C:\Program Files (x86)\TVAP".

I didn't see an unlock button on atomicparseley.exe. I ran it as administrator and I'll see what happens on the next reboot which might not be until Tuesday (can't put Final Four at risk).

TVAP seems to run fine after the startup. It's just the annoyance of the error messages which don't seem to match in number to anything. On the last reboot, it looks like there were three files pending but there were at least 6 error messages.

Thanks, Bill
 

dlflannery

Moderator
TVAP Ver. 0.91, QSF Filtering

7 April 2010 Ver. 0.91
Added: Automatic application of dimension filters during QSF based on configuration file “VRDDimensionFilters.txt” located in the user's TVAP data folder (%APPDATA%\TVAP). Each line in this text file must contain three items, formatted as follows:
<match string> :: <filter width> :: <filter height>
Examples:
Tonight Show :: 1920 :: 1080
WDTNDT :: 1280 :: 720

If the video name contains the match string, the corresponding dimensions will be applied.

Syntax rules:
  • Matches are NOT case sensitive.
  • The double-colon delimiters “::” MUST be used to separate the three items.
  • Quotes are not needed, and should not be used, on the match string (unless they are part of the string).
  • Spaces before or after each item are ignored. (The spaces on each side of the double-colon delimiters in the examples are just for a neat look.)
  • If a line is improperly formatted it is ignored. The TVAP log gives messages indicating whether a filter was used during QSF of a file, and the dimensions used.
  • The first match found will be used. Thus if you have two match strings like “WDTNDT” and “WDTNDT2”, place the line with the longer match string higher in the file list.
If the dimensions file doesn't exist filtering is not done but operation is otherwise normal.

The dimensions file is read at the start of each QSF operation -- Thus you can modify it and changes will be in effect without restarting TVAP.

Note: If the file being processed has NO scenes with the filter dimensions you specify, VideoReDo will pop up a message box, which you must click. This obviously halts automatic processing until you click it.
 

tannebil

New member
I rebooted and got no error messages this time.

Curiouser and curiouser. I'm traveling again but will try some more reboots next week to see if I can reproduce the problem.
 

tannebil

New member
How do you use dimension filters? Is it for situations where you have transfer both SD and HD versions of a program and only want to process one of them?
 

phd

Super Moderator
A stream can sometimes contain a mix of dimensions which cannot be processed directly.

They could be all SD as well.

QSF with filters will separate out the video segments that are the dimension you specify.
 

dlflannery

Moderator
How do you use dimension filters? Is it for situations where you have transfer both SD and HD versions of a program and only want to process one of them?
No. phd described it correctly, of course. VideoReDo will throw an error if you try to process a file in which the video dimensions change. Doing a QSF with a user-selected dimension filter has always been the solution to this.

Here are a few examples of people encountering this issue:

http://www.videoredo.net/msgBoard/showthread.php?t=17324

http://www.videoredo.net/msgBoard/showthread.php?t=6110

http://www.videoredo.net/msgBoard/showthread.php?t=17938
 

dlflannery

Moderator
Yes, it has been reported. I've never seen it myself but since my digital cable is Time Warner, most channels cannot be downloaded to my PC due to their copy protection policies. If you search this thread for posts by user goodtrips you will see an example.

My impression is mixed dimensions are much more common with sources other than TiVo, which is why I added the filtering feature to VAP first.

If you don't encounter mixed dimension sources, the new feature will be transparent to you. Otherwise you will have a tool to let you continue to use TVAP with them. You will know when you encounter one by the VRD error message that pops up.
 
Top Bottom