VideoReDo-Autoprocessor Program

dlflannery

Moderator
Thanks. I haven't identified a specific need for such files. Posters reporting problems earlier in this thread stopped responding long ago (while I was still trying to define their problem). In this VAP sub-forum I want to address only problems that occur ONLY when VAP is involved. Problems that can be reproduced using VRD alone (interactive or batch) should be reported in the appropriate VRD sub-forums. Also I would prefer that VAP problems be reported by starting a new thread specific to the problem rather than in this thread.
 

dlflannery

Moderator
How do I configure VAP to shrink/convert the files to something smaller so I can watch it on my Android phone? Thanks.
You can set up a post-process script to use HandBrake to transcode the video into other formats/sizes.

Search for Post-Process and VAPpostProcess.cmd in the VAP documentation here.
This should just be a matter of selecting an appropriate VRD Output Profile, or making a modified custom Output Profile to use. It might pay to look and search in the other VRD sub-forums as this is not a new topic. The answer does depend on the properties of your input file and the video and audio parameters accepted by your phone.

Some people do prefer to use HandBrake in a VAPpostProcess script file apparently. I believe a sample script is distributed with VAP but this is based only on memory since I'm not able to access my PC right now. For more info on this search on "vappostprocess" in the PDF doc.

If you want more help or discussion on this, please start a new thread and give details on your input files and phone.
 

rgracie

Member
Trying to decide if VAP will work for my setup..

My current setup is a bit of a hodgepog setup.

1) Recordings from my CETON Tuner via SageDCT (in .ts format) get dump into VideoRecordings folder. (All these folders need qsf) Initiated by SageTV.
1a)Recordings from my Hauppauge Tuners (in .mpeg format) get dump into VideoRecordings folder. (these files do not need qsf but I do it anyway) Initiated by SageTV.
2) WatchDirectory watches the folder for new .ts and .mpeg files, and once the recording is complete, (unlocked by SageTV) fires off a batch file.

In this batch file....
3) The file is checked for size (must be larger then 500megs) and than ran through VRD Quick Stream Fix.
4) Then ran through ComSkip.
5) Then ran back through VRD to cut the commercials out.
6) Form here, and this is the important part, the batch files checks the name of the recoding (the name is checked from right to left) and compares it to a parameter file and pulls the storage path used to copy the completed file to.

Does this setup sound like it can be reproduced by solely using VAP?
 

dlflannery

Moderator
Maybe. It depends on some details.
What format and extension are your recorded files? Do they already contain embedded metadata?

Does the 500 MB minimum mean smaller files are not processed at all, or just not run thru QSF?

Please furnish some example input file names and some details on how they are parsed to determine output folder.
VAP puts files in output folders with names based on metadata (and can rename the output file itself based on metadata). VAP gets the metadata three ways:
1. Reads embedded metadata if available in input .wtv and .mp4 files.
2. Looks it up for TV series shows using theTVDB.com provided it has key elements to search by.
3. Parses it from input file name using operator-specified templates (similar to regular expressions).

Your method of selecting the output folder is not possible via VAP but perhaps a suitable metadata-based scheme can be implemented.

VAP determines when a monitored file has finished recording by checking the file size every few seconds. If the size doesn't grow for a certain number of seconds VAP assumes the file is done recording. If one of your software components keeps the file locked in such a way that it interferes with VAP, it could cause problems. VAP only has to read the input file to do its processing, so I doubt there will be issues.

To accomplish the 500 MB filtering (assuming you mean don't process the small files at all), you could use a simplified version of your current batch file that just checks the size and moves only the larger files to another folder that would be the folder monitored (watched by) VAP. As long as the source and destination folders for such moves are on the same disk directory, they will be practically instantaneous.

So I think the primary issue is whether VAP can create the output folder system you want. Answers to my above questions should help determine whether this is possible.
 

rgracie

Member
Thanks for the response.

1) They are either .mpeg or .ts and no imbedded metadata. Pure stream recordings.
2) Anything under 500 megs is not processed at all. I can loose this option if not possible, it's just this helps stop broken recording from being processed. (just about every failure ends up with a file that is under 500)

Example:

AlaskaTheLastFrontier-S04E03-SpringForward-37047950-0

Here is some example code used in the bat file:

Code:
:GetUntilDash

SET SEARCHNAME=

:GetUntilDash_loop

IF "%HAYSTACK%" EQU "" GOTO :EOF
   IF "%HAYSTACK:~0,1%" EQU "-" GOTO :SearchFile
      SET SEARCHNAME=%SEARCHNAME%%HAYSTACK:~0,1%
      SET HAYSTACK=%HAYSTACK:~1%

GOTO :GetUntilDash_loop

:SearchFile
:RetrySearch

@echo On

:: Get the Show Params
for /f "tokens=2-11 delims=;" %%A in ('findstr /b /i "%SEARCHNAME%" "C:\FTPSyncFiles\Bat Files\SHOWPARMS.txt"') do ( 

set sQSF=%%A
set sCOM=%%B
set sPAT=%%C
set sM3UPath=%%~C
set sRAR=%%D
set sUPL=%%E
set sSFV=%%F
set sDEL=%%G
set sCUT=%%H
The sPAT var contains the folder path

Here is what the show SHOWPARMS.txt looks like inside

Code:
UnderTheDome;False;"C:\Program Files (x86)\ComSkip\comskip.ini";"\\SERVER\Recorded TV\Under The Dome";False;"";False;False;True
TheMiddle;False;"C:\Program Files (x86)\ComSkip\comskip.ini";"\\SERVER\Recorded TV\The Middle";False;"";False;False;True
TheBigBangTheory;False;"C:\Program Files (x86)\ComSkip\comskipTBBT.ini";"\\SERVER\Recorded TV\TBBT";False;"";False;False;True
Hope this helps...

Also, is it possible to select which comskip ini file is used based on the show?
 

dlflannery

Moderator
VAP can't use different comskip.ini files per showname. If that is a showstopper then I guess our discussion is over, at least for now. You are not the first person who has wanted this so I may look at incorporating that feature. VAP already can apply customized OutputProfiles and process flows (i.e., QSF? AdScan? Autocut-after-adscan?) based on matching string fragments in the input file names. Custom comskip.ini files might be implemented by adding a "comskip ini file index number", e.g., 0, 1, 2 etc., which would select a comskip#.ini file in the user's VAP data folder.

Other than the comskip ini issue I think VAP can do what you want although I would have a few other issues to discuss if you want to proceed. If you want to try it and can verify VAP does what you want except for the custom ini files, I will definitely add that feature.
 

rgracie

Member
Oh I'm definitely interested in getting all my processing under one program. As you can imagine, the up-keep on my current system can be a bit of a pain. I have found over the years using COMSKIP, what works on one channel/show doesn't always work on another.

So I loaded VAP and wow.... I'm VERY impressed indeed! (Nice Work).

I don't really know how to setup the different shows upload folders yet, but in the interim, I've setup VAP to do the QSF/AdScan(Comskip's)and AutoCut. I've redirected WatchDirectory to watch the "Finished" folder to fire a modified bat file to move the TV show to the appropriate folder. It's not as fast as my current setup, but once I (we ;-)) can figure out the balance of the details and how too's, I firmly believe your solution will be much faster, cleaner and significantly easier to maintain.

I'm really impressed with you work, is there a donation system in place to help support your efforts?
 

dlflannery

Moderator
Great! And I have the custom Comskip ini file feature breathing now -- just need to test it some more.

If your example input file name is typical, VAP's input file name parsing will be able to get the Series Name, Episode Name, Season# and Episode#. I can verify that on my machine by just renaming an input file and trying it. Thus you can have output folders based on the series name as you seem to be doing already. But you can also have season# subfolders and/or rename the videos themselves, for example to get rid of the long number at the end of the names (unless you want to keep that).

One potential issue is the //server syntax in the output paths. These are given to the VRD COM functions as part of the output file path and that type path frequently seems to cause problems. However the workaround of mounting that path as a Windows drive letter and using that version of paths in VAP configuration usually works. No one has ever figured out exactly what the issue is here, but you might experiment with this if you haven't already.

Too tired to go further tonight, so: to be continued!
 
Last edited:

rgracie

Member
Sounds great dlflannery, I take it moss doesn't grow under your feet.. ;)

So are you saying currently VAP has the ability to handle show names and folder locations? Are there any tutorials around on this topic?


I'm not sure what you mean by "typical" but SageTV outputs file names as such..
AlaskaTheLastFrontier-S04E03-SpringForward-37047950-0
SageTV always outputs the show name like this AlaskaTheLastFrontier, but doesn't always output the other data like EP or Show name.

and yes, I would LOVE to be able to clean up the recordings name string, so that would be a big bonus!


Thanks again for your help.
 

dlflannery

Moderator
Yes, VAP can currently create/use output folders based on metadata items. For example I renamed a test file to
Code:
AlaskaTheLastFrontier-S04E03-SpringForward-37047950-0.mpg
In Advanced Configuration, on the Metadata Templates tab, I then:
1. Entered a "Template for parsing metadata from input file names" with
Code:
Extension = .mpg and Template={title}-S{seasonnum}E{epnum}-{eptitle}-
note the final dash in the template string.

2. Entered a TV Series Output file name/path renaimg template:
Code:
J:\Recorded TV\{title}\{eptitle}_s{seasonnum}_e{epnum}
the J: drive is a USB2 (i.e. slow) hard drive.
Note the Tag Grabber which helps you insert tags and prevents misspelling of them.

Metadata parsing only occurs when a file is first detected in VAP's monitored folder or if you select the file in the Monitored Files list and click "Reset", which I did.
VAP processed the file with QSF, Adscan, Use Comskip and Auto Cut. The output file was J:\Recorded TV\AlaskaTheLastFrontier\SpringForward_s04_e03.mpg

If I had used just:
Code:
J:\Recorded TV\{title}\{eptitle}
for the Output template then the output file would have been: J:\Recorded TV\AlaskaTheLastFrontier\SpringForward.mpg

You can have multiple input file parsing templates. So you could use just
Code:
Extension = .mpg and Template={title}-
as the second template. If uses the first template that matches so if the input file name only has the title (show or series name) the it will match the second template.

There are two many variations on these two templates to cover them all here. For example if you capitalize an output template tag (e.g., {Eptitle}) and that meta item isn't present, the output folder will default to the basic VAP output folder. If the tags aren't capitalized then empty strings are inserted in that case.

If you haven't done so already, I recommend you read the first few sections of the VAP-readme.pdf, In particular how VAP can semi-automate reviewing the commercial cuts made by Comskip (or VRD AdDetective) when you don't have Autocuts configured.
 

rgracie

Member
That is very nice, I think I will be able to make this work. I do have 3 more questions that don't seem to be covered in the .pdf (at least I've not seem them)

1) How do I insert the date as a folder for those shows without any EP info? Eg: \\SERVER\Recorded TV\Battle Creek\03-15-2015\BattleCreek-S01E03-MansBestFriend-36807625-0.ts

Code:
J:\Recorded TV\Battle Creek\{currentdate}\{eptitle}
2) I can't remember the other questions right now.... :confused:
 

rgracie

Member
Oh now I remember.... well one of them at least... :cool:

So I take it that I would have to create duplicate (.mpeg and .ts) templates, or is there a way to look for both in the same template?
 

rgracie

Member
After all this, it doesn't look like I would be able to send the completed file to a specified folder of my choice.....

Currently my folder structure looks something like this:

Z:\Almost Human\02-03-2014\AlmostHuman-25596991-0-cln.ts

but VAP would create the folder structure based on the file name which would look something like:

Z:\almosthuman\AlmostHuman.ts

There is no way the wife will like that, especial with long show names... I suspect no way around this?? I see there is a "string translator" for the TVDB lookup, would that work?
 

dlflannery

Moderator
...... How do I insert the date as a folder for those shows without any EP info? Eg: \\SERVER\Recorded TV\Battle Creek\03-15-2015\BattleCreek-S01E03-MansBestFriend-36807625-0.ts

Code:
J:\Recorded TV\Battle Creek\{currentdate}\{eptitle}
..........
I'm confused. Doesn't the example you gave have the EP name (MansBestFriend)? I'll assume that was a typo(?).

Do you really want the current date (the day the processing takes place) or could you use or prefer the Original Air Date?

I think if you configure a series title translation, e.g.,
Code:
BattleCreek ==> Battle Creek
and parse the {seasonnum} and {epnum} from the input file name, this will allow the TVDB lookup to succeed and then the tiles/names will be properly spaced and you will also have the original air date, which could be used as your date. Be sure that theTVDB.com lookups are not disabled on the Other tab of Advanced Config.

I will try this later myself.
 

rgracie

Member
Thank you for trying to hammer this out with me.

I'm sorry, I was just using the "J:\Recorded TV\Battle Creek\{currentdate}\{eptitle}" as an example, and any correct date (air or capture or process) will work.

I need some way to organize the shows on the server that only have the show name in the recording. (Episode ID and or Names are not guaranteed)

and I have some shows that look like this"

OnceUponaTime-AnInsidersGuide-36405421-0.ts
Just a note, you may want to update the comskip.exe executable as it appears that the packaged version does not utilize the Thread_count setting. I was only getting 10-16% CPU utilization, until I updated the comskip.exe to the latest version, and now can push 30-35%
 

dlflannery

Moderator
For an input file name of the form: Title-S##E##-<anything more>, Use this parsing template:
Code:
{title}-S{seasonnum}E{epnum}-
Add a Series Name translation such as: AlaskaTheLastFrontier==>Alaska The Last Frontier (i.e., insert the spaces)

I tried this on the AlaskaTheLastFrontier-S04E03-SpringForward-37047950-0 .mpg file using the following renaming template:

J:\Recorded TV\{title}\{month}-{day}-{year}\{eptitle}_s{seasonnum}_e{epnum}

The output file path was:
J:\Recorded TV\Alaska The Last Frontier\10-19-2014\Spring Forward_s04_e03.mpg

Obviously you can adjust the output naming template as desired. The {month}-{day}-{year} tags are for the Original Air Date.

So this should handle cases for which the series title and season/ep numbers can be parsed out, with or without the show name in the file title. Note you don't want to parse the show name (ep name) from the file name. If you do, it will stick as the "unspaced" version. Rather you let the TVDB search succeed based on just the (translated) series title and the season/ep numbers. This gives you a whole raft of metadata from theTVDB.com, including the OAD.

As for cases where the input file name has the series name but not the season/ep #'s, I don't have a solution. VAP will go ahead and process them but will put the output file on the default output folder and will not rename the file provided you capitalized at least one of the missing meta-tags in the file name portion of the template, e.g., {Eptitle}. Renaming would work if you put in a suitable string translation for the episode name, but that's not workable for obvious reasons.

The TVDB lookup will also succeed based on just the series and episode names (i.e., without the season/ep #'s) but the episode name would have to have spaces.

I realized the distributed version of (free) Comskip needed to be updated and the latest version will be in the next release.
 

rgracie

Member
Ok Here is what I've got, and the problems that I'm running into.

1) The TVDB connection always fails. I've tested this at least 15 times over the last 2 days and it never works.
2) The output file never gets moved to the listed folder (ends up in the default folder with only the ep name (SindependenceDay_s06_e12.ts)).

I'm sure I've got something, probably many things wrong, but here are some screen shots of my setup..


3-17-2015 10-39-38 AM.png
3-17-2015 10-40-13 AM.png
3-17-2015 10-44-22 AM.png

The Log:

3/17/2015 11:03:31 AM theTVDB metaData search failed for: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:31 AM series/episode search fail message: Failed get one series match
3/17/2015 11:03:31 AM series/season#/ep# search fail message: theTVDB connection failed
3/17/2015 11:03:31 AM Series Name = "Drugs Inc". Episode Name = "SindependenceDay", Air Date = ""
3/17/2015 11:03:35 AM Getting input file encoding and video width for: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:35 AM 1280 is video width for: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:35 AM MPEG2 is encoding for: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:35 AM Started QSF, input: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:36 AM Using filter dimensions from MediaInfo: 1280 X 720
3/17/2015 11:03:36 AM Using profile: MPEG-2 Transport Stream
3/17/2015 11:03:37 AM VideoReDo said: INFO: VideoReDo version 5.1.1.726 - Mar 6 2015
3/17/2015 11:03:37 AM VideoReDo said: INFO: VideoReDo opened file/project: E:\VideoCaptures\DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:38 AM VideoReDo said: INFO: Applying QSF filter: 1280X720
3/17/2015 11:03:46 AM VideoReDo completed QSF on input: E:\VideoCaptures\DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:50 AM Getting input file encoding and video width for: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:50 AM 1280 is video width for: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:50 AM MPEG2 is encoding for: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:03:50 AM Started ComSkip detection followed by autocuts, input: DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:04:41 AM VideoReDo completed Comskip ad detection on input: E:\VideoCaptures\QSF\DrugsInc-S06E12-SindependenceDay-36770989-0.ts
3/17/2015 11:04:41 AM Failed to create metadata-based output folder due to missing metadata item: Year
3/17/2015 11:04:41 AM Template was: Z:\{title}\{month}-{day}-{year}\
3/17/2015 11:04:41 AM Input file was: DrugsInc-S06E12-SindependenceDay-36770989-0.vprj
3/17/2015 11:04:41 AM Meta-renamed output file: SindependenceDay_s06_e12.vprj
3/17/2015 11:04:44 AM Started auto-cuts, input: DrugsInc-S06E12-SindependenceDay-36770989-0
3/17/2015 11:04:44 AM Failed to create metadata-based output folder due to missing metadata item: Year
3/17/2015 11:04:44 AM Template was: Z:\{title}\{month}-{day}-{year}\
3/17/2015 11:04:44 AM Input file was: SindependenceDay_s06_e12.vprj
3/17/2015 11:04:44 AM Using profile: MPEG-2 Program Stream
3/17/2015 11:04:45 AM VideoReDo said: INFO: VideoReDo version 5.1.1.726 - Mar 6 2015
3/17/2015 11:04:56 AM VideoReDo completed manual cuts on: DrugsInc-S06E12-SindependenceDay-36770989-0
3/17/2015 11:04:56 AM Meta-renamed output file: SindependenceDay_s06_e12.ts
 
Top Bottom