Post-download script
4 posters
Page 1 of 1
Post-download script
Here's a sample script for processing newly acquired comic files, similar to the one manders2600 developed, that will covert, tag, and rename the files. This one is in bash, so suitable for Mac or Linux only, probably. It uses the new export-to-zip feature, so a lot less file management. Suitable for running by hand, or via some other program.
- Code:
#!/bin/bash
# post-processing script for comics.
#make bash globbing case-insensitive
shopt -s nocaseglob
# Where you want the processed comics to go
DEST="/data/downloads/tagged_comics"
LOGFILE="$DEST/log.txt"
# You might need to change this
COMICTAGGER="/Applications/ComicTagger.app/Contents/MacOS/ComicTagger"
#-----------------------------------------------------
log ()
{
if [ -n "$1" ]
then
data="$@"
echo "[$(date +"%D %T")] $data" >> $LOGFILE
echo "[$(date +"%D %T")] $data"
else
while read data
do
echo "[$(date +"%D %T")] $data" >> $LOGFILE
echo "[$(date +"%D %T")] $data"
done
fi
}
#-----------------------------------------------------
touch $LOGFILE
# The first argument ($1) should be the folder with the comics
if [ "$1" == "" ]; then
WORKDIR="."
else
WORKDIR="$1"
fi
LOOK_FOR_COMICS=`ls "$WORKDIR"/*.cb[rz] 2> /dev/null`
if [ "$LOOK_FOR_COMICS" == "" ] ; then
echo "No comics here in $WORKDIR" 2>&1 |log
exit
fi
log "#####################################################"
log " # Processing comics in $WORKDIR"
log "#####################################################"
log "-------------Converting CBRs to CBZ------------------"
$COMICTAGGER -e --delete-rar "$WORKDIR"/*.cb[rz] 2>&1 | log
log "-------------Tagging with CR style-------------------"
$COMICTAGGER -s -o -f -m "year=" -t cr --nooverwrite --verbose "$WORKDIR/"*.cb[rz] 2>&1 | log
log "-------------Copying to CBI--------------------------"
$COMICTAGGER -c cr -t cbl --nooverwrite "$WORKDIR/"*.cb[rz] 2>&1 | log
log "-------------Renaming--------------------------------"
# Do the rename twice, since the filename overrides the internal metadata,
# and the tagging may have failed...
$COMICTAGGER -r -f -t cr "$WORKDIR/"*.cb[rz] 2>&1| log
$COMICTAGGER -r -t cr "$WORKDIR/"*.cb[rz] 2>&1| log
# Don't move anything if we're working in the current directory
if [ "$WORKDIR" == "$1" -a "$WORKDIR" != "." ]; then
log "-------------Relocating------------------------------"
mv "$WORKDIR/"*.cb[rz] $DEST 2>&1 | log
rmdir "$WORKDIR" 2>&1 | log
fi
log "Done!"
Making it a launchd service
Hi, i've been tinkering with comictagger for an automated tagging of my comics, and with the excellent help of your script i've managed to create a system that works fine:
This works with OSX 10.9 (and i suppose with earlier versions too) or anything that uses the launchd daemon.
We are gonna use a modified version of the script to tag our comics, and then we are gonna set up a daemon that will run periodically or whenever a new folder is detected in the _unTagged folder:
The Script:
EDIT: Some more edits to be able to invoke it as a daemon
I saved the script as "tagcomics" in my /opt/local/bin and chmod'd it to +x
> sudo chmod +x tagcomics
> sudo mv bagandtag /opt/local/bin/tagcomics
The Daemon setup
The setup for the daemon is pretty simple, really, but can be confusing. We need to create a .plist file that defines our daemon:
And we save it to /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
and we have to assign it the proper privileges.(unable to post links yet, just google it )
> sudo chown root /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
> sudo chmod 644 /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
Make sure you get the correct plist parameters before changing the permissions.
If you need to tinker with the plist, more info at:(link)
type man launchd at terminal
Once everything is set to your liking, you just have to get it started up with:
>sudo launchctl load /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
And when you need to edit the .plist file, you first have to
> sudo launchctl unload /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
And that's all, i think.
EDIT: Thank you very much to comictagger for his help in the modifications to the script, that line that finds the untagged comics, its his magic, boys! ^.^
EDIT2: Added some more refination to the script, it works better when periodically checking for tagged files, we just search for the ones that are still untagged. Still not improved in the all or nothing logic used to move the folders with the cbz files.
This works with OSX 10.9 (and i suppose with earlier versions too) or anything that uses the launchd daemon.
We are gonna use a modified version of the script to tag our comics, and then we are gonna set up a daemon that will run periodically or whenever a new folder is detected in the _unTagged folder:
The Script:
- Code:
#!/bin/bash
# post-processing script for comics.
#logging function Very usefull
#-----------------------------------------------------
log ()
{
if [ -n "$1" ]
then
data="$@"
echo "[$(date +"%D %T")] $data" >> $LOGFILE
echo "[$(date +"%D %T")] $data"
else
while read data
do
echo "[$(date +"%D %T")] $data" >> $LOGFILE
echo "[$(date +"%D %T")] $data"
done
fi
}
#-----------------------------------------------------
# Where you want the processed comics to go
DEST="/Where/do/we/put/our/tagged/files"
LOGFILE="$DEST/_ComicTagger_log_new.txt"
# Check to see if we are already working (we just want one process working at the same time
LOCKFILE="$DEST/comicTaggerLock"
# where's the find command
FIND="/usr/bin/find"
GREP="/usr/bin/grep"
CUT="/usr/bin/cut"
log "--------------------STARTING------------------"
if [ -f "$LOCKFILE" ]; then
log "Shit, we're second!"
exit
else
log "Lets stablish a beach head"
touch "$LOCKFILE"
fi
#make bash globbing case-insensitive
shopt -s nocaseglob
# You might need to change this
COMICTAGGER="/Applications/ComicTagger.app/Contents/MacOS/ComicTagger"
touch $LOGFILE
# The first argument ($1) should be the folder with the comics
if [ "$1" == "" ]; then
WORKDIR="."
else
WORKDIR="$1"
fi
# step 1, search for untagged files in the workdir
#Set the field separator to new line
IFS=$'\n'
UNTAGGED=`$FIND "$WORKDIR" -name "*.cb*" -exec $COMICTAGGER -p --terse {} + |$GREP "\[ none \]" |$CUT -f1 -d:`
log "-------Files to Process----"
if [ "$UNTAGGED" ]; then
log "Hay ficheros que taggear"
# log "$UNTAGGED"
else
log "No hay ficheros que taggear"
fi
#Step 2, tag files as UNTAGGED
log "-------Let's tag this files------"
# log "------------ Renaming Untagged comics for further processing --------------"
# log "$UNTAGGED"
for i in $UNTAGGED
do
if [[ $i == *\(UNTAGGED\)* ]]
then
log "$i: No need to rename";
else
NEWNAME="${i%.*} (UNTAGGED).${i##*.}"
mv $i $NEWNAME
# log "\"$i\": renamed to " \"$NEWNAME\"
#echo "$i: Need to Rename";
fi
done
#step 3, process just the files with the untagged tag
log "#####################################################"
log " # Processing untagged comics in $WORKDIR"
log "#####################################################"
# Convert to cbz if needed
$FIND "$WORKDIR" -name "*.cbr" -exec $COMICTAGGER -e --delete-rar {} + 2>&1 #| log
# Tag just the untagged ones
$FIND "$WORKDIR" -name "*\(UNTAGGED\)*" -exec $COMICTAGGER -s -o -f -m "year=" -t cbl --nooverwrite --noabort --verbose {} + 2>&1 #| log
# Step 4, rename files to proper series and number, etc...
# Renaming
log "-------------Renaming--------------------------------"
$FIND "$WORKDIR" -name "*.cb*" -exec $COMICTAGGER -r -f -t cbl {} + 2>&1 # | log
#Double Renaming
$FIND "$WORKDIR" -name "*.cb*" -exec $COMICTAGGER -r -t cbl {} + 2>&1 # | log
# Step 5, tag again the untagged ones
#Set the field separator to new line
IFS=$'\n'
UNTAGGED=`$FIND $WORKDIR -name "*.cb*" -exec $COMICTAGGER -p --terse {} + |"$GREP" "\[ none \]" |"$CUT" -f1 -d:`
# log "let's refind the untagged ones------"
# log "$UNTAGGED"
for i in $UNTAGGED
do
if [[ $i == *\(UNTAGGED\)* ]]
then
log "$i: No need to rename";
else
#we add an (UNTAGGED) to the filename so we can easily check for untagged comics
NEWNAME="${i%.*} (UNTAGGED).${i##*.}"
mv $i $NEWNAME
#log "\"$i\": renamed to " \"$NEWNAME\"
#echo "$i: Need to Rename";
fi
done
# Step 6, if all comics tagged, move them to _tagged
EMPTYFOLDER=`$FIND $WORKDIR -name "*.cb*"`
# Very important to use the variables for the binaries, so when called from launchd, they get the right result.
UNTAGGED=`$FIND $WORKDIR -name "*.cb*" -exec $COMICTAGGER -p --terse {} + |"$GREP" "\[ none \]" |"$CUT" -f1 -d:`
# log "--------Final Untagged files---------"
# log "$UNTAGGED"
# log "--------End of untagged files--------"
# Check for not null string as a result of finding cb* files in workdir
# TO-DO : Detect which folders are completely tagged and which are not, so we can move them individually
if [ "$EMPTYFOLDER" ]; then
if [ ! "$UNTAGGED" ] ; then
log "-------------Relocating------------------------------"
mv "$WORKDIR/"* $DEST 2>&1 | log
else
#not completely tagged, leave the files for further process
log "--------- Not Relocating--------"
fi
fi
rm "$LOCKFILE"
log "-------------------FINISHED--------------------"
EDIT: Some more edits to be able to invoke it as a daemon
I saved the script as "tagcomics" in my /opt/local/bin and chmod'd it to +x
> sudo chmod +x tagcomics
> sudo mv bagandtag /opt/local/bin/tagcomics
The Daemon setup
The setup for the daemon is pretty simple, really, but can be confusing. We need to create a .plist file that defines our daemon:
- Code:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>ExitTimeOut</key>
<integer>600</integer>
<key>RunAtLoad</key>
<true/>
<key>Label</key>
<string>com.Whatever.comicTaggerScript</string>
<key>ProgramArguments</key>
<array>
<string>/opt/local/bin/tagcomics</string>
<string>/Path/to/untagged/folder</string>
</array>
<key>StartInterval</key>
<integer>3200</integer>
<key>StandardOutPath</key>
<string>/var/log/comicTagger.log</string>
<key>StandardErrorPath</key>
<string>/var/log/comicTagger.log</string>
<key>Debug</key>
<false/>
</dict>
</plist>
And we save it to /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
and we have to assign it the proper privileges.(unable to post links yet, just google it )
> sudo chown root /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
> sudo chmod 644 /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
Make sure you get the correct plist parameters before changing the permissions.
If you need to tinker with the plist, more info at:(link)
type man launchd at terminal
Once everything is set to your liking, you just have to get it started up with:
>sudo launchctl load /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
And when you need to edit the .plist file, you first have to
> sudo launchctl unload /Users/myUserName/Library/LaunchAgents/com.wathEverSuitsYou.comicTaggerScript.plist
And that's all, i think.
EDIT: Thank you very much to comictagger for his help in the modifications to the script, that line that finds the untagged comics, its his magic, boys! ^.^
EDIT2: Added some more refination to the script, it works better when periodically checking for tagged files, we just search for the ones that are still untagged. Still not improved in the all or nothing logic used to move the folders with the cbz files.
shadowcharly- Posts : 3
Join date : 2014-02-12
Would love a windows version
All these options seem pretty linux and OSX specific and I was hoping there would be a windows option, I don't know enough about scripting to do it myself but I know this code worked for me when ran manually from the cmd
- Code:
./comictagger.exe -s -t cr -f -o $DOWNLOADED_COMIC
dlrdlrdlr- Posts : 2
Join date : 2015-04-01
A Windows PowerShell Version
Below is some code that works in Windows PowerShell. I'm not a PowerShell programmer so I don't know what version this will or will not work with . I wrote it in 8.1.
It will work very much like the script at the top of this page but I like to put my folders into a folder and process them all so the main essentially traverses the sub-folders in the folder that you define either as a parameter or your current folder.
If you want to use it like the first post from comictagger, I think it will work if you just change the main call at the bottom of the script to processComics and pass the source variable (haven't actually tried it)
It will work very much like the script at the top of this page but I like to put my folders into a folder and process them all so the main essentially traverses the sub-folders in the folder that you define either as a parameter or your current folder.
If you want to use it like the first post from comictagger, I think it will work if you just change the main call at the bottom of the script to processComics and pass the source variable (haven't actually tried it)
- Code:
# Param([string]$source=".")
# Unfortunately the line above must be the first line if you want to use a named param like -source
# If you do not define your source in your call, it will default to your current directory
# Global variables
# I'm lazy and redefine source here for testing, you can uncomment this and hardcode it here if you want
#[string]$source="D:\comics\unprocessed"
# Set your destination of where the processed comics will go
[string]$dest="D:\comics\processed"
# Set to where you want your log file written - currently set to be created where your comics are sent
[string]$logfile="$dest\log.txt"
# This starts the logging of all messages
Start-Transcript -path $logfile
# Function to process the current working directory that is passed to it.
# See comments below on how the main processes the folder
function processComics($WORKDIR) {
# Set this to the path of your executable - in Windows it is usually here
[string]$comictagger="C:\Program Files (x86)\ComicTagger\comictagger.exe"
# Use Write-OUtput if you want to input text to the std out and to the log
# Notice how two lines down you can insert your variable directly into the quotes
Write-Output "#####################################################"
Write-Output " # Processing comics in $WORKDIR"
Write-Output "#####################################################"
Write-Output "-------------Remove Text Files------------------"
# This will remove the .txt files that are sometimes in the folders that catalog what is in the folder
Remove-Item "$WORKDIR/*.txt" -recurse -force | Out-Host
# Remove the Archive attribute if it is set on the files in this folder
# it will change all read onlys in the folder if you process it
removeReadOnly $WORKDIR
Write-Output "-------------Converting CBRs to CBZ------------------"
# This is pretty much the same as the other scripts but I do not convert to comic lover, change as needed but follow format
& $COMICTAGGER -e --delete-rar "$WORKDIR/*.cb*" 2>&1 | Out-Host
Write-Output "-------------Tagging with CR style-------------------"
& $COMICTAGGER -s -o -f -t cr -1 --nooverwrite --verbose --noabort "$WORKDIR/*.cb*" 2>&1 | Out-Host
Write-Output "-------------Renaming--------------------------------"
# Do the rename twice, since the filename overrides the internal metadata,
# and the tagging may have failed...
& $COMICTAGGER -r -f -t cr "$WORKDIR/*.cb*" 2>&1| Out-Host
& $COMICTAGGER -r -t cr "$WORKDIR/*.cb*" 2>&1| Out-Host
Write-Output "-------------Relocating------------------------------"
# Moves all of the files to your destination
Move-Item -Path "$WORKDIR/*.cb*" -Destination $DEST 2>&1 | Out-Host
# Call the funciton to remove the directory if it is empty
removeEmptyDirectory $WORKDIR
}
# Function will check the folder to see if it is empty and if so will remove it
function removeEmptyDirectory($WORKDIR){
# Checks if the folder is empty of files if it is it will remove the folder
If((Get-ChildItem $WORKDIR -force | Select-Object -First 1 | Measure-Object).Count -eq 0)
{
Remove-Item $WORKDIR | Out-Host
}
Else
{
Write-Output "***************************************************************************************************************"
Write-Output "*********************** $WORKDIR IS NOT EMPTY ***********************************"
Write-Output "***************************************************************************************************************"
}
}
# Will set all the attributes in the current working folder to normal - removing any read only attributes
# This will not recurse so it will only do the current working directory and no subs of that directory - hopefully safer
function removeReadOnly($WORKDIR){
Get-Item $WORKDIR | foreach {$_.Attributes = 'Normal'}
}
function main(){
# Grab a collection of all folders within your source and loop through them checking for comics in each one
$dirs = Get-ChildItem $source | Where-Object { $_.Attributes -band [System.IO.FileAttributes]::Directory }
foreach ($dir in $dirs)
{
$fullpath=$dir.FullName + "\*.*"
# Get count of how many comic files ar in the directory
$comic_count=(Get-ChildItem $fullpath -include *.cbr,*.cbz).Count
Write-Output "the count of comics in $dir.FullName is $comic_count"
# If there are any comics in the folder, pass the foldername to the processComics function
# If not try to remove the directory if it is empty
If ($comic_count -gt 0) {
processComics $dir.FullName
#removeReadOnly $dir.FullName
}
Else
{
removeEmptyDirectory $dir.FullName
}
}
# Stop and close the log file
Stop-Transcript
}
main
tomdelise- Posts : 2
Join date : 2015-05-18
Re: Post-download script
I cleaned up the script a little so it ins't as bad formed as it was. I also was going to play with the great move2folder script in the next thread that takes the tagged comics and creates directories and sub-directories from that BUT I didn't have to. The script will work as it is written in PowerShell if you have python 2.7 installed. I removed the check that will write out an error if in windows since it works just fine.
The cleaned up script uses 3 folders. One to place the unprocessed comics that are in their own folders. It will tag the comics and move them all into a processed folder then the move2folder will create all the folders and subs in the final destination folder moving the files if they are tagged and there isn't a copy already there.
You can also setup this to work like the post above by using Windows Task Scheduler. If you put all of your unprocessed comic folders into your unprocessed folder you can run this nightly from your scheduler. The only thing that I believe is still a problem is getting the script to pause when your ComicVine ID locks for a bit.
The cleaned up script uses 3 folders. One to place the unprocessed comics that are in their own folders. It will tag the comics and move them all into a processed folder then the move2folder will create all the folders and subs in the final destination folder moving the files if they are tagged and there isn't a copy already there.
You can also setup this to work like the post above by using Windows Task Scheduler. If you put all of your unprocessed comic folders into your unprocessed folder you can run this nightly from your scheduler. The only thing that I believe is still a problem is getting the script to pause when your ComicVine ID locks for a bit.
- Code:
# Param([string]$source=".")
# Unfortunately the first line has to be there if you want to use a named param like -source
# If you do not define your source in your call, it will default to your current directory
# Global variables
# I'm lazy and redefine source here for testing, you can uncomment this and hardcode it here if you want
[string]$source="D:\comics\unprocessed"
# Set your folder of where the tagged comics will go
[string]$middle="D:\comics\processed"
# Set the folder of where the tagged comics will go and create folders from the tags
[string]$finalDest="D:\comics\comicsdone"
# Path to where the move2folder or other post run script you want to run is
[string]$scripts="D:\comics\privatescripts"
# Set this to the path of your executable - in Windows it is usually here
[string]$comictagger="C:\Program Files (x86)\ComicTagger\comictagger.exe"
# Set to where you want your log file written - currently set to be created where your comics are sent
[string]$logfile="$dest\log.txt"
# Function to process the current working directory that is passed to it.
# See comments below on how the main processes the folder
function processComics($workdir) {
# Use Write-OUtput if you want to input text to the std out and to the log
# Notice how two lines down you can insert your variable directly into the quotes
Write-Output "#####################################################"
Write-Output " # Processing comics in $workdir"
Write-Output "#####################################################"
Write-Output "-------------Remove Text Files------------------"
# This will remove the .txt files that are sometimes in the folders that catalog what is in the folder
Remove-Item "$workdir/*.txt" -recurse -force | Out-Host
# Remove the Archive attribute if it is set on the files in this folder
# it will change all read onlys in the folder if you process it
removeReadOnly $workdir
Write-Output "-------------Converting CBRs to CBZ------------------"
# This is pretty much the same as the other scripts but I do not convert to comic lover, change as needed but follow format
& $comictagger -e --delete-rar "$workdir/*.cb*" 2>&1 | Out-Host
Write-Output "-------------Tagging with CR style-------------------"
& $comictagger -s -o -f -t cr -1 --nooverwrite --verbose --noabort "$workdir/*.cb*" 2>&1 | Out-Host
Write-Output "-------------Renaming--------------------------------"
# Do the rename twice, since the filename overrides the internal metadata,
# and the tagging may have failed...
& $comictagger -r -f -t cr "$workdir/*.cb*" 2>&1| Out-Host
& $comictagger -r -t cr "$workdir/*.cb*" 2>&1| Out-Host
Write-Output "-------------Relocating------------------------------"
# Moves all of the files to your destination
Move-Item -Path "$workdir/*.cb*" -Destination $middle 2>&1 | Out-Host
# Call the funciton to remove the directory if it is empty
removeEmptyDirectory $workdir
}
# Function will check the folder to see if it is empty and if so will remove it
function removeEmptyDirectory($workdir){
# Checks if the folder is empty of files if it is it will remove the folder
If((Get-ChildItem $workdir -force | Select-Object -First 1 | Measure-Object).Count -eq 0)
{
Remove-Item $workdir | Out-Host
}
Else
{
Write-Output "***************************************************************************************************************"
Write-Output "*********************** $workdir IS NOT EMPTY ***********************************"
Write-Output "***************************************************************************************************************"
}
}
# Will set all the attributes in the current working folder to normal - removing any read only attributes
# This will not recurse so it will only do the current working directory and no subs of that directory - hopefully safer
function removeReadOnly($workdir){
Get-Item $workdir | foreach {$_.Attributes = 'Normal'}
}
function main(){
# Grab a collection of all folders within your source and loop through them checking for comics in each one
$dirs = Get-ChildItem $source | Where-Object { $_.Attributes -band [System.IO.FileAttributes]::Directory }
foreach ($dir in $dirs)
{
$fullpath=$dir.FullName + "\*.*"
# Get count of how many comic files ar in the directory
$comic_count=(Get-ChildItem $fullpath -include *.cbr,*.cbz).Count
Write-Output "the count of comics in $dir.FullName is $comic_count"
# If there are any comics in the folder, pass the foldername to the processComics function
# If not try to remove the directory if it is empty
If ($comic_count -gt 0) {
processComics $dir.FullName
#removeReadOnly $dir.FullName
}
Else
{
removeEmptyDirectory $dir.FullName
}
}
# Run comictagger with the python script to create folders and subfolders based on tags
& $comictagger -S "$scripts\move2folder.py" "$middle" "$finalDest" 2>&1| Out-Host
}
# This starts the logging of all messages
Start-Transcript -path $logfile
main
# Stop and close the log file
Stop-Transcript
tomdelise- Posts : 2
Join date : 2015-05-18
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum