For 0.9.1-beta, OS X 10.8 - copying CR tags to CBL using the CLI cause ComicBookLover to hang on import.

View previous topic View next topic Go down

For 0.9.1-beta, OS X 10.8 - copying CR tags to CBL using the CLI cause ComicBookLover to hang on import.

Post  maxx on Fri Dec 07, 2012 7:57 pm

I checked the files and they indeed had the CBL comments and they looked fine to me. But something about the way they are causes CBL to hang when importing them. I will try with more files and using the Python source as well just in case.

Using a plugin for CR called "ComicBookInfo Generator", the files import, though the script doesn't do a good job at mapping values.

Here's the result of the same files with ComicTagger and ComicBookInfo Generator

ComicTagger
Code:
{"lastModified": "2012-12-07 18:43:57.739625", "ComicBookInfo/1.0": {"publisher": "IDW Publishing", "publicationYear": "2010", "publicationMonth": "9", "title": "1 (of 5)", "series": "5 Days To Die", "tags": [], "comments": "Ray Crisara, a tough cop who\u2019s survived a terrible car accident, discovers his wife is dead and his daughter in critical condition. Believing this was a planned hit by the city\u2019s drug lord he\u2019s been pursuing, Ray\u2019s brain injury leaves him five days before he dies to track the killer or be there for his little girl\u2026 Ray\u2019s clock is ticking.", "volume": "2010", "credits": [{"person": "Andy Schmidt", "role": "Writer"}, {"person": "Chee Yang Ong", "role": "Penciller"}, {"person": "Chee Yang Ong", "role": "Inker"}, {"person": "Robbie Robbins", "role": "Letterer"}, {"person": "David Finch", "role": "Cover"}, {"person": "Bob Schreck", "role": "Editor"}], "genre": "crime", "issue": "1"}, "appID": "ComicTagger/0.1"}

And with ComicBookInfo Generator
Code:

{
  "lastModified": "2012-12-07 18:45:33 +0000",
  "appID": "ComicRack",
  "ComicBookInfo/1.0": {
    "issue": "1",
    "genre": "",
    "publicationYear": 2010,
    "comments": "Ray Crisara, a tough cop whos survived a terrible car accident, discovers his wife is dead and his daughter in critical condition. Believing this was a planned hit by the citys drug lord hes been pursuing, Rays brain injury leaves him five days before he dies to track the killer or be there for his little girl Rays clock is ticking.",
    "credits": [
      {
        "role": "Inker",
        "person": "Chee Yang Ong"
      },
      {
        "role": "Writer",
        "person": "Andy Schmidt",
        "primary": "YES"
      },
      {
        "role": "Letterer",
        "person": "Robbie Robbins"
      },
      {
        "role": "Editor",
        "person": "Bob Schreck"
      },
      {
        "role": "Cover Artist",
        "person": "David Finch"
      },
      {
        "role": "Penciller",
        "person": "Chee Yang Ong"
      },
      {
        "role": "Artist",
        "person": "Chee Yang Ong",
        "primary": "YES"
      }
    ],
    "title": "1 (of 5)",
    "publisher": "IDW Publishing",
    "publicationMonth": 9,
    "series": "5 Days To Die",
    "rating": 0.0,
    "volume": 2010
  }
}


Reading through the thread in the forum post I linked and looking at the output of ComicTagger, it seems to be an issue with the text encoding.
Here's the Python script for that plugin where they do a conversion to ASCII...



Code:

'''
ComicBookInfo Generator Plug-In for ComicRack by TomWaitsForNoMan
This plugin utilises two external libraries:
DotNetZip - *******dotnetzip.codeplex.com/
Json.NET - *******json.codeplex.com/
'''

#@Name Generate ComicBookInfo
#@Hook Books

import clr
import System
import System.IO
from System.IO import *
import datetime

#clr.AddReference("System.Collections")
from System.Collections import Hashtable

clr.AddReference("System.Windows.Forms")
from System.Windows.Forms import MessageBox, MessageBoxButtons, DialogResult

clr.AddReference("Ionic.Zip.dll")
from Ionic.Zip import ZipFile #@UnresolvedImport

clr.AddReference("Newtonsoft.Json.dll")
from Newtonsoft.Json import * #@UnresolvedImport






def CBIGen( books ):

    dlgResult = MessageBox.Show( "Generate ComicBookInfo for " + str( len( books ) ) + " comics. Confirm?", "Confirm", MessageBoxButtons.YesNo )
    if dlgResult == DialogResult.Yes:
        for book in books:
            if book.FilePath:
                bookFileInfo = FileInfo( book.FilePath )
                if bookFileInfo.Extension.lower() == ".cbz":
                    try:
                   
                        appID = 'ComicRack'
                        lastModified = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S +0000')
                   
                        #general tags
                        #we have to strip accents here because ComicBookLover can't handle accented characters
                        publisher = latin1_to_ascii(book.Publisher)
                        series = latin1_to_ascii(book.Series)
                        volume = book.Volume
                        issue = book.Number
                        title = latin1_to_ascii(book.Title)
                        publicationYear = book.Year
                        publicationMonth = book.Month
                        rating = book.Rating
                        genre = latin1_to_ascii(book.AlternateSeries)
                        comments = latin1_to_ascii(book.Summary)

                       
                        #credits
                        #There's no fixed way in which multiple artists are delimited in ComicRack credits, I have seen both
                        #comma-separated and slash-separated lists.  So for now just stick them all in one field
                        creditsDict = {'Writer': latin1_to_ascii(book.Writer),
                                        'Penciller': latin1_to_ascii(book.Penciller),
                                        'Inker': latin1_to_ascii(book.Inker),
                                        'Colorist': latin1_to_ascii(book.Colorist),
                                        'Letterer': latin1_to_ascii(book.Letterer),
                                        'Cover Artist': latin1_to_ascii(book.CoverArtist),
                                        'Editor': latin1_to_ascii(book.Editor),
                                        'Artist': latin1_to_ascii(book.Penciller)} #we have to strip accents again or ComicBookLover will break
                       
                        creditsList = []
                       
                        for role, names in creditsDict.iteritems():
                            if names != '':
                                nameList = names.split(', ')
                                for i, name in enumerate(nameList):
                                    if i == 0 and role in ['Writer','Artist']:
                                        creditsList.append( {'person': name, 'role': role, 'primary' : 'YES'} )
                                    else:
                                        creditsList.append( {'person': name, 'role': role} )
                       
                        cbiHash = Hashtable()
                       
                        cbiHash['appID'] = appID
                        cbiHash['lastModified'] = lastModified
                       
                        genHash = Hashtable()
                       
                        genHash['publisher'] = publisher
                        genHash['series'] = series
                        if volume != -1:
                            genHash['volume'] = volume
                        genHash['issue'] = issue
                        genHash['title'] = title
                        genHash['publicationYear'] = publicationYear
                        genHash['publicationMonth'] = publicationMonth
                        genHash['rating'] = rating
                        genHash['genre'] = genre
                        genHash['comments'] = comments

                       
                        if len(creditsList) > 0:
                            genHash['credits'] = creditsList
                       
                        cbiHash['ComicBookInfo/1.0'] = genHash
                       
                        json = JsonConvert.SerializeObject(cbiHash, Formatting.Indented)
                   
                       
                        with ZipFile.Read(book.FilePath) as zip:
                            zip.Comment = json
                            zip.Save()
                       
                    except Exception, ex:
                        MessageBox.Show( "Threw an exception: " + str( ex ) + " while processing " + book.FilePath + "." )
                       
                       
                       
#taken from *******code.activestate.com/recipes/251871-latin1-to-ascii-the-unicode-hammer/
def latin1_to_ascii (unicrap):
    """This takes a UNICODE string and replaces Latin-1 characters with
        something equivalent in 7-bit ASCII. It returns a plain ASCII string.
        This function makes a best effort to convert Latin-1 characters into
        ASCII equivalents. It does not just strip out the Latin-1 characters.
        All characters in the standard 7-bit ASCII range are preserved.
        In the 8th bit range all the Latin-1 accented letters are converted
        to unaccented equivalents. Most symbol characters are converted to
        something meaningful. Anything not converted is deleted.
    """
    xlate={0xc0:'A', 0xc1:'A', 0xc2:'A', 0xc3:'A', 0xc4:'A', 0xc5:'A',
        0xc6:'Ae', 0xc7:'C',
        0xc8:'E', 0xc9:'E', 0xca:'E', 0xcb:'E',
        0xcc:'I', 0xcd:'I', 0xce:'I', 0xcf:'I',
        0xd0:'Th', 0xd1:'N',
        0xd2:'O', 0xd3:'O', 0xd4:'O', 0xd5:'O', 0xd6:'O', 0xd8:'O',
        0xd9:'U', 0xda:'U', 0xdb:'U', 0xdc:'U',
        0xdd:'Y', 0xde:'th', 0xdf:'ss',
        0xe0:'a', 0xe1:'a', 0xe2:'a', 0xe3:'a', 0xe4:'a', 0xe5:'a',
        0xe6:'ae', 0xe7:'c',
        0xe8:'e', 0xe9:'e', 0xea:'e', 0xeb:'e',
        0xec:'i', 0xed:'i', 0xee:'i', 0xef:'i',
        0xf0:'th', 0xf1:'n',
        0xf2:'o', 0xf3:'o', 0xf4:'o', 0xf5:'o', 0xf6:'o', 0xf8:'o',
        0xf9:'u', 0xfa:'u', 0xfb:'u', 0xfc:'u',
        0xfd:'y', 0xfe:'th', 0xff:'y',
        0xa1:'!', 0xa2:'{cent}', 0xa3:'{pound}', 0xa4:'{currency}',
        0xa5:'{yen}', 0xa6:'|', 0xa7:'{section}', 0xa8:'{umlaut}',
        0xa9:'{C}', 0xaa:'{^a}', 0xab:'<<', 0xac:'{not}',
        0xad:'-', 0xae:'{R}', 0xaf:'_', 0xb0:'{degrees}',
        0xb1:'{+/-}', 0xb2:'{^2}', 0xb3:'{^3}', 0xb4:"'",
        0xb5:'{micro}', 0xb6:'{paragraph}', 0xb7:'*', 0xb8:'{cedilla}',
        0xb9:'{^1}', 0xba:'{^o}', 0xbb:'>>',
        0xbc:'{1/4}', 0xbd:'{1/2}', 0xbe:'{3/4}', 0xbf:'?',
        0xd7:'*', 0xf7:'/'
        }

    r = ''
    for i in unicrap:
        if xlate.has_key(ord(i)):
            r += xlate[ord(i)]
        elif ord(i) >= 0x80:
            pass
        else:
            r += str(i)
    return r

if __name__ == '__main__':
    s = unicode('','latin-1')
    for c in range(32,256):
        if c != 0x7f:
            s = s + unicode(chr(c),'latin-1')
    plain_ascii = latin1_to_ascii(s)
   
    print 'INPUT type:', type(s)
    print 'INPUT:'
    print s.encode('latin-1')
    print
    print 'OUTPUT type:', type(plain_ascii)
    print 'OUTPUT:'
    print plain_ascii

Thanks!

maxx

Posts : 1
Join date : 2012-12-06

View user profile

Back to top Go down

Re: For 0.9.1-beta, OS X 10.8 - copying CR tags to CBL using the CLI cause ComicBookLover to hang on import.

Post  ComicTagger on Sat Dec 08, 2012 2:07 pm

Maxx, no doubt this is an encoding problem, as you suggest.

Off the bat, I just want to make sure of the data flow.

Here's what I think you are saying:

1. start with the comic archive in question, with no metadata, and open it in ComicRack on Windows
2. Use the ComicVineScraper plugin to fill in the metadata and save it

(I assume these first two steps happened a while ago)

3. Copy the comic archive to Mac OSX
4. Use ComicTagger to convert the CR tags to CBL tags (via command line)
5. Open the comic in CBL, and it hangs.

Correct?

Right now, I just tested just grabbing that data (for "5 Days to Die #1") from ComicVine directly with ComicTagger, saving the tags to CBL format, and then opening in CBL 1.5 and it worked just fine. But maybe there is an issue with encoding coming from CVScraper....? Will test later once you confirm the data flow.

Actually, can you run this, and paste the output here??
Code:

ComicTagger -p --raw  "Five Days to Die #1.cbz"
(Change the filename as needed, of course).

This will dump out all raw tag blocks. Assuming the original CR tags are there, this will show me the input ComicInfo.xml (CR tags).

If this is a one-off problem, you can always do what I just did with something like this:

Code:

ComicTagger -s -t cbl -f -o  --verbose "Five Days to Die #1.cbz"

I am curious if you will have problems with the import after that?

avatar
ComicTagger
Admin

Posts : 208
Join date : 2012-12-02

View user profile http://comictagger.forumotion.com

Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum