Converting XML plists to the old NeXTSTEP (ASCII) format

General discussion for players of Oolite.

Moderators: winston, another_commander

User avatar
Diziet Sma
---- E L I T E ----
---- E L I T E ----
Posts: 6310
Joined: Mon Apr 06, 2009 12:20 pm
Location: Aboard the Pitviper S.E. "Blackwidow"

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by Diziet Sma » Sun Jan 27, 2013 7:25 pm

JazHaz wrote:EDIT: also tried dragging and dropping the .py file onto python.exe, tried executing the .py file too.
According to Svengali, you should be able to drag and drop the file you wish to convert onto the xml2ns.py script, and it will do the conversion, even saving the new file into the same folder as the original xml plist.
Most games have some sort of paddling-pool-and-water-wings beginning to ease you in: Oolite takes the rather more Darwinian approach of heaving you straight into the ocean, often with a brick or two in your pockets for luck. ~ Disembodied

User avatar
Svengali
---- E L I T E ----
---- E L I T E ----
Posts: 2365
Joined: Sat Oct 20, 2007 2:52 pm

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by Svengali » Sun Jan 27, 2013 7:51 pm

Diziet Sma wrote:According to Svengali, you should be able to drag and drop the file you wish to convert onto the xml2ns.py script, and it will do the conversion, even saving the new file into the same folder as the original xml plist.
Yep. The new file gets "-ns" as suffix, so it won't override the original one. The dropping will only work if the Pathes are set (usually Python does it on it's own when the installer is used).

User avatar
Svengali
---- E L I T E ----
---- E L I T E ----
Posts: 2365
Joined: Sat Oct 20, 2007 2:52 pm

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by Svengali » Sun Jan 27, 2013 8:20 pm

GGShinobi wrote:
Diziet Sma wrote:I'd like to formally nominate Svengali's version of the script for inclusion as the new official version. How say ye?
But what I like is that Svengali uses spaces instead of tabs! :mrgreen: so I think I equally like both versions! :mrgreen:
Usually I wouldn't have posted the script, because I think that we don't need multiple versions. Commander Jettisons script works well and the tweaked version only changes a very minor thing and is just my personal preference. And you have already made a 'Linux' version for it .-)

btw: The script uses tabs, like the orginal script. All I've changed is the handling of quotes to get rid of them for keynames in the sublevels (e.g. role). Parent keynames (like entity identifiers) always get quotes. The exceptions are keynames which are containing a minus (e.g. escort-ship) or keynames with a dot (e.g. filenames). Together with the colorscheme it gives a good overview about outdated keynames, mistakes, or simply a warning to check things.

Image
PS: Nonsensical comment -> should be escort_role .-)

User avatar
GGShinobi
---- E L I T E ----
---- E L I T E ----
Posts: 291
Joined: Tue Dec 25, 2012 7:20 pm

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by GGShinobi » Mon Jan 28, 2013 6:33 am

Diziet Sma wrote:
GGShinobi wrote:I had a little problem with Svengali's version:

Code: Select all

bash: ./xml2ns-svengali.py: /usr/bin/python^M: bad interpreter: No such file or directory
So I had to resolve it by running

Code: Select all

dos2unix xml2ns-svengali.py
Weird.. how did you create your copy? I copy/pasted into Geany, saved it, and it ran fine.. (but now I think about it, I've set Geany to auto-strip CR characters from line endings.. :mrgreen:)
I pressed "quote", copy-pasted the whole text into a console with vim running (in paste-mode (:set paste)) and got rid of the "quote"-tags and so...
Svengali wrote:btw: The script uses tabs, like the orginal script.
Really? Then it seems that the tabs are being converted by the bulletin board to spaces when one presses "quote". I just double checked it, I get spaces.
Svengali wrote:All I've changed is the handling of quotes to get rid of them for keynames in the sublevels (e.g. role). Parent keynames (like entity identifiers) always get quotes. The exceptions are keynames which are containing a minus (e.g. escort-ship) or keynames with a dot (e.g. filenames). Together with the colorscheme it gives a good overview about outdated keynames, mistakes, or simply a warning to check things.
Hmm, sounds and looks reasonable. But I haven't decided yet which version I prefer (I have used them only for one test yet), so I won't vote on this. At the moment, I like both equally :)

EDIT: I've now used both versions and it seems that vim likes the Svengali-version more: the syntax is properly highlighted there. Unfortunately, not so with the other version. :( So I vote for the Svengali-Version! :mrgreen:
忍 knowing that enough is enough, you'll always have enough.

Running Oolite 1.77 on Ubuntu Linux 12.04 LTS

User avatar
Diziet Sma
---- E L I T E ----
---- E L I T E ----
Posts: 6310
Joined: Mon Apr 06, 2009 12:20 pm
Location: Aboard the Pitviper S.E. "Blackwidow"

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by Diziet Sma » Sat Jan 18, 2014 4:18 am

In the course of using this to convert the Q-Bomb Detector equipment.plist in readiness for 1.79, I found that since I have Python 2.7 and Python 3.3 installed, (not uncommon in many Linux machines these days) I was getting error messages when trying to run the script.

Some quick google-fu turned up two solutions:
Python 2 will happily run alongside Python 3. You need to specify python2 in order to run this version.

Any program requiring Python 2 needs to point to /usr/bin/python2, instead of /usr/bin/python, which points to Python 3.

To do so, open the program or script in a text editor and change the first line.

The line will show one of the following:

Code: Select all

#!/usr/bin/env python
or

Code: Select all

#!/usr/bin/python
In both cases, just change python to python2 and the program will then use Python 2 instead of Python 3.

Another way to force the use of python2 without altering the scripts is to call it explicitely with python2, i.e.

Code: Select all

python2 myScript.py
I've now edited the Linux version of the script in the Wiki page using the first method, to automatically force use of Python 2 in multi-Python systems.
Most games have some sort of paddling-pool-and-water-wings beginning to ease you in: Oolite takes the rather more Darwinian approach of heaving you straight into the ocean, often with a brick or two in your pockets for luck. ~ Disembodied

User avatar
cim
Quite Grand Sub-Admiral
Quite Grand Sub-Admiral
Posts: 4018
Joined: Fri Nov 11, 2011 6:19 pm

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by cim » Sat Jan 18, 2014 10:04 am

Diziet Sma wrote:I've now edited the Linux version of the script in the Wiki page using the first method, to automatically force use of Python 2 in multi-Python systems.
That is going to vary between distributions, I think. My system (Debian stable) has available "python", "python2.6" and "python2.7", with "python" being a symlink to "python2.7", which is not quite the same arrangement as ArchLinux.

I'm not sure if there is a solution that works everywhere, though, and at least that change makes it clear it's a python2 script.

User avatar
Diziet Sma
---- E L I T E ----
---- E L I T E ----
Posts: 6310
Joined: Mon Apr 06, 2009 12:20 pm
Location: Aboard the Pitviper S.E. "Blackwidow"

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by Diziet Sma » Sat Jan 18, 2014 12:28 pm

Well mostly, that's just where I first found the info, and I've learned that Arch links tend to have the best-informed and clearest explanations & solutions to many Linux problems..

In fact, I haven't fired up my Arch partition for a couple of weeks.. I have those two versions installed in my Mint partition.. 2.7 is for Oolite-related things, and 3.3 is because I'm teaching myself Python in preparation for making some iPhone/Android apps.
Most games have some sort of paddling-pool-and-water-wings beginning to ease you in: Oolite takes the rather more Darwinian approach of heaving you straight into the ocean, often with a brick or two in your pockets for luck. ~ Disembodied

User avatar
cim
Quite Grand Sub-Admiral
Quite Grand Sub-Admiral
Posts: 4018
Joined: Fri Nov 11, 2011 6:19 pm

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by cim » Sat Jan 18, 2014 4:03 pm

Diziet Sma wrote:In the course of using this to convert the Q-Bomb Detector equipment.plist in readiness for 1.79
Oh ... hang on. I may not have been clear enough here. Oolite 1.79 will still be able to read XML format plists. The only case where Openstep is required is when sending me a manifest.plist to add to the index, and that's only because the index needs to be in one format or the other, and I haven't got automatic conversion between the two set up yet.

User avatar
Diziet Sma
---- E L I T E ----
---- E L I T E ----
Posts: 6310
Joined: Mon Apr 06, 2009 12:20 pm
Location: Aboard the Pitviper S.E. "Blackwidow"

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by Diziet Sma » Sun Jan 19, 2014 2:37 am

cim wrote:
Diziet Sma wrote:In the course of using this to convert the Q-Bomb Detector equipment.plist in readiness for 1.79
Oh ... hang on. I may not have been clear enough here. Oolite 1.79 will still be able to read XML format plists
Ahh.. I wasn't quite clear enough either.. it had been on my mind for a while to make the change anyway.. (future-proofing, and all that.. not to mention being easier to read), and making an OXZ version was enough motivation to kill two birds with one stone..
Most games have some sort of paddling-pool-and-water-wings beginning to ease you in: Oolite takes the rather more Darwinian approach of heaving you straight into the ocean, often with a brick or two in your pockets for luck. ~ Disembodied

Cdr. Jettison
Average
Average
Posts: 12
Joined: Wed Jun 01, 2011 2:26 pm

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by Cdr. Jettison » Thu Jun 15, 2017 11:36 am

Suddenly realised that this old script still gets some use. So, after a long hiatus, I updated it to work with Python 3. Actually, since the script is so basic, the very first version from the top post probably could be run by Python 3 unchanged.

While being at this, I also incorporated the change by Svengali (many thanks!) and made some other cosmetic changes. Python 2 interpreters should still be able to run the script correctly as long as the input data is in ASCII. Actually, I had the code to make it fully compatible with Python 2, but at the last minute decided against applying it to keep things simple: it would require the Python 2 users to install an additional module.

Here is the new version:

Code: Select all

#!/usr/bin/python
'''Convert plist from XML to OpenStep format'''
# Version: 2
# Changes:
#   * convert to Python 3 retaining some Python 2 compatibility
#   * quote strings less (thanks to Svengali)
#   * optimize handling of lists: make the 1st item special, not the last one
#   * more comments
#   * assorted cosmetic changes

import sys

# this script requires at least Python 2.6
assert sys.version_info >= (2, 6)

import re
from plistlib import readPlist  # using deprecated method to retain
                                # compatibility with Python 2

# Change these as needed.
# plist file name can also be passed as parameter on command line
FILENAME = 'shipyard.plist' # name of the plist file to be converted
NEWSUFFIX = '-ns'           # added to FILENAME to produce output file name
NONWORD = re.compile(r'\W') # if keys contain this pattern, they'll be
			    # put in quotes (the keys on the outmost
			    # level are always quoted)

def write_dict(fs, dct, lvl):
    '''Writes dictionary into plist file fs. Recursion as needed'''
    fs.write('{\n')
    for k, v in sorted(dct.items()):
        q = '"' if lvl == 0 or NONWORD.search(k) else ''
        fs.write((lvl+1)*'\t'+'{quote}{key}{quote} = '.format(
            quote=q, key=k))
        if isinstance(v, list):
            write_list(fs, v, lvl+1)
        elif isinstance(v, dict):
            write_dict(fs, v, lvl+1)
        else:
            q = '"' if isinstance(v, str) else ''
            fs.write('{quote}{val}{quote}'.format(
                quote=q,
                val=v if not isinstance(v, bool) else 'yes' if v else 'no'))
        fs.write(';\n')
    fs.write(lvl*'\t'+'}')

def write_list(fs, lst, lvl):
    '''Writes list as array into plist file fs. Recursion as needed'''
    fs.write('(')
    first = True
    for i in lst:
        if first:
            first = False
        else:
            fs.write(',')
        fs.write('\n' + (lvl+1)*'\t')
        if isinstance(i, list):
            write_list(fs, i, lvl+1)
        elif isinstance(i, dict):
            write_dict(fs, i, lvl+1)
        else:
            q = '"' if isinstance(i, str) else ''
            fs.write('{quote}{item}{quote}'.format(
                quote=q, item=i))
    fs.write('\n' + lvl*'\t' + ')')

def main(args):
    '''Main conversion routine'''
    # Either get the plist file name from command line or use the default
    filename = args[0] if args else FILENAME

    # Intentionally using deprecated API to retain compatibility with Python 2
    pl = readPlist(filename)
    level = 0

    with open(filename + NEWSUFFIX, 'w') as f:
        if isinstance(pl, list):
            write_list(f, pl, level)
        elif isinstance(pl, dict):
            write_dict(f, pl, level)
        else:
            f.write('// Parsing error!')
        f.write('\n')

if __name__ == '__main__':
    sys.exit(main(sys.argv[1:]))
Should I replace the old one on box.com? I'll also strip the CR characters before uploading this time, promise.:roll:

cag
Dangerous
Dangerous
Posts: 84
Joined: Fri Mar 17, 2017 1:49 am

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by cag » Fri Oct 20, 2017 1:18 am

Should I replace the old one on box.com?
Maybe just add a link for Python3 version, so Python2 users don't scream and Python3 users can get up & running faster! :cry:
"Better to be thought a fool, boy, than to open your trap and remove all doubt." - Grandma [over time, just "Shut your trap... fool"]
"The only stupid questions are the ones you fail to ask." - Dad
How do I...? Nevermind.

cag
Dangerous
Dangerous
Posts: 84
Joined: Fri Mar 17, 2017 1:49 am

Re: Converting XML plists to the old NeXTSTEP (ASCII) format

Post by cag » Fri Oct 27, 2017 1:57 am

Have the .plist format rules changed Jan/2014?

I've added this script to one I'm writing (to generate effectdata.plist files) and get errors due to missing semi-colons in the output. I've gotten

Code: Select all

20:31:25.268 [plist.parse.failed]: Failed to parse Resources/Config/shipdata.plist as a property list.
Parse failed at line 58 (char 2205) - unexpected character (wanted ';' or '}')
when lacking one at the end of a ship definition, and even

Code: Select all

21:31:38.971 [startup.exception]: ***** Unhandled exception during startup: NSInvalidArgumentException (Tried to add nil value for key 'food' to dictionary).
when missing ';' at the end of lists or dictionaries!

The wiki documentation is inconsistent about this, so, I just add a ';' after every closing brace/bracket:

Code: Select all

...

def write_dict(fs, dct, lvl):
...
    fs.write(lvl*'\t'+'};')    
    # added semi-colon  ^  after closing brace

def write_list(fs, lst, lvl):
...
    fs.write('\n' + lvl*'\t' + ');')    
    # added semi-colon           ^  after closing bracket

...
and the parser is much happier.
"Better to be thought a fool, boy, than to open your trap and remove all doubt." - Grandma [over time, just "Shut your trap... fool"]
"The only stupid questions are the ones you fail to ask." - Dad
How do I...? Nevermind.

Post Reply