Using xml software lists in sdlmame 0.165

In May 2015, mamedev finally decided to merge MAME and MESS into one monolithic build. Now if you install sdlmame through your Linux distro's package manager, you can emulate arcade hardware (MAME) as well as hundreds of home computer systems and consoles (MESS) including the c64, apple2, amiga, original macintosh, etc.

One oddity is that in Archlinux, at least, the installation of sdlmame does not include hashes for using software lists, which contain lists of known software for each hardware platform in MESS (note that software lists for many platforms like apple2 are not yet complete). The path for software lists is $HOME/.mame/hash/ which contains xml files using the schema defined below:

http://www.mess.org/mess/swlist_format

After installing the sdlmame binary package, you can find the xml software lists from the mamedev/mame repo on github at the following link:

https://github.com/mamedev/mame/tree/master/hash

You can right-click to save the software list .xml file you are interested in, or if you want to download all lists, you can just download a .zip of the entire tree from the main project page:

https://github.com/mamedev/mame/archive/master.zip

After extracting the archive to your hard drive, navigate to the location of mame-master and copy all the contents of the hash subfolder to $HOME/.mame/hash

If you are using a GUI frontend to sdlmame (I recommend QMC2) don't forget to specify the path to the hash directory in the global configuration. Now if you select a certain HW platform, say apple2 you should see a list of all known software for that platform:

Let's say I want to play Archon for the apple2 -- if I have the necessary floppy disk image(s) in my ROM directory (this path must be configured in $HOME/.mame/mame.ini), I can launch it from the software list on the right of the QMC2 screen by highlighting the program and pressing ENTER or double-clicking it.

The problem I encountered, however, is that launching programs from the MESS software list requires that the name of the local disk image file exactly match the one specified in the xml file. If you have downloaded apple2 disk images from ROM sites, you will know that for any given program, there is rarely just one canonical file; instead there are often several different images available in differing formats like .do .dsk .nib etc. In addition, the CRC32 and SHA1 hash values for downloaded disk images are often different from the those specified in apple2.xml

When invoking sdlmame from the command line, it is able to recognize disk images stored inside .zip archives, but when trying to launch programs from the software list in QMC2, .zip archives are not recognized (you will get file not found errors). If the compressed disk images are extracted and renamed to exactly match the name of the rom/images specified in the xml file, you will be able to launch the program in QMC2 even if the CRC's and SHA1 hashes don't match, although you will get a warning in the Emulator Log tab.

In contrast, when launching some floppy images using sdlmame from the command line, images within .zip archives launch just fine:

[archjun@latitude630 apple2]$ sdlmame apple2e -flop1 /MULTIMEDIA/ROMs/apple2/"Empire - Wargame of The Century (1987)(Interstel)(Disk 1 of 2 Side A).zip"
Value 1 not supported for option sound - falling back to auto
Average speed: 98.29% (239 seconds)

It would be much more convenient to launch disk/cart/rom images through the QMC2 GUI, but unless I am using the exact same image specified in the software list xml file, I will get a file not found error.

We can easily add a new entry to the apple2.xml software list, however. Let's take a look at an existing entry for archon2:

We can copy this template and change the fields software name, description, year, publisher, size (in bytes), rom name, crc, and sha1. As an example, I will create an entry for Snooper Troops Case of the Granite Point Ghost.

I originally used cksum from GNU coreutils to calculate the CRC32:

[archjun@latitude630 apple2]$ cksum "Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk" 
20123348 143360 Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk

The highlighted value is in decimal, but sdlmame expects the checksum to be in hex:

[archjun@latitude630 apple2]$ printf '%x\n' $(cksum "Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk" | awk '{ print $1 }')
1330ed4

I was mystified when QMC2 kept complaining that the expected crc in the xml file failed to match the calculated checksum. Reading this post from 2011 by Box.com engineers, I learned that cksum has a strange way of calculating crc32 compared to other implementations:

We discovered that Unix cksum differs from other CRC generators because it uses the polynomial table twice; it puts the file's bytes through the polynomial, but then also puts the length of the file through the polynomial as well, which is not mentioned anywhere in any documentation.

I tried another command line crc32 utility called crctk which generated a proper checksum:

[archjun@latitude630 apple2]$ crctk -c "Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk"
[Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk] 1EE9FEC2 (note this crc32 differs from 20123348 returned by cksum from coreutils)

I want the hex crc32 to be in lowercase, however, so I first pipe it to awk to print the last column and then pipe to tr from GNU coreutils to convert from upper to lowercase:

[archjun@latitude630 apple2]$ crctk -c "Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk" | awk '{ print $NF }' | tr [:upper:] [:lower:]
1ee9fec2

Now I will add the following entry to apple2.xml using a text editor:

From the QMC2 menu, Tools -> Clean Up -> Clear Software List Cache, and then click Reload all information from the software list buttons on the right to apply changes to the apple2.xml file.

Note that the software name must be at most 8 letters as well as unique within the xml file. Now launching the new disk image snptrpgp from the software list works. Note that the rom name field must be populated by a non-archived disk/cartridge image, meaning you cannot use a .zip file here. This is odd, because when launching MESS-supported systems from sdlmame on the command line, it is possible to specify a disk image that is located inside a zip archive.

There are perhaps hundreds of software titles for the apple2 and c64 which are absent from the MESS software lists, so I inquired in the #messdev IRC channel on EFNET about the possibility of adding titles to the apple2.xml software list and submitting a pull request to the mamedev/mame github repo. I was told that mame developer R. Belmont is responsible for adding titles to the list for apple2, and that he would populate the xml file soon.

Until then, it is probably better for me to launch random MESS roms and disk images from the command line instead of tediously creating software entries in the xml software lists.