Living life in the tty

I have a an old Acer Aspire One D257 with 2GB of RAM which I purchased back in 2011 and have happily run a succession of Linux distro's like Debian, Ubuntu, and Archlinux on it. On purpose, I have only used lightweight Desktop Environments like LXDE and Openbox on this netbook to keep the system resources used to a minimum.

Unfortunately, In the past year or two, everyday browsing on the netbook has become too slow, with modern browsers easily taking up more than 1 GB of RAM with just a few tabs open (Chrome/Chromium is a memory hog). For example, if I have gmail, Trello, Slack, and a Google Drive document open in separate tabs, the Intel Atom processor has a hard time keeping up. This doesn't even take into account non-browser apps I might be running, like QuodLibet music player or daemons for Dropbox and SpiderOakONE. Due to sluggish performance, I decided to say get rid of the overhead for an X11 graphical desktop.

My Acer D257 has found a new lease on life as a tty-only machine which I ssh into from my primary notebook. I was partly inspired by reading K. Mandla's blog which chronicles his adventures testing command line apps on obsolete hardware. After connecting over ssh, I launch a GNU Screen session with the following script:

	#!/bin/bash
	#screen_netbook.sh
	#This script launches Screen with various tabs and starts various programs
	#such as htop, cmus, etc. as well as custom scripts in separate tabs.
	#It is intended to be used in tty-only sessions on netbooks that don't
	#have enough memory and CPU to handle X11 graphical sessions
	# Meaning of GNU Screen option flags:
	# -A adapt the sizes of all windows to the size of the current terminal
	# -d -m launch GNU Screen in detached mode; creates a new session but
	# doesn't attach to it
	# -S specify session name
	# -p preselect a window (by no. or name)
	# -X send cmd to running screen session
	# to list running Screen sessions, 'screen -ls'
	# to reconnect to a Screen session, 'screen -r sessionName'
	# launch a GNU Screen session named 'netbook' in detached mode
	screen -AdmS netbook -t SHELL
	# create separate tabs in session 'netbook' and launch programs in tabs
	screen -S netbook -X screen -t HTOP htop
	screen -S netbook -X screen -t CMUS cmus
	screen -S netbook -X screen -t IRSSI irssi
	screen -S netbook -X screen -t EMACS env TERM=xterm-256color emacs
	# On tab 0, launch SpiderOakONE and dropboxd daemons for cli
	screen -S netbook -p 0 -X stuff "cd $HOME/bin^M"
	screen -S netbook -p 0 -X stuff "./tty_startup.sh^M"

view raw screen_netbook.sh hosted with ❤ by GitHub

I then connect to the detached session with screen -r (you can see a list of detached sessions with screen -ls)

In the screenshot, you can see several GNU Screen tabs created by the script above. cmus is playing Internet radio, htop monitors CPU and memory for each process, and irssi with bitlbee daemon allows me to use irc to connect to Google Chat. In addition, I have the dropboxd and SpiderOakONE daemons running in headless mode so that my files on Dropbox and Spideroak get synced even when I'm not in a graphical session. Dropbox offers a script, dropbox.py for running dropboxd from the command line. This script has been packaged in AUR for Archlinux in the dropbox-cli package. In the case of SpiderOakONE, you can launch the daemon from the command line using the --headless option.

Here is a simple script that launches both daemons in the background when in a tty session:

	#!/bin/bash
	# This script launches various daemons for a non-graphical
	# session
	SpiderOakONE --headless &
	dropbox-cli start &

view raw tty_startup.sh hosted with ❤ by GitHub

In addition, I am running an NFS server for my LAN from this netbook -- all of this within 376 MB! In the screenshot above, the GNU Screen caption line shows the time, DoW, month, date, hostname, and a list of tabs that will expand to the right while properly reflowing even if the number of tabs goes off the right edge. You can find this caption setting in my .screenrc file from my dotfiles repo on Github.

In a series of posts I will talk more about cmus command line music player (that also supports Internet radio) and setting up irssi + bitlbee so you can connect to instant messaging services using an IRC client.

Using xml software lists in sdlmame 0.165

In May 2015, mamedev finally decided to merge MAME and MESS into one monolithic build. Now if you install sdlmame through your Linux distro's package manager, you can emulate arcade hardware (MAME) as well as hundreds of home computer systems and consoles (MESS) including the c64, apple2, amiga, original macintosh, etc.

One oddity is that in Archlinux, at least, the installation of sdlmame does not include hashes for using software lists, which contain lists of known software for each hardware platform in MESS (note that software lists for many platforms like apple2 are not yet complete). The path for software lists is $HOME/.mame/hash/ which contains xml files using the schema defined below:

http://www.mess.org/mess/swlist_format

After installing the sdlmame binary package, you can find the xml software lists from the mamedev/mame repo on github at the following link:

https://github.com/mamedev/mame/tree/master/hash

You can right-click to save the software list .xml file you are interested in, or if you want to download all lists, you can just download a .zip of the entire tree from the main project page:

https://github.com/mamedev/mame/archive/master.zip

After extracting the archive to your hard drive, navigate to the location of mame-master and copy all the contents of the hash subfolder to $HOME/.mame/hash

If you are using a GUI frontend to sdlmame (I recommend QMC2) don't forget to specify the path to the hash directory in the global configuration. Now if you select a certain HW platform, say apple2 you should see a list of all known software for that platform:

Let's say I want to play Archon for the apple2 -- if I have the necessary floppy disk image(s) in my ROM directory (this path must be configured in $HOME/.mame/mame.ini), I can launch it from the software list on the right of the QMC2 screen by highlighting the program and pressing ENTER or double-clicking it.

The problem I encountered, however, is that launching programs from the MESS software list requires that the name of the local disk image file exactly match the one specified in the xml file. If you have downloaded apple2 disk images from ROM sites, you will know that for any given program, there is rarely just one canonical file; instead there are often several different images available in differing formats like .do .dsk .nib etc. In addition, the CRC32 and SHA1 hash values for downloaded disk images are often different from the those specified in apple2.xml

When invoking sdlmame from the command line, it is able to recognize disk images stored inside .zip archives, but when trying to launch programs from the software list in QMC2, .zip archives are not recognized (you will get file not found errors). If the compressed disk images are extracted and renamed to exactly match the name of the rom/images specified in the xml file, you will be able to launch the program in QMC2 even if the CRC's and SHA1 hashes don't match, although you will get a warning in the Emulator Log tab.

In contrast, when launching some floppy images using sdlmame from the command line, images within .zip archives launch just fine:

[archjun@latitude630 apple2]$ sdlmame apple2e -flop1 /MULTIMEDIA/ROMs/apple2/"Empire - Wargame of The Century (1987)(Interstel)(Disk 1 of 2 Side A).zip"
Value 1 not supported for option sound - falling back to auto
Average speed: 98.29% (239 seconds)

It would be much more convenient to launch disk/cart/rom images through the QMC2 GUI, but unless I am using the exact same image specified in the software list xml file, I will get a file not found error.

We can easily add a new entry to the apple2.xml software list, however. Let's take a look at an existing entry for archon2:

	<software name="archon2">
	<description>Archon II: Adept</description>
	<year>1985</year>
	<publisher>Electronic Arts</publisher>
	<part name="flop1" interface="floppy_5_25">
	<dataarea name="flop" size="143360">
	<rom name="archon ii - adept (1985)(electronic arts)[cr racketeers].dsk" size="143360" crc="fcf7b2e8" sha1="9b6e91644b23e08138dd125a11ce51aef0e1138d" offset="0x0000" />
	</dataarea>
	</part>
	</software>

view raw apple2_example1.xml hosted with ❤ by GitHub

We can copy this template and change the fields software name, description, year, publisher, size (in bytes), rom name, crc, and sha1. As an example, I will create an entry for Snooper Troops Case of the Granite Point Ghost.

I originally used cksum from GNU coreutils to calculate the CRC32:

[archjun@latitude630 apple2]$ cksum "Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk" 
20123348 143360 Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk

The highlighted value is in decimal, but sdlmame expects the checksum to be in hex:

[archjun@latitude630 apple2]$ printf '%x\n' $(cksum "Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk" | awk '{ print $1 }')
1330ed4

I was mystified when QMC2 kept complaining that the expected crc in the xml file failed to match the calculated checksum. Reading this post from 2011 by Box.com engineers, I learned that cksum has a strange way of calculating crc32 compared to other implementations:

We discovered that Unix cksum differs from other CRC generators because it uses the polynomial table twice; it puts the file's bytes through the polynomial, but then also puts the length of the file through the polynomial as well, which is not mentioned anywhere in any documentation.

I tried another command line crc32 utility called crctk which generated a proper checksum:

[archjun@latitude630 apple2]$ crctk -c "Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk"
[Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk] 1EE9FEC2 (note this crc32 differs from 20123348 returned by cksum from coreutils)

I want the hex crc32 to be in lowercase, however, so I first pipe it to awk to print the last column and then pipe to tr from GNU coreutils to convert from upper to lowercase:

[archjun@latitude630 apple2]$ crctk -c "Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk" | awk '{ print $NF }' | tr [:upper:] [:lower:]
1ee9fec2

Now I will add the following entry to apple2.xml using a text editor:

	<software name="snptrpgp">
	<description>Snooper Troops Case of the Granite Point Ghost</description>
	<year>1982</year>
	<publisher>Spinnaker Software</publisher>
	<part name="flop1" interface="floppy_5_25">
	<dataarea name="flop" size="143360">
	<rom name="Snooper Troops and the Case of- The Granit Point Ghost (1982)(Spinmaker Software Corp).dsk" size="143360" crc="1ee9fec2" sha1="fedc1df0bbf84b1c9569ba497b320535f4b2752c" offset="0x0000" />
	</dataarea>
	</part>
	</software>

view raw apple2_example2.xml hosted with ❤ by GitHub

From the QMC2 menu, Tools -> Clean Up -> Clear Software List Cache, and then click Reload all information from the software list buttons on the right to apply changes to the apple2.xml file.

Note that the software name must be at most 8 letters as well as unique within the xml file. Now launching the new disk image snptrpgp from the software list works. Note that the rom name field must be populated by a non-archived disk/cartridge image, meaning you cannot use a .zip file here. This is odd, because when launching MESS-supported systems from sdlmame on the command line, it is possible to specify a disk image that is located inside a zip archive.

There are perhaps hundreds of software titles for the apple2 and c64 which are absent from the MESS software lists, so I inquired in the #messdev IRC channel on EFNET about the possibility of adding titles to the apple2.xml software list and submitting a pull request to the mamedev/mame github repo. I was told that mame developer R. Belmont is responsible for adding titles to the list for apple2, and that he would populate the xml file soon.

Until then, it is probably better for me to launch random MESS roms and disk images from the command line instead of tediously creating software entries in the xml software lists.

Truncating a file when the noclobber option is set

On a recent service call, a client asked me if there was a way to get rid of an enormous log file (> 10 gigabytes) without using rm which would cause high I/O load on a production system. I told the client to use the redirect operator to truncate the file to an empty file.

In normal cases, simply doing

> fileName

which is equivalent to

cat /dev/null > fileName

will do the trick.

When the client tried this, however, he got the following error message:

File exists

A quick google search revealed that this error is returned when the noclobber option is set in .bashrc or the configuration file for your shell of choice (ksh, csh, etc).

To override noclobber just once, you would use the >| operator in bash:

>| fileName

The server in question happened to be using C-shell, however, so the noclobber redirect override should be >!

>! fileName


References:

http://www.linuxhowtos.org/Tips%20and%20Tricks/Protecting%20files%20with%20noclobber.htm

http://stackoverflow.com/questions/2423281/how-to-empty-truncate-a-file-on-linux-that-already-exists-and-is-protected-i