2016년 3월 26일 토요일

Turning off OpenGL debug mode in Pyglet for Brain Workshop

I recently read an article on Infoworld by Serdar Yegulalp about performance optimizations for Python. The following tip caught my eye:

Pyglet, a handy library for creating windowed graphical applications, automatically enables a debug mode, which dramatically impacts performance until it’s explicitly disabled.
I checked the Pyglet docs for proof of this and found the following:

http://pyglet.readthedocs.org/en/latest/programming_guide/options.html

options = {'xsync': True, 'debug_gl_trace': False, 'debug_media': False, 'debug_gl': True, 'debug_font': False, 'font': ('gdiplus', 'win32'), 'audio': ('directsound', 'pulse', 'openal', 'silent'), 'debug_win32': False, 'xlib_fullscreen_override_redirect': False, 'debug_trace_depth': 1, 'debug_lib': False, 'debug_x11': False, 'debug_trace_args': False, 'debug_gl_trace_args': False, 'shadow_window': True, 'debug_texture': False, 'debug_trace_flush': True, 'darwin_cocoa': False, 'vsync': None, 'search_local_libs': True, 'debug_trace': False, 'debug_graphics_batch': False, 'graphics_vbo': True}

Apparently OpenGL debugging is turned on by default!

From the docs:
Global dict of pyglet options. To change an option from its default, you must import pyglet before any sub-packages. For example:

    import pyglet
    pyglet.options['debug_gl'] = False

debug_gl
    If True, all calls to OpenGL functions are checked afterwards for errors using glGetError. This will severely impact performance, but provides useful exceptions at the point of failure. By default, this option is enabled if __debug__ is (i.e., if Python was not run with the -O option). It is disabled by default when pyglet is “frozen” within a py2exe or py2app library archive.

I therefore added the highlighted snippet at line 814 in the Brainworkshop 4.8.7 source inside of a try-except block:

try:
    # workaround for pyglet.gl.ContextException error on certain video cards.
    os.environ["PYGLET_SHADOW_WINDOW"]="0"
    import pyglet

    # disable OpenGL debugging mode which can hurt performance
    pyglet.options['debug_gl'] = False

    from pyglet.gl import *
    if NOVBO: pyglet.options['graphics_vbo'] = False
    from pyglet.window import key
except:
    quit_with_error(_('Error: unable to load pyglet.  If you already installed pyglet, please ensure ctypes is installed.  Please visit %s') % WEB_PYGLET_DOWNLOAD)


I'm not sure how to rigorously measure the changes in execution time, however. Although Brain Workshop feels faster and is using less cpu according to htop, I would like to use something like the Python timeit module to measure a baseline for the original version and then time the edited code.

2016년 3월 19일 토요일

ASUS U36JC Notebook Intel H55 chipset does not support 8GB DDR3 modules

I have happily used a 13.3" ASUS U36JC notebook since 2011. When I bought it I made sure that 8GB DDR3 of total memory was installed (2 x 4GB DDR3). Back then, 8GB was a large amount of memory, but as the years have gone by, I have required more and more memory (Chrome has become a memory hog, and I use virtualization via KVM and virtualbox much more frequently).

Recently at work I have been put on an Openstack project so I want to be able to play with Openstack at home on my personal notebook as well. Although simple all-in-one(AIO) installations will work with just 8GB of RAM, standalone physical installations using the Ubuntu installer for Openstack Kilo require at least 12GB, while installation of the Kolla project (which containerizes all Openstack components within Docker) requires > 8GB to build and run ~80 containers.

I used dmidecode to query the system BIOS for max memory supported and info on the memory modules currently populating the memory banks.

[fedjun@u36jfed23 ~]$ sudo dmidecode -t 16
# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.6 present.

Handle 0x0041, DMI type 16, 15 bytes
Physical Memory Array
    Location: System Board Or Motherboard
    Use: System Memory
    Error Correction Type: None
    Maximum Capacity: 16 GB
    Error Information Handle: Not Provided
    Number Of Devices: 4


DMI type 16 shows the Physical Memory Array. According to the information above, the motherboard has 4 memory slots and can support a total of 16GB of RAM. This means that each memory bank can support up to 4GB RAM. Note that although the Intel H55 chipset can support up to 4 physical memory banks, there are only two physical memory slots offered on the ASUS U36JC. According to the following Superuser forum thread, this is not an unusual occurrence:

http://serverfault.com/questions/137491/dmidecode-showing-more-ram-slots-than-available

But since the mobo can theoretically support up to 16GB DDR3, I decided to make a gamble and purchased two Samsung Electronics 8GB DDR3L PC-12800 memory modules:


The picture above shows the 8GB DDR3L (low-power 1.35V) memory modules installed on my ASUS U36JC notebook. Here is the information on the memory modules provided by dmidecode reading from the BIOS:

[fedjun@u36jfed23 ~]$ sudo dmidecode -t 17
[sudo] password for fedjun:
# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.6 present.

Handle 0x0042, DMI type 17, 28 bytes
Memory Device
    Array Handle: 0x0041
    Error Information Handle: Not Provided
    Total Width: 64 bits
    Data Width: 64 bits
    Size: 8192 MB
    Form Factor: SODIMM
    Set: None
    Locator: DIMM0
    Bank Locator: BANK 0
    Type: DDR3
    Type Detail: Synchronous
    Speed: 1067 MHz
    Manufacturer: 80CE
    Serial Number: 312BBD50
    Asset Tag: Unknown
    Part Number: M471B1G73DB0-YK0 
    Rank: Unknown

Handle 0x0044, DMI type 17, 28 bytes
Memory Device
    Array Handle: 0x0041
    Error Information Handle: Not Provided
    Total Width: Unknown
    Data Width: Unknown
    Size: No Module Installed
    Form Factor: DIMM
    Set: None
    Locator: DIMM1
    Bank Locator: BANK 1
    Type: Unknown
    Type Detail: None
    Speed: Unknown
    Manufacturer: [Empty]
    Serial Number: [Empty]
    Asset Tag: Unknown
    Part Number: [Empty]
    Rank: Unknown

Handle 0x0045, DMI type 17, 28 bytes
Memory Device
    Array Handle: 0x0041
    Error Information Handle: Not Provided
    Total Width: 64 bits
    Data Width: 64 bits
    Size: 8192 MB
    Form Factor: SODIMM
    Set: None
    Locator: DIMM2
    Bank Locator: BANK 2
    Type: DDR3
    Type Detail: Synchronous
    Speed: 1067 MHz
    Manufacturer: 80CE
    Serial Number: 312BBD72
    Asset Tag: Unknown
    Part Number: M471B1G73DB0-YK0 
    Rank: Unknown

Handle 0x004A, DMI type 17, 28 bytes
Memory Device
    Array Handle: 0x0041
    Error Information Handle: Not Provided
    Total Width: Unknown
    Data Width: Unknown
    Size: No Module Installed
    Form Factor: DIMM
    Set: None
    Locator: DIMM3
    Bank Locator: BANK 3
    Type: Unknown
    Type Detail: None
    Speed: Unknown
    Manufacturer: [Empty]
    Serial Number: [Empty]
    Asset Tag: Unknown
    Part Number: [Empty]
    Rank: Unknown


DMI type 17 shows the Memory Device. Banks 0 and 2 are populated and Banks 1 and 3 are empty (and do not physically exist on the U36JC Intel H55 chipset motherboard).

BIOS detects the memory modules and Linux will boot; however, once a memory address above 4GB is accessed, the system will reboot or a kernel panic will result. I contacted ASUS and asked if a BIOS upgrade could fix this issue, but ASUS tech support responded that this is a physical limitation of the H55 chipset as implemented on the mobo for the ASUS U36JC.

It seems that the maximum amount of memory supported in each memory bank is 4GB!

2016년 3월 12일 토요일

Uncompressed audio playback using Python 2 pyglet

Python 2's pyglet module is quite popular for making games as it can render graphics files like png and also playback uncompressed audio. pyglet is not capable of playing compressed audio, however; for this task it has traditionally relied on the avbin plugin. Unfortunately, avbin is not under active development anymore and many programs using pyglet require ancient versions of avbin to be installed on your system.

Take, for instance, the dual N-back training game Brain Workshop (BW). Written in Python 2 and pyglet, it uses both uncompressed wav and compressed ogg audio files. For playing ogg files, BW requires avbin 7, which was released in 2008. This is a problem because most Linux distributions don't provide avbin in their default repositories (and even if they did, they would use the latest version avbin 10/11). In another post, I will describe some changes I made to Brain Workshop's python source to remove the dependency on avbin. The trick is to first convert all compressed audio to wav and then to use pyglet's built-in audio playback function.

When attempting to play wav files with pyglet, you must make sure that there are no audio tags encoded in the file. If any tags exist, pyglet will give a WAVEFormatException complaining that the format is invalid. Let's say you convert ogg to wav using ffmpeg:

$ ffmpeg -i Concert.ogg Concert.wav

A file converted in this way will be tagged with the string Lavf56.40.101 in the field Software:


The screenshot above is from Audacity, an open source digital audio workstation. If you select edit metadata tags from the File menu you will be able to see what information, if any, is encoded as metadata. Clear Lavf56.40.101 and re-export the file to wav (Microsoft PCM, 16 bit) and you should be able to play the file with pyglet:


import pyglet
source = pyglet.media.load('Concert.wav')
source.play()

2016년 3월 5일 토요일

Canon Pixma MG2490 as CUPS print server on Fedora 23

A machine with Fedora 23 installed will act as a print server for both Linux and Windows clients.

What is really nice is that using ipp, CUPS can now share printers over the network to Windows clients without messing with Samba configuration!

At first, I naively thought that simply installing cups and xsane would be suffcient:

sudo dnf install cups xsane

However these packages and their dependencies are not sufficient for getting the MG2490 to print or scan.

Initially I also tried installing the package cnijfilter-mg2400-series.rpm.tar.gz but again, this package alone is not sufficient for setting up the printer/scanner. I therefore removed this package and was able to get the printer/scanner working just fine with the pure open-source drivers.

Referring to /var/log/dnf.log Here are all the packages I installed on Fedora 23 server for the Canon Pixma MG2490:

cups
gutenprint
gutenprint-libs
gutenprint-foomatic
gutenprint-cups

foomatic
foomatic-db
system-config-printer
(optional, but recommended)
sane
sane-backends-drivers-scanners
sane-backends
xsane (optional, but recommended)

Once everything is installed, start the cups systemd service:

systemctl start cups

Make sure the MG2490 is connected by USB cable to your CUPS print server and then run system-config-printer as root. Select "Add" and choose printer and your MG2490 should appear in a list. Choose the "USB MG2490" entry. The gutenprint-foomatic package includes a postscript PPD file for the Canon MG2400 series, so your printer should appear in the list under Canon printers





Since you will be sharing this printer on the network, in the printer properties make sure it is shared for all users. One of the dialog boxes will also offer to set up rules for CUPS and IPP in firewalld (how convenient!).
If you don't use system-config-printer, however, you will have to add several rules to firewalld manually using the gui firewall-config or on the command line with firewall-cmd.

In your default zone, open up port 631/tcp (for CUPS remote administration http interface) and also allow ipp-client traffic.There might be other settings required, but these are the ones that occur off the top of my head (I just used system-config-printer because I'm lazy).

Now from the server machine, open a browser and navigate to http://localhost:631, login to Printer Administration with root:password
 for your server, and print a test page. You can also print a test page from system-config-printer.

Other Linux clients can find the CUPS server if you enter it's IP address in the system-config-printer "Network Printer -> Find Network Printer" dialog box.

To add the Canon MG2490 for Windows clients, click on Control Panel -> Add New Printer, and select Network Printer.

In the address bar, enter the IPP address of your printer listed in CUPS. You can find this address in the CUPS web admin page on port 631 or through system-config-printer. In my case, the IPP address is:

ipp://192.168.30.6:631/printers/Canon_MG2400_series
But in the "Add Network Printer" dialog box on Windows clients, change this to http:

http://192.168.30.6:631/printers/Canon_MG2400_series

Note that the Windows client must have windows drivers for the MG2490 installed. Windows users will be prompted to select the printer driver. They should select the driver named "MG 2400 series". In the case of Windows 10, however, it will automatically detect the printer and install all drivers for the printer and scanner automatically! Once the printer is set up, have the Windows user print a test page. In my experience the initial test page took > 1 minute to print, but subsequent print jobs from Windows clients were processed much more quickly (< 30 sec).

It is such a relief to be able to set up a Linux print server for Windows clients without messing with Samba shares. Thank you, Linux + IPP!