2014년 12월 5일 금요일

Fixing an issue with RAM statistics in archey 0.1-11 (archbey) Archbang banner script

The banner script used in Archlinux is called archey:

[archjun@arch ~]$ sudo pacman -Ss archey
[sudo] password for archjun: 
community/archey3 0.5-3
    Output a logo and various system information

However Archbang has taken this script and renamed it archbey, adding a 'b' for 'bang', I think. archbey is a python2 script located in /usr/bin that creates the following banner every time a terminal window is opened:

               .                
               #.               OS: Archbang x86_64
              /;#               Hostname: arch
              #;##              Kernel: 3.17.4-1-ARCH
             /###'              Uptime: 0:32
            ;#\   #;            Window Manager: Openbox
           +###  .##            Packages: 1269
          +####  ;###           RAM: -2523 MB / 3947 MB
         ######  #####;         CPU: Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz
        #######  ######         Shell: Bash
       ######## ########        Root FS: 9.2G / 24G (ext4)
     .########;;########\        
    .########;   ;#######       
    #########.   .########`     
   ######'           '######    
  ;####                 ####;   
  ##'                     '##   
 #'                         `#  


As you can see, however, the 'RAM' value listed above is incorrect. Let's take a look at the output of free -m (memory stats in MB):

[archjun@arch ~]$ free -m
              total        used        free      shared  buff/cache   available
Mem:           3947        1081        1721         155        1145        2467
Swap:           999           0         999

We are using a 1081 MB of RAM, but the archbey script is giving is a negative value which is definitely wrong.

Let's take a look at the python2 script /usr/bin/archbey to see how it is calculating RAM usage:

#!/usr/bin/env python2
#
# archey [version 0.1-11]
...
# Modified for ArchBang -sHyLoCk
...
import os, sys, subprocess, optparse, re
from subprocess import Popen, PIPE
...
# RAM Function
def ram_display():
 raminfo = Popen(['free', '-m'], stdout=PIPE).communicate()[0].split('\n')
 ram = ''.join(filter(re.compile('M').search, raminfo)).split()
 used = int(ram[2]) - int(ram[5]) - int(ram[6])
 output ('RAM', '%s MB / %s MB' % (used, ram[1]))

We can see that the output of free -m is being stored in the variable raminfo, which is then parsed with a regex that searches for a line starting with 'M'.

This line of text is then turned into a list of strings using the built-in function split(), which splits a line into words at every whitespace character and inserts them into a list.

the variable used is then calculated by accessing specific indexes from list ram (starting at index 0). 

Let's look at the relevant code snippets in action:

[archjun@arch ~]$ python2
Python 2.7.8 (default, Sep 24 2014, 18:26:21) 
[GCC 4.9.1 20140903 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.Popen(['free', '-m'], stdout=subprocess.PIPE).communicate()[0].split('\n')
['              total        used        free      shared  buff/cache   available', 'Mem:           3947        1036        1772         149        1139        2518', 'Swap:           999           0         999', '']
>>> raminfo = subprocess.Popen(['free', '-m'], stdout=subprocess.PIPE).communicate()[0].split('\n')
>>> import re
>>> ram = ''.join(filter(re.compile('M').search, raminfo)).split()
>>> print ram
['Mem:', '3947', '1050', '1757', '149', '1139', '2503']

We can see from the output above that

ram[0] is the string 'Mem'
ram[1] is 3947 (total)
ram[2] is 1050 (used)
ram[3] is 1757 (free)
ram[4] is 149 (shared)
ram[5] is 1139 (buff/cache)
ram[6] is 2503 (available)

It is apparent that the formula
used = int(ram[2]) - int(ram[5]) - int(ram[6])
from /usr/bin/archbey is incorrect.

If we want to see how much memory is being used, we should simply look at ram[2] alone, which would give us 1050.

free comes from the package procps-ng:

[archjun@arch ~]$ sudo pacman -Qo free
[sudo] password for archjun: 
/usr/bin/free is owned by procps-ng 3.3.10-1

(FYI, pacman -Qo fileName is equivalent to rpm -qf fileName in RHEL/CentOS)

Was procps-ng updated recently?

[archjun@arch ~]$ sudo cat /var/log/pacman.log |grep procps-ng
[2013-03-30 02:57] upgraded procps-ng (3.3.5-1 -> 3.3.7-1)
[2013-05-21 15:40] [PACMAN] upgraded procps-ng (3.3.7-1 -> 3.3.7-2)
[2013-06-05 12:57] [PACMAN] upgraded procps-ng (3.3.7-2 -> 3.3.8-1)
[2013-06-27 13:50] [PACMAN] upgraded procps-ng (3.3.8-1 -> 3.3.8-2)
[2013-09-21 13:35] [PACMAN] upgraded procps-ng (3.3.8-2 -> 3.3.8-3)
[2013-12-10 10:38] [PACMAN] upgraded procps-ng (3.3.8-3 -> 3.3.9-1)
[2014-01-29 08:45] [PACMAN] upgraded procps-ng (3.3.9-1 -> 3.3.9-2)
[2014-05-03 11:49] [PACMAN] upgraded procps-ng (3.3.9-2 -> 3.3.9-3)
[2014-11-12 23:36] [PACMAN] upgraded procps-ng (3.3.9-3 -> 3.3.10-1)

Apparently it was upgraded on Nov. 12th, about 3 weeks ago.

I wasn't able to find any mention from the procps-ng project page that fields in free were added or changed. Taking a look at the man page for free, it says the following about the field used:

Used memory (calculated as total - free - buffers - cache)

I think the archbey script was trying to manually replicate the calculation above. In that case,

used = int(ram[2]) - int(ram[5]) - int(ram[6])

should be changed to

used = int(ram[1]) - int(ram[3]) - int(ram[5])

But this is a waste of CPU time because ram[2] already contains the value of used from free -m!

We can see that
int(ram[2])
is almost identical to
int(ram[1]) - int(ram[3]) - int(ram[5])

>>> int(ram[1]) - int(ram[3]) - int(ram[5])
1051
>>> int(ram[2])
1050

I am not sure why the values aren't identical, however (rounding error from buff/cache?)

Conclusion

I have fixed the archbey banner by editing lines 112 and 113 in /usr/bin/archbey as follows:

used = int(ram[2])
output ('RAM Used', '%s MB / %s MB' % (used, ram[1]))

Now the terminal banner properly lists RAM usage (of course, archbey or archey3 must be included in your ~/.bashrc file):

               .                
               #.               OS: Archbang x86_64
              /;#               Hostname: arch
              #;##              Kernel: 3.17.4-1-ARCH
             /###'              Uptime: 0:37
            ;#\   #;            Window Manager: Openbox
           +###  .##            Packages: 1269
          +####  ;###           RAM Used: 1114 MB / 3947 MB
         ######  #####;         CPU: Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz
        #######  ######         Shell: Bash
       ######## ########        Root FS: 9.2G / 24G (ext4)
     .########;;########\        
    .########;   ;#######       
    #########.   .########`     
   ######'           '######    
  ;####                 ####;   
  ##'                     '##   
 #'                         `#  

This script could use some loving; for one thing, all the indentation is non-kosher by Python standards -- default indentation should be 4 spaces, but this script uses just a single space throughout. By contrast, archey on github is properly indented with  4 spaces as default for code inside functions and classes.

The archbey script was based off of archey 0.1-11 but today archey(3) is at version 0.5-3. Unfortunately, no package owns archbey:

[archjun@arch ~]$ sudo pacman -Qo archbey
[sudo] password for archjun: 
error: No package owns /usr/bin/archbey

It is a script that was added on top of the base Archlinux install by the Archbang installer, so it is not updated by pacman -Syyu like archey would be.

I could always just get rid of archbey and install archey3 instead for creating a pretty Archlinux terminal banner.