Patch to sphinx-apidoc to autogenerate inheritance graphs in documentation

Some days ago I started to look into the Python code of a free software project.

The initial idea was to generate the documentation by using some automatic tool. I tested three tools: doxygen, epydoc, and sphinx.

Doxygen was the one I knew the best, as I used it in my previous C/C++ projects (and it supports Python), epydoc and sphinx are targeted to Python.

Every tools has pros and cons:

• Doxygen: very automatic, good inheritance graphs (generated by graphviz), not very eye-pleasant HTML
• Epydoc: very automatic and complete generation of API documentation, no (evident to me) support to graphs, and even less good looking HTML
• Sphinx: very good looking HTML, no direct and automatic generation of the API documentation and graphs

The best combination could be: the graphs done by doxygen, and the beauty of the sphinx generated HTML.

Sphinx has some useful tools, like sphinx-apidoc, that doesn’t have the direct opportunity to ask the system to generate the graphs (you have to manually modify conf.py and the various *.rst files).

So I did a little patch that adds the -G (--graphs) option. If you don’t want the graphs, just use sphinx-apidoc as you always did, if you want them, add -G to the commandline.

You can find the bitbucket page here and the message in the sphinx-dev Google group here.

debbugs-monkey, a greasemonkey script to navigate in Debian Bug reports

Some days ago I did my very first GreaseMonkey script, to ease the navigation in the Debian BTS bug reports.
I noticed that bug reports can have many messages, and the messages themselves sometimes tend to be very long, because of verbose logs posted to help the maintainers to understand the problems and fix them.

GreaseMonkey is a Firefox extension that allows to load scripts (written in Javascript) that can “do useful things” when the user navigate in the website the script refers to.

This useful feature is supported also by Chrome (Chromium, Iron, etc..) and other browsers.

My script (called Debbugs-monkey) is tested on Firefox. Here are the two links: the git repository and the install link.

The usage is very simple. After installing (and eventually activating) the script, when you go on a bug report page in the DBTS (the address is something like http://bugs.debian.org/cgi-bin/bugreport.cgi?blahblahblah) in the right bottom corner of the page (regardless of the scrolling) there are some letters:

• T: goes to the Top of the page
• F: goes to the First message
• P: goes to the Previous message
• N: goes to the Next message
• L: goes to the Last message
• B: goes to the Bottom of the page

I know, at the moment it’s very ugly, maybe I’ll add some fancyness (like a border, or icons instead of letters), but the important thing is that it works (at least for me, but if you have some problems using it, or feature requests, please tell me).

Managing your music library with beets

I found some time ago (thanks to a dent on identi.ca that I don’t find at the moment, but if I’ll find it, I’ll update this post with the direct link) this little piece of Python called beets.

What is this guy? It’s simply (but not only) a music collection organizer for obsessive-compulsive music geeks that does a very large amount of work in a beautiful automatic way, and it does it GOOD.

After installing it via aptitude install beets, I created a configuration file called ~/.beetsconfig with these lines in it:

[beets]
directory: /mnt/BigBoy/Music
library: ~/.musiclibrary.blb

[paths]
default: $albumartist/$year - $album/$track - $title
singleton: Non-Album/$artist/$title
comp: Compilations/$album/$track - $title

directory is where the cleaned && tagged && fixed files will go, library is the SQLite file containing all the metadata (that can be later on queried by beets himself, and this is another good feature of beets), and the [paths] section holds the patterns I like for the directory structure of my music collection.

After that, let’s assume that I just bought a CD by a fictitious rockstar called “Rockstar”, and the album is called “New Album”. I rip it by using A Better CD Encoder (abcde) (maybe it will be subject of a post, sometimes), and I obtain a bunch of mp3/ogg/flac/whatever I like. Let’s assume also that the ID3V{1,2} tags aren’t really correct (even though abcde can tag the files in the right way, but I digress, sorry..).

I just have to tell beets where to find the directory containing these new files, and it’ll do what it has to:

$ beets import ~/abcde_ripped_music/RockStar-NewAlbum

What beets will do is to search (by using filenames and/or already tagged metadata) in the MusicBrainz database and when it finds the right entries, it’ll correct the tags and the filenames, it’ll add the metadata in the library SQLite database, and it’ll copy the files from the source to the destination directory, beautifully renamed like I want.

If something goes wrong (for example if beets is in doubt which entry is correct) it stops and asks the user some hint. And the user can manually search for the right entry, or can skip the album, or can add it to the library as it is without modifications, and so on.

After that I can make things like beets ls some_value_to_search_for to have the listing of my library, or by searching ID3 tags, I can maintain my database up to date by adding/removing/modifying entries, I can obtain the actual filename of some songs/albums, for example to fed them to a pipe in a shell script, I can play music with a client that uses the MPD protocol, or with a web interface from an HTML5-enabled browser, I can make statistics with beets stats, and the list of possibilities can be very long…

beets has tons and tons of options to be given in the commandline and in the configuration file (I suggest you to read the documentation and experiment a lot).

I hope that some of you will start using this little gem in the Free Software world.

P.S.: And I’ll appreciate and welcome any comment by beets users, with tricks and/or configuration advices!

Build ffmpeg from git (just for fun, just because we can…)

I have all my git repositories under ~/GIT-REPOS/, so I created a new directory and cloned the ffmpeg repo under it:

$ mkdir ~/GIT-REPOS/ffmpeg.git && cd ~/GIT-REPOS/ffmpeg.git
$ git clone git://source.ffmpeg.org/ffmpeg.git .

As stated in the documentation, it’s possible to build ffmpeg elsewhere, just by calling the configure script in the destination folder, adding the relative path. So I created another directory, where I’ll build the binaries, and launched configure, with the --prefix option to decide where ffmpeg will be later installed (I prefer to install compiled software in a directory under my home, to not mess up my /usr/):

$ mkdir ~/GIT-REPOS/ffmpeg-build && cd ~/GIT-REPOS/ffmpeg-build
$ ../ffmpeg.git/configure --prefix=~/Programs/ffmpeg

Here you may have to install some development libraries, just read the output and install what you need (configure will be very noisy && informative about that). If everything goes well, it’s time to build! Grab a cup of coffee/tea/whatever_you_want, and execute:

$ make

After a while, hopefully you’ll have the compiled binaries, just install them (in the directory you chose before in the configure step) this way:

$ make install

That’s it! Here is what my newly compiled ffmpeg says:

$ cd ~/Programs/ffmpeg/bin
$ ./ffmpeg -version
ffmpeg version N-37208-g01fcbdf
built on Jan 27 2012 13:31:16 with gcc 4.6.2
configuration: --prefix=~/Programs/ffmpeg
libavutil 51. 34.101 / 51. 34.101
libavcodec 53. 60.100 / 53. 60.100
libavformat 53. 31.100 / 53. 31.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 60.100 / 2. 60.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 6.100 / 0. 6.100

Now it’s time to investigate all the options of configure (like, for example, a ton of --{dis,en}able-something)

How Doxygen can help the understanding of some piece of code

In my spare time I tend to look at the code of the free software I use, and lately I’m digging in the TeXmaker sources. TeXmaker is my favourite LaTeX editor, written in C++ with the Qt framework (and not depending from the KDE libraries, but only from Qt, whereas, for example, Kile needs almost all the KDE environment, which is bad for me, because I’m using fluxbox and trying to keep my machines slim and clean).

First of all (maybe I didn’t searched thoroughly enough…) I couldn’t find the upstream code repository (SVN, git, whatever) with the code in development, but only the released tarballs in the download page of the official website.

I am currently trying to work on a very little modification (and if I’ll succeed, obviously my patch will be proposed to upstream), but I found almost immediately one problem: the code has no comments. Yeah, right, no comments. So I summoned Doxygen asking for help

After decompressing the tarball (the code is all in the “root” directory of the source tree), I created a basic doxyfile with the command

$ doxygen -g


that creates a template called Doxyfile (because I didn’t give a filename after the -g option). After that I made some changes:

PROJECT_NAME    = "TeXmaker Source Code"
EXTRACT_ALL     = YES
EXTRACT_PRIVATE = YES
EXTRACT_STATIC  = YES
INPUT           = .
SOURCE_BROWSER  = YES
HAVE_DOT        = YES

The options I changed are almost self-explaining (and the file is packed with comments for each and every setting).

Launching doxygen with this file in the main TeXmaker source directory creates a beautifully cross-referenced HTML documentation, and launching make in the latex subdirectory compiles a ~500 pages PDF file with the same reference.

Thank you Doxygen, thank you TeXmaker coders for this great piece of software (but let me say that it’s sad it’s not commented at all).

And now, let’s go back reading, experimenting, hacking, and coding!

Solving a wicd problem when connecting to an unsecured wifi after using a WPA/WPA2 one

So, here I am after some time (university is leeching my “spare” time and energies…), and this morning I want to write a couple of lines about a workaround for a problem I had with wicd, the network manager I use.

In short, I have a WPA2 enabled wifi router at home, and the wifi at my university is unsecured. I usually tend to hibernate my netbook instead of shut it down and some time ago a problem emerged. If I tried to connect at university after having connected at home without shutdown/reboot my netbook, wicd failed to connect.

I found out that I had to rmmod and then modprobe/insmod the module controlling my wifi card (a Broadcom). After some chat with the Debian maintainer of the package I solved with a workaround (also explained in the bug report for #629377 in the DBTS).

wicd can use shell scripts before and after connecting and disconnecting, simply by placing them in the relevant directory among these four: /etc/wicd/scripts/{pre,post}{dis,}connect. So I just modified the skeleton script found in the wicd official wiki adding these lines at the end:

if [ "${essid}" == "$MyUniversityESSID" ]; then
    echo "#629377 fix: wl module reload"
    rmmod wl
    modprobe wl
    sleep 10
fi

and dropping it under /etc/wicd/scripts/preconnect.

In this way, if I’m trying to connect to my university’s wifi, wicd will remove and reload the module for my wireless adapter. The sleep 10 is needed to make wicd wait for the module to completely load before trying the connection (otherwise it will fail, and obviously the timeout can be adjusted).

What HTML documentation is installed in my Debian box?

So, here I am after a little period of “silence”
Yesterday evening I was thinking about the question that you can see in the title of this post…and this little script came out:

#!/bin/bash
actualtime=`date +%Y/%m/%d\ %H:%M:%S`
# The directory $HOME/Documents *MUST* exist (change to a directory of your choice)
indexfile="$HOME/Documents/Debian_Installed_HTML_Documentation.html"

cd "/usr/share/doc"

echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" > $indexfile
echo "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.1//EN\"" >> $indexfile
echo "\"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd\">" >> $indexfile
echo "<html lang=\"en\" dir=\"ltr\" xmlns=\"http://www.w3.org/1999/xhtml\">" >> $indexfile
echo "<head>Debian Installed HTML Documentation (generated $actualtime, `find | grep "/index.html$" | wc -l` links)</head><br />" >> $indexfile
echo "<body>" >> $indexfile

find | grep "/index.html$" | sort -u | sed -e 's,^./,,g' -e 's,\(^.*\)/index.html$,<a href="/usr/share/doc/\1/index.html">\1</a><br />,g' >> $indexfile

echo "</body>" >> $indexfile
exit 0

The core of the script is the find | grep | sort | sed thing (yes, I know, grep can be removed by using the -name option of find, but this is the first version, it works, it’s a very tiny piece of code, so I don’t bother too much about optimization this time).

At the same time I don’t bother too much about making a good looking HTML page, for now it just searches index.html files under the documentation directory, and after an almost clean (I hope, I’m not a web{designer,developer,whatever}) XHTML 1.1 DTD it spits out links to every one of them.

Now I’m thinking about something like that for PDF files too, but HTML hyperlinks to PDF files aren’t so good, because the browser will “download” (even if in a temporary directory) the file before feeding it to evince, epdfview, okular, etc.. resulting in another copy everytime I’ll click.

So, I hope you enjoyed my little naive evening hack!

Happy Documentation Searching!

PS: and Happy New Year!

Configuring sort order in Newsbeuter (Use the Source, Luke…)

Hi!
This morning I reattached to my GNU screen session, launching newsbeuter, my favourite TUI rss aggregator, in a new screen window.

Then I found myself doing a G-u (to reverse sort on unread articles number), because I like having my feeds containing unread articles at the top of the window.

So I thought: “Hell! there must be an option for the config file that does this every time I fire up newsbeuter! Let’s RTFM!”

Reading here, I saw feed-sort-order that seems to be my buddy, but there’s no indication on how to do reverse sort, or the list of sort orders. So, instead of better searching the documentation, I chose to look directly at the source.

In the github repository, I searched the src/ subdirectory, finding that in the file feedlist_formaction.cpp there is the process_operation() method which reveals that the option is always feed-sort-order and the value I was looking for is unreadarticlecount-asc.

The Source is strong, use the Source, Luke!

And… what if I’ll take a look at Python?

Ok, in the last few days I didn’t blog because I had my second last exam at university (which went well), and now I’m back! I was thinking that I can spend my Xmas holidays playing around with Python (yes, I like having geek fun!)

I’m now reading Dive into Python 3 (a freely downloadable book) and sometimes I take a peek to the official Python documentation.

If someone has some advice to give to a C/C++/Java/bash/whatever (and wannabe-Perl, among other things) programmer who wants to learn Python, leave me a comment!

Bye!

Script to synchronize Minecraft savegames between different PCs

Hello!
I just wrote a little script to keep in sync my Minecraft savegames between my two machines (NB = netbook, DT = desktop) and I want to explain a little how it works.

First of all: my DT has sshd configured to permit passwordless login from my NB by using a keypair.

#!/bin/bash
remote_user="user1"
local_user="user2"
local_wlan0="$(/sbin/ifconfig wlan0 | grep "inet addr" | sed 's,.*inet addr:\(.*[0-9]\) *Bcast.*$,\1,')"

Here I declare two variables that keep the username in the two machines (remote -> DT, and local -> NB), and I get the LAN IP of the NB.

if [ "$local_wlan0" == "192.168.1.2" ]; then
   echo "Home LAN!"
   remote_host="192.168.1.3"
elif [ "$(echo $local_wlan0 | cut -c -6)" == "10.14." ]; then
   echo "University LAN!"
   remote_host=$(RIP)
else
   echo "Unsupported LAN, exiting"
   exit 1
fi

Here I find which network the NB is connected to (RIP is a script of mine, that extracts the DT Remote IP from my local maildir on the NB. My DT sends me an email when his external IP changes. Yes, I got rid of dyndns services in this way )

remote_dir="/home/$remote_user/.minecraft/saves"
local_dir="/home/$local_user/.minecraft/saves"
remote_saves="$(ssh $remote_host ls $remote_dir)"
local_saves="$(ls $local_dir)"

I get the list of savegames under the ~/.minecraft/saves/ directory.

for savegame in $(ls $local_dir);
do
   echo "Working on local savegame '$savegame'"
   echo $remote_saves | grep $savegame &> /dev/null
   if [ "$?" == "0" ]; then
      echo "Found on remote machine"
      local_time=$(stat -c %Y "$local_dir/$savegame")
      remote_time=$(ssh $remote_host stat -c %Y \"$remote_dir/$savegame\")
      let diff_time=$local_time-$remote_time
      if [ "$diff_time" -lt "0" ]; then
         echo "Remote newer than local"
         scp -p -r "$remote_host:$remote_dir/$savegame/" "$local_dir/"
      elif [ "$diff_time" -gt "0" ]; then
         echo "Local newer than remote"
         scp -p -r "$local_dir/$savegame/" "$remote_host:$remote_dir/"
      else
         echo "Local equals remote, doing nothing"
      fi
   else
      echo "Not found on remote machine"
   fi
done
exit 0

In the last snippet of code, I check that the current savegame on the NB exists also on the DT, and then I copy (with scp) the older copy over the new copy (wherever it is).

The second version will use rsync instead of scp (to transmit only the minimal quantity of bytes) and will do the “reverse check” (if a savegame is only on the DT, it will copy on the NB).

Obviously if you have some advice, comment!

Bye bye!