in English

My music compositions

My music compositions can be heard at nandoflorestan.bandcamp.com. Also, this blog is much less active now that I am twitting...

What Oracle should do to MySQL

What is probably going to happen now that Sun and MySQL belong to Oracle is:

1) They are going to get a world-class boring buzzword-compliant website.

2) All this.

Java's opinion of Python

I read some guy talking about Java, "the greatest language in the world", just like an ugly American might say "the greatest country in the world". He has motivated me to become an ugly Pythonista.

A Javer and a Cee-Sharper meet a Pythonista and consider Python for a brief moment.

Oracle Certified Java Programmer: ― I guess writing code in Python is quicker, but it might create maintenance nightmares later. Python is optimized for productivity and Java for maintainability.

Microsoft Certified Professional: ― Actually, the absence of great IDEs for Python, such as Visual Studio, might slow down the production of code.

Oracle Certified Java Programmer: ― "Rigid" languages such as Java make you write more code, but that code stays legible till the end of the project lifecycle.

Python apologist guy: ― You are wrong. I chose Python exactly because it is the most readable language available...

Microsoft Certified Professional: ― Nah, creating properties is very hard without an IDE. Another problem in Python is the lack of Generics. I am proud of writing in a language that has LINQ. Python is also missing a nice Reflection library such as the ones in C# and Java! That shows how much more powerful these languages are.

Oracle Certified Java Programmer: ― Yeah, I like Java because I get to program in XML. Hey, in Python you never know the type of a variable or parameter. Java is more explicit, therefore Java code is more readable.

Python apologist guy realizes the hopelessness of it: ― You are right, of course, but you can always use Hungarian Notation: s_name = 'Nando'; i_age = 33;

Microsoft Certified Professional: ― I don't like Python or Java. I prefer the C family of languages, invented by Microsoft, that includes C, C++ and C#.

Python apologist guy: ― Wait, in Python you can continue to type a semicolon at the end of every line.

Oracle Certified Java Programmer: ― Hmm, it must have copied Groovy. But that makes Python better than I figured...

Python apologist guy tries to ignore him: ― You can have strongly typed variables too. And braces. See this example:

# MyClass.py
# Proud author: nosklo (Clovis Fabricio)
class MyClass(object):#{
    def __init__(self, s_name, i_age):#{
        assert isinstance(s_name, str);
        assert isinstance(i_age, int);
        if (i_age > 20):#{
            print s_name;
        #}
    #}
#}

Microsoft Certified Professional: ― Hmm, the eye sees the #{ combination easily. OK, this code is readable. But I like .net because it is extremely multi-language.

Python apologist guy: ― Don't worry, you can even have goto if you wish. Here is a VB-like alternative:

# MyClass.py. Proud author: nosklo (Clovis Fabricio)
class MyClass(object):
    def __init__(self, s_name, i_age):
        assert isinstance(s_name, str)
        assert isinstance(i_age, int)
        if (i_age > 20):
            print s_name
        #end if
    #end function
#end class

Microsoft Certified Professional: ― You can't convince me because Python is an interpreted language, and I only like compiled languages, like C#. Furthermore, I like when application blocks are ready so I don't have to write them myself. ― Goes away.

OCJP waits until MCP is far enough. ― Never mind him, he never even understood the importance of Checked Exceptions. Hey, I see there are 2 classes in this file. This must be a maintenance nightmare! The only true way is one class per file.

Python apologist guy: ― Yeah, namespaces are one honking great idea ― let's do more of those!

Oracle Certified Java Programmer: ― As much as I might like your language, I could never give up world-class enterprise-ready buzzword compliance and a BIG company backing us up.

Python apologist guy: ― Good for you!

Ruby developer: ― Excuse me, I couldn't help but hear your conversation. I just wanted to say that if you ever need a dynamic language, consider Ruby ― it is more powerful.

Python apologist guy turns his back to OCJP and attacks Ruby developer: ― Die, heretic scum!

Schemaless Python databases

3 new persistence options for Python

As I struggle to create an object-oriented database for Python on top of Tokyo Cabinet -- Pykesto -- I have found two other nice persistence mechanisms:

Kineta is a dynamic, schemaless, relational database written in Python on top of Berkeley DB. It is almost done, I have tested it a bit and was impressed. It is full of different ideas, sometimes I felt it was too different from what I am used to, but if you really think about them and try to be fair, you realize these are all good ideas. Really, take a look at Kineta.

The last one -- Pryvilege -- is a Prevayler for Python. For those unfamiliar with the Prevayler concept, this is an extremely fast, transparent, ACID persistence mechanism for small databases that fit in memory. In such a case you can be free of all the overhead of traditional databases and just use your objects in the most natural way. Behind the scenes, the framework writes a log of all the operations you do. Every night or so a checkpoint is created, too (allowing the log to be reset). When the system is turned off and on again, the checkpoint is loaded and the log is applied so your objects come back to the state they were before. This might take a lot of RAM, but RAM can be much cheaper than developing with traditional databases.

Pryvilege is in an alpha state, but is already functional.

EDIT: One more! buzhug is a pure-Python database engine.

And there exists a B-plus Tree implemented in Python.

Installing Tokyo Cabinet and its Python driver

Tokyo Cabinet is the fastest database available, and it comes in several flavors. Here is how to install it in Ubuntu and use it with Python.

After realizing that CouchDB is not appropriate when you know you will need ad hoc queries (which doesn't mean CouchDB isn't very cool), I am trying out Tokyo Cabinet.

Here is a post about it, using Ruby.

Tokyo Cabinet offers several types of database, but the version that comes in Ubuntu 8.10 is missing the fixed-length and the table database, so I had to compile it:

sudo apt-get install checkinstall build-essential libbz2-dev
# Dependencies installed; now compile:
./configure --prefix=/usr
make clean
make
# checkinstall -R    # would create an RPM package
sudo checkinstall -D # creates and installs a Debian package
# Now you have a package that you can install AND uninstall, instead of
# sudo make install

Now the library, programs and headers are available, so we can install the Python driver. The most well-known is the old pytc which also doesn't offer all kinds of databases. tc is the Python driver we need. Here is the author's blog.

I download the latest code from github, uncompress it, enter its directory, then:

./setup.py install

That will succeed in Ubuntu 8.10 as long as the header files are found.

Unfortunately this driver is only nearly complete. As of 2008-04-03, the table database lacks a close() method, and the query API is being finished (you cannot execute queries yet).

Anyway, what software are we going to write using Tokyo Cabinet? How about an ambitious object-oriented database for Python apps? Something to replace ZODB? I am talking about pykesto and I wish you would help me write it.

However, it will take time to get there, and we might have to start by creating a higher-level interface to Tokyo Cabinet, because right now instantiating, for instance, the table database involves passing ugly flags like this:

import tc
t = tc.TDB('test.tdb', tc.TDBOWRITER | tc.TDBOCREAT)
# The above means open the file 'test.tdb' for writing and
# create it if it does not exist.
t.put('row id', {'col id':'value'})
t.get('row id')
# Out: {'col id': 'value'}
t.close()

Another reason to create a higher-level API is that the query API for the table database also involves lots of flags. I am sure this can easily be made Pythonic. I will put the code at pykesto as soon as I have it.

Running Kurso de Esperanto on Ubuntu 8.10

Today I decided to learn Esperanto. Best way to start seems to be Kurso, a gratis program. So...

...I downloaded Kurso3.0.deb and installed it. You might have 3 issues running this program in Ubuntu:

1) No way to start the program

Installing the program does not add a menu entry for it. So click the Gnome Panel, pick "Add to panel...", then "Custom Application Launcher", then type:

Name: Kurso
Command: kurso

You may also click the button at the left to choose a nice icon. That's it, now you can easily launch the program.

2) /usr/share/kurso/tradukoj

The second problem that might happen is a message about lacking write access to the directory /usr/share/kurso/tradukoj. Another symptom is, clicking on the Settings button does not display the configuration screen.

A solution would be to type the following in a console:

cd /usr/share/kurso/
ls -l # prints out the contents of the kurso directory
sudo chmod 666 tradukoj
ls -l

This sets write permissions on that directory. Now you can restart the program, the message is gone and you can open the Settings screen.

3) Sounds problem

You might get no sound for playback or recording. The solution is:

a) Install mpg321:

sudo apt-get install mpg321

b) Open the Settings screen, switch to the Sound/Internet tab, and leave it like this:

MP3 player:    mpg321
WAV recorder:  aplay
WAV player:    arecord --duration=3 --rate=44100
Browser:       firefox
e-mail client: thunderbird

If you are really paying attention, you might notice that it doesn't make sense to pick "aplay" to record and "arecord" to play. This is not my mistake; in this version (3.0) of Kurso, they messed up the 2 labels.

There you go, fully functional Kurso.

Web dev in Python: must read

If you are a Python web developer, here is some stuff you absolutely cannot miss.

WSGI middleware considered harmful
WSGI Framework Components
Restish, a new web framework
Spawning, a WSGI server which supports multiple processes, multiple threads, non-blocking HTTP IO, and automatic graceful upgrading of code
CherryPy, an easy HTTP framework
Webpyte, a reimplementation of TurboGears
Genshi, a great templating system
Tutorial: CherryPy + Genshi
CouchDB with Python. Couchdb is a document-oriented database
Introduction to Twisted, an asynchronous network framework
The best e-mail address validation module you are going to find

Easily start the screensaver in Ubuntu Linux

How to start the Gnome screen saver immediately

The command to start the Gnome screensaver is:

gnome-screensaver-command -a

I tried to make this happen using keyboard shortcuts first, but couldn't find a way. So I went with the next best thing: a launcher in the Gnome panel.

Right click the Gnome panel and click "Add to panel...". Pick "Custom Application Launcher". Another window appears. Fill it like this:

Type: Application
Name: Screensaver
Command: gnome-screensaver-command -a
Icon: <pick any icon...>

Hit OK and now you can start the screensaver by clicking the new icon.

Tuples, lists, dicts and objects: how fast?

Comparison of tuple, list, dict and object instantiation performance in Python.

It is well-known that in Python tuples are faster than lists, and dicts are faster than objects. I wondered how much so. After some tests, the conclusions are:

1. Use a tuple when you need a tuple and a list when you need a list. A list can have items appended or removed, while a tuple is immutable. If you won't be adding or removing items, use a tuple and your code will run a little faster.

On the other hand, if you make a tuple with the intention of being fast, and then some code has to create another tuple out of your tuple, when it could have added or removed an item from a list instead, then you are actually being slower.

2. Dicts can be twice as fast as objects (but they do much less). Object oriented programming is based on the fact that uniting data and behaviour in a single place leads to better factored, more reusable code. So go on and make classes and objects, it is the only civilized thing to do! However, note that dicts are quite faster than objects (to instantiate).

Needless to say, premature optimization is the root of all evil, so keep these facts in mind only when you are optimizing some code, not when you are writing something new.

Here is a little module, and the tests are in its docstring if you want to reproduce them:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

'''speed_test.py

Comparison of tuple, list, dict and object instantiation performance.

Usage: (results are from Python 2.5.2 in Ubuntu)


python -m timeit -s 'from speed_test import *' 'a = {}'
10000000 loops, best of 3: 0.0792 usec per loop

python -m timeit -s 'from speed_test import *' 'a = []'
10000000 loops, best of 3: 0.081 usec per loop

python -m timeit -s 'from speed_test import *' 'a = object()'
1000000 loops, best of 3: 0.231 usec per loop

This object() instantiation is almost useless, done only to get a rough idea.

Conclusion:
For empty containers, lists and dicts are equivalent.


python -m timeit -s 'from speed_test import *' 'a = (x, x)'
10000000 loops, best of 3: 0.184 usec per loop

python -m timeit -s 'from speed_test import *' 'a = [x, x]'
1000000 loops, best of 3: 0.271 usec per loop

python -m timeit -s 'from speed_test import *' 'a = {x:x, y:x}'
1000000 loops, best of 3: 0.474 usec per loop

python -m timeit -s 'from speed_test import *' 'a = TestObj(x, x)'
1000000 loops, best of 3: 1.17 usec per loop

Conclusion:
For storing 2 values, tuples are 50% faster than lists.
Lists are twice as fast as dicts, but not fair: dicts also store keys.
Dicts are 250% faster than objects.


python -m timeit -s 'from speed_test import *' 'a = (x, x, x, x)'
1000000 loops, best of 3: 0.268 usec per loop

python -m timeit -s 'from speed_test import *' 'a = [x, x, x, x]'
1000000 loops, best of 3: 0.357 usec per loop

python -m timeit -s 'from speed_test import *' 'a = {x:x, y:x, z:x, v:x}'
1000000 loops, best of 3: 0.79 usec per loop

python -m timeit -s 'from speed_test import *' 'a = TestFour(x, x, x, x)'
1000000 loops, best of 3: 1.81 usec per loop

Conclusion:
When storing 4 values, tuples are 33% faster than lists.
And dicts are 2 times faster than objects.

The difference diminishes as you store more and more values.
'''

__author__ = 'Nando Florestan'

x = 42 # we use variables because constants would be much faster.
y = 43 # you don't use so many constants in the real world.
z = 44
v = 45


class TestObj(object):
    def __init__(self, a, b):
        self.a = a
        self.b = b


class TestFour(object):
    def __init__(self, a, b, c, d):
        self.a = a
        self.b = b
        self.c = c
        self.d = d

I dream of a db4o for Python

I wish Python would have an object database that wouldn't eat up all the RAM.

Such a project could also be defined as "Durus with queries that don't have to first instantiate objects".

It would need a custom file format, without using Python's pickles, to be able to query without instantiating. And I wouldn't mind losing some dynamic-language-like flexibility if I could have this.

I have been using the SQLAlchemy ORM because it uses very little memory, it is very powerful and very fast. It does everything you need and more. However...

For some projects, I feel the difficulties in defining SQLALchemy models (and dealing with the relational database behind it). I want to use Python types, not relational database types. I would switch to an object database, if only it would query and use little RAM.

(In Python, making a tuple is faster than making a list is faster than making a dict is MUCH faster than instantiating an object.)

If I were to start such a db4o-like project (maybe 2010?) I would have to wrap my head around pointers to objects and their activation. So at first I might use SQLAlchemy as a backend. A table of properties... belongs to a table of objects... which know their Python type. This would probably enable one to make those queries without first instantiating.

About this scheme, ronny tells me to take a look at RDF -- neat for dealing with not clearly defined object graphs.

(This is not my area of expertise...)

If you are a real good, but real good Python developer, go ahead and do it! But heads up, you're gonna face DAMN tricky stuff about reference cycles and efficient collections.

This project is for the future. Python 3.0 on it!

Backalaika, an easy backup solution

A Python utility that I have written lets you define backup profiles, filter filenames, compress to zip, tar.gzip or tar.bz2, perform differential, incremental or full backups etc.

Backalaika is not a Russian musical instrument. It is a backup solution for small offices, available at SourceForge.

The original idea was to be implemented in C# a few years ago and indeed a crummy prototype was even uploaded to the CVS repository in SourceForge.

That architecture was unnecessarily complex. The current program is written in a smarter language (Python) with fewer features, but is much more useable. You can download it here.

Read the documentation (on the program itself).

Python 2.6.1 on Ubuntu

Python 2.6.1 is the new stable Python version, but Ubuntu 8.10 Intrepid Ibex does not come with it. Here is how to install it though.

First of all, you must learn about Python 2.6.

Excited about the new features? Then now realize that while Python 2.6 prepares you for 3.0, it also makes your code incompatible with 2.5 if you use any of the new features! You must keep this in mind when you decide what to use!

Most library authors won't bother updating their code with Python 2.6 features, as this would mean incompatibility with Python 2.4 and 2.5 (as well as 3.0). Python 2.6 is a version that is only compatible with itself.

In other words, there are two antagonic forces here: On the one hand, you want to use the latest features of the language and write modern code that won't have to be heavily rewritten/updated soon (and using Python 2.6 is the best option for that right now, because nobody is using Python 3.0 yet); on the other hand, the lack of libraries and batteries that you can use, as well as other people using your code, may dictate that you must stay in previous versions for a long time.

We would be talking about using Python 3.0 here if only there were any libraries for it already...

Latest Zope/Plone still use Python 2.4!

In these times, simple libraries that are more likely to be ported sooner to 3.0 make a good choice.

If you have to stay in Python 2.5... you still can import these:

from __future__ import with_statement
from __future__ import absolute_import
# http://docs.python.org/whatsnew/pep-328.html

On the other hand, if you are targeting Python 2.6, why would you not use all its features? Therefore, add this to the top of your files:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import absolute_import
# http://docs.python.org/whatsnew/pep-328.html
from __future__ import print_function   # deletes the print statement
from __future__ import unicode_literals # unicode by default

To sum up, Python 2.6 is for people who can get away with using Python 2.6. These will benefit from having fewer changes to make when the time to upgrade to 3.0 comes, and from using new language features right now. But there is a small price to pay in library installation complexity, as we shall see soon. (Keep reading.)

Installing Python 2.6.1 on Ubuntu Intrepid Ibex

Download the sources:

wget http://python.org/ftp/python/2.6.1/Python-2.6.1.tar.bz2

Decompress them:

tar -xjf Python-2.6.1.tar.bz2
cd Python-2.6.1/

Install dependencies:

sudo apt-get install checkinstall build-essential libc6-dev tk8.4-dev libgdbm-dev
sudo apt-get install libdb-dev libreadline-dev libsqlite3-dev libncurses5-dev
sudo apt-get install libbz2-dev libssl-dev # and possibly a few others

Configure, make, test:

# I install it at /progs because I am afraid of messing up other directories
mkdir /progs
./configure --prefix=/progs --enable-ipv6  # --enable-shared
make
# It finishes with this complaint:
# Failed to find the necessary bits to build these modules:
# bsddb185           sunaudiodev
# These seem not to be a problem. I continue anyway.
make test # (optional, takes 15 minutes or so)
# 327 tests OK.
# 33 tests skipped: (...)
# Those skips are all expected on linux2.

Now you can do one of two things. Either create a .deb package:

checkinstall -D --pkgname=python2.6.1 --pkgversion=2.6.1 --maintainer=nandoflorestan@gmail.com--inspect --backup=yes --install=no make altinstall

And make sure you answer YES when asked to create a default set of docs!

...or install directly:

make install

The final step of the installation is to make your new python and python2.6 binaries available in the path, such that they will be found before python2.5. However, you had better do this for your user only, lest your whole Ubuntu operating system suffer (since all Linux distros depend heavily on Python). Being cautious this way, I create symbolic links to python and python2.6 from the bin directory of my user:

cd /home/nando/python/bin
ln -s /progs/bin/python2.6
ln -s /progs/bin/python

Now you can test the interpreter:

python

This tutorial is based on these.

Now let us install a couple libraries so that our new Python 2.6 may be minimally useful:

Library hell

Now if you decide to go ahead and use 2.6, you are in library hell -- at least when the library you need is not pure Python. For instance, to be able to manipulate images, you must install PIL yourself, and it may not be trivial:

sudo apt-get install libjpeg62 libjpeg62-dev libfreetype6 libfreetype6-dev tk-dev tcl-dev
wget http://effbot.org/downloads/Imaging-1.1.6.tar.gz
tar xvzf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6
sudo /progs/bin/python2.6 setup.py build_ext -i

The last command fails in Ubuntu Intrepid Ibex, saying: "_imagingtk.c:20:16: error: tk.h: No such file or directory"

If you run "locate tk.h", you discover the missing file is really at: /usr/include/tcl8.4/tk.h

Because this is a gcc problem and I am just a Python guy, I don't know how to solve it. I wonder if I really need this _imagingtk stuff... I don't use TK... So I just uninstall the python-tk and tk packages. This way it succeeds and just says:

*** TKINTER support not available

An alternative solution to the above problem might be to just turn TKINTER support off by commenting out these lines in setup.py (starting at 328):

    '''
    elif feature.tcl and feature.tk:
        exts.append(Extension(
            "_imagingtk", ["_imagingtk.c", "Tk/tkImaging.c"],
            libraries=[feature.tcl, feature.tk]
            ))
    '''

So now I test PIL:

/progs/bin/python2.6 selftest.py # runs the tests

Unfortunately one important test fails, showing a new problem for me to solve:

  File "PIL/Image.py", line 375, in _getdecoder
    raise IOError("decoder %s not available" % decoder_name)
IOError: decoder jpeg not available
1 items had failures:
   1 of  57 in selftest.testimage
***Test Failed*** 1 failures.
*** 1 tests of 57 failed.

Since all required libraries are installed, I don't know what is going on. Googling the problem, I see people have solved this by repeating the whole process after everything is in place. Yeah I know, this is stupid. Anyway, I go ahead and delete the whole directory and repeat the steps and this time it works:

cd ..
rm -r Imaging-1.1.6
tar xvzf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6/
/progs/bin/python2.6 setup.py build_ext -i
/progs/bin/python2.6 selftest.py # runs the tests
/progs/bin/python2.6 setup.py install

Library hell indeed. If it took one working day to install each library. Fortunately, pure Python libraries written for 2.5 seem to work just as well with 2.6:

Setuptools

Installing setuptools is quite easy:

wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg
sh setuptools-0.6c9-py2.6.egg

After this, I find that I can run both bpython and ipython interactive interpreters, just reusing their old eggs that were already installed in my system for Python 2.5. I just change the shebang at the top of the commands to this:

#!/usr/bin/env python

This way, our Python 2.6 can be picked automatically (depending on $PATH).

cherrypy, genshi and most other packages just work too.

Conclusion

If you are writing an application: make an effort to use Python 2.6 right now.
If you are writing a library: (very hard) try to support both Python 2.5 and Python 3.0.

Don LaFontaine is dead, who is gonna do movie trailers now?

I vote Pablo Francisco. View these youtubed videos in order:

Who was Don LaFontaine

The 3 tenors do a movie trailer

Pablo Francisco made a carreer out of this...

Music links

Online ear tests for anyone, not just musicians

Color and music

Wireless network is down on Ubuntu startup/boot

A quick tip for Hardy users.

After finally finding and buying a PCI wireless card that has a little penguin logo on the package, I found out that it worked, but I had to do "sudo ifdown wlan0; ifup wlan0" on every boot.

The solution for this is:

sudo nano /etc/network/interfaces

...and add a line to your interface definition that reads:

pre-up sleep 7

...this way the network will start 7 seconds later, but with success. (Some systems might need more than 7 seconds, just increase the value.)

Linux on Frankenstein PC hardware still has too many of these annoyances and I don't see them going away at all. Using free software instead of a Mac that just works does have a cost, we have to admit it.

For more information on the interfaces file:

man interfaces

"Exposé" in Ubuntu Hardy

Easily switch between applications in Ubuntu Hardy

First you have to turn on Visual Effects:

Gnome menu System / Preferences / Appearance
Visual Effects tab
Turn on Compiz by selecting "Normal" (or even "Extra").

There is an additional application that can be installed for you to configure Compiz:

Open a terminal
sudo apt-get install compizconfig-settings-manager
Gnome menu System, Preferences, Advanced Desktop Effects Settings
Scroll down to Window Management

By default, the alt-tab combination is assigned to "Application Switcher", which is cool, shows the reduced windows linearly... but you may prefer to disable it and enable the Ring Switcher instead. I have done so and assigned alt-tab and shift-alt-tab to it. The advantage in my opinion is a clearer view of the applications while switching.

I also tweak the Ring Switcher so it becomes fast and displays larger thumbnails.

In addition, I enable the Compiz equivalent of Exposé: Scale / Window Picker. This one deserves to be accessible by keyboard and by screen corner. I also increment its speed.

If you use multiple desktops, note that Super-E activates Expo which allows you to choose one of the desktops and move apps among them.

Sadly, beyond these I think there is little that really would improve your productivity as opposed to being just eye candy.

MuseScore

Maybe it is not quite there yet, but MuseScore is the most promising free-software music score editor I have ever seen. Here is a review. There is a Ubuntu package for it.

Web development reading

A collection of the most interesting things I have been reading about web development.

A free ebook about Greasemonkey
WYMeditor: finally a javascript wysiwyg editor that shouldn't generate buggy XHTML.
Google Gears makes the web browser a richer client.
Google App Engine is Python HTTP hosting focused on scalability.

Our daily nazism — parents beware

What are our children being prepared for? "Death Note", seen by millions, is nazi. The story was a huge opportunity for moral thought, but only nazi ethos has a chance to appear.

I am talking about Death Note, anime series. The manga series has been read by millions in Japan alone. There is also a Japanese film and a few other franchises.

I wouldn't normally lose any of my time with nonsense such as anime series, but my 14-year-old daughter likes it and I decided to accompany her, plus in the first few episodes I was curious myself. Why?

Light is a popular, good-looking student who only gets A grades. A genius, really. One day he finds a weapon, thrown unto this world by a Shinigami (a God of Death) for his amusement. The weapon basically lets Light kill anyone he wants, without even meeting the victim. Light decides that he will use this weapon to kill every criminal that appears on the news. This way, he thinks, he will be the unknown creator of a new world, without crime.

Crime rates drop quickly and crowds thank the mysteryous Kira ("killer") for it.

Is it nazi enough for you? But there is more.

The premise of the story seems a golden opportunity to talk about ethics. It would have been easy to discuss (and, in the process, teach children) about:

how wrong it is to take justice in one's own hands;
how wrong it is to be accuser and judge at the same time;
why we need the due process of law;
how deceiving the news (and the rest of the culture industry) can be;
how anybody can end up committing a crime (nobody is immune);
how the law is not always right and may define crimes that shouldn't be;
how the law may not define some actions as crimes when they should be;
the principle that punishment should be proportional to the crime;
etc. etc.

Instead what happens?

The mass murderer, who thinks of himself as a god, is indeed called Light;
He just kills everybody, he is not interested in finding out whether the news are even true;
He is Japan's best student (!!!) but somehow has no idea that his concept of justice is barbaric, absolutely uncivilized;
The series carefully avoids any moral evaluation, except Kira's own reasoning to do as he does;
Nobody else has high dreams;
Kira is the main character, no other character comes close;
Eventually other people find a way to follow the mass murderer's example;
The Gods of Death don't ever interfere for moral reasons. They interfere either for fun or because they are in love with a female character. They don't really think there is an enormous mess going on. In the end the plot just forgets about the Shinigami, they almost disappear.
The whole series becomes just a thriller alla Sherlock Holmes versus Moryarty. Except that the killer is more competent than the cops.
When the blockbusterish thriller ends, the series ends. There was nothing else to say.
Due process of law is never mentioned.
What is a crime? This is never questioned.
How should punishment be done? This is never asked.
etc. etc.

Do you think I am exaggerating? Do you think nobody would really identify with Kira? Think again.

Many people already think like Kira and would do the same. I have witnessed bad taste television shows in Brazil in which the angry host talked about a criminal (not a convicted criminal, but one that had just been arrested by the police), showing sensationalist disgust to what the criminal had done, saying things like "when he was doing it, he didn't think about human rights, so why should he have human rights now?"

This is how many people think. We are surrounded, outnumbered by the stupid.

I was curious enough to go to IMDb. I know most of the posts there are noise; there is no place to find idiots as the IMDb forums. But I have also found extremely intelligent comments there before. Maybe it depends on the film.

In this case, I could find next to nobody interested in any moral issues raised by Death Note. Most people just talked about the cat/mouse game. When they did talk about ethics, they were always examples of what I am saying:

Example 1

I have a hunch as to why Light may not have ultimately succeeded in wiping out criminals. Rather why he ended up getting caught. It's because he never got creative with his killings. He killed solely by heart attacks.

Example 2

"The real problem started when Light decided to kill an innocent. IE Lind "L". For starters, he's no longer got moral authority. Before that you could make a philosophical argument that although he wasn't legally supposed to kill criminals with the Death Note, from a moral standpoint he was the only one with the power to protect the world and humanity from it's own evil elements, and therefore he had to act. But the moment he kills "Lind" (who as far as Light knew was only doing his job), he's just another killer who takes pleasure in the act."

Example 3

YEah you shouldn't feel sorry for Misa b/c she even states that she doesn't care if Light only uses her and she's happy w/ just that.

Example 4

death note is not about good vs. evil L died because Light is better than him and Light died because the death note is bad luck.

in my opinion Light is a good person that happened to pick up the death note and use it to make the world a better place (he can be seen as a supper hero who’s only power is to kill people just by knowing their face and name, some evil guy who wants to take over the world, or a victim). I didn’t really see treat misa bad he just got mad at her when she did something stupid (it was her fault why they got cot by L besides she wanted to help light. Also I thank he fell in love with her when they lost their memory. If he didn’t like her would have killed her or break up with her near the end of the series in stead of going to marry her.

L might seem like the good guy but he’s not really a good person. he’s just a detective for fun, he doesn’t care if what he’s doing is bad or good, he will do any thing just to solve a case like lie, stalling, killing, letting good people die, takes advantage of people, breaks the law, forces people to do bad thing (forcing light to take advantage of misa) siding with criminals. He also hinted he is/was a pervert. You see his evil side and love of sweets in mello and his thinking, logic, loner side in near.

If it was about good vs. evil it would have been more black and white about it.

If any thing the most good person is light dad and the most bad person is the mob boss

Conclusion

See what I mean? People can watch this obscenity and not even be aware that they are becoming nazis.

In the last (bloody) episode, the good cops prevail. OK. But notice how this is done. Kira, surrounded, wounded, once again explains at length what he thinks (more nazi propaganda). Now the good cops are going to respond. I expect some education, finally, would anyone pleeease explain to Kira his mistake? Nobody does. The cop just says "you are wrong, you are just another serial killer and that is all".

But I have already shown that the moral question is not understood by the target audience. I don't feel it is understood by the author either.

Artistically, this shit isn't. It is just another anime. Economically animated (this means many, many stills), silly story, unidimensional characters (who never change), everything you can expect from the culture industry. No investigator appears smoking — but most of them are children (because children is the target audience of this crap and they like to see children), so they have children's vices such as eating too much chocolate. When a female character finally appears, of course she is a model, blonde, subservient, dumb, and her voice sounds like a woodpecker with a toothache. Intelligent characters are always socially inept, even Light (who starts out popular) becomes lonesome; the detective who finally defeats Kira is a 10-year-old genius who only plays alone with his toys. In other words, all significant people must always be alone. If there ever was a serial killer generator, this is it.

Trash that only children would take; however it is rated TV-14. Much of the music (which always sounds stupid and is repeated in every episode) is stolen from "Carmina Burana" by Carl Orff, himself possibly a Nazi and in any case a favourite composer of the original Nazis. ("Carmina Burana" is a very simple score by 20th century standards; nazis don't take complexity well, as "Death Note" helps to show.) Some of the credits music is heavy metal, which is known to always be Nazi anyway.

Some cultural references are thrown in without any real significance or effect:

Plainchant in the music is in the wrong places and leads to nothing.
The main Shinigami character likes to eat apples and Kira provides apples to him. Everybody knows what apples mean, but like I said, this has no significance or effect. It is like an afterthought: "wow, I finally finished writing this damn script — oh wait, I forgot to drop a couple of biblical references in it".

Anyhow, now that Death Note contains apples, people are saying it is cult.

Desu nôto is one of the most irresponsible things I have ever seen. My advice is: take care of your children and never be afraid to say "no TV"!

A FUD campaign against XHTML

HTML was a total mess and XHTML fixed that. But now some people want to unfix it.

This was the first time I ever saw stupid people in the Python community. They don't use XHTML because it is not perfect.

Well, as any web designer or web programmer in the world knows, Internet Explorer has always tried and will always try to destroy any standard. And yes, XHTML has its own shortcomings. It is still infinitely better than the pure mess that HTML 4 was.

This FUD is going to increase from now on, because now some people want to sell us that "XHTML has failed" and "HTML 5 will be great".

I did not know there were so many morons who in 2008 still haven't understood this:

Transcribed from #Pylons, 2008-03-15

<mitsuhiko> but as browsers don't implement it currently i have to keep a page compatible to xhtml and html browsers

<mitsuhiko> which is so problematic from so many point of views

<mitsuhiko> starting with different css semantics, different cdata semantics, different javascript behavior etc.

<mitsuhiko> steg: i want to understand why so many people are blindly promoting xhtml without understanding it

<NandoFlorestan> defending common HTML in 2008 sucks

<zepolen> defending xhtml in 2008 sucks, as its more of a 2015 language

<mitsuhiko> NandoFlorestan: show me a page of you where you're doing xhtml

<NandoFlorestan> mitsuhiko, anywhere in oui.com.br

<dpn`> what i know about xhtml i like

<devinus> every modern browser supports XHTML sent as text/html

<mitsuhiko> devinus: nope

<dpn`> mostly that because it is xml you can do all sorts of dandy thing with it

<devinus> so can it everyone, this argument is soooo 1999

<frostschutz> it's amazingly difficult to do some table layouts with just divs

<steg> let's tally NandoFlorestan's fantastic argument so far: "Defending HTML sucks", "People who don't use XHTML are lazy" oh and here's the absolute killer, the one that convinces me most of all... apparently we "drool over table lay outs and are ultraconservative Microsoft-worshiping Bush-adoring XHTML-bashing bastards"

<steg> NandoFlorestan: can you add to this impressive repertoire?

<frostschutz> oui.com.br does not look like xhtml

<mitsuhiko> NandoFlorestan: you're neither having any problematic cdata sections there nor javascript that plays with the dom

<steg> actually wait you sound foreign

<steg> no self-respecting american would be called "Nando"

<zepolen> i bet you support ron paul

<steg> does calling you Nando count as name-calling? ;)

<zepolen> call him obama

NandoFlorestan looks up this "Ron Paul" guy

<NandoFlorestan> "Congressman Ron Paul is the leading advocate for freedom in our nation's capital." Sounds good

<zepolen> wrong definition

<zepolen> ron paul: master and commander, jesus's 3rd coming

<zepolen> when the terrorists attack, Ron Paul will FIGHT....for a better fiscal policy

<devinus> lol

<devinus> Ron Paul

<devinus> and Obama both have XHTML sites

<devinus> McCain has invalid HTML

<zepolen> NandoFlorestan: what is your website

<NandoFlorestan> oui.com.br

<NandoFlorestan> thank you for such genuine interest in my musings

<NandoFlorestan> such a wonderful audience

<stepz_> xhtml error handling is not exactly ideal for building robust stuff

<mitsuhiko> zepolen: validation is not important, the main problem is if the xml is syntactically incorrect

<mitsuhiko> then the browser must abort rendering with a syntax error

<stepz_> html4 works fine, has no real problems and is more copmatible

<NandoFlorestan> I just wanted to understand what was the reason for the rage against this Ron Paul, he sounds perfectly reasonable to me in the interview

<mitsuhiko> actually america needs yet another gw bush

<NandoFlorestan> america is not a country

<NandoFlorestan> america is a continent.

<NandoFlorestan> the country is called United States of America.

<NandoFlorestan> There are many other countries in America.

<zepolen> if you have a problem with me you will have to take it up with my attorney

<NandoFlorestan> I didnt know there were so many trolls in #pylons...

<maqr[hax]> NandoFlorestan: i really hope that's your real name

SQLAlchemy has a new, easy declarative layer

A transcript from the #sqlalchemy IRC channel

<zzzeek_> deitarion: we just added a new declarative layer which removes a little of the verbosity, not sure if thats where you saw "too much boilerplate"

<justsomedood> woot, declarative layer part of SA? So not an extension?

<zzzeek_> its in ext/

<zzzeek_> we've had activemapper its just terrilby out of date

<justsomedood> way out of date

<zzzeek_> this is way smaller than activemapper

<zzzeek_> its 90 lines

<justsomedood> I ran into a lot of problems with it, and sent some fixes to johnathan but it ended up being too much work

<zzzeek_> i doubt this one has any big problems

<zzzeek_> since it does almost nothing :)

<justsomedood> hehehe

<Gedd> justsomedood: what kind of problems?

<justsomedood> It deferred class loading / table creation to make sure all dependencies were met before it did, that didn't always work

<justsomedood> I ran into cases where if I didn't define my classes in the correct order it would never create them, or get stuck in a loop

<justsomedood> I haven't used it since 3.10-ish I think though

<Gedd> ah ok

<Gedd> that explains it :)

<NandoFlorestan> zzzeek_, the synonym stuff in the declarative layer is sweet!

<NandoFlorestan> I think this may be an Elixir killer

<jek> nah

<Gedd> NandoFlorestan: FWIW, Elixir handles synonyms too

<NandoFlorestan> hmm, didnt know that.

<zzzeek_> NandoFlorestan: elixir has a ton of stuff this does not

<justsomedood> So on this one, when defining "complex" relations(), with a primaryjoin on a table/object that hasn't been defined yet you'd probably have to make that at the end with a class_mapper()?

<zzzeek_> you can stick it on later, like User.someprop = relation(SomeOtherObject, priamryjoin=foo.c.bar==bat.c.foo)

<zzzeek_> theres no class_mapper() needed here really since the class goes right to the mapper.....its also there as class.__mapper__

<justsomedood> k

<justsomedood> cool

<justsomedood> it would definitly condense my models and make them easier for others to read, and I wouldn't have to worry about the syntax changing

<justsomedood> me likes

<NandoFlorestan> that is why I like it: I already know the syntax

Pylons without Buffet, and a critique of TurboGears

Ben Bangert has decided to remove Buffet from Pylons. Of course it will still support all those templating languages, just not through Buffet.

The replacement? Just a few basic render functions. They make it much easier to see how you'd modify parts if you needed to tweak options — something you cannot do if Buffet is in the equation.

Here is ltbarcly's version of the same thing.

ltbarcly agrees. Why do we need one function that knows how to render every possible type of template? Just have a function called "render_genshi", one called "render_mako" etc., then you can do things like streaming in genshi, or form filling, and use all the caching etc in mako. Buffet doesn't do anything for you really — except give you the lowest common denominator.

He thinks, as I do, that dot notation for template names is stupid. If you write a common path instead, such as "test/blah.html", it looks for "test/blah/html"... Who needs the additional text replacement code — who ever needed dot notation for templates?

(Mitsuhiko answers: Kid was the one that needed dot notation, because it compiled the template, then imported it.)

I wish TurboGears 2.0 does without Buffet, too...

Do you really need 21 MB of RAM for a Python website?

I made a little experiment and found that using CherryPy 3.1, plus the packages I need, instead of the whole TurboGears stack, can save a lot of RAM.

How can a TurboGears website use 1/3 less RAM?

(If you host at WebFaction, your account is mainly constrained on memory usage.)

My TurboGears 1.0.4.2 application is using 23,888 KB on startup, then 27,512 KB after many requests for different pages. Pylons is more or less the same, since a Hello World takes up 21 MB.

(Python 2.5, Ubuntu 7.10. I understand this varies a lot based on OS.)

I decided to see what would happen if I rolled my own "framework" on top of CherryPy 3.1, which is vastly superior to CherryPy 2.3 — pity TG hasn't upgraded!

So I tried a memory consumption test for a CherryPy 3.1 based website.

I started with the CherryPy Hello World, then added import by import, measuring the impact in memory usage. The total memory used is annotated besides the corresponding import.

You can see memory usage with the following Linux shell commands:

ps aux
ps aux | grep python
ps aux | grep hello.py

Don't confuse virtual memory with resident memory. I am worried about the latter only.

The answer: A CherryPy 3.1 Hello World amounts to mere 7 MB. After I import everything I remember that I need (including Genshi, SQLAlchemy and Routes), I'm using 14 MB.

Here is the (very simple) test for you to reproduce.

Why this enormous difference?

Robert Brewer, CherryPy author, explains to me at #cherrypy:

well, I know Sylvain went on a tear recently to make sure we weren't importing things too early
so CP 2.x -> 3.1 might be a good chunk of the difference by itself
I also ripped out some of the stdlib modules from CP and rewrote them myself in order to be faster (e.g. no sense importing the whole MIME module when you're just doing http headers--a lot of the mime stuff is for email)

Of course, I owe my whole Python web programming experience to TurboGears. It provides sane defaults, especially for beginners. I learned Kid, then Genshi. I learned SQLObject, then SQLAlchemy. I also learned how to use Routes as the TG URL dispatcher. And there was the help of a large community.

But now that I am not a beginner, just using plain CherryPy could be a very similar experience.

Based on the result of this simple test, I am thinking of assembling a small TurboGears-like web framework. I would really just steal the things I really use. Make them work under CherryPy 3.1 (TG really should have upgraded.)

TurboGears 2.0 is being implemented on top of Pylons for good reasons... but CherryPy still seems to have several advantages over Paste or even WebOb, so it is not such an easy decision. At least for me.

How to set up a nice Python development environment

A recommended way to start developing in Python in Linux

Newbies to the Python programming language usually ask for an IDE. First of all, Python is a language that does not really need an IDE. You can just use a text editor like Kate, with a very simple text completion based on words that already exist in the file being edited. I have been using 2 very good simple IDEs for Python: Eric 4 and Geany.

(Sometimes I try to use Eclipse/PyDev too, but I hate it when it hangs!)

Instead of worrying about an IDE, make sure you install ipython. It is an enhanced Python interpreter, full of features such as tab completion, help, documentation viewing etc. However, ipython usage is beyond the scope of this post.

As a new Python developer there are some things that you should be worried about, but maybe you don't know. These things gradually led me to set up a development environment with these characteristics:

No root access. Come on, you know better than becoming root for such mundane tasks as running or even editing your programs.
No installation from your distro's repositories. Tempting as it may be to just sudo apt-get install turbogears, as a developer you want to easily control which versions you are using, instead of just accepting an old version from the repositories. setuptools is a nice solution for that.
No installation as root. As a Python developer you want to be able to swap packages and versions easily. Therefore, we shall install everything to ~/python/lib ("~" means your home directory, such as /home/nando).

If you are doing web development, note that this approach can also be used in shared hosting accounts where you have no root access!

The following tutorial shows how to accomplish these goals in Linux. We use nano as a text editor, but you can use any text editor you want. You also should know about file names that start with a dot: they are usually hidden from the user in Linux, and used for configuration and for programs to store user data. In Nautilus, the Gnome file manager, you can toggle visibility of hidden files by pressing Control-H.

Step 0: Uninstall

First of all, uninstall any special Python packages that you may have installed from your distribution's repositories, such as TurboGears, SQLObject, SQLAlchemy, Kid, Genshi, Django etc.

From your distro, keep only your Python, or better yet, just what your distro uses. There is really no need to custom install Python itself. Hopefully you already have Python 2.5, 2.6 and 3.0...

You can also apt-get install setuptools -- that is ok.

Step 1: Prepare program locations

Open a console to create the following directories:

mkdir ~/python
mkdir ~/python/lib
mkdir ~/python/bin

The last 2 directories is where we will install all Python packages, using setuptools. Now we want to tell Python to look up those directories when searching for some Python package. So edit your .bashrc file:

nano ~/.bashrc

At the top of that file, add these 2 lines:

export PYTHONPATH=~/python/lib:$PYTHONPATH
export PATH=~/python/bin:$PATH

That adds the libraries to the Python path and the startable programs to the shell path. This will happen the next time the shell starts and reads the .bashrc file.

We also want that to take effect now. So go ahead and paste those 2 lines to your console, too.

To see your current settings use these commands:

echo $PATH
echo $PYTHONPATH

Step 2: Configure setuptools

Now we want setuptools to use those 2 directories too, so when we say easy_install TurboGears it will be installed in ~/python/lib.

You can always use easy_install --prefix ~/python/lib TurboGears, but it is tedious to specify the prefix everytime. So let us make that a default place. Edit your ~/.pydistutils.cfg file:

nano ~/.pydistutils.cfg

In that file insert:

[install]
# directory for packages:
install_lib = ~/python/lib
# directory for scripts:
install_scripts = ~/python/bin

Now easy_install will install everything to ~/python/lib and ~/python/bin!

Step 3: Install your stuff

Here is an example of using Python setuptools to install the necessary eggs for developing an old version of Webpyte:

easy_install -UZ ipython
easy_install -UZ "TurboGears >= 1.0.4 , < 2.0"
easy_install -UZ "SQLAlchemy >= 0.4.2"
easy_install -UZ "Routes >=1.7"
easy_install -UZ "Genshi >=0.4.4" "Markdown" "TurboMail >=2.1"

Here is an explanation: -U means "find the latest available version"; -Z means "unzip the egg file".

Because Python is able to run zipped programs, they have created this glorified zip file (with a standard structure) that the call an egg. Each egg contains programs and metadata -- information about itself. For instance, it knows its dependencies. The easy_install command takes care of dependencies for you.

Because it is easier to see the source files if they aren't zipped, I like to use easy_install -Z, which extracts the eggs. So you see, an installed egg can be either a zip file (.egg) or a directory.

But where do these eggs come from? From the Cheese Shop, now known as Python Package Index: http://pypi.python.org -- this is the place where you search for nice packages!

Step 4: Uninstalling and upgrading

There is no remove command. To uninstall an egg, you just delete its directory and erase its line from the text file easy-install.pth.

Usually you don't need to remove anything. setuptools supports multiple versions of the same package. The easy-install.pth file determines which version is being used. Therefore, to upgrade to the newest turbogears, it is usually enough to just type:

easy_install TurboGears

Anyway, this is how I currently work in Ubuntu Linux. If you have any suggestions, I would like to hear them.

Maybe an even better solution?

Weeks after writing this article I learned that virtualenv exists and has a nice reputation. I haven't tried it yet, but it looks good.

Webpyte, a website system written in Python

I have published the source code for Webpyte, my new website system, based on the excellent web framework TurboGears. If you are a Python web developer, join us! Here is the project home at Google Code.

Python is the 6th most popular language

In 2007, Python surpassed Perl and C# in popularity, becoming the 6th most popular programming language in the world, according to the Tiobe Index.

Typography: Helvetica and Arial

The history of the design of Helvetica and the subsequent scourge of Arial is pretty interesting.

Resources for learning to program in Python

Python is a great language for beginners. (It is also a great language for any application: imperative programming, object-oriented programming and functional programming.)

How to think like a (Python) programmer, a free e-book, teaches the basics in a very intelligent way.
Dive into Python is the most famous e-book. It is even an installable package in Ubuntu Linux.
Style guide for Python code (PEP 8)
Docstring conventions (PEP 257)
Idiomatic Python

Enlightening blog posts

For advanced programmers

What's new in Python
The Python debugger
The Python Challenge is for advanced programmers to improve their skills. You solve problems, then compare implementations.
SQLAlchemy is a great ORM.
Python coroutines

Here comes another bubble

Have you seen this videoclip?

What's wrong with CS research

What's wrong with Computer Science research

Same subject: Hackers and painters

Beautiful short story by Isaac Asimov

http://www.multivax.com/last_question.html

...and since we are in a science fiction mood: Are we living in a computer simulation?

Project management: C²I²

The C2I2 Hypothesis

The Big Rewrite

Interesting talks at Google Video

The Paradox of Choice - Why More Is Less

How Open Source Projects Survive Poisonous People (And You Can Too)

Practical Common Lisp is a great book by Peter Seibel

Advanced Python

Restoring Grub

If you install Ubuntu, then Windows in a different partition, the GRUB boot loader vanishes. But don’t panic and reinstall Ubuntu — you can recover it.

Boot to the Ubuntu Live CD, open up a Console window and type:

# Create a directory called “mounted”:
mkdir mounted
# In there, mount the partition where Ubuntu is installed:
sudo mount /dev/sda2 mounted
# (For the above step you need to find the partition name on your own.)
# However, replace the /dev directory in the mounted partition with the
# current /dev directory:
sudo mount –bind /dev mounted/dev
# Change the root for now
sudo chroot mounted
# Finally, install grub to the first SATA disk:
sudo grub-install –recheck /dev/sda

For more information:

https://help.ubuntu.com/community/RecoveringUbuntuAfterInstallingWindows

https://wiki.ubuntu.com/phbc50/howtos/how-to_reinstall_grub

Favorite internet videos

Most of them at youtube...

Orchestration by Berlioz

This website contains excerpts from Berlioz's Grand Traité d’Instrumentation et d’Orchestration Modernes.

Grip, for ripping CDs

Grip is a great CD ripper/encoder for GNU/Linux. If you have never tried it, give it a go. It rips to wave using cdparanoia or others. It can simultaneously encode the ripped tracks to the format of your choice. Supports cddb too. Easy to use.

I usually encode to the ogg vorbis format (...)

Ogg Vorbis is much better than MP3 (roughly the same audio quality for half the file size) and is free of patents.

On Windows, Winamp has always been able to play ogg files. If you don't like Winamp there is plenty of other options too -- ogg vorbis really is the successor to MP3.

Autopackage: packaging software for multiple distros

Autopackage looks very useful for free software developers.

Zero Install is another perspective.

Where to change your MSN / Passport password

If you ever need to change your Passport / MSN password, you might lose half an hour searching for this function. It is buried here.