OLPC's kernel hacker Dilinger, and other contributors, have written some scripts to build Debian images to run on the XO. The main changes involve grabbing a compatable kernel, installing a Forth script so that OpenFirmware can boot the kernel, and generate JFFS2 images suitable for flashing onto the XO's NAND.
You can get a copy of the scripts from git, as well as some pre-built Debain Lenny images containing Gnome, KDE, LXDE or Sugar desktop environments. There's also a bare image that contains a minimal Debian Lenny install, ready for installing the desktop environment of choice.
I've been using a DebXO image on my old B3 XO. It's been used for streaming video from it's webcam with gstreamer, serving webpages with Apache, and most recently it's become a DAAP music server, serving my music collection from a USB HDD over a wireless network. Despite the XO's limited compute power and RAM, it has been more than capable at these tasks.