Planet Lisp

Gábor Melis: Micmac Initial Release

Fri, 05 Feb 2010 23:00:00 GMT

From a failed experiment today I salvaged Micmac, a statistical library wannabe, that for now only has Metropolis-Hastings MCMC and Metropolis Coupled MCMC implemented. The code doesn't weigh much but I think it gets the API right. In other news MGL v0.0.6 was released.

Ryan Davis: more heat-maps using vecto and ch-image

Fri, 05 Feb 2010 20:40:24 GMT

This is a follow-up to my post last year about simplistic heat-maps using Vecto. To recap, I’m trying to make heat maps for google maps overlays.

Here’s how it works in a nutshell:

From javascript I pass to the server the lat/lng region currently shown on the google map, and what size heat map to generate, in pixels.
lisp pulls weights from my database within the given lat/lng region
lisp iterates over the db results, mapping lat/lng to x/y coordinates for the final heat map image
lisp uses the list of mapped (x y weight) to draw the heat map in png
javascript throws the png on top of the google map

I tried a few things based upon the comments I got back from the helpful lisp community.

used zpng to get direct pixel access, and calculated each pixel’s color using a weighted average of nearby points using distance. This didn’t produce good images, and was pretty slow.
used zpng to get direct pixel access, and calculated each pixel’s color using the gravity formula against nearby points. This didn’t produce good images, and was very slow.

I did some more research and learned about the Generic Mapping Tools and bicubic interpolation. The GMT is a set of C programs, similar to the Imagemagick suite. GMT showed one way to draw heat maps in the Image Presentations tutorial. It spoke of gridded data sets, and that gave me one more vecto-based idea: split the desired heat-map into a grid and color each square in the grid based upon an average of the weights mapped in that square. This is a neat effect, but not what I was going for:

This is reasonably fast, taking about 1 second on my dev server. To quickly find what weights belong in which grid square, I make a spatial index of all the weights, using an r-tree from the spatial-trees library.

The next method I tried was to use interpolation to get a smooth look. I found Cyrus Harmon’s ch-image library supports image interpolation, and got to it. As Patrick Stein noted elsewhere, ch-image isn’t easy to install. It’s not asdf-installable, and the project page doesn’t list all its dependencies. For future reference, here’s what I think I needed to install it:

(asdf-install:install "http://cyrusharmon.org/static/releases/ch-asdf_0.2.14.tar.gz")
(asdf-install:install "http://cyrusharmon.org/static/releases/ch-util_0.3.10.tar.gz")
(asdf-install:install "http://cyrusharmon.org/static/releases/smarkup_0.4.2.tar.gz")
(asdf-install:install "http://mirror.its.uidaho.edu/pub/savannah/cl-bibtex/cl-bibtex-1.0.1.tar.gz")
(asdf-install:install "http://cyrusharmon.org/static/releases/clem_0.4.1.tar.gz")
(asdf-install:install "http://cyrusharmon.org/static/releases/ch-image_0.4.1.tar.gz")

Armed with ch-image, now the drawing process becomes:

draw a small image, coloring pixels based upon weights
enlarge the small image with interpolation

The first step is very similar to the code I wrote to make the grid version above. Instead of drawing a rectangle, I draw a pixel using ch-image’s pixel access functions. This was a little weird because ch-image’s coordinate system has 0,0 at the top left of the image. I’m still not sure how to best choose the size of this smaller image, but ultimately it should depend on my data. For now I just have it hard-coded be 20x smaller than the desired size:

Yep, that’s pretty small. Applying a transform to scale it up to the desired size using bilinear interpolation yields:

It looks pretty good and takes about a half-second to draw. If you click into the larger version, you can see some discontinuities in there, which is a well-known result of bilinear interpolation. However, based upon other graphics I’ve seen, what I really want is bicubic interpolation. Luckily, ch-image has this built in:

Oops, maybe not so luckily. I can certainly see the kinds of look I’m wanting in all the garbled stuff, but ch-image is freaking out somewhere there.

Bilinear it is! Here’s a screenshot of the overlay in place on the map:

It’s pretty fast, and looks pretty nice, and is fairly close to the look I wanted. I probably still have some off-by-one errors somewhere, and need to check the git repos for the ch-* libs to see if there might be newer versions than the tarballs I installed. I still count this as great progress for 5 hours of coding and research. Huzzah for the much-maligned lisp libraries!

Erik Winkels: Google AI Challenge 2010

Fri, 05 Feb 2010 15:43:00 GMT

For those who've missed it: Google AI Challenge 2010

There's no Common Lisp starter pack (edit: since this is getting misinterpreted I meant a CL starter pack for the AI Challenge) but after a particular nasty adventure spelunking for days through an archaic database at work, fighting through the 20 years of cruft that has been build up inside of it and finally finding and fixing the problem I decided that I needed a break and wrote an initial version of a Common Lisp (SBCL) client.

Perhaps someone with connections at the University of Waterloo and some work can finish it and make it an official starter pack? I've got to go back to work ;-)

Relevant links:

Lispjobs: Lisp Programmer (Raleigh, NC)

Thu, 04 Feb 2010 20:49:53 GMT

The candidate will be a well-versed Lisp developer with expert system expertise. Candidates do not have to have Rules experience for this role.
The development group consists of a small focused team of professionals, responsible for the design, development, and maintenance of the Convergence Point service. ConVergence Point is offered to clients through a 24×7x365 Software as a Service (SaaS) model.

Lispjobs: ITA jobs

Wed, 03 Feb 2010 21:57:28 GMT

It’s been a while since we mentioned ITA.

You might look at the full time and internship jobs at ITA that mention Lisp.

Update: currently, only one of positions listed, an internship, is primarily a Lisp job.

Lispjobs: Freelance Lispworks programmer, Holland

Wed, 03 Feb 2010 03:52:08 GMT

MBA Michael Bailey Associates zoekt “Developer”

Peter Van Eynde: CLC v7

Mon, 01 Feb 2010 20:38:31 GMT

With a lot of help from Faré and James Y Knight we have now the incredible disappearing common-lisp-controller v7. We went from 617 to 267 lines which in fact are only 193 lines excluding comments and empty lines...

Using the new features of cl-asdf version 1.500 and higher the implementation of CLC is now merely setting a few variables and putting a few files into place. As an added bonus I added clbuild integration, so now people can download and update libraries with ease.

Still todo:

there is an bug (#567912) that the first call to (clc:clc-require :<foo>) will fail with clisp due to a loop where there is no looping...
We still need to find a way to precompile libraries
Because asdf is now taking care of placing the fasl file in the correct place, we cannot remove fasls for a specific implementation or library, so we nuke them all. Not optimal.

.

Still a nice improvement, I hope to find some time at fosdem to work on the other issues...

Erik Winkels: A Black Tie and a Clysma

Mon, 01 Feb 2010 09:41:00 GMT

I've uploaded one-and-a-half project to GitHub that may or may not be useful to other people:

Black Tie packages the noise functions that were previously in Okra as a seperate library. It is not very extensive yet and has only been optimized for SBCL. At the moment it includes functions for generating Perlin noise, simplex noise and Voronoi diagrams.

Clysma is a toy project that at the moment mostly serves as an example of using CL-GTK2. I don't really know where I want to go with it but for now it allows you to stroll through a running CL session and do some sightseeing.

Andreas Fuchs: cl-beanstalk: A queue service client

Sun, 31 Jan 2010 16:49:52 GMT

Over the weekend, I wrote a little client library to a queue server that I’ve grown very fond of over the last year, beanstalk. It’s a very simple queue server, but it comes with a nice feature (delayed jobs) that I’ve had a use for recently.

The queue server is nicely engineered (written in C, works with queues a few million jobs deep), and very fast; it has guards in the protocol against worker failure, and it was a pleasure to implement: The whole thing is just 320 lines of code, including comments.

You can get the source (and a tiny example) at the cl-beanstalk github repository.

Hope this is useful for anyone else - I am planning on using this in autobench myself, to distribute work across several build hosts.

Andy Hefner: McPixel, a frivolous lisp hack.

Sun, 31 Jan 2010 07:26:03 GMT

Yesterday, I had the idea that it would be easy and entertaining to hack together a toy application for drawing and animating pixel art. Some hours later, McPixel was born. It provides an editable grid of pixels, a color palette, and allows you to string sequences of images together into animations. It's a toy, but someone might find it amusing. Obligatory screenshot:

Features:

80's retro McCLIM user interface.
No documentation or online help.
Maximally frustrating color sliders, using IHS colorspace.
Intuitive modeless drawing UI, achieved by including only a pencil tool.
Saves/loads files only in its own format, based on unportable s-expressions.
Requires X11 and SBCL with threads, to keep out the riffraff.
User-editable brush shape.
Remap Color and Silhouette commands, for tracing previous frames.
Realtime animated preview plays while you work.

I'm not sure I'll ever put this to productive use, if a productive use for such a thing even exists, but there you have it.

Update: I've added the ability to export an animated GIF file, using Xach Beane's Skippy library. Here's the "turtle.anim" example file, exported as a GIF: