Nicolas MartyanoffCounting lines with Common Lisp

· 7 days ago

A good line counting program has two features: it only counts non-empty lines to get a fair estimate of the size of a project, and it groups line counts by file type to help see immediately which languages are used.

A long time ago I got frustrated with two well known line counters. Sloccount spits out multiple strange Perl warnings about locales, and most of the output is a copyright notice and some absurd cost estimations. Cloc has fourteen Perl packages as dependencies. Writing a simple line counter is an interesting exercise; at the time I was discovering Common Lisp, so I wrote my own version.

I made a few changes years after years, but most of the code stayed the same. I thought it would be interesting to revisit this program and present it part by part as a demonstration of how you can use Common Lisp to solve a simple problem.

We are going to write the program bottom-up, starting with the smallest building blocks and progressively building upon them.

The program

The program is written in Common Lisp. The most convenient way of storing and executing it is a single executable file stored in a directory being part of the PATH environment variable. In my case, the script will be called locc, for “line of code counter”, and will be stored in the ~/bin directory.

We start the file with a shebang indicating how to execute the file. We use the SBCL implementation because it is stable and actively developed. It also makes it easy to execute a simple file:

#!/usr/bin/sbcl --script

Finding files

Our line counter will operate on directories, so it has to be able to list files in them. Path handling functions are very disconcerting at first. Common Lisp was designed a long time ago, and operating systems were different at the time. Let us dig in!

First let us write a simple function to check if a pathname object is a directory pathname:

(defun directory-path-p (path)
  "Return T if PATH is a directory or NIL else."
  (declare (type (or pathname string) path))
  (and (not (pathname-name path))
       (not (pathname-type path))))

Then we write a function to identify hidden files and directories since we do not want to include them:

(defun hidden-path-p (path)
  "Return T if PATH is a hidden file or directory or NIL else."
  (declare (type pathname path))
  (let ((name (if (directory-path-p path)
                  (car (last (pathname-directory path)))
                  (file-namestring path))))
    (and (plusp (length name))
         (eq (char name 0) #\.))))

As you can see we use DIRECTORY-PATH-P to extract the basename of the path, then check if it starts with a full stop (only if it is not empty of course).

Finally we can write the function to actually list files in a directory recursively:

(defun directory-path (path)
  "If PATH is a directory pathname, return it as it is. If it is a file
pathname or a string, transform it into a directory pathname."
  (declare (type (or pathname string) path))
  (if (directory-path-p path)
      (make-pathname :directory (append (or (pathname-directory path)
                                            (list :relative))
                                        (list (file-namestring path)))
                     :name nil :type nil :defaults path)))

(defun find-files (path)
  "Return a list of all files contained in the directory at PATH or any of its
  (declare (type pathname path))
  (flet ((list-directory (path)
            (make-pathname :defaults (directory-path path)
                           :type :wild :name :wild))))
    (let ((paths nil)
          (children (list-directory (directory-path path))))
      (dolist (child children paths)
        (unless (hidden-path-p child)
          (if (directory-path-p child)
              (setf paths (append paths (find-files child)))
              (push child paths)))))))

We use the DIRECTORY standard function with a path containing a wildcard component to list the files in a directory, and do so recursively.

Counting lines

Now that we have files, we can start counting lines. Let us first write a function to count the number of non-empty lines in a file.

(defun count-file-lines (path)
  "Count the number of non-empty lines in the file at PATH. A line is empty if
it only contains space or tabulation characters."
  (declare (type pathname path))
  (with-open-file (stream path :element-type '(unsigned-byte 8))
    (do ((nb-lines 0)
         (blank-line t))
      (let ((octet (read-byte stream nil)))
          ((or (null octet) (eq octet #.(char-code #\Newline)))
           (unless blank-line
             (incf nb-lines))
           (when (null octet)
             (return-from count-file-lines nb-lines))
           (setf blank-line t))
          ((and (/= octet #.(char-code #\Space))
                (/= octet #.(char-code #\Tab)))
           (setf blank-line nil)))))))

We open the file to obtain a steam of octets, and read it octet by octet, keeping track of whether the current line is blank or not. Note how we make sure to count the last line even if it does not end with a newline character.

Reading a file one octet could be a disaster for performances. Fortunately SBCL file streams are buffered, something we can easily check by running our program with strace -e trace=openat,read. We would not rely on this property if we wanted our program to work on multiple Common Lisp implementations, but this is a non issue here.

Identifying the file type

Counting lines is one thing, but we need to identify their content. The simplest way is to do so based on the file extension.

Obviously we will want to ignore various files which are known not to contain text content, so we start by building a hash table containing these extensions:

(defparameter *ignored-extensions*
  (let ((extensions '("a" "bin" "bmp" "cab" "db" "elc" "exe" "gif" "gz"
                      "jar" "jpeg" "jpg" "o" "pcap" "pdf" "png" "ps" "rar"
                      "svg" "tar" "tgz" "tiff" "zip"))
        (table (make-hash-table :test 'equal)))
    (dolist (extension extensions table)
      (setf (gethash extension table) t)))
  "A hash table containing all file extensions to ignore.")

We then create another hash table to associate a type symbol to each known file extension:

(defparameter *extension-types*
  (let ((pairs '(("asm" . assembly) ("s" . assembly)
                 ("adoc" . asciidoc)
                 ("awk" . awk)
                 ("h" . c) ("c" . c)
                 ("hpp" . cpp) ("cpp" . cpp) ("cc" . cpp)
                 ("css" . css)
                 ("el" . elisp)
                 ("erl" . erlang)
                 ("go" . go)
                 ("html" . html) ("htm" . html)
                 ("ini" . ini)
                 ("hs" . haskell)
                 ("java" . java)
                 ("js" . javascript)
                 ("json" . json)
                 ("tex" . latex)
                 ("lex" . lex)
                 ("lisp" . lisp)
                 ("mkd" . markdown) ("md" . markdown)
                 ("rb" . ruby)
                 ("pl" . perl) ("pm" . perl)
                 ("php" . php)
                 ("py" . python)
                 ("sed" . sed)
                 ("sh" . shell) ("bash" . shell) ("csh" . shell)
                 ("zsh" . shell) ("ksh" . shell)
                 ("scm" . scheme)
                 ("sgml" . sgml)
                 ("sql" . sql)
                 ("texi" . texinfo)
                 ("texinfo" . texinfo)
                 ("vim" . vim)
                 ("xml" . xml) ("dtd" . xml) ("xsd" . xml)
                 ("yaml" . yaml) ("yml" . yaml)
                 ("y" . yacc)))
        (table (make-hash-table :test 'equal)))
    (dolist (pair pairs table)
      (setf (gethash (car pair) table) (cdr pair))))
  "A hash table containing a symbol identifying the type of a file for
  each known file extension.")

With these hash tables, the function identifying the type of a file is trivial:

(defun identify-file-type (path)
  "Return a symbol identifying the type of the file at PATH, or UNKNOWN if the
file extension is not known."
  (declare (type pathname path))
  (let ((extension (pathname-type path)))
    (unless (gethash extension *ignored-extensions*)
      (gethash extension *extension-types* 'unknown))))

Collecting file information

Up to this point, we wrote several functions without connecting them. But we now have all the building blocks we need. Let us use them to accumulate information about the files in a list of directories.

(defun collect-line-counts (directory-paths)
  "Collect the line count of all files in the directories located at one of the
paths in DIRECTORY-PATHS and return them grouped by file type as an
association list."
  (declare (type list directory-paths))
  (let ((line-counts (make-hash-table)))
    (dolist (directory-path directory-paths)
      (dolist (path (find-files directory-path))
            (let ((type (identify-file-type path)))
              (when (and type (not (eq type 'unknown)))
                (let ((nb-lines (count-file-lines path)))
                  (incf (gethash type line-counts 0) nb-lines))))
          (error (condition)
            (format *error-output* "~&error while reading ~A: ~A~%"
                    path condition)))))
    (let ((line-count-list nil))
      (maphash (lambda (type nb-lines)
                 (push (cons type nb-lines) line-count-list))

We get to use all previous functions. We iterate through directory paths to find non-hidden files using FIND-FILES, then we use IDENTIFY-FILE-TYPE to obtain a type symbol and COUNT-FILE-LINES to count the number of non-empty lines in each file. Results are accumulated by file type in the LINE-COUNTS hash table. During this process, we handle errors that may occur while reading files with a message on the error output. Finally we transform the hash table into an association list and return it.

Presenting results

Executing all these functions in the Lisp REPL is quite practical during developement, but the default pretty printer is not really what you expect for the final command line tool:

REPL output

So let us write a function to format this association list:

(defun format-line-counts (line-counts &key (stream *standard-output*))
  "Format the line counts in the LINE-COUNTS association list to STREAM."
  (declare (type list line-counts)
           (type stream stream))
  (dolist (entry (sort line-counts '> :key 'cdr))
    (let ((type (car entry))
          (nb-lines (cdr entry)))
      (format stream "~12A  ~8@A~%"
              (string-downcase (symbol-name type)) nb-lines))))

We print the list sorted in descending order, meaning that the file type with the most number of lines comes first. Of course the output is padded to make sure numbers are all aligned.

Finalizing the program

The only thing left to do is the entry point of the script. This is the only place where we need to call a non-standard function in order to access command line arguments. If no command line argument were passed to the script, we look for files in the current directory.

(let ((paths (or (cdr sb-ext:*posix-argv*) '("."))))
   (collect-line-counts paths)))

Shell output

Much better!

There does not seem to be any performance issue: on the Linux kernel source tree, this program is almost 11 times faster than sloccount. It would be interesting to profile the program to make sure IO is the bottleneck and improve inefficient parts, but this code is fast enough for my needs.

As you can see, it is not that hard to use Common Lisp to solve problems.

Marco AntoniottiThe LETV and LETV* Macros

· 15 days ago


It has been a long time since I posted something here.  I have been busy with my day job and bogged down in a major rewrite of something (more on this hopefully very soon now (tm)) that is full of rabbit's holes.

I was able to get out of one of these rabbit's holes with this little hack I cooked up that allows you to, possibly, write more concise code.

This little hack introduces two handy Common Lisp macros, LETV and LETV* that allow you to mix regular LET and MULTIPLE-VALUE-BIND forms in a less verbose way. The amount of indentation needed is also reduced.

The syntax of LETV (or LETV*) is very "loopy", with a nod to SML/OCaml/F#/Haskell/Julia. The current syntax is the following:

    letv     ::= 'LETV' [vars] [IN [body]]
    letvstar ::= 'LETV*' [vars] [IN [body]]

    vars     ::= var | var vars

    var      ::= ids '=' <form> [decls]

    decls    ::= decl | decl decls
    decl     ::= OF-TYPE idtypes

    ids      ::= <symbol> | '(' <symbol> + ')'
    idtypes  ::= <type designator> | '(' <type designator> + ')'

    body     ::= [<declarations>] <form> * 

(I know: the grammar is not completely kosher, but I trust you will understand it.)

The two macros expand in forms that mimic the semantics of let, LET* and MULTIPLE-VALUE-BIND. All type declarations, if present, are properly handled via locally forms. LETV expands into a LET, with variable initialized by PSETQs and multiple-VALUE-SETQs. LETV* expands into a form that is an interleaving of LET* and MULTIPLE-VALUE-BIND.

The library exports only the two symbols LETV and LETV*. The other "symbols" mentioned (=, IN, OF-TYPE) are checked as in LOOP; therefore you can use different styles to write your code, as you would when writing LOOPs.

The library is available here.


  1. Simple case:
      (letv x = 42 in (format t "The answer is ~D.~%" x))
  2. Same with declaraion:
      (letv x = 42 of-type fixnum
    	in (format t "The answer is ~D." x))
  3. Simple case with MULTIPLE-VALUE-BIND:
      (letv (v found) = (gethash 'the-key *my-hash-table*)
            in (if found
                   (format t "Found THE-KEY, doing stuff.~%")
                   (error "THE-KEY not found.")))
  4. Mixing things up:
      (letv (v found) = (gethash 'the-key *my-hash-table*)
                      of-type (fixnum boolean)
             x = 42
             in (if found
                     (format t "Found THE-KEY, adding to the answer ~D.~%"
                             (+ v x))
                     (error "THE-KEY not found.")))
  5. With LETV*:
      (letv* (v found) = (gethash 'the-key *my-hash-table*)
                       of-type (fixnum boolean)
             x = (when found (+ v 42))
             in (if found
                    (format t "Found THE-KEY, adding to the answer ~D.~%"
                    (error "THE-KEY not found.")))
  6. The other way around.
      (letv x = 42 of-type integer
            (v found) = (gethash 'the-key *my-hash-table*)
                      of-type (fixnum boolean)
            in (if found
                   (format t "Found THE-KEY, adding to the answer ~D.~%"
                           (+ v x))
                   (error "THE-KEY not found.")))
  7. A more compelling example.
      (letv* (p found) = (gethash 'the-key *points-table*) of-type (point)
             (x y) = (if found
                         (unpack-point p)
                         (values :missing :missing))
             (declare (type point p)
                      (type real x y))  ; Adding declarations here also works.
             (do-stuff-with x y)
             (do-things-with p))

All the examples are meant to illustrate the use of LETV and LETV*.


LETV and LETV* are obviously not the first macros of this kind floating around: others are available and all have their niceties. But I never claimed not to suffer from NIH syndrome.

LETV and LETV* do not do "destructuring" or "pattern matching". That is a different can of worms; but you can check cl-unification for a library (always by yours truly) that provides facilities in that sense.


Nicolas HafnerLevel Editor Update is Live!

· 16 days ago

The first major update for Kandria is now live on all platforms! It includes the level editor, a modding system, some new sample levels, and bugfixes!

Level Editor

The level editor received a big overhaul and is now a lot more accessible. There's also official documentation now to help get you started and explain all the tools and shortcuts available. If you want to dig in, just update your game and navigate to Mod Manager > Create Mod, which should place you directly into the editor in a new world of your own!

Sharing Mods

You can play levels made by others and download them directly from within the game. To browse existing levels, simply go to Mod Manager > Discover. From there you can select and install mods. Once installed, you can play their world under the Worlds tab.

We've provided a number of sample levels for you to play and edit. Open them up and see how they work, maybe you'll get inspired to make your own!

Join in on our Discord community event! We'll raffle out 20 Steam keys for Kandria to anyone that uploads a new level. You can freely give the key out to a friend so they can play the level you made!


Here's the detailed changelog for this version:

  • Fix issues with multiple worlds corrupting save files
    Reported by Dieting Hippo

  • Fix issue with animation editor crashing the game
    Reported by Jeremiah Fieldhaven

  • Fix issue with history dialog crashing the game
    Reported by Dieting Hippo

  • Fix issue with placing lava in an empty world crashing the game
    Reported by Hempuli

  • Fix issue with floating doors crashing the game on navmesh construction
    Reported by Hempuli

  • Fix issue with world field edits leading to a crash on editor exit
    Reported by Hempuli

  • Fix issue with text alignment on mixed fonts like Nicokaku
    Reported by Pir

  • Add double-click tile selection
    Reported by Hempuli

  • Make Ctrl+Z work, not just Z
    Reported by Dieting Hippo

  • Select entities on mouse down rather than up
    Reported by Hempuli

  • Fix rectangle tool being off by one at edges
    Reported by Hempuli

  • Fix roaming NPCs hard-setting their velocity, preventing bounce and other velocity based mechanics
    Reported by Hempuli

  • Hide the bsize field from edits as its duplicity with size is confusing
    Reported by Hempuli

  • Don't overwrite the keymap.lisp if the shipped one is never, merge them instead.
    Reported by Hempuli

  • Fix multi-tile picking being screwed when not selecting tiles from bottom left to top right
    Reported by Hempuli, Dieting Hippo

  • Fix moving platforms moving faster when climbing them

  • Fix default mod preview image size
    Reported by Jeremiah Fieldhaven

  • Fix various minor usability issues
    Reported by Jeremiah Fieldhaven

  • Fix issue related to spawning of child entities in new worlds
    Reported by Tim White

  • Fix input field for spawner types
    Reported by Dieting Hippo

  • Fix mod dependency management
    Reported by Jeremiah Fieldhaven

  • Fix issues with the history dialog
    Reported by Dieting Hippo

  • Fix issues with the animation editor
    Reported by Jeremiah Fieldhaven

  • Fix issue related to empty GI
    Reported by Goldberg

There's a bunch more changes that were done internally. If you would like to see that kind of detail, you can check the source code repository.

Nicolas HafnerNext Kandria Update on March 8th!

· 23 days ago

Let's get the important news out of the way: the next major update for Kandria will launch on Wednesday, March 8th, at 15:00 CET. The update will include a number of improvements, new features, more content, and a community event!

Level Editing

The biggest part of the update is, no doubt, the polished level editor. The editor has always been a part of Kandria since release, but now it's nicely polished and much easier to use!

I've also included documentation to make it much easier for you to get a handle on how the editor works and how to get started with using it. I'm very excited to see what people can come up with!

And yes, the entire Kandria world was created in this editor, so it is fully capable of creating some very complex levels!

Partial Modding Support

But, the editor itself is not all by far. Not only can you create levels, you can also browse levels others have made easily directly from within the game, and upload your own as well! We've integrated with the great service to make levels and mods available to everyone, regardless of how they purchased Kandria.

Worlds are distributed as just another kind of mod, so the UI for browsing and managing mods has also been developed already. And, if you're very adventurous you can even get started with code mods for Kandria, too.

More Content

In order to give you some idea of what you can do with the editor, and to give more casual players something new to nibble on, too, I've also included a bunch small, new worlds as well.

You can simply download those through the mod manager's discovery tab when the update is out!

Community Event: Dare to Share

Finally, we will be holding a community content event: we will raffle out 20 Steam keys for Kandria to anyone that uploads a new level. You can freely give the key out to a friend so they can play the level you made!

If you're interested in participating, please join our Discord. We'll be sharing more details on how exactly the raffle will work closer to the release of the modding support.


Kandria will be the featured game for Speed Bump this month! A couple of speedrunners will be taking a closer look at the game and competing for time. Some runs have already started to appear on our leaderboard, as well! As a speedrunning fan and amateur runner myself, I'm really excited to see what kinds of tech they can figure out and how far they can optimise the run. The time has already come down quite considerably from my initial estimates, and I can definitely see things becoming quite competitive in the future.

Closing Thoughts

I'll have more news about the future development of Kandria for you next month. Suffice to say, we're not done yet! For now, just remember that the update will launch on Wednesday, March 8th, at 15:00 CET along with a 15% discount on Steam. Please continue to share the page with your friends and communities!

Tim BradshawTwo tiny Lisp evaluators

· 25 days ago

Everyone who has written Lisp has written tiny Lisp evaluators in Lisp: here are two more.

Following two recent articles I wrote on scope and extent in Common Lisp, I thought I would finish with two very tiny evaluators for dynamically and lexically bound variants on a tiny Lisp.

The language

The tiny Lisp these evaluators interpret is not minimal: it has constructs other than lambda, and even has assignment. But it is pretty small. Other than the binding rules the languages are identical.

  • λ & lambda are synonyms and construct procedures, which can take any number of arguments;
  • quote quotes its argument;
  • if is conditional expression (the else part is optional);
  • set! is assignment and mutates a binding.

That is all that exists.

Both evaluators understand primitives, which are usually just functions in the underlying Lisp: since the languages are Lisp–1s, you could also expose other sorts of things of course (for instance true and false values). You can provide a list of initial bindings to them to define useful primitives.


Both evaluators rely on my iterate and spam hacks: they could easily be rewritten not to do so.

The dynamic evaluator

A procedure is represented by a structure which has a list of formals and a body of one or more forms.

(defstruct (procedure
             (lambda (p s d)
               (declare (ignore d))
               (print-unreadable-object (p s)
                 (format s "λ ~S" (procedure-formals p))))))
  (formals '())
  (body '()))

The evaluator simply dispatches on the type of thing and then on the operator for compound forms.

(defun evaluate (thing bindings)
  (typecase thing
     (let ((found (assoc thing bindings)))
       (unless found
         (error "~S unbound" thing))
       (cdr found)))
     (destructuring-bind (op . arguments) thing
       (case op
         ((lambda λ)
          (matching arguments
            ((head-matches (list-of #'symbolp))
             (make-procedure :formals (first arguments)
                             :body (rest arguments)))
             (error "bad lambda form ~S" thing))))
          (matching arguments
            ((list-matches (any))
             (first arguments))
             (error "bad quote form ~S" thing))))
          (matching arguments
            ((list-matches (any) (any))
             (if (evaluate (first arguments) bindings)
                 (evaluate (second arguments) bindings)))
            ((list-matches (any) (any) (any))
             (if (evaluate (first arguments) bindings)
                 (evaluate (second arguments) bindings)
               (evaluate (third arguments) bindings)))
             (error "bad if form ~S" thing))))
          (matching arguments
            ((list-matches #'symbolp (any))
             (let ((found (assoc (first arguments) bindings)))
               (unless found
                 (error "~S unbound" (first arguments)))
               (setf (cdr found) (evaluate (second arguments) bindings))))
             (error "bad set! form ~S" thing))))
          (applicate (evaluate (first thing) bindings)
                     (mapcar (lambda (form)
                               (evaluate form bindings))
                             (rest thing))
    (t thing)))

The interesting thing here is that applicate needs to know the current set of bindings so it can extend them dynamically.

Here is applicate which has a case for primitives and procedures

(defun applicate (thing arguments bindings)
  (etypecase thing
     ;; a primitive
     (apply thing arguments))
     (iterate bind ((vtail (procedure-formals thing))
                    (atail arguments)
                    (extended-bindings bindings))
        ((and (null vtail) (null atail))
         (iterate eval-body ((btail (procedure-body thing)))
           (if (null (rest btail))
               (evaluate (first btail) extended-bindings)
               (evaluate (first btail) extended-bindings)
               (eval-body (rest btail))))))
        ((null vtail)
         (error "too many arguments"))
        ((null atail)
         (error "not enough arguments"))
         (bind (rest vtail)
               (rest atail)
               (acons (first vtail) (first atail)

The thing that makes this evaluator dynamic is that the bindings that applicate extends are those it was given: procedures do not remember bindings.

The lexical evaluator

A procedure is represented by a structure as before, but this time it has a set of bindings associated with it: the bindings in place when it was created.

(defstruct (procedure
             (lambda (p s d)
               (declare (ignore d))
               (print-unreadable-object (p s)
                 (format s "λ ~S" (procedure-formals p))))))
  (formals '())
  (body '())
  (bindings '()))

The evaluator is almost identical:

(defun evaluate (thing bindings)
  (typecase thing
     (let ((found (assoc thing bindings)))
       (unless found
         (error "~S unbound" thing))
       (cdr found)))
     (destructuring-bind (op . arguments) thing
       (case op
         ((lambda λ)
          (matching arguments
            ((head-matches (list-of #'symbolp))
             (make-procedure :formals (first arguments)
                             :body (rest arguments)
                             :bindings bindings))
             (error "bad lambda form ~S" thing))))
          (matching arguments
            ((list-matches (any))
             (first arguments))
             (error "bad quote form ~S" thing))))
          (matching arguments
            ((list-matches (any) (any))
             (if (evaluate (first arguments) bindings)
                 (evaluate (second arguments) bindings)))
            ((list-matches (any) (any) (any))
             (if (evaluate (first arguments) bindings)
                 (evaluate (second arguments) bindings)
               (evaluate (third arguments) bindings)))
             (error "bad if form ~S" thing))))
          (matching arguments
            ((list-matches #'symbolp (any))
             (let ((found (assoc (first arguments) bindings)))
               (unless found
                 (error "~S unbound" (first arguments)))
               (setf (cdr found) (evaluate (second arguments) bindings))))
             (error "bad set! form ~S" thing))))
          (applicate (evaluate (first thing) bindings)
                     (mapcar (lambda (form)
                               (evaluate form bindings))
                             (rest thing)))))))
    (t thing)))

The differences are that when constructing a procedure the current bindings are recorded in the procedure, and it is no longer necessary to pass bindings to applicate.

applicate is also almost identical:

(defun applicate (thing arguments)
  (etypecase thing
     ;; a primitive
     (apply thing arguments))
     (iterate bind ((vtail (procedure-formals thing))
                    (atail arguments)
                    (extended-bindings (procedure-bindings thing)))
        ((and (null vtail) (null atail))
         (iterate eval-body ((btail (procedure-body thing)))
           (if (null (rest btail))
               (evaluate (first btail) extended-bindings)
               (evaluate (first btail) extended-bindings)
               (eval-body (rest btail))))))
        ((null vtail)
         (error "too many arguments"))
        ((null atail)
         (error "not enough arguments"))
         (bind (rest vtail)
               (rest atail)
               (acons (first vtail) (first atail)

The difference is that the bindings it extends when binding arguments are the bindings which the procedure remembered, not the dynamically-current bindings, which it does not even know.

The difference between them

Here is the example that shows how these two evaluators differ.

With the dynamic evaluator:

? ((λ (f)
     ((λ (x)
        ;; bind x to 1 around the call to f
   ((λ (x)
      ;; bind x to 2 when the function that will be f is created
      (λ () x))

The binding in effect is the dynamically current one, not the one that was in effect when the procedure was created.

With the lexical evaluator:

? ((λ (f)
     ((λ (x)
        ;; bind x to 1 around the call to f
   ((λ (x)
      ;; bind x to 2 when the function that will be f is created
      (λ () x))

Now the binding in effect is the one that existed when the procedure was created.

Something more interesting is how you create recursive procedures in the lexical evaluator. With suitable bindings for primitives, it’s easy to see that this can’t work:

((λ (length)
   (length '(1 2 3)))
 (λ (l)
   (if (null? l)
       (+ (length (cdr l)) 1))))

It can’t work because length is not in scope in the body of length. it will work in the dynamic evaluator.

The first fix, which is similar to what Scheme does with letrec, is to use assignment to mutate the binding so it is correct:

((λ (length)
   (set! length (λ (l)
                  (if (null? l)
                      (+ (length (cdr l)) 1))))
   (length '(1 2 3)))

Note the initial value of length is never used.

The second fix is to use something like the U combinator (you could use Y of course: I think U is simpler to understand):

((λ (length)
   (length '(1 2 3)))
 (λ (l)
   ((λ (c)
      (c c l 0))
    (λ (c t s)
      (if (null? t)
          (c c (cdr t) (+ s 1)))))))

Source code

These two evaluators, together with a rudimentary REPL which can use either of them, can be found here.

Tim BradshawDynamic binding without special in Common Lisp

· 25 days ago

In Common Lisp, dynamic bindings and lexical bindings live in the same namespace. They don’t have to.

Common Lisp has two sorts of bindings for variables: lexical binding and dynamic binding. Lexical binding has lexical scope — the binding is available where it is visible in source code — and indefinite extent — the binding is available as long as any code might reference it. Dynamic binding has indefinite scope — the binding is available to any code which runs between when the binding is established and when control leaves the form which established it — and dynamic extent — the binding ceases to exist when control leaves the binding form.

These are really two very different things. However CL places both of these kinds of bindings into the same namespace, relying on special declarations and proclamations to tell the system which sort of binding to create and reference for a given name.

That doesn’t have to be the case: it’s possible in CL to completely isolate these two namespaces from each other. This means you could write code where all variable references were to lexical bindings and where dynamic bindings were created and referenced by a completely different set of operators. Here is an example of that. Following practice in some old Lisps I will call this ‘fluid’ binding. I will also use / to delimit the names of fluid variables simply to distinguish them from normal variables.

(defun inner (varname value)
  (setf (fluid-value varname) value))

(defun outer (varname value)
   (lambda ()
      (fluid-value varname)
        (inner varname (1+ value))
        (fluid-value varname))))
   (list varname)
   (list value)))

And now

> (outer '/v/ 1)

Here are a set of operators for dealing with these fluid variables:

fluid-value accesses the value of a fluid variable.

fluid-boundp tells you if a name is bound as a fluid variable.

call/fluid-bindings calls a function with one or more fluid variables bound.

define-fluid (not used above) defines a global value for a fluid variable.

Well, of course you can do something like this using an explicit binding stack and a single special variable to hang it from. But that’s not how this works: these ‘fluid variables’ are just CL’s dynamic variables:

(defun call/print-base (f base)
  (call/fluid-bindings  f '(*print-base*) (list base)))
> (call/print-base
   (lambda ()

So how does this work? Well fluid-value and fluid-boundp are obvious:

(defun fluid-value (s)
  (symbol-value s))

(defun (setf fluid-value) (n s)
  (setf (symbol-value s) n))

(defun fluid-boundp (s)
  (boundp s))

And the trick now is that CL gives you enough mechanism to bind named dynamic variables yourself, that mechanism being progv, which

[…] allows binding one or more dynamic variables whose names may be determined at run time […]

So now call/fluid-bindings just uses progv:

(defun call/fluid-bindings (f fluids values)
  (progv fluids values (funcall f)))

And finally define-fluid looks like this:

(defmacro define-fluid (var &optional (value nil)
                            (doc nil docp))
     (setf (fluid-value ',var) ,value)
     ,@(if docp
           `((setf (documentation ',var 'variable) ',doc))

The interesting thing here is that there are no special declarations or proclamations: you can create and bind new fluid variables without any recourse to special at all, in a way which is completely compatible with the existing dynamic variables, because fluid variables are dynamic variables.

So one way of thinking about special is that it is a declaration that says ‘for this variable name, access the namespace of dynamic bindings rather than lexical bindings’. This is not really what special was of course in Lisps before CL — it was historically closer to an instruction to use the interpreter’s variable binding mechanism in compiled code — but you can think of it this way in CL, where the interpreter and compiler do not have separate binding rules.

And, of course, using something like the above, you could write code in CL where all variable bindings were lexical and dynamic variables lived entirely in their own namespace. For instance this works fine:

(defun f ()
  (let ((x 2))
     (lambda ()
       (values x (fluid-value 'x)))
     '(x) '(3))))
> (f)

The reference to x as a variable refers to its lexical binding, while (fluid-value 'x) refers to its dynamic binding.

Whether writing code like that would be useful I am not sure: I think that the *-convention for dynamic variables is perfectly fine in fact. But it is perhaps interesting to see that you can think of dynamic bindings in CL this way.

Nicolas MartyanoffCustom Font Lock configuration in Emacs

· 28 days ago

Font Lock is the builtin Emacs minor mode used to highlight textual elements in buffers. Major modes usually configure it to detect various syntaxic constructions and attach faces to them.

The reason I ended up deep into Font Lock is because I was not satisfied with the way it is configured for lisp-mode, the major mode used for both Common Lisp and Emacs Lisp code. This forced me to get acquainted with various aspects of Font Lock in order to change its configuration. If you want to change highlighting for your favourite major mode, you will find this article useful.

Common Lisp highlighting done wrong

The core issue of Common Lisp highlighting in Emacs is that a lot of it is arbitrary and inconsistent:

  • The mode highlights what it calls “definers” and “keywords”, but it does not really make sense in Common Lisp. Why would WITH-OUTPUT-TO-STRING be listed as a keyword, but not CLASS-OF?
  • SIGNAL uses font-lock-warning-face. Why would it be a warning? Even stranger, why would you use this warning face for CHECK-TYPE?
  • Keywords and uninterned symbols are all highlighted with font-lock-builtin-face. But they are not functions or variables. They are not even special in any way, and their syntax already indicates clearly their nature. Having so many yellow symbols everywhere is really distracting.
  • All symbols starting with & are highlighted using font-lock-type-face. But lambda list arguments are not types, and symbols starting with & are not always lambda list arguments.
  • All symbols preceded by ( whose name starts with DO- or WITH- are highlighted as keywords. There is even a comment by RMS stating that it is too general. He is right.

Beyond these issues, the mode sadly uses default Font Lock faces instead of defining semantically appropriate faces and mapping them to existing ones as default values.

The chances of successfully driving this kind of large and disruptive change directly into Emacs are incredibly low. Even if it was to be accepted, the result would not be available until the next release, which could mean months. Fortunately, Emacs is incredibly flexible and we can change all of this ourselves.

Note that you may not agree with the list of issues above, and this is fine. The point of this article is to show you how you can change the way Emacs highlights content in order to match your preferences. And you can do that for all major modes!

Font Lock configuration

Font Lock always felt a bit magic and it took me some time to find the motivation to read the documentation. As is turned out, it can be used for very complex highlighting schemes, but basic features are not that hard to use.

The main configuration of Font Lock is stored in the font-lock-defaults buffer-local variable. It is a simple list containing the following entries:

  • A list of symbols containing the value to use for font-lock-keywords at each level, the first symbol being the default value.
  • The value used for font-lock-keywords-only. If it is nil, it enables syntaxic highlighting (strings and comments) in addition of search-based (keywords) highlighting.
  • The value used for font-lock-keywords-case-fold-search. If true, highlighting is case insensitive.
  • The value used for font-lock-syntax-table, the association list controlling syntaxic highlighting. If it is nil, Font Lock uses the syntax table configured with set-syntax-table. In lisp-mode this would mean lisp-mode-syntax-table.
  • All remaining values are bindings using the form (VARIABLE-NAME . VALUE) used to set buffer-local values for other Font Lock variables.

The part we are interested about is search-based highlighting which uses regular expressions to find specific text fragments and attach faces to them.

Values used for font-lock-keywords are also lists. Each element is a construct used to specify one or more keywords to highlight. While these constructs can have multiple forms for more complex use cases, we will only use the two simplest ones:

  • (REGEXP . FACE) tells Font Lock to use FACE for text fragments which match REGEXP. For example, you could use ("\\_<-?[0-9]+\\_>" . font-lock-constant-face) to highlight integers as constants (note the use of \_< and \_> to match the start and end of a symbol; see the regexp documentation for more information).
  • (REGEXP (GROUP FACE)...) is a bit more advanced. When REGEXP matches a subset of the buffer, Font Lock assigns faces to the capture group identified by their number. You could use this construction to detect a complex syntaxic element and highlight some of its parts with different faces.

Simplified Common Lisp highlighting

We are going to configure keyword highlighting for the following types of values:

  • Character literals, e.g. #\Space.
  • Function names in the context of a function call for standard Common lisp functions.
  • Standard Common Lisp values such as *STANDARD-OUTPUT* or PI.

Additionally, we want to keep the default syntaxic highlighting configuration which recognizes character strings, documentation strings and comments.


Let us start by defining new faces for the different values we are going to match:

(defface g-cl-character-face
  '((default :inherit font-lock-constant-face))
  "The face used to highlight Common Lisp character literals.")

(defface g-cl-standard-function-face
  '((default :inherit font-lock-keyword-face))
  "The face used to highlight standard Common Lisp function symbols.")

(defface g-cl-standard-value-face
  '((default :inherit font-lock-variable-name-face))
  "The face used to highlight standard Common Lisp value symbols.")

Nothing complicated here, we simply inherit from default Font Lock faces. You can then configure these faces in your color theme without affecting other modes using Font Lock.


To detect standard Common Lisp functions and values, we are going to need a regular expression. The first step is to build a list of strings for both functions and values. Easy to do with a bit of Common Lisp code!

(defun standard-symbol-names (predicate)
  (let ((symbols nil))
    (do-external-symbols (symbol :common-lisp)
      (when (funcall predicate symbol)
        (push (string-downcase (symbol-name symbol)) symbols)))
    (sort symbols #'string<)))
(standard-symbol-names #'fboundp)
(standard-symbol-names #'boundp)

The STANDARD-SYMBOL-NAMES build a list of symbols exported from the :COMMON-LISP package which satisfy a predicate. The first call gives us the name of all symbols bound to a function, and the second all which are bound to a value.

The astute reader will immediately wonder about symbols which are bound both a function and a value. They are easy to find by calling INTERSECTION on both sets of names: +, /, *, -. It is not really a problem: we can highlight function calls by matching function names preceded by (, making sure that these symbols will be correctly identified as either function symbols or value symbols depending on the context.

We store these lists of strings in the g-cl-function-names and g-cl-value-names (the associated code is not reproduced here: these lists are quite long; but I posted them as a Gist).

With this lists, we can use the regexp-opt Emacs Lisp function to build optimized regular expressions matching them:

(defvar g-cl-font-lock-keywords
  (let* ((character-re (concat "#\\\\" lisp-mode-symbol-regexp "\\_>"))
         (function-re (concat "(" (regexp-opt g-cl-function-names t) "\\_>"))
         (value-re (regexp-opt g-cl-value-names 'symbols)))
    `((,character-re . 'g-cl-character-face)
       (1 'g-cl-standard-function-face))
      (,value-re . 'g-cl-standard-value-face))))

Characters literals are reasonably easy to match.

Functions are a bit more complicated since we want to match the function name when it is preceded by an opening parenthesis. We use a capture capture (see the last argument of regexp-opt) for the function name and highlight it separately.

Values are always matched as full symbols: we do not want to highlight parts of a symbol, for example MAP in a symbol named MAPPING.

Final configuration

Finally we can define the variable which will be used for font-lock-defaults in the initialization hook; we copy the original value from lisp-mode, and change the keyword list for what is going to be our own configuration:

(defvar g-cl-font-lock-defaults
    nil                                 ; enable syntaxic highlighting
    t                                   ; case insensitive highlighting
    nil                                 ; use the lisp-mode syntax table
    (font-lock-mark-block-function . mark-defun)
    (font-lock-extra-managed-props help-echo)
	 . lisp-font-lock-syntactic-face-function)))

To configure font-lock-defaults, we simply set it in the initialization hook of lisp-mode:

(defun g-init-lisp-font-lock ()
  (setq font-lock-defaults g-cl-font-lock-defaults))
(add-hook 'lisp-mode-hook 'g-init-lisp-font-lock)


Let us compare highlighting for a fragment of code before and after our changes:

Before After

The differences are subtle but important:

  • All standard functions are highlighted, helping to distinguish them from user-defined functions.
  • Standard values such as *ERROR-OUTPUT* are highlighted.
  • Character literals are highlighted the same way as character strings.
  • Keywords are not highlighted anymore, avoiding the confusion with function names.


That was not easy; but as always, the effort of going through the documentation and experimenting with different Emacs components was very rewarding. Font Lock does not feel like a black box anymore, opening the road for the customization of other major modes.

In the future, I will work on a custom color scheme to use more subtle colors, with the hope of reducing the rainbow effect of so many major modes, including lisp-mode.

ABCL DevABCL 1.9.1 "never use a dot oh"

· 29 days ago

    If one has been hesitating about using the latest ABCL because one "never uses a dot oh release", we have now sloughed off abcl-1.9.1 for  your appraisal from the depths of a Bear's long winter nap.  Now one can use the somewhat less buggy version of the Tenth Edition of Armed Bear Common Lisp, available, as usual, at <> or (shortly) via Maven <>.

      Please note that when running on openjdk17 and later runtimes, there is quite a bit of necessary fiddling with command line arguments to"open" various modules to the Bear's introspective gaze. For example, see <> for the switches necessary to use CFFI to successfully run the CL+SSL test suite.

      As a reward for your patience, we mention the following humble improvements:

        CFFI compatibility

          We improved the compiler to use wide indices for stack frame locals  for the ALOAD, ASTORE, ILOAD, ISTORE, LLOAD, and LSTORE opcodes, which  among other goodness, allows CFFI-TESTS to compile again.  In addition, we have updated the jar artifact used by CFFI to jna-5.12.1 which includes support for native linkage on the Apple Silicon and  other exotic architectures.

            (Alan Ruttenberg) Ability to discriminate generic function execution on sub-types of MOP:SPECIALIZER

              Following SBCL, CCL, and ECL, the Bear now accepts subtypes of MOP:SPECIALIZER as a generic function discriminator.

                    Overhauled relationship to later openjdk threading models

                      Going back to the future of the original green thread models, recent openjdk releases have started offering lightweight threading implementations.  For the Bear, the ability to spawn virtual threads is indicated by the presence of :VIRTUAL-THREADS in CL:*FEATURES*. Setting the special THREADS:*THREADING-MODEL* to :VIRTUAL results in THREADS:MAKE-THREADS spawning virtual threads.  On some openjdks one may have to pass a command-line switch the JVM to enable virtual threading.

                        (Uthar) Implement array types for JAVA:JNEW-RUNTIME-CLASS

                          Support for creating synthetic Java classes at runtime via JAVA:JNEW-RUNTIME-CLASS has been improved by introducing machinery for dealing with array types.

                            (Alejandrozf) Compiler uses of signals to fallback to interpreted form

                              We fall back to use an interpreted form for compilation results larger than 65535 bytes, which fixes the loading of the FirCAS computer algebra system.

                                (Alejandrozf) Further fixes to COMPILE-FILE-PATHNAME

                                  We have restored the ability to compile Maxima by hopefully untanglingthe last kinks in the reworking of our COMPILE-FILE-PATHNAME logic.

                                    (Tarn W. Burton) Avoid NIL in simple LOOP from CL:FORMAT directives

                                      Simplify simple CL:LOOP requires only compound forms, in the place where returning NIL is neither permitted nor desired.

                                            Broad testing and tweaks across Java Long Term Support (LTS) binaries

                                              We have extensively tested against recent openjdk8, openjdk11,openjdk17, and openjdk19 binaries.  Unfortunately, for openjdk17+ runtimes one needs to pass additional runtime command line switches to the hosting JVM which "open" Java modules to reflective actions.

                                                    Fuller details

                                                      More details can be found in <>.
                                                          Enjoy, and please CONS responsibly...

                                                          Nicolas MartyanoffCommon Lisp implementations in 2023

                                                          · 30 days ago

                                                          Much has been written on Common Lisp; there is rarely one year without someone proclaming the death of the language and how nobody uses it anymore. And yet it is still here, so something must have been done right.

                                                          Common Lisp is not a software, it is a language described by the ANSI INCITS 226-1994 standard; there are multiple implementations available, something often used as argument for how alive and thriving the language is.

                                                          Let us see what the 2023 situation is.

                                                          General information

                                                          Implementation License Target Last release
                                                          SBCL Public domain Native 2023/01 (2.3.1)
                                                          CCL Apache 2.0 Native 2021/05 (1.12.1)
                                                          ECL LGPL 2.1 Native (C translation) 2021/02 (21.2.1)
                                                          ABCL GPL2 Java bytecode 2023/02 (1.9.1)
                                                          CLASP LGPL 2.1 Native (LLVM) 2023/01 (2.1.0)
                                                          CMUCL Public domain Native 2017/10 (21c)
                                                          GCL LGPL2 Native (C translation) 2023/01 (2.6.14)
                                                          CLISP GPL Bytecode 2010/07 (2.49)
                                                          Lispworks Proprietary Native 2022/06 (8.0.1)
                                                          Allegro Proprietary Native 2017/04 (10.1)

                                                          Note that all projects may have small parts with different licenses. This is particularily important for CLASP which contains multiple components imported from other projects.

                                                          I was quite surprised to see so many projects with recent releases. Clearly a good sign. Let us look at each implementation.



                                                          Steel Bank Common Lisp was forked from CMUCL in December 1999 and has since massively grown in popularity; it is currently the most used implementation by far. Unsurprisingly given its popularity, SBCL is supported by pretty much all Common Lisp libraries and tools out there. It is well known for generating fast native code compared to other implementations.

                                                          The most important aspect of SBCL is that it is actively maintained: its developers release new versions on a monthly basis, bringing each time a small list of improvements and bug fixes. Activity has actually increased these last years, something uncommon in the Common Lisp world.


                                                          Clozure Common Lisp has a long and complex history and has been around for decades. It is a mature implementation; it has two interesting aspects compared to SBCL:

                                                          • The compiler is much faster.
                                                          • Error messages tend to be clearer.

                                                          This is why I currently use it to test my code along SBCL. And according to what I have heard, this is a common choice among developers.

                                                          The main issue with CCL is that the project is almost completely abandonned. Git activity has slowed down to a crawl in the last two years, and none of the original maintainers from Clozure seem to be actively working on it. It remains nonetheless a major implementation.


                                                          Embeddable Common Lisp is a small implementation which can be used both as a library or as a standalone program. It contains a bytecode interpreter, but can also translate Lisp code to C to be compiled to native code.

                                                          While development is slow, improvements and bug fixes are still added on a regular basis. Clearly an interesting project: I could see myself using ECL to write plugins into an application able to call a C library.


                                                          Armed Bear Common Lisp is quite different from other implementations: it produces Java bytecode and targets the Java Virtual Machine, making it a useful tool in Java ecosystems.

                                                          While it has not found the same success as Clojure, ABCL is still a fully featured Common Lisp implementation which passes almost the entire ANSI Common Lisp test suite.

                                                          Developement is slow nowadays but there are still new releases with lots of bug fixes. Also note that two of the developers are able to provide paid support.


                                                          CLASP is a newcomer in the Common Lisp world (new meaning it is less than a decade old). Developed by Christian Schafmeister for his research work, this implementation has been used as an exemple of how alive and kicking Common Lisp, mainly due to two excellent presentations.

                                                          While very promising, CLASP suffers from its young age: trying to run the last release on my code resulted in a brutal error with now details and no backtrace. However I have no doubt that CLASP will get a lot better: it is actively maintained and used in production, two of the necessary ingredients for a software to stay relevant.


                                                          GNU Common Lisp is described as the official Common Lisp implementation for the GNU project. While it clearly does not have the popularity of other implementations, it is still a maintained project.

                                                          Trying to use it, I quickly realized it is not fully compliant with the standard. For example it will fail when evaluating a call to COMPILE-FILE with the :VERBOSE key argument.

                                                          Hopefully development will continue.


                                                          CLISP is almost as old as I am; it was the first implementation I used a long time ago, and it still works. While it has all the usual features (multithreading, FFI, MOP, etc.), there is no real reason to use it compared to other implementations.

                                                          Even if it was to have any specific feature, CLISP is almost completely abandonned. While there are has been a semblant of activity a few years ago, active development pretty much stopped around 2012; the last release was more than 12 years ago.


                                                          Moving to proprietary implementations; Lispworks has been around for more than 30 years and the company producing it still release new versions on a regular basis.

                                                          While Lispworks supports most features you would expect from a commercial product (native compiler, multithreading, FFI, GUI library, various graphical tools, a Prolog implementation...), it is hampered by its licensing system.

                                                          The free “Personal Edition” limits the program size and the amount of time it can run, making it pretty much useless for anything but evaluation. The professional and enterprise licenses do not really make sense for anyone: you will have to buy separate licenses for every single platform at more than a thousand euros per license (with the enterprise version being 2-3 times more expensive). Of course you will have to buy a maintenance contract on a yearly basis... but it does not include technical support. It will have to be bought with “incident packs” costing thousands of euros; because yes, paying for a product and a maintenance contract does not mean they will fix bugs, and you will have to pay for each of them.

                                                          I do not have anything personal against commercial software, and I strongly support developers being paid for their work. But this kind of licensing makes Lispworks irrelevant to everyone but those already using their proprietary libraries.


                                                          Allegro Common Lisp is the other well known proprietary implementation. Developped by Franz Inc., it is apparently used by multiple organizations including the U.S. Department of Defense.

                                                          Releases are uncommon, the last one being almost 6 years ago. But Allegro is a mature implementation packed with features not easily replicated such as AllegroCache, AllegroServe, libraries for multiple protocols and data formats, analysis tools, a concurrent garbage collector and even an OpenGL interface.

                                                          Allegro suffers the same issue as Lispworks: the enterprise-style pricing system is incredibly frustrating. The website advertises a hefty $599 starting price (which at least includes technical support), but there is no mention of what it contains. Interested developpers will have to contact Franz Inc. to get other prices. A quick Google search will reveal rumours of enterprise versions priced above 8000 dollars. No comment.


                                                          Researching Common Lisp implementations has been interesting. While it is clear that the language is far from dead, its situation is very fragile. Proprietary implementations are completely out of touch with the needs of most developers, leaving us with a single open source, actively maintained, high performance implementation: SBCL. Unless of course they are willing to deal with the JVM to use ABCL.

                                                          It might me interesting to investigate a possible solution to keep CCL somehow alive, with patches being merged and releases being produced. I sent a patch very recently, let us see what can be done!

                                                          Tim BradshawHow to understand closures in Common Lisp

                                                          · 30 days ago

                                                          The first rule of understanding closures is that you do not talk about closures. The second rule of understanding closures in Common Lisp is that you do not talk about closures. These are all the rules.

                                                          There is a lot of elaborate bowing and scraping about closures in the Lisp community. But despite that a closure isn’t actually a thing: the thing people call a closure is just a function which obeys the language’s rules about the scope and extent of bindings. Implementors need to care about closures: users just need to understand the rules for bindings. So rather than obsessing about this magic invisible thing which doesn’t actually exist in the language, I suggest that it is far better simply to think about the rules which cover bindings.

                                                          Angels and pinheads

                                                          It’s easy to see why this has happened: the CL standard has a lot of discussion of lexical closures, lexical and dynamic environments and so on. So it’s tempting to think that this way of thinking about things is ‘the one true way’ because it has been blessed by those who went before us. And indeed CL does have objects representing part of the lexical environment which are given to macro functions. Occasionally these are even useful. But there are no objects which represent closures as distinct from functions, and no predicates which tell you if a function is a closure or not in the standard language: closures simply do not exist as objects distinct from functions at all. They were useful, perhaps, as part of the text which defined the language, but they are nowhere to be found in the language itself.

                                                          So, with the exception of the environment objects passed to macros, none of these objects exist in the language. They may exist in implementations, and might even be exposed by some implementations, but from the point of the view of the language they simply do not exist: if I give you a function object you cannot know if it is a closure or not.

                                                          So it is strange that people spend so much time worrying about these objects which, if they even exist in the implementation, can’t be detected by anyone using the standard language. This is worrying about angels and pinheads: wouldn’t it be simpler to simply understand what the rules of the language actually say should observably happen? I think it would.

                                                          I am not arguing that the terminology used by the standard is wrong! All I am arguing is that, if you think you want to understand closures, you might instead be better off understanding the rules that give rise to them. And when you have done that you may suddenly find that closures have simply vanished into the mist: all you need is the rules.


                                                          Common Lisp is steeped in history: it is full of traces of the Lisps which went before it. This is intentional: one goal of CL was to enable programs written in those earlier Lisps — which were all Lisps at that time of course — to run without extensive modification.

                                                          But one place where CL didn’t steep itself in history is in exactly the areas that you need to understand to understand closures. Before Common Lisp (really, before Scheme), people spent a lot of time writing papers about the funarg problem and describing and implementing more-or-less complicated ways of resolving it. Then Scheme came along and decided that this was all nonsense and that it could just be made to go away by implementing the language properly. And the Common Lisp designers, who knew about Scheme, said that, well, if Scheme can do this, then we can do this as well, and so they also made it the problem vanish, although not in quite such an extreme way as Scheme did.

                                                          And this is now ancient history: these predecessor Lisps to CL are all at least 40 years old now. I am, just, old enough to have used some of them when they were current, but for most CL programmers these questions were resolved before they were born. The history is very interesting, but you do not need to steep yourself in it to understand closures.


                                                          So the notion of a closure is part of the history behind CL: a hangover from the time when people worried about the funarg problem; a time before they understood that the whole problem could simply be made to go away. So, again, if you think you want to understand closures, the best approach is to understand something else: to understand bindings. Just as with closures, bindings do not exist as objects in the language, although you can make some enquiries about some kinds of bindings in CL. They are also a concept which exists in many programming languages, not just CL.

                                                          A binding is an association between a name — a symbol — and something. The most common binding is a variable binding, which is an association between a name and a value. There are other kinds of bindings however: the most obvious kind in CL is a function binding: an association between a name and a function object. And for example within a (possibly implicit) block there is a binding between the name of the block and a point to which you can jump. And there are other kinds of bindings in CL as well, and the set is extensible. The CL standard only calls variable bindings ‘bindings’, but I am going to use the term more generally.

                                                          Bindings are established by some binding construct and are usually not first-class objects in CL: they are just as vaporous as closures and environments. Nevertheless they are a powerful and useful idea.

                                                          What can be bound?

                                                          By far the most common kind of binding is a variable binding: an association between a name and a value. However there are other kinds of bindings: associations between names and other things. I’ll mention those briefly at the end, but in everything else that follows it’s safe to assume that ‘binding’ means ‘variable binding’ unless I say otherwise.

                                                          Scope and extent

                                                          For both variable bindings and other kinds of bindings there are two interesting questions you can ask:

                                                          • where is the binding available?
                                                          • when is the binding visible?

                                                          The first question is about the scope of the binding. The second is about the extent of the binding.

                                                          Each of these questions has (at least) two possible answers giving (at least) four possibilities. CL has bindings which use three of these possibilities and the fourth in a restricted case: two and a restricted version of a third for variable bindings, the other one for some other kinds of bindings.

                                                          Scope. The two options are:

                                                          • the binding may be available only in code where the binding construct is visible;
                                                          • or the binding may be available during all code which runs between where the binding is established and where it ends, regardless of whether the binding construct is visible.

                                                          What does ‘visible’ mean? Well, given some binding form, it means that the bindings it establishes are visible to all the code that is inside that form in the source. So, in a form like (let ((x 1)) ...) the binding of x is visible to the code that replaces the ellipsis, including any code introduced by macroexpansion, and only to that code.

                                                          Extent. The two options are:

                                                          • the binding may exist only during the time that the binding construct is active, and goes away when control leaves it;
                                                          • or the binding may exist as long as there is any possibility of reference.

                                                          Unfortunately the CL standard is, I think, slightly inconsistent in its naming for these options. However I’m going to use the standard’s terms with one exception. Here they are.


                                                          • when a binding is available only when visible this called lexical scope;
                                                          • when a binding available to all code within the binding construct this is called indefinite scope1;


                                                          The term from the standard I am not going to use is dynamic scope, which it defines to mean the combination of indefinite scope and dynamic extent. I am not going to use this term because I think it is confusing: although it has ‘scope’ in its name it concerns both scope and extent. Instead I will introduce better, commonly used, terms below for the interesting combinations of scope and extent.

                                                          The four possibilities for bindings are then:

                                                          • lexical scope and dynamic extent;
                                                          • lexical scope and indefinite extent;
                                                          • indefinite scope and dynamic extent;
                                                          • indefinite scope and indefinite extent.

                                                          The simplest kind of binding

                                                          So then let’s ask: what is the simplest kind of binding to understand? If you are reading some code and you see a reference to a binding then what choice from the above options will make it easiest for you to understand whether that reference is valid or not?

                                                          Well, the first thing is that you’d like to be able to know by looking at the code whether a reference is valid or not. That means that the binding construct should be visible to you, or that the binding should have lexical scope. Compare the following two fragments of code:

                                                          (defun simple (x)
                                                            (+ x 1)


                                                          (defun confusing ()
                                                            (+ *x* 1)

                                                          Well, in the first one you can tell, just by looking at the code, that the reference to x is valid: the function, when called, establishes a binding of x and you can see that when reading the code. In the second one you just have to assume that the reference to *x* is valid: you can’t tell by reading the code whether it is or not.

                                                          Lexical scope makes it easiest for people reading the code to understand it, and in particular it is easier to understand than indefinite scope. It is the simplest kind of scoping to understand for people reading the code.

                                                          So that leaves extent. Well, in the two examples above definite or indefinite extent makes no difference to how simple the code is to understand: once the functions return there’s no possibility of reference to the bindings anyway. To expose the difference we need somehow to construct some object which can refer to a binding after the function has returned. We need something like this:

                                                          (defun maker (x)
                                                            <construct object which refers to binding of x>)
                                                          (let ((o (maker 1)))
                                                            <use o somehow to cause it to reference the binding of x>)

                                                          Well, what it this object going to be? What sort of things reference bindings? Code references bindings, and the objects which contain code are functions3. What we need to do is construct and return a function:

                                                          (defun maker (x)
                                                            (lambda (y)
                                                              (+ x y)))

                                                          and then cause this function to reference the binding by calling it:

                                                          (let ((f (maker 1)))
                                                            (funcall f 2))

                                                          So now we can, finally, ask: what is the choice for the extent of the binding of x which makes this code simplest to understand? Well, the answer is that unless the binding of x remains visible to the function that is created in maker, this code can’t work at all. It would have to be the case that it was simply not legal to return functions like this from other functions. Functions, in other words, would not be first-class objects.

                                                          Well, OK, that’s a possibility, and it makes the above code simple to understand: it’s not legal and it’s easy to see that it is not. Except consider this small variant on the above:

                                                          (defun maybe-maker (x return-identity-p)
                                                            (if return-identity-p
                                                              (lambda (y)
                                                                (+ x y))))

                                                          There is no way to know from reading this code whether maybe-maker will return the nasty anonymous function or the innocuous identity function. If it is not allowed to return anonymous functions in this way then there is no way of knowing whether

                                                          (funcall (maybe-maker 1 (zerop (random 2)))

                                                          is correct or not. This is certainly not simple: in fact it is a horrible nightmare. Another way of saying this is that you’d be in a situation where

                                                          (let ((a 1))
                                                            (funcall (lambda ()

                                                          would work, but

                                                          (funcall (let ((a 1))
                                                                     (lambda ()

                                                          would not. There are languages which work that way: those languages suck.

                                                          So what would be simple? What would be simple is to say that if a binding is visible, it is visible, and that’s the end of the story. In a function like maker above the binding of x established by maker is visible to the function that it returns. Therefore it’s visible to the function that maker returns: without any complicated rules or weird special cases. That means the binding must have indefinite extent.

                                                          Indefinite extent makes it easiest for people reading the code to understand it when that code may construct and return functions, and in particular it is easier to understand than dynamic extent, which makes it essentially impossible to tell in many cases whether such code is correct or not.

                                                          And that’s it: lexical scope and indefinite extent, which I will call lexical binding, is the simplest binding scheme to understand for a language which has first-class functions4.

                                                          And really that’s it: that’s all you need to understand. Lexical scope and indefinite extent make reading code simple, and entirely explain the things people call ‘closures’ which are, in fact, simply functions which obey these simple rules.

                                                          Examples of the simple binding rules

                                                          One thing I have not mentioned before is that, in CL, bindings are mutable, which is another way of saying that CL supports assignment: assignment to variables is mutation of variable bindings. So, as a trivial example:

                                                          (defun maximum (list)
                                                            (let ((max (first list)))
                                                              (dolist (e (rest list) max)
                                                                (when (> e max)
                                                                  (setf max e)))))

                                                          This is very easy to understand and does not depend on the binding rules in detail.

                                                          But, well, bindings are mutable, so the rules which say they exist as long as they can be referred to also imply they can be mutated as long as they can be referred to: anything else would certainly not be simple. So here’s a classic example of this:

                                                          (defun make-incrementor (&optional (value 0))
                                                            (lambda (&optional (increment 1))
                                                              (prog1 value
                                                                (incf value increment))))

                                                          And now:

                                                          > (let ((i (make-incrementor)))
                                                              (print (funcall i))
                                                              (print (funcall i))
                                                              (print (funcall i -2))
                                                              (print (funcall i))
                                                              (print (funcall i))

                                                          As you can see, the function returned by make-incrementor is mutating the binding that it can still see.

                                                          What happens when two functions can see the same binding?

                                                          (defun make-inc-dec (&optional (value 0))
                                                             (lambda ()
                                                               (prog1 value
                                                                 (incf value)))
                                                             (lambda ()
                                                               (prog1 value
                                                                 (decf value)))))

                                                          And now

                                                          > (multiple-value-bind (inc dec) (make-inc-dec)
                                                              (print (funcall inc))
                                                              (print (funcall inc))
                                                              (print (funcall dec))
                                                              (print (funcall dec))
                                                              (print (funcall inc))

                                                          Again, what happens is the simplest thing: you can see simply from reading the code that both functions can see the same binding of value and they are therefore both mutating this common binding.

                                                          Here is an example which demonstrates all these features: an implementation of a simple queue as a pair of functions which can see two shared bindings:

                                                          (defun make-queue ()
                                                            (let ((head '())
                                                                  (tail nil))
                                                               (lambda (thing)
                                                                 ;; Push thing onto the queue
                                                                 (if (null head)
                                                                     ;; It's empty currently so set it up
                                                                     (setf head (list thing)
                                                                           tail head)
                                                                   ;; not empty: just adjust the tail
                                                                   (setf (cdr tail) (list thing)
                                                                         tail (cdr tail)))
                                                               (lambda ()
                                                                  ((null head)
                                                                   ;; empty
                                                                   (values nil nil))
                                                                  ((null (cdr head))
                                                                   ;; will be empty: don't actually need this case but it is
                                                                   ;; cleaner
                                                                   (values (prog1 (car head)
                                                                             (setf head '()
                                                                                   tail nil))
                                                                   ;; will still have content
                                                                   (values (pop head) t)))))))

                                                          make-queue will return two functions:

                                                          • the first takes one argument which it appends to the queue;
                                                          • the second takes no argument and either the next element of the queue and t or nil and nil if the queue is empty.

                                                          So, with this little function to drain the queue

                                                          (defun drain-and-print (popper)
                                                            (multiple-value-bind (value fullp) (funcall popper)
                                                              (when fullp
                                                                (print value)
                                                                (drain-and-print popper))

                                                          we can see this in action

                                                          > (multiple-value-bind (pusher popper) (make-queue)
                                                              (funcall pusher 1)
                                                              (funcall pusher 2)
                                                              (funcall pusher 3)
                                                              (drain-and-print popper))

                                                          A less-simple kind of binding which is sometimes very useful

                                                          Requiring bindings to be simple usually makes programs easy to read and understand. But it also makes it hard to do some things. One of those things is to control the ‘ambient state’ of a program. A simple example would be the base for printing numbers. It’s quite natural to say that ‘in this region of the program I want numbers printed in hex’.

                                                          If all we had was lexical binding then this becomes a nightmare: every function you call in the region you want to cause printing to happen in hex needs to take some extra argument which says ‘print in hex’. And if you then decide that, well, you’d also like some other ambient parameter, you need to provide more arguments to every function5. This is just horrible.

                                                          You might think you can do this with global variables which you temporarily set: that is both fiddly (better make sure you set it back) and problematic in the presence of multiple threads6.

                                                          A better approach is to allow dynamic bindings: bindings with indefinite scope & dynamic extent. CL has these, and at this point history becomes unavoidable: rather than have some separate construct for dynamic bindings, CL simply says that some variable bindings, and some references to variable bindings, are to be treated as having indefinite scope and dynamic extent, and you tell the system which bindings this applies to withspecial declarations / proclamations. CL does this because that’s very close to how various predecessor Lisps worked, and so makes porting programs from them to CL much easier. To make this less painful there is a convention that dynamically-bound variable names have *stars* around them, of course.

                                                          Dynamic bindings are so useful that if you don’t have them you really need to invent them: I have on at least two occasions implemented a dynamic binding system in Python, for instance.

                                                          However this is not an article on dynamic bindings so I will not write more about them here: perhaps I will write another article later.

                                                          What else can be bound?

                                                          Variable bindings are by far the most common kind. But not the only kind. Other things can be bound. Here is a partial list7:

                                                          • local functions have lexical scope and indefinite extent;
                                                          • block names have lexical scope and definite extent (see below);
                                                          • tag names have lexical scope and definite extent (see below);
                                                          • catch tags have indefinite scope and definite extent;
                                                          • condition handlers have indefinite scope and definite extent;
                                                          • restarts have indefinite scope and definite extent.

                                                          The two interesting cases here are block names and tag names. Both of these have lexical scope but only definite extent. As I argued above this makes it hard to know whether references to them are valid or not. Look at this, for example:

                                                          (defun outer (x)
                                                            (inner (lambda (r)
                                                                     (return-from outer r))
                                                          (defun inner (r rp)
                                                            (if rp
                                                              (funcall r #'identity)))

                                                          So then (funcall (outer nil) 1) will: call inner with a function which wants to return from outer and nil, which will cause inner to call that function, returning the identity function, which is then called by funcall with argument 1: the result is 1.

                                                          But (funcall (outer t) 1) will instead return the function which wants to return from outer, which is then called by funcall which is an error since it is outside the dynamic extent of the call to outer.

                                                          And there is no way that either a human reading the code or the compiler can detect that this is going to happen: a very smart compiler might perhaps be able to deduce that the internal function might be returned from outer, but probably only because this is a rather simple case: for instance in

                                                          (defun nasty (f)
                                                            (funcall f (lambda ()
                                                                         (return-from nasty t))))

                                                          the situation is just hopeless. So this is a case where the binding rules are not as simple as you might like.

                                                          What is simple?

                                                          For variable bindings I think it’s easy to see that the simplest rule for a person reading the code is lexical binding. The other question is whether that is simpler for the implementation. And the answer is that probably it is not: probably lexical scope and definite extent is the simplest implementationally. That certainly approximates what many old Lisps did8. It’s fairly easy to write a bad implementation of lexical binding, simply by having all functions retain all the bindings, regardless of whether they might refer to them. A good implementation requires more work. But CL’s approach here is that doing the right thing for people is more important than making the implementor’s job easier. And I think that approach has worked well.

                                                          On the other hand CL hasn’t done the right thing for blocks and tags: There are at least three reasons for this.

                                                          Implementational complexity. If the bindings had lexical scope and indefinite extent then you would need to be able to return from a block which had already been returned from, and go to a tag from outside the extent of the form that established it. That opens an enormous can of worms both in making such an implementation work at all but also handling things like dynamic bindings, open files and so on. That’s not something the CL designers were willing to impose on implementors.

                                                          Complexity in the specification. If CL had lexical bindings for blocks and tags then the specification of the language would need to describe what happens in all the many edge cases that arise, including cases where it is genuinely unclear what the correct thing to do is at all such as dealing with open files and so on. Nobody wanted to deal with that, I’m sure: the language specification was already seen as far too big and the effort involved would have made it bigger, later and more expensive.

                                                          Conceptual difficulty. It might seem that making block bindings work like lexical variable bindings would make things simpler to understand. Well, that’s exactly what Scheme did with call/cc and call/cc can give rise to some of the most opaque code I have ever seen. It is often very pretty code, but it’s not easy to understand.

                                                          I think the bargain that CL has struck here is at least reasonable: to make the common case of variable bindings simple for people, and to avoid the cases where doing the right thing results in a language which is harder to understand in many cases and far harder to implement and specify.

                                                          Finally, once again I think that the best way to understand how closures in CL is not to understand them: instead understand the binding rules for variables, why they are simple and what they imply.

                                                          1. indefinite scope is often called ‘dynamic scope’ although I will avoid this term as it is used by the standard to mean the combination of indefinite scope and dynamic extent. 

                                                          2. Dynamic extent could perhaps be called ‘definite extent’, but this is not the term that the standard uses so I will avoid it. 

                                                          3. Here and below I am using the term ‘function’ in the very loose sense that CL usually uses it: almost none of the ‘functions’ I will talk about are actually mathematical functions: they’re what Scheme would call ‘procedures’. 

                                                          4. For languages which don’t have first-class functions or equivalent constructs, lexical scope and definite extent is the same as lexical scope and indefinite extent, because it is not possible to return objects which can refer to bindings from the place those bindings were created. 

                                                          5. More likely, you would end up making every function have, for instance an ambient keyword argument whose value would be an alist or plist which mapped between properties of the ambient environment and values for them. All functions which might call other functions would need this extra argument, and would need to be sure to pass it down suitably. 

                                                          6. This can be worked around, but it’s not simple to do so. 

                                                          7. In other words ‘this is all I can think of right now, but there are probably others’. 

                                                          8. Very often old Lisps had indefinite scope and definite extent in interpreted code but lexical scope and definite extent in compiled code: yes, compiled code behaved differently to interpreted code, and yes, that sucked. 

                                                          Stelian IonescuDistributing binaries with Common Lisp and foreign libraries

                                                          · 32 days ago
                                                          The ability to create a new binary executable by simply “dumping” to disk the code in a running Common Lisp executable is one of the features that makes Common Lisp well suited for rapid development. That ease of development breaks down when using foreign libraries (usually written in C/C++): in the most popular Common Lisp implementations, the Foreign Function Interface loads foreign libraries using dlopen() and stores their paths in the Lisp image, so that the resulting binary can start from their previous state.

                                                          Nicolas MartyanoffReading files faster in Common Lisp

                                                          · 37 days ago

                                                          While Common Lisp has functions to open, read and write files, none of them takes care of reading and returning the entire content. This is something that I do very regularly, so it made sense to add such a function to Tungsten. It turned out to be a bit more complicated than expected.

                                                          A simple but incorrect implementation

                                                          The simplest implementation relies on the FILE-LENGTH function which returns the length of a stream (which of course only makes sense for a file stream). The Hyperspec clearly states that “for a binary file, the length is measured in units of the element type of the stream”. Since we are only reading binary data, everything is fine.

                                                          Let us write the function:

                                                          (defun read-file (path)
                                                            (declare (type (or pathname string) path))
                                                            (with-open-file (file path :element-type 'core:octet)
                                                              (let ((data (make-array (file-length file) :element-type 'core:octet)))
                                                                (read-sequence data file)

                                                          Note that CORE:OCTET is a Tungsten type for (UNSIGNED-BYTE 8).

                                                          The function works as expected, returning the content of the file as an octet vector. But it is not entirely correct.

                                                          This implementation only works for regular files. Various files on UNIX will report a length of zero but can still be read. Now you might protest that it would not make sense to call READ-FILE on a device such as /dev/urandom, and you would be right. But a valid example would be pseudo files such as those part of procfs. If you want to obtain memory stats about your process on Linux, you can simply read /proc/self/statm. But this is not a regular file and READ-FILE will return an empty octet vector.

                                                          Doing it right and slow

                                                          The right way to read a file is to read its content block by block until the read operation fails because it reached the end of the file.

                                                          Let us re-write READ-FILE:

                                                          (defun read-file (path)
                                                            (declare (type (or pathname string) path))
                                                            (let ((data (make-array 0 :element-type 'core:octet :adjustable t))
                                                                  (block-size 4096)
                                                                  (offset 0))
                                                              (with-open-file (file path :element-type 'core:octet)
                                                                  (let* ((capacity (array-total-size data))
                                                                         (nb-left (- capacity offset)))
                                                                    (when (< nb-left block-size)
                                                                      (let ((new-length (+ capacity (- block-size nb-left))))
                                                                        (setf data (adjust-array data new-length)))))
                                                                  (let ((end (read-sequence data file :start offset)))
                                                                    (when (= end offset)
                                                                      (return-from read-file (adjust-array data end)))
                                                                    (setf offset end))))))

                                                          This time we rely on an adjustable array; we iterate, making sure we have enough space in the array to read an entire block each time. When the array is too short, we use ADJUST-ARRAY to extend it, relying on its ability to reuse the underlying storage instead of systematically copying its content.

                                                          Finally, once READ-SEQUENCE stops returning data, we truncate the array to the right size and return it.

                                                          This function worked correctly and I started using it regularly. Recently I started working with a file larger than usual and realized that READ-FILE was way too slow. With a NVMe drive, I would expect to be able to read a 10+MB file almost instantaneously, but it took several seconds.

                                                          After inspecting the code to find what could be so slow, I started to wonder about ADJUST-ARRAY; while I thought SBCL would internally extend the underlying memory in large blocks to minimize allocations, behaving similarly to realloc() in C, it turned out not to be the case. While reading the code behind ADJUST-ARRAY, I learned that it precisely allocates the required size. As a result, this implementation of READ-FILE performs one memory allocation for each 4kB block. Not a problem for small files, slow for larger ones.

                                                          A final version, correct and fast

                                                          Since I understood what the problem was, fixing it was trivial. When there is not enough space to read a block, we extend the array by at least 50% of its current size. Of course this is a balancing act: for example doubling the size at each allocation would reduce even more the number of allocations, but would increase the total amount of memory allocated. The choice is up to you.

                                                          (defun read-file (path)
                                                            (declare (type (or pathname string) path))
                                                            (let ((data (make-array 0 :element-type 'core:octet :adjustable t))
                                                                  (block-size 4096)
                                                                  (offset 0))
                                                              (with-open-file (file path :element-type 'core:octet)
                                                                  (let* ((capacity (array-total-size data))
                                                                         (nb-left (- capacity offset)))
                                                                    (when (< nb-left block-size)
                                                                      (let ((new-length (max (+ capacity (- block-size nb-left))
                                                                                             (floor (* capacity 3) 2))))
                                                                        (setf data (adjust-array data new-length)))))
                                                                  (let ((end (read-sequence data file :start offset)))
                                                                    (when (= end offset)
                                                                      (return-from read-file (adjust-array data end)))
                                                                    (setf offset end))))))

                                                          This last version reads a 250MB file in a quarter of a second, while the original version took almost two minutes. Much better!

                                                          Quicklisp newsFebruary 2023 Quicklisp dist update now available

                                                          · 37 days ago

                                                           New projects: 

                                                          • asdf-dependency-graph — A minimal wrapper around `dot` available at least on Linux systems to generate dependency-graphs. — MIT
                                                          • aws-sdk-lisp — AWS-SDK for Common Lisp — BSD 2-Clause
                                                          • chlorophyll — ANSI escape code library for Common Lisp — Expat
                                                          • ciao — OAuth 2.0 Client for Common Lisp — LGPLv3
                                                          • cl-cmark — Common Lisp bindings to libcmark, the CommonMark reference implementation — BSD-2-Clause
                                                          • cl-jingle — jingle -- ningle with bells and whistles — BSD 2-Clause
                                                          • cl-modio — A client library for the API. — zlib
                                                          • csv-validator — Validates tabular CSV data using predefined validations, similar to its Python counterpart 'Great Expectations'. — BSD-3
                                                          • history-tree — Store the history of a browser's visited paths. — BSD 3-Clause
                                                          • jzon — A correct and safe(er) JSON RFC 8259 parser with sane defaults. — MIT
                                                          • lisp-pay — Wrappers over multiple Payment Processor APIs — MIT
                                                          • mito-attachment — Mito mixin class for file management — LLGPL
                                                          • nclasses — Simplify class like definitions with define-class and friends. — Public Domain
                                                          • njson — NJSON is a JSON handling framework with the focus on convenience and brevity. — BSD-3 Clause
                                                          • nsymbols — A set of convenience functions to list class, variable, function, and other symbols. — BSD-3 Clause
                                                          • py4cl2-cffi — CFFI based alternative to PY4CL2, primarily developed for performance reasons. — MIT
                                                          • symath — A simple symbolic math library for Common Lisp — MIT

                                                          Updated projects: 3b-bmfont, 40ants-asdf-system, abstract-arrays, adp, ahungry-fleece, amb, april, babel, bdef, binary-parser, binpack, bp, cells, cffi, ci, cl+ssl, cl-advice, cl-bcrypt, cl-bnf, cl-cffi-gtk, cl-collider, cl-colors2, cl-confidence, cl-conspack, cl-containers, cl-cxx, cl-data-structures, cl-dbi, cl-digraph, cl-enumeration, cl-etcd, cl-fix, cl-gamepad, cl-git, cl-glib, cl-gobject-introspection, cl-gobject-introspection-wrapper, cl-graph, cl-gserver, cl-hamcrest, cl-i18n, cl-indentify, cl-jpeg, cl-lambdacalc, cl-mathstats, cl-megolm, cl-migratum, cl-mixed, cl-naive-store, cl-oju, cl-opencl-utils, cl-patterns, cl-pdf, cl-prevalence, cl-protobufs, cl-rashell, cl-rdkafka, cl-rfc4251, cl-ssh-keys, cl-steamworks, cl-store, cl-str, cl-telegram-bot, cl-trie, cl-unification, cl-utils, cl-vorbis, cl-webkit, cl-wol, cl-xkb, cl-yacc, clack, clad, clast, clath, clavier, clazy, climacs, climc, clingon, clip, clog, closer-mop, cluffer, clx, cmd, colorize, common-doc, consfigurator, croatoan, cserial-port, ctype, database-migrations, defenum, definer, dense-arrays, deploy, depot, dexador, djula, doc, docparser, docs-builder, dsm, duologue, eclector, extensible-compound-types, fare-csv, fare-utils, file-attributes, file-select, filesystem-utils, find-port, fiveam-matchers, float-features, for, functional-trees, glacier, gtirb-capstone, gtirb-functions, harmony, hashtrie, helambdap, http2, hu.dwim.syntax-sugar, in-nomine, journal, js, jsonrpc, lack, lass, lift, lisp-unit2, lmdb, local-time, log4cl-extras, lunamech-matrix-api, maiden, markup, mcclim, metabang-bind, mfiano-utils, mgl-mat, mgl-pax, mito, mk-defsystem, mmap, more-cffi, multiposter, music-spelling, mutility, nail, ndebug, new-op, nfiles, nhooks, nkeymaps, nodgui, north, numerical-utilities, numericals, nyxt, omglib, ook, osc, osicat, parachute, pero, petalisp, plot, plump, polymorphic-functions, postmodern, promise, py4cl2, qtools, random-state, read-number, replic, rove, sc-extensions, sel, serapeum, shop3, simple-neural-network, sketch, slime, sly, speechless, spinneret, statistics, stepster, stumpwm, swank-client, swank-crew, teepeedee2, ten, testiere, tfeb-lisp-hax, tooter, trace-db, trivial-backtrace, trivial-coerce, trivial-download, trivial-mimes, trivial-package-locks, trivial-shell, trivial-timeout, type-i, typo, uax-9, vellum, vellum-postmodern, vk, vp-trees, wayflan, with-contexts, with-user-abort, wookie, workout-timer, xhtmlambda, yason, zippy.

                                                          Removed projects: cl-sane, cluster, vellum-binary.

                                                          To get this update, use (ql:update-dist "quicklisp"). Enjoy!

                                                          Nicolas HafnerBuilding the Future - February Kandria Update

                                                          · 45 days ago

                                                          It's already been a month since Kandria released! Woah, time sure flies these days, huh?

                                                          Kandria Launch

                                                          Well, in case you missed the launch, the game is now officially available on Steam, Itch, and directly on our Website.

                                                          The reviews we've gotten have been very positive, both from press, on Steam, and on the streams I've caught! I'm really happy that people have been enjoying the game, and am very thankful for all of the support and well wishes.

                                                          The first two weeks after launch were spent furiously fixing things, with a patch being released almost every day. Things have calmed down a lot since, and while there's a few polish things left over that we know about, overall the game is now very stable, and even got the official Steam Deck Verified rating!

                                                          But, the work isn't done yet.

                                                          Upcoming Attractions

                                                          Back in the Kickstarter we reached three stretch goals. One of those was an extra questline, which is already in the released game. But, the two other goals I'm hammering out now:

                                                          • Level Editor
                                                            This is actually already in... sort of. If you open the game and press the section/tilde key below Escape, you'll toggle the editor during gameplay. If you want, you can mess about with it! But, it's still a bit rough around the edges and especially sharing levels with others is not easy enough for my tastes. So, that's what this update is about. Alongside the update we'll also organise a little contest with rewards, so look forward to that!

                                                          • Modding Support
                                                            And this one is quite exciting as well. Part of the modding support update is the source code release of Kandria, which has already happened. But, the more important and difficult parts are a stable API and mod loading mechanism, the integration with the API, and a convenient user interface to manage the mods. An extremely rudimentary API already exists now, but it will be quite a while before things become stable.

                                                          Alongside both of those updates I'll also be writing documentation on the editor and the modding, to ensure people have an easier entry point. The editor should be usable without any coding knowledge, too.

                                                          I can't promise any dates for the two features, but if things go really well the editor will be out sometime next month!

                                                          But, there is yet another update: there'll be a Japanese translation for Kandria! As someone that's been trying (and mostly failing) to learn Japanese for many years, this is quite exciting for me. If all goes well it should be out by June.

                                                          And that's all I can say for now. If you're interested in the level editor and modding stuff, please hop on by our Discord! You'll be able to ask questions there and chat with other interested folks.

                                                          Nicolas MartyanoffCustom Common Lisp indentation in Emacs

                                                          · 60 days ago

                                                          While SLIME is most of the time able to indent Common Lisp correctly, it will sometimes trip on custom forms. Let us see how we can customize indentation.

                                                          In the process of writing my PostgreSQL client in Common Lisp, I wrote a READ-MESSAGE-CASE macro which reads a message from a stream and execute code depending on the type of the message:

                                                          (defmacro read-message-case ((message stream) &rest forms)
                                                            `(let ((,message (read-message ,stream)))
                                                               (case (car ,message)
                                                                  (backend-error (cdr ,message)))
                                                                  (error 'unexpected-message :message ,message)))))

                                                          This macro is quite useful: all message loops can use it to automatically handle error responses, notices, and signal unexpected messages.

                                                          But SLIME does not know how to indent READ-MESSAGE-CASE, so by default it will align all message forms on the first argument:

                                                          (read-message-case (message stream)
                                                                               (unless password
                                                                                 (error 'missing-password))
                                                                               (write-password-message password stream)))

                                                          While we want it aligned the same way as HANDLER-CASE:

                                                          (read-message-case (message stream)
                                                              (unless password
                                                                (error 'missing-password))
                                                              (write-password-message password stream)))

                                                          Good news, SLIME indentation is defined as a list of rules. Each rule associates an indentation specification (a S-expression describing how to indent the form) to a symbol and store it as the common-lisp-indent-function property of the symbol.

                                                          You can obtain the indentation rule of a Common Lisp symbol easily. For example, executing (get 'defun 'common-lisp-indent-function) (e.g. in IELM or with eval-expression) yields (4 &lambda &body). This indicates that DEFUN forms are to be indented as follows:

                                                          • The first argument of DEFUN (the function name) is indented by four spaces.
                                                          • The second argument (the list of function arguments) is indented as a lambda list.
                                                          • The rest of the arguments are indented based on the lisp-body-indent custom variable, which controls the indentation of the body of a lambda form (two spaces by default).

                                                          You can refer to the documentation of the common-lisp-indent-function Emacs function (defined in SLIME of course) for a complete description of the format.

                                                          We want READ-MESSAGE-CASE to be indented the same way as HANDLER-CASE, whose indentation specification is (4 &rest (&whole 2 &lambda &body)) (in short, an argument and a list of lambda lists). Fortunately there is a way to specify that a form must be indented the same way as another form, using (as <symbol>).

                                                          Let us first define a function to set the indentation specification of a symbol:

                                                          (defun g-common-lisp-indent (symbol indent)
                                                            "Set the indentation of SYMBOL to INDENT."
                                                            (put symbol 'common-lisp-indent-function indent))

                                                          Then use it for READ-MESSAGE-CASE:

                                                          (g-common-lisp-indent 'read-message-case '(as handler-case))

                                                          While it is in general best to avoid custom indentation, exceptions are sometimes necessary for readability. And SLIME makes it easy.

                                                          TurtleWareMethod Combinations

                                                          · 66 days ago

                                                          Table of Contents

                                                          1. Introduction
                                                          2. Defining method combinations - the short form
                                                          3. Defining method combinations - the long form
                                                            1. The Hooker
                                                            2. The Memoizer
                                                          4. Conclusions

                                                          Update [2023-01-23]

                                                          Christophe Rhodes pointed out that "The Hooker" method combination is not conforming because there are multiple methods with the same "role" that can't be ordered and that have different qualifiers:

                                                          Note that two methods with identical specializers, but with different qualifiers, are not ordered by the algorithm described in Step 2 of the method selection and combination process described in Section 7.6.6 (Method Selection and Combination). Normally the two methods play different roles in the effective method because they have different qualifiers, and no matter how they are ordered in the result of Step 2, the effective method is the same. If the two methods play the same role and their order matters, an error is signaled. This happens as part of the qualifier pattern matching in define-method-combination.


                                                          So instead of using qualifier patterns we should use qualifier predicates. They are not a subject of the above paragraph because of its last sentence (there is also an example in the spec that has multiple methods with a predicate). So instead of

                                                          (define-method-combination hooker ()
                                                            (... (hook-before (:before*)) ...) ...)

                                                          the method combination should use:

                                                          (defun hook-before-p (method-qualifier)
                                                            (typep method-qualifier '(cons (eql :before) (cons t null))))
                                                          (define-method-combination hooker ()
                                                            (... (hook-before hook-before-p) ...) ...)

                                                          and other "hook" groups should also use predicates.

                                                          Another thing worth mentioning is that both ECL and SBCL addressed issues with the qualifier pattern matching and :arguments since the publication of this blog post.


                                                          Method combinations are used to compute the effective method for a generic function. An effective method is a body of the generic function that combines a set of applicable methods computed based on the invocation arguments.

                                                          For example we may have a function responsible for reporting the object status and each method focuses on a different aspect of the object. In that case we may want to append all results into a list:

                                                          (defgeneric status (object)
                                                            (:method-combination append))
                                                          (defclass base-car ()
                                                            ((engine-status :initarg :engine :accessor engine-status)
                                                             (wheels-status :initarg :wheels :accessor wheels-status)
                                                             (fuel-level :initarg :fuel :accessor fuel-level))
                                                            (:default-initargs :engine 'ok :wheels 'ok :fuel 'full))
                                                          (defmethod status append ((object base-car))
                                                            (list :engine (engine-status object)
                                                                  :wheels (wheels-status object)
                                                                  :fuel (fuel-level object)))
                                                          (defclass premium-car (base-car)
                                                            ((gps-status :initarg :gps :accessor gps-status)
                                                             (nitro-level :initarg :nitro :accessor nitro-level))
                                                            (:default-initargs :gps 'no-signal :nitro 'low))
                                                          (defmethod status append ((object premium-car))
                                                            (list :gps (gps-status object)
                                                                  :nitro (nitro-level object)))
                                                          CL-USER> (status (make-instance 'premium-car))
                                                          (:GPS NO-SIGNAL :NITRO LOW :ENGINE OK :WHEELS OK :FUEL FULL)
                                                          CL-USER> (status (make-instance 'base-car))
                                                          (:ENGINE OK :WHEELS OK :FUEL FULL)

                                                          The effective method may look like this:

                                                          (append (call-method #<method status-for-premium-car>)
                                                                  (call-method #<method status-for-base-car>   ))

                                                          Note that append is a function so all methods are called. It is possible to use other operators (for example a macro and) and then the invocation of particular methods may be conditional:

                                                          (and (call-method #<method can-repair-p-for-premium-car>)
                                                               (call-method #<method can-repair-p-for-base-car>   ))

                                                          Defining method combinations - the short form

                                                          The short form allows us to define a method combination in the spirit of the previous example:

                                                          (OPERATOR (call-method #<m1>)
                                                                    (call-method #<m2>)

                                                          For example we may want to return as the second value the count of odd numbers:

                                                          (defun sum-and-count-odd (&rest args)
                                                            (values (reduce #'+ args)
                                                                    (count-if #'oddp args)))
                                                          (define-method-combination sum-and-count-odd)
                                                          (defclass a () ())
                                                          (defclass b (a) ())
                                                          (defclass c (b) ())
                                                          (defgeneric num (o)
                                                            (:method-combination sum-and-count-odd)
                                                            (:method sum-and-count-odd ((o a)) 1)
                                                            (:method sum-and-count-odd ((o b)) 2)
                                                            (:method sum-and-count-odd ((o c)) 3)
                                                            (:method :around ((o c))
                                                              (print "haa!")
                                                          (num (make-instance 'b)) ;; (values 3 1)
                                                          (num (make-instance 'c)) ;; (values 6 2)

                                                          Note that the short form supports also around methods. It is also important to note that effective methods are cached, that is unless the generic function or the method combination changes, the computation of the effective method may be called only once per the set of effective methods.

                                                          Admittedly these examples are not very useful. Usually we operate on data stored in instances and this is not a good abstraction to achieve that. Method combinations are useful to control method invocations and their results. Here is another example:

                                                          (defmacro majority-vote (&rest method-calls)
                                                            (let* ((num-methods (length method-calls))
                                                                   (tie-methods (/ num-methods 2)))
                                                              `(prog ((yes 0) (no 0))
                                                                  ,@(loop for invocation in method-calls
                                                                          append `((if ,invocation
                                                                                       (incf yes)
                                                                                       (incf no))
                                                                                     ((> yes ,tie-methods)
                                                                                      (return (values t yes no)))
                                                                                     ((> no ,tie-methods)
                                                                                      (return (values nil yes no))))))
                                                                  (error "we have a tie! ~d ~d" yes no))))
                                                          (define-method-combination majority-vote)
                                                          (defclass a () ())
                                                          (defclass b (a) ())
                                                          (defclass c (b) ())
                                                          (defclass d (c) ())
                                                          (defgeneric foo (object param)
                                                            (:method-combination majority-vote)
                                                            (:method majority-vote ((o a) param) nil)
                                                            (:method majority-vote ((o b) param) t)
                                                            (:method majority-vote ((o c) param) t)
                                                            (:method majority-vote ((o d) param) nil))
                                                          (foo (make-instance 'a) :whatever) ; (values nil 0 1)
                                                          (foo (make-instance 'b) :whatever) ; #<error tie 1 1>
                                                          (foo (make-instance 'c) :whatever) ; (values t 2 0)
                                                          (foo (make-instance 'd) :whatever) ; #<error tie 2 2>

                                                          Defining method combinations - the long form

                                                          The long form is much more interesting. It allows us to specify numerous qualifiers and handle methods without any qualifiers at all.

                                                          The Hooker

                                                          Here we will define a method combination that allows us to define named hooks that are invoked before or after the method. It is possible to have any number of hooks for the same set of arguments (something we can't achieve with the standard :before and :after auxiliary methods):

                                                          (defun combine-auxiliary-methods (primary around before after)
                                                            (labels ((call-primary ()
                                                                       `(call-method ,(first primary) ,(rest primary)))
                                                                     (call-methods (methods)
                                                                       (mapcar (lambda (method)
                                                                                 `(call-method ,method))
                                                                     (wrap-after (the-form)
                                                                       (if after
                                                                           `(multiple-value-prog1 ,the-form
                                                                              ,@(call-methods after))
                                                                     (wrap-before (the-form)
                                                                       (if before
                                                                              ,@(call-methods before)
                                                                     (wrap-around (the-form)
                                                                       (if around
                                                                           `(call-method ,(first around)
                                                                                         (,@(rest around)
                                                                                          (make-method ,the-form)))
                                                              (wrap-around (wrap-after (wrap-before (call-primary))))))
                                                          (define-method-combination hooker ()
                                                            ((normal-before (:before))
                                                             (normal-after  (:after)
                                                                            :order :most-specific-last)
                                                             (normal-around (:around))
                                                             (hook-before   (:before *))
                                                             (hook-after    (:after  *)
                                                                            :order :most-specific-last)
                                                             (hook-around   (:around *))
                                                             (primary () :required t))
                                                            (let ((around (append hook-around normal-around))
                                                                  (before (append hook-before normal-before))
                                                                  (after  (append normal-after hook-after)))
                                                              (combine-auxiliary-methods primary around before after)))

                                                          With this we may define a generic function and associated methods similar to other functions with an extra feature - we may provide named :before, :after and :around methods. Named auxiliary methods take a precedence over unnamed ones. Only after that the specialization is considered. There is one caveat - PCL-derived CLOS implementations (clasp, cmucl, ecl, sbcl) currently ([2023-01-18 śro]) have a bug preventing wildcard qualifier pattern symbol * from working. So better download ccl or wait for fixes. Here's an example for using it:

                                                          ;;; The protocol.
                                                          (defgeneric note-buffer-dimensions-changed (buffer w h)
                                                            (:method (b w h)
                                                              (declare (ignore b w h))
                                                          (defgeneric change-dimensions (buffer w h)
                                                            (:method-combination hooker))
                                                          ;;; The implementation of unspecialized methods.
                                                          (defmethod change-dimensions :after (buffer w h)
                                                            (note-buffer-dimensions-changed buffer w h))
                                                          ;;; The stanard class.
                                                          (defclass buffer ()
                                                            ((w :initform 0 :accessor w)
                                                             (h :initform 0 :accessor h)))
                                                          ;;; The implementation for the standard class.
                                                          (defmethod change-dimensions ((buffer buffer) w h)
                                                            (print "... Changing the buffer size ...")
                                                            (setf (values (w buffer) (h buffer))
                                                                  (values w h)))
                                                          (defmethod note-buffer-dimensions-changed ((buffer buffer) w h)
                                                            (declare (ignore buffer w h))
                                                            (print "... Resizing the viewport ..."))
                                                          ;;; Some dubious-quality third-party code that doesn't want to interfere with
                                                          ;;; methods defined by the implementation.
                                                          (defmethod change-dimensions :after system (buffer w h)
                                                            (print `(log :something-changed ,buffer ,w ,h)))
                                                          (defmethod change-dimensions :after my-hook ((buffer buffer) w h)
                                                            (print `(send-email! :me ,buffer ,w ,h)))
                                                          CL-USER> (defvar *buffer* (make-instance 'buffer))
                                                          CL-USER> (change-dimensions *buffer* 10 30)
                                                          "... Changing the buffer size ..." 
                                                          "... Resizing the viewport ..." 
                                                          (LOG :SOMETHING-CHANGED #<BUFFER #x30200088220D> 10 30) 
                                                          (SEND-EMAIL! :ME #<BUFFER #x30200088220D> 10 30) 

                                                          The Memoizer

                                                          Another example (this time it will work on all implementations) is optional memoization of the function invocation. If we define a method with the qualifier :memoize then the result will be cached depending on arguments. The method combination allows also "normal" auxiliary functions by reusing the function combine-auxiliary-methods from the previous section.

                                                          The function ensure-memoized-result accepts the following arguments:

                                                          • test: compare generations
                                                          • memo: a form that returns the current generation
                                                          • cache-key: a list composed of a generic function and its arguments
                                                          • form: a form implementing the method to be called

                                                          When the current generation is nil that means that caching is disabled and we remove the result from the cache. Otherwise we use the test to compare the generation of a cached value and the current one - if they are the same, then the cached value is returned. Otherwise it is returned.

                                                          (defparameter *memo* (make-hash-table :test #'equal))
                                                          (defun ensure-memoized-result (test memo cache-key form)
                                                            `(let ((new-generation ,memo))
                                                               (if (null new-generation)
                                                                     (remhash ,cache-key *memo*)
                                                                   (destructuring-bind (old-generation . cached-result)
                                                                       (gethash ,cache-key *memo* '(nil))
                                                                     (apply #'values
                                                                            (if (,test old-generation new-generation)
                                                                                 (setf (gethash ,cache-key *memo*)
                                                                                       (list* new-generation (multiple-value-list ,form))))))))))

                                                          The method with the qualifier :memoize is used to compute the current generation key. When there is no such method then the function behaves as if the standard method combination is used. The method combination accepts a single argument test, so it is possible to define different predicates for deciding whether the cache is up-to-date or not.

                                                          (define-method-combination memoizer (test)
                                                            ((before (:before))
                                                             (after  (:after) :order :most-specific-last)
                                                             (around (:around))
                                                             (memoize (:memoize))
                                                             (primary () :required t))
                                                            (:arguments &whole args)
                                                            (:generic-function function)
                                                            (let ((form (combine-auxiliary-methods primary around before after))
                                                                  (memo `(call-method ,(first memoize) ,(rest memoize)))
                                                                  (ckey `(list* ,function ,args)))
                                                              (if memoize
                                                                  (ensure-memoized-result test memo ckey form)

                                                          Now let's define a function with "our" method combination. We will use a counter to verify that values are indeed cached.

                                                          (defparameter *counter* 0)
                                                          (defgeneric test-function (arg &optional opt)
                                                            (:method-combination memoizer eql))
                                                          (defmethod test-function ((arg integer) &optional opt)
                                                            (list* `(:counter ,(incf *counter*)) arg opt))
                                                          CL-USER> (test-function 42)
                                                          ((:COUNTER 1) 42)
                                                          CL-USER> (test-function 42)
                                                          ((:COUNTER 2) 42)
                                                          CL-USER> (defmethod test-function :memoize ((arg integer) &optional (cache t))
                                                                     (and cache :gen-z))
                                                          #<STANDARD-METHOD TEST-FUNCTION :MEMOIZE (INTEGER)>
                                                          CL-USER> (test-function 42)
                                                          ((:COUNTER 3) 42)
                                                          CL-USER> (test-function 42)
                                                          ((:COUNTER 3) 42)
                                                          CL-USER> (test-function 42 nil)
                                                          ((:COUNTER 4) 42)
                                                          CL-USER> (test-function 42)
                                                          ((:COUNTER 3) 42)
                                                          CL-USER> (test-function 43)
                                                          ((:COUNTER 5) 43)
                                                          CL-USER> (test-function 43)
                                                          ((:COUNTER 5) 43)
                                                          CL-USER> (defmethod test-function :memoize ((arg (eql 43)) &optional (cache t))
                                                                     (and cache :gen-x))
                                                          #<STANDARD-METHOD TEST-FUNCTION :MEMOIZE ((EQL 43))>
                                                          CL-USER> (test-function 43)
                                                          ((:COUNTER 6) 43)
                                                          CL-USER> (test-function 43)
                                                          ((:COUNTER 6) 43)
                                                          CL-USER> (test-function 42)
                                                          ((:COUNTER 3) 42)


                                                          Method combinations are a feature that is often overlooked but give a great deal of control over the generic function invocation. The fact that ccl is the only implementation from a few that I've tried which got method combinations "right" doesn't surprise me - I've always had an impression that it shines in many unexpected places.

                                                          Nicolas MartyanoffANSI color rendering in SLIME

                                                          · 67 days ago

                                                          I was working on the terminal output for a Common Lisp logger, and I realized that SLIME does not interpret ANSI escape sequences.

                                                          This is not the end of the world, but having at least colors would be nice. Fortunately there is a library to do just that.

                                                          First let us install the package, here using use-package and straight.el.

                                                          (use-package slime-repl-ansi-color
                                                            :straight t)

                                                          While in theory we are supposed to just add slime-repl-ansi-color to slime-contribs, it did not work for me, and I add to enable the minor mode manually.

                                                          If you already have a SLIME REPL hook, simply add (slime-repl-ansi-color-mode 1). If not, write an initialization function, and add it to the SLIME REPL initialization hook:

                                                          (defun g-init-slime-repl-mode ()
                                                            (slime-repl-ansi-color-mode 1))
                                                          (add-hook 'slime-repl-mode-hook 'g-init-slime-repl-mode)

                                                          To test that it works as intended, fire up SLIME and print a simple message using ANSI escape sequences:

                                                          (let ((escape (code-char 27)))
                                                            (format t "~C[1;33mHello world!~C[0m~%" escape escape))

                                                          While it is tempting to use the #\Esc character, it is part of the Common Lisp standard; therefore we use CODE-CHAR to obtain it from its ASCII numeric value. We use two escape sequences, the first one to set the bold flag and foreground color, and the second one to reset display status.

                                                          If everything works well, should you see a nice bold yellow message:

                                                          ANSI escape sequence rendering

                                                          LispjobsDevOps Engineer | HRL Laboratories | Malibu, CA

                                                          · 70 days ago

                                                          Job posting:

                                                          HRL has been on the leading edge of technology, conducting pioneering research and advancing the state of the art. This position is integrated with a growing team of scientists and engineers on HRL's quantum computing research program.

                                                          GENERAL DESCRIPTION:

                                                          As a DevOps/DevSecOps engineer, you’ll be focused on maintaining reliable systems for testing and delivery of HRL’s quantum software. (You will not be directly responsible for developing the software or its tests.)

                                                          Specifically, you will be responsible for:

                                                          * Monitoring the status of CI/CD infrastructure on open and air-gapped networks.

                                                          * Building and maintaining infrastructure for synchronizing software between open and air-gapped networks.

                                                          * Working closely with developers and IT staff to ensure continued reliability of integration and deployment infrastructure.

                                                          * Tracking and vetting software dependencies.

                                                          * Looking for and implementing improvements to DevSecOps practices.

                                                          Among other candidate requirements, we highly value expertise in Lisp, Python, and C++.

                                                          CONFIDENTIALITY NOTICE: The information transmitted in this email, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential, proprietary and/or privileged material exempt from disclosure under applicable law. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this message in error, please contact the sender immediately and destroy any copies of this information in their entirety.

                                                          Nicolas MartyanoffSwitching between implementations with SLIME

                                                          · 71 days ago

                                                          While I mostly use SBCL for Common Lisp development, I regularly switch to CCL or even ECL to run tests.

                                                          This is how I do it with SLIME.

                                                          Starting implementations

                                                          SLIME lets you configure multiple implementations using the slime-lisp-implementations setting. In my case:

                                                          (setq slime-lisp-implementations
                                                             '((sbcl ("/usr/bin/sbcl" "--dynamic-space-size" "2048"))
                                                               (ccl ("/usr/bin/ccl"))
                                                               (ecl ("/usr/bin/ecl"))))

                                                          Doing so means that running M-x slime will execute the first implementation, i.e. SBCL. There are two ways to run other implementations.

                                                          First you can run C-u M-x slime which lets you type the path and arguments of the implementation to execute. This is a bit annoying because the prompt starts with the content of the inferior-lisp-program variable, i.e. "lisp" by default, meaning it has to be deleted manually each time. Therefore I set inferior-lisp-program to the empty string:

                                                          (setq inferior-lisp-program "")

                                                          Then you can run C-- M-x slime (or M-- M-x slime which is easier to type) to instruct SLIME to use interactive completion (via completing-read) to let you select the implementations among those configured in slime-lisp-implementations.

                                                          To make my life easier, I bind C-c C-s s to a function which always prompt for the implementation to start:

                                                          (defun g-slime-start ()
                                                            (let ((current-prefix-arg '-))
                                                              (call-interactively 'slime)))

                                                          Using C-c C-s as prefix for all my global SLIME key bindings helps me remember them.

                                                          Switching between multiple implementations

                                                          Running the slime function several times will create multiple connections as expected. Commands executed in Common Lisp buffers are applied to the current connection, which is by default the most recent one.

                                                          There are two ways to change the current implementation:

                                                          1. Run M-x slime-next-connection.
                                                          2. Run M-x slime-list-connections, which opens a buffer listing connections, and lets you choose the current one with the d key.

                                                          I find both impractical: the first one does not let me choose the implementation, forcing me to run potentially several times before getting the one I want. The second one opens a buffer but does not switch to it.

                                                          All I want is a prompt with completion. So I wrote one.

                                                          First we define a function to select a connection among existing one:

                                                          (defun g-slime-select-connection (prompt)
                                                            (let* ((connections-data
                                                                    (mapcar (lambda (process)
                                                                              (cons (slime-connection-name process) process))
                                                                      (lambda (string)
                                                                        (let* ((process (alist-get string minibuffer-completion-table
                                                                                                   nil nil #'string=))
                                                                               (contact (process-contact process)))
                                                                          (if (consp contact)
                                                                              (format "  %s:%s" (car contact) (cadr contact))
                                                                            (format "  %S" contact))))))
                                                                   (connection-name (completing-read prompt connections-data)))
                                                              (let ((connection (cl-find connection-name slime-net-processes
                                                                                         :key #'slime-connection-name
                                                                                         :test #'string=)))
                                                                (or connection
                                                                    (error "Unknown SLIME connection %S" connection-name)))))

                                                          Then use it to select a connection as the current one:

                                                          (defun g-slime-switch-connection ()
                                                            (let ((connection (g-slime-select-connection "Switch to connection: ")))
                                                              (slime-select-connection connection)
                                                              (message "Using connection %s" (slime-connection-name connection))))

                                                          I bind this function to C-c C-s c.

                                                          In a perfect world, we could format nice columns in the prompt and highlight the current connection, but the completing-read interface is really limited, and I did not want to use an external package such as Helm.

                                                          Stopping implementations

                                                          Sometimes it is necessary to stop an implementations and kill all associated buffers. It is not something I use a lot; but when I need it, it is frustrating to have to switch to the REPL buffer, run slime-quit-lisp, then kill the REPL buffer manually.

                                                          Adding this feature is trivial with the g-slime-select-connection defined earlier:

                                                          (defun g-slime-kill-connection ()
                                                            (let* ((connection (g-slime-select-connection "Kill connection: "))
                                                                   (repl-buffer (slime-repl-buffer nil connection)))
                                                              (when repl-buffer
                                                                (kill-buffer repl-buffer))
                                                              (slime-quit-lisp-internal connection 'slime-quit-sentinel t)))

                                                          Finally I bind this function to C-c C-s k.

                                                          It is now much more comfortable to manage multiple implementations.

                                                          Tim BradshawA case-like macro for regular expressions

                                                          · 72 days ago

                                                          I often find myself wanting a simple case-like macro where the keys are regular expressions. regex-case is an attempt at this.

                                                          I use CL-PPCRE for the usual things regular expressions are useful for, and probably for some of the things they should not really be used for as well. I often find myself wanting a case like macro, where the keys are regular expressions. There is a contributed package for Trivia which will do this, but Trivia is pretty overwhelming. So I gave in and wrote regex-case which does what I want.

                                                          regex-case is a case-like macro. It looks like

                                                          (regex-case <thing>
                                                            (<pattern> (...)
                                                             <form> ...)
                                                            (otherwise ()
                                                             <form> ...))

                                                          Here <pattern> is a literal regular expression, either a string or in CL-PPCRE’s s-expression parse-tree syntax for them. Unlike case there can only be a single pattern per clause: allowing the parse-tree syntax makes it hard to do anything else. otherwise (which can also be t) is optional but must be last.

                                                          The second form in a clause specifies what, if any, variables to bind on a match. As an example

                                                          (regex-case line
                                                            ("fog\\s+(.*)\\s$" (:match m :registers (v))

                                                          will bind m to the whole match and v to the substring corresponding to the first register. You can also bind match and register positions. A nice (perhaps) thing is that you can not bind some register variables:

                                                          (regex-case line
                                                            (... (:registers (_ _ v))

                                                          will bind v to the substring corresponding to the third register. You can use nil instead of _.

                                                          The current state of regex-case is a bit preliminary: in particular I don’t like the syntax for binding thngs very much, although I can’t think of a better one. Currently therefore it’s in my collection of toys: it will probably migrate from there at some point.

                                                          Currently documentation is here and source code is here.

                                                          Nicolas HafnerKandria is now out!

                                                          · 72 days ago

                                                          Kandria is now finally available for purchase and play!

                                                          I recommend buying it on Steam, as the algorithm there will help us bring the game in front of more people, as well. However, if that isn't a possibility for you, there's also options on and through Xsolla on our webpage:

                                                          I am also live on Steam, Twitch, and YouTube right now, to celebrate the launch! Come on and hang out in the chat:

                                                          I hope you all enjoy the game, and thank you very much for sticking with us for all this time!

                                                          Nicolas HafnerKandria launches tomorrow!

                                                          · 73 days ago

                                                          ​Kandria launches tomorrow, on Wednesday the 11th, at 15:00 CET / 9:00 EST!

                                                          There'll be a launch stream for the occasion as well. It'll be live on Twitch! I'll be happy to answer any questions you may have about the game, and hope to see you there!​

                                                          Last opportunity to wishlist the game, too:

                                                          vindarelThese Years in Common Lisp: 2022 in review

                                                          · 74 days ago

                                                          And 2022 is over. The Common Lisp language and environment are solid and stable, yet evolve. Implementations, go-to libraries, best practices, communities evolve. We don’t need a “State of the Ecosystem” every two weeks but still, what happened and what did you miss in 2022?

                                                          This is my pick of the most exciting, fascinating, interesting or just cool projects, tools, libraries and articles that popped-up during that time (with a few exceptions that appeared in late 2021).

                                                          This overview is not a “State of the CL ecosystem” (HN comments (133)) that I did in 2020, for which you can find complementary comments on HN.

                                                          I think this article (of sorts) is definitely helpful for onlookers to Common Lisp, but doesn’t provide the full “story” or “feel” of Common Lisp, and I want to offer to HN my own perspective.

                                                          And, suffice to say, I tried to talk about the most important things, but this article (of sorts) is by no means a compilation of all new CL projects or all the articles published on the internet. Look on Reddit, Quicklisp releases, Github, and my favourite resources:

                                                          If I had to pick 3 achievements they would be:

                                                          • SBCL developments: SBCL is now callable as a shared library. See below in “Implementations”.
                                                          • a new 3D graphics project: Kons-9: “The idea would be to develop a system along the lines of Blender/Maya/Houdini, but oriented towards the strengths of Common Lisp”. And the project progresses at a good pace.
                                                          • CLOG, the Common Lisp Omnificent GUI. It’s like a GUI framework to create web apps. Based on websockets, it offers a light abstraction to create fully-dynamic web applications, in Common Lisp. It has lots of demos to create websites, web apps, games, and it ships a complete editor. For development, we can connect our Lisp REPL to the browser, and see changes on the fly. The author had a similar commercial product written in Ada, discovered Common Lisp, and is now super active on this project.

                                                          Let’s go for more.

                                                          Thanks to @k1d77a, @Hexstream, @digikar and @stylewarning for their feedback.

                                                          Table of Contents


                                                          A newcomer to Lisp came, asked a question, and suddenly he created a super useful rendering of the specification. Check it out!

                                                          But that’s not all, he also started work on a new Common Lisp editor, built in Rust and Tauri, see below.

                                                          We continue to enrich the Common Lisp Cookbook. You are welcome to join, since documention is best built by newcomers for newcomers.

                                                          A resurrected project:



                                                          We saw achievements in at least 7 8 implementations.

                                                          New implementation! It’s 2022 and people start new CL implementations.

                                                          • NPT - an implementation of ANSI Common Lisp in C.

                                                          See also:

                                                          • LCL, Lua Common Lisp - The goal of this project is to provide an implementation of Common Lisp that can be used wherever an unmodified Lua VM is running.
                                                            • not a complete implementation.

                                                          They are doing great work to revive a Lisp machine:

                                                          Medley Interlisp is a project aiming to restore the Interlisp-D software environment of the Lisp Machines Xerox produced since the early 1980s, and rehost it on modern operating systems and computers. It’s unique in the retrocomputing space in that many of the original designers and implementors of major parts of the system are participating in the effort.

                                                          Paolo Amoroso blog post: my encounter with Medley Interlisp.


                                                          I won’t list expired job announces, but this year Lispers could apply for jobs in: web development(WebCheckout, freelance announces), cloud service providers (Keepit), big-data analysis (Ravenpack, and chances are they are still hiring)), quantum computing (HLR Laboratories), AI (Mind AI, SRI International), real-time data aggregration and alerting engines for energy systems (3E); for a startup building autism tech (and using CLOG already); there have been a job seeking to rewrite a Python backend to Common Lisp (RIFFIT); there have been some bounties; etc.

                                                          Prior Lisp experience was not 100% necessary. There were openings for junior and senior levels, remote and not remote (Australia for “a big corp”, U.S., Spain, Ukraine...).

                                                          Comes a question:

                                                          I remind the reader that most Lisp jobs do not have a public job posting, instead candidates are often found organically on the community channels: IRC, Twitter, Discord, Reddit... or teams simply train their new developer.

                                                          In 2022 we added a few companies to the ongoing, non-official list on awesome-lisp-companies. If your company uses Common Lisp, feel free to tell us on an issue or in the comments!

                                                          For example, “is entirely Lisp”.

                                                          Lisp was a conscious decision because it allows a small team to be incredibly productive, plus the fact that it’s a live image allows you to connect to it over the internet and poke and prod the current state, which has really allowed a much clearer understanding of the data.

                                                          They post SLY screenshots on their Twitter^^

                                                          Evacsound (HN):

                                                          We’re using CL in prod for an embedded system for some years now, fairly smooth sailing. It started out as an MVP/prototype so implementation was of no concern, then gained enough velocity and market interest that a rewrite was infeasible. We re-train talent on the job instead.

                                                          Pandorabots, or barefootnetworks, designing the Intel Tofino programmable switches, and more.


                                                          Language libraries

                                                          Editors, online editors, REPLs, plugins

                                                          New releases:


                                                          See also lisp-actors, which also does networking. It looks like more of a research project, as it doesn’t have unit-tests nor documentation, but it was used for the (stopped) Emotiq blockchain.



                                                          More choices: awesome-cl#databases.

                                                          Delivery tools

                                                          There has been outstanding work done there. It is also great to see the different entities working on this. That includes SBCL developers, Doug Katzman of Google, and people at HRL Laboratories (also responsible of Coalton, Haskell-like on top of CL).

                                                          Have you ever wanted to call into your Lisp library from C? Have you ever written your nice scientific application in Lisp, only to be requested by people to rewrite it in Python, so they can use its functionality? Or, maybe you’ve written an RPC or pipes library to coordinate different programming languages, running things in different processes and passing messages around to simulate foreign function calls.

                                                          [...] If you prefer using SBCL, you can now join in on the cross-language programming frenzy too.


                                                          Kandria launches on Steam on the 11th of January, in two days!

                                                          🎥 Kandria trailer.

                                                          Graphics, GUIs

                                                          We saw the release of fresh bindings for Gtk4.

                                                          We had bindings for Qt5... but they are still very rough, hard to install so far.



                                                          But an awesome novelty of 2022 is Kons-9.

                                                          Kons-9, a new 3D graphics project

                                                          🚀 A new 3D graphics project: Kons-9.

                                                          The idea would be to develop a system along the lines of Blender/Maya/Houdini, but oriented towards the strengths of Common Lisp.

                                                          I’m an old-time 3D developer who has worked in CL on and off for many years.

                                                          I don’t consider myself an expert [...] A little about me: · wrote 3D animation software used in "Jurassic Park" · software R&D lead on "Final Fantasy: The Spirits Within" movie · senior software developer on "The Hobbit" films.

                                                          Interfaces with other languages

                                                          • py4cl2-cffi: CFFI based alternative to py4cl2.
                                                            • it does one big new thing: it supports passing CL arrays by reference. That means we actually have access to numpy, scipy, and friends.
                                                            • “If py4cl2-cffi reaches stability, and I find that the performance of (i) cffi-numpy, (ii) magicl, as well as (iii) a few BLAS functions I have handcrafted for numericals turn out to be comparable, I might no longer have to reinvent numpy.” @digikar
                                                          • Small update to RDNZL (CL .NET bridge by Edi Weitz)
                                                            • forked project, added support for Int16, fixed Int64, re-building the supporting DLLs.
                                                            • see also: Bike
                                                          • jclass: Common Lisp library for Java class file manipulation

                                                          For more, see awesome-cl.

                                                          Numerical and scientific

                                                          • 🚀 new Lisp Stats release
                                                            • “emphasis on plotting and polishing of sharp edges. data-frames, array operations, documentation.”
                                                            • HN comments (55)
                                                            • ” I’ve been using lisp-stat in production as part of an algorithmic trading application I wrote. It’s been very solid, and though the plotting is (perhaps was, in light of this new release) kinda unwieldy, I really enjoyed using it. Excited to check out the newest release.”
                                                            • “For example, within Lisp-Stat the statistics routines [1] were written by an econometrician working for the Austrian government (Julia folks might know him - Tamas Papp). It would not be exaggerating to say his job depending on it. These are state of the art, high performance algorithms, equal to anything available in R or Python. So, if you’re doing econometrics, or something related, everything you need is already there in the tin.”
                                                            • “For machine learning, there’s CLML, developed by NTT. This is the largest telco in Japan, equivalent to ATT in the USA. As well, there is MGL, used to win the Higgs Boson challenge a few years back. Both actively maintained.”
                                                            • “For linear algebra, MagicCL was mention elsewhere in the thread. My favourite is MGL-MAT, also by the author of MGL. This supports both BLAS and CUBLAS (CUDA for GPUs) for solutions.”
                                                            • “Finally, there’s the XLISP-STAT archive. Prior to Luke Tierney, the author of XLISP-Stat joining the core R team, XLISP-STAT was the dominate statistical computing platform. There’s heaps of stuff in the archive, most at least as good as what’s in base R, that could be ported to Lisp-Stat.”
                                                            • “Common Lisp is a viable platform for statistics and machine learning. It isn’t (yet) quite as well organised as R or Python, but it’s all there.”
                                                          • numericals - Performance of NumPy with the goodness of Common Lisp
                                                          • MGL-MAT - a library for working with multi-dimensional arrays which supports efficient interfacing to foreign and CUDA code. BLAS and CUBLAS bindings are available.
                                                          • hbook - Text-based histograms in Common Lisp inspired by the venerable HBOOK histogramming library from CERN.

                                                          New releases:

                                                          • Maxima 5.46 was released.
                                                            • “Maxima is a Computer Algebra System comparable to commercial systems like Mathematica and Maple. It emphasizes symbolic mathematical computation: algebra, trigonometry, calculus, and much more.”
                                                            • see its frontends, for example WxMaxima.

                                                          Call to action:


                                                          Screenshotbot (Github) was released. It is “a screenshot testing service to tie with your existing Android, iOS and Web screenshot tests”.

                                                          It is straightforward to install with a Docker command. They offer more features and support with their paid service.

                                                          LicensePrompt was released. It is “a single place to track all recurring software and IT expenses and send relevant reminders to all interested people”. It’s built in CL, interface with HTMX.


                                                          • jingle: Common Lisp web framework with bells and whistles (based on ningle)
                                                            • jingle demo: OpenAPI 3.x spec, Swagger UI, Docker and command-line interface app with jingle.
                                                          • ciao: Ciao is an easy-to-use Common Lisp OAuth 2.0 client library. It is a port of the Racket OAuth 2.0 Client to Common Lisp.
                                                          • stepster: a web scraping library, on top of Plump and Clss (new in QL)
                                                          • openrpc: Automatic OpenRPC spec generation, automatic JSON-RPC client building
                                                          • HTTP/2 implementation in Common Lisp


                                                          • cl-cookieweb: my project skeleton to start web projects. Demo in video. I am cheating, the bulk of it was done in 2021.
                                                            • “Provides a working toy web app with the Hunchentoot web server, easy-routes, Djula templates, styled with Bulma, based on SQLite, with migrations and an example table definition.”
                                                            • if you don’t know where to start for web dev in CL, enjoy all the pointers of this starter kit and find your best setup.
                                                            • see also this web template by @dnaeon, and check out all his other Lisp libraries.


                                                          • 👍 lisp-pay: Wrappers around various Payment Processors (Paypal, Stripe, Coinpayment)
                                                          • lunamech-matrix-api: Implementation of the Matrix API, LunaMech a Matrix bot


                                                          • Ackfock - a platform of mini agreements and mini memos of understanding (built with CLOG, closed source).
                                                          • todolist-cl: a nice looking todolist with a web UI, written in Common Lisp (and by a newcomer to CL, to add credit)

                                                          I don’t have lots of open-source apps to show. Mines are running in production and all is going well. I share everything on my blog posts. I also have an open-source one in development, but that’s for the 2023 showcase :D


                                                          🚀 The awesome novelty of 2022 I spoke of in the introduction is CLOG, the Common Lisp Omnificent GUI:

                                                          The CLOG system browser

                                                          I know of one open-source consequent CLOG app: mold-desktop, in development.

                                                          I’m developing a programmable desktop and a bookmarks manager application with CLOG. I think I know one of the things that make CLOG user interfaces so easy to develop. It is that they are effortlessly composable. That’s it for now :)


                                                          New releases

                                                          There are lots of awesome projects in music composition, including OpusModus and OpenMusic which saw new releases. I also like to cite ScoreCloud, a mobile app built with LispWorks, where you whistle, sing or play your instrument, and the app writes the music score O_o

                                                          See awesome-cl and Cliki for more.

                                                          (re) discoveries





                                                          Around the language


                                                          Call for action:

                                                          Screencasts and podcasts

                                                          New videos by me:

                                                          by Gavin Freeborn:

                                                          KONS-9 series:

                                                          CLOG series:

                                                          CL study group:


                                                          and of course, find 3h48+ of condensed Lisp content on my Udemy video course! (I’m still working on new content, as a student you get updates).

                                                          Aside screncasts, some podcasts:

                                                          Other discussions


                                                          Learning Lisp

                                                          Common Lisp VS ...

                                                          Thanks everyone, happy lisping and see you around!

                                                          Nicolas MartyanoffImproving Git diffs for Lisp

                                                          · 75 days ago

                                                          All my code is stored in various Git repositories. When Git formats a diff between two objects, it generates a list of hunks, or groups of changes.

                                                          Each hunk can be displayed with a title which is automatically extracted. Git ships with support for multiple languages, but Lisp dialects are not part of it. Fortunately Git lets users configure their own extraction.

                                                          The first step is to identify the language using a pattern applied to the filename. Edit your Git attribute file at $HOME/.gitattributes and add entries for both Emacs Lisp and Common Lisp:

                                                          *.lisp diff=common-lisp
                                                          *.el diff=elisp

                                                          Then edit your Git configuration file at $HOME/.gitconfig and configure the path of the Git attribute file:

                                                              attributesfile = ~/.gitattributes

                                                          Finally, set the regular expression used to match a top-level function name:

                                                          [diff "common-lisp"]
                                                          [diff "elisp"]

                                                          For Lisp dialects, we do not just identify function names: it is convenient to identify hunks for all sorts of top-level definitions. We use a regular expression which captures the first symbol of the form and the name that follows.

                                                          Of course you can modifiy these expressions to identify more complex top-level forms. For example, for Emacs Lisp, I also want to identify use-package expressions.

                                                          You can see the result in all tools displaying Git diffs, for example in Magit with Common Lisp code:

                                                          Common Lisp diff

                                                          Or for my Emacs configuration file:

                                                          Emacs Lisp diff

                                                          Hunk titles, highlighted in blue, now contain the type and name of the top-level construction the changes are associated with.

                                                          A simple change, but one which really helps reading diffs.

                                                          Nicolas HafnerKandria releases in one week on January 11!

                                                          · 79 days ago

                                                          In case you missed the yearly update last week: Kandria will release in one week from today, on January 11th, 15:00 CET / 09:00 EST. I hope you're as excited to play it as we are to finally get it into your hands!

                                                          Please remember to wishlist it on Steam to make sure you don't miss it!

                                                          Nicolas Hafner2022 for Kandria in Review

                                                          · 84 days ago

                                                          It's that time of the year again! The end of it. And what a year it's been for Kandria. We're now less than two weeks away from the release. Yikes! Or should I say, woah! Well, let's take a moment and look at some of all of the things that happened, before we look at what the future may possibly hold in store for us. At least, if I have anything to say about it.

                                                          Honestly, so many things happened that I barely remember most of them. I had to go back through the monthly reviews to remember all of it. But then again, I've always been rather terrible at remembering things that far back in any chronologically complete manner. I won't be going over stuff in chronological order, either, but instead will touch on a bunch of individual topics. Let's start out with


                                                          In 2022 we were present in person at quite a number of conferences:

                                                          • European Lisp Symposium in Portugal

                                                          • Digital Dragons in Poland

                                                          • Develop: Brighton in England

                                                          • Gamescom & Devcom in Germany

                                                          • HEROFest in Switzerland

                                                          We got a lot of useful feedback from random people trying the game out at the events, and also got to meet a lot of friendly and great developers from around the World. That all said, these conferences are also quite taxing and costly. We got the booth sponsored for all of them, but travel expenses are still not nothing, not to mention the work time. Travelling is also quite exhausting to me in general, so I hope I won't have to zip around the place quite as much next year.

                                                 Swiss Games booth at Gamescom

                                                          However, I can already say that - unforeseen circumstances notwithstanding - I will be at the European Lisp Symposium in the Netherlands, and Tokyo Game Show in Japan.

                                                          Kickstarter & Steam Next Fest

                                                          In July we had a big double whammy of our Kickstarter and the Steam Next Fest, both launched at the same time. This was also our first big attempt at pushing for some marketing. We tried out Facebook ads, which weren't of much use at all. We also contacted a number of streamers and influencers, a few of which actually gave the new demo we had at the time a shot. It was a lot of fun to watch them play through it and chat with them as they did so.

                                                 gif I made in an attempt to illustrate Kandria's large vertical map

                                                          Leading up to release and during it I imagine we'll have a few more such stream appearances. If you see a streamer playing Kandria, please don't hesitate to notify us in the Discord and I'd be happy to drop by in chat. Assuming I'm not asleep at the time, of course!

                                                          Anyway, the Kickstarter went rather well for us, and we managed to get funded in the first week. After that it was mostly coasting along, giving an update every now and again to keep spirits up and push for those stretch goals (more on that later).

                                                          I'm really happy that things weren't as hectic as they are often described as being, as I was still able to focus on developing the game. Losing a month of work would have made things quite a bit more troublesome later on.

                                                          Still, I'm also very well aware that the reason things went so well for us is mostly down to the fact that we had a rather low goal set, and that we had a lot of support from the programming, and especially Common Lisp community, many of which chipped in rather large sums. Thank you all very much!

                                                          I'm not sure that launching your Kickstarter alongside the Next Fest was a good idea. It's definitely a good idea to have a demo available for your Kickstarter, so that people can trust in your abilities to deliver a complete product, but I don't know if the cross-promotion idea worked out. It might be better to have the next fest part way through the Kickstarter or even at the end of it, or entirely separate them, to have two marketing beats rather than one bigger one. Still, it's impossible to say whether it would have gone better or worse overall if we had done it differently, so I'm not complaining. More just thinking about how I'd do this if there's going to be another similar thing in the future.


                                                          At the start of the year we didn't even have the full game map ready yet, let alone all the assets, quests, or dialogue. A number of important features were also missing still, both in the engine and in the game itself.

                                                          Thinking back to it now it is kind of insane how much of the game was still missing. I know there's folks that can put together a full game in less than a year total, but that's usually much larger teams, or far smaller games.

                                                          There were quite a few painful stretches of arduous work. Filling out the entire map with interesting challenges was one, then going back and tiling it all was another. And finally going back again to add details and flairs everywhere was yet another. But, the game feels a lot more livelier and interesting now, so it was definitely worth all that extra work.

                                                          If you want to read up on all the nitty gritty of the development that happened during the year, you can browse back in time on the blog, or for even more detail, hop on by the mailing list.

                                                 running on the Steam Deck

                                                          One of the coolest parts for me was finally getting a Steam Deck (they're still not officially available in Switzerland), and seeing Kandria just... work for the most part. Having it be portable is really, really sweet. And it only took a couple of tweaks with the menuing to make it all run well. I would still love to also have the game on Switch, but we'll have to see about that later down the road.

                                                          Working up to Release

                                                          The game's been pretty much done since the end of November, and in the remaining time since then I've been working on translating the game into German. That took quite a bit of work, there's some 60'000 words, and I'm not the best at translating to begin with. The first draft of that is now done, and we should be good ready to get the game out there in both English and German by the release date.

                                                          Unfortunately there won't be any other languages for the foreseeable future. My funds have run very dry, and I need to save up again to be able to support the development of the next game (more on that later). However, if you're interested in localising the game yourself, you'll be able to do so soon. Please keep your eyes and ears open!


                                                          Aside from localisation work I've also ironed out some more bugs, cleaned up some stuff in the code base, developed an independent key distribution system so I can sell copies without being attached to a third-party platform, and added some more minor enhancements and changes along the way.

                                                          I really hope the release will go well, as far as I know there's only very minor outstanding issues.

                                                          The Release

                                                          So. Kandria is releasing on Wednesday, January 11th, 15:00 CET. It'll be released on Steam,, and our website. All versions will be DRM-free, though we get the biggest cut of the revenue if you buy it directly from our website.

                                                          In addition to this, the Soundtrack will also be available on Steam,, and Bandcamp, and on various streaming services such as Spotify.

                                                          If you were a backer of the Kickstarter campaign, you will receive your keys for the game and OST in the coming days.

                                                          Immediately on release I will be streaming the game on Steam and Twitch, so please join me there for a little celebration. After that I will be looking at any and all feedback that's coming in, and working on patches to address any fires that may be unveiled. And after I've addressed what I can, I think I'll take some more holidays to recenter myself and consider the coming year properly.


                                                          Even after the release, my work on Kandria will not be done quite yet. There's two big post-release updates thanks to the Kickstarter stretch goals that will be coming:

                                                          1. Level Editor. The initial release already includes the development level editor, but it is a bit rough around the edges and needs more usability and stability improvements. Once that is done, there'll be another big patch update along with a community event to encourage people to make and share their own levels.

                                                          2. Modding Support. While the game's source code will be available on release already, the second post-launch update will focus on two things: presenting an explicit API for mods, documentation for people to write their own mods for Kandria, and an in-game mod browser supported by

                                                          I cannot yet make any promises about when these updates will land, especially as I also need to start gearing up work for the next game project. That's right, I'm already planning and working on the next game, and I'm really excited about it. I don't want to reveal anything about it yet, but I think you'll be positively surprised when I do!

                                                          Since things are still a bit under covers at the moment I don't know if I'll be able to keep doing monthly roundups like this, though rest assured that I will keep you in the loop with any important developments, don't you worry about that.


                                                          I wanted to reserve this last section right at the very end of both the article and the year here just for you. Thank you so much. I know this is sappy, and I know this is cliche, and I know it is all of these things and many others, but I do genuinely feel blessed at this moment to have you reading about my work, and following along for such a long time. And the better you know me, the better you'll know how rare it is for me to express such genuine positivity, so I hope you will take it to heart and believe me when I say that I am very thankful to you, and I hope that you'll continue to follow my endeavours in the future as well.

                                                          Before I go, I have one last favour to ask of you: please share Kandria with your friends, colleagues, and groups. I know it may not seem like much, and I know it can feel awkward, but it is invaluable for someone like me that's just starting out in this industry. Even just a few more people can make a big difference. So please, share the Steam page, itch site, or our website with people.

                                                          And again, thank you. I hope you have a great new year.

                                                          Nicolas MartyanoffConfiguring SLIME cross-referencing

                                                          · 86 days ago

                                                          The SLIME Emacs package for Common Lisp supports cross-referencing: one can list all references pointing to a symbol, move through this list and jump to the source code of each reference.

                                                          Removing automatic reference jumps

                                                          While cross-referencing is very useful, the default configuration is frustrating: moving through the list in the Emacs buffer triggers the jump to the reference under the cursor. If you are interested in a reference in the middle of the list, you will have to move to it, opening multiple buffers you do not care about as a side effect. I finally took the time to fix it.

                                                          Key bindings for slime-ref-mode mode are stored in the slime-xref-mode-map keymap. After a quick look in slime.el, it is easy to remove bindings for slime-xref-prev-line and slime-xref-next-line:

                                                          (define-key slime-xref-mode-map (kbd "n") nil)
                                                          (define-key slime-xref-mode-map [remap next-line] nil)
                                                          (define-key slime-xref-mode-map (kbd "p") nil)
                                                          (define-key slime-xref-mode-map [remap previous-line] nil)

                                                          If you are using use-package, it is even simpler:

                                                          (use-package slime
                                                            (:map slime-xref-mode-map
                                                                 ([remap next-line])
                                                                 ([remap previous-line]))))

                                                          Changing the way references are used

                                                          SLIME supports two ways to jump to a reference:

                                                          1. With return or space, it spawns a buffer containing the source file and close the cross-referencing buffer.
                                                          2. With v, it spawns the source file buffer but keeps the cross-referencing buffer open and keeps it current.

                                                          This is not practical to me, so I made a change. The default action, triggered by return, now keeps the cross-referencing buffer open and switches to the source file in the same window. This way, I can switch back to the cross-referencing buffer with C-x b to select another reference without spawning buffers in other windows (I do not like having my windows hijacked by commands).

                                                          To do that, I need a new function:

                                                          (defun g-slime-show-xref ()
                                                            "Display the source file of the cross-reference under the point
                                                          in the same window."
                                                            (let ((location (slime-xref-location-at-point)))
                                                              (slime-goto-source-location location)
                                                              (with-selected-window (display-buffer-same-window (current-buffer) nil)
                                                                (goto-char (point))

                                                          Note the use of g-recenter-window, a custom function to move the current point at eye level. Feel free to use the builtin recenter function instead.

                                                          I then bind the function to return and remove other bindings:

                                                          (define-key slime-xref-mode-map (kbd "RET") 'g-slime-show-xref)
                                                          (define-key slime-xref-mode-map (kbd "SPC") nil)
                                                          (define-key slime-xref-mode-map (kbd "v") nil)

                                                          Much better now!

                                                          Tim BradshawThe empty list

                                                          · 98 days ago

                                                          My friend Zyni pointed out that someone has been getting really impressively confused and cross on reddit about empty lists, booleans and so on in Common Lisp, which led us to a discussion about what the differences between CL and Scheme really are here. Here’s a summary which we think is correct.

                                                          A peculiar object in Common Lisp1

                                                          In Common Lisp there is a single special object, nil.

                                                          • This represents both the empty list, and the special false value, all other objects being true.
                                                          • This object is a list and is the only list object which is not a cons.
                                                          • As such this object is an atom, and again it is the only list object which is an atom.
                                                          • You can take the car and cdr of this object: both of these operations return the object itself.
                                                          • This object is also a symbol, and it is the only object which is both a list and a symbol.
                                                          • The empty list when written as an empty list, (), is self-evaluating.

                                                          Some comments.

                                                          • It is necessary that there be a special empty-list object which is a list but not a cons: the things which are not necessary are that it be a symbol, and that it represent falsity.
                                                          • Combining the empty list and the special false object can lead to particularly good implementations perhaps.
                                                          • The implementation of this object is always going to be a bit weird.
                                                          • It is clear that the empty list cannot be any kind of compound form so requiring it to be quoted — requiring you to write '() really — serves no useful purpose. Nevertheless I (Tim) would probably rather CL did that.
                                                          • Not having to quote nil on the other hand is not at all strange: any symbol can be made self-evaluating simply by (defconstant s 's), for instance.
                                                          • The graph of types in CL is a DAG, not a tree: it is not at all strange that there is an object whose type is both list and symbol.

                                                          Some entirely mundane things in Common Lisp

                                                          • There is a symbol, t which represents the canonical true value. Nothing is magic about this symbol in any way: it could be defined by (defconstant t 't).
                                                          • There is a type, boolean which could be defined by (deftype boolean () '(member nil t)), except that it is required that boolean be a recognisable subtype of symbol. All implementations we have tried recognise (member nil t) as a subtype of symbol, but the standard does not require them to do so. Additionally (type-of 't) must return boolean we think.
                                                          • There is a type, null, which could be defined by (deftype null () '(member nil)) or (deftype null () '(eql nil)), with the same caveats as above, and (type-of nil) should return null.
                                                          • There are types named t (top of the type graph) and nil (bottom of type graph).

                                                          These mundane things are just that: they don’t require implementational magic at all.

                                                          Three peculiar objects in Scheme

                                                          In Scheme there is an object, ().

                                                          • () is the special object that represents the empty list.
                                                          • It does not represent false.
                                                          • It is not a symbol.
                                                          • It is the only list object which is not a pair (cons): list? is true of it but pair? is false.
                                                          • You can’t take the car or cdr of it.
                                                          • It is not self-evaluatiing.

                                                          There is another object, #f.

                                                          • #f is the distinguished false value and is the only false value in Scheme, all other objects being true.
                                                          • It is not a symbol or a list but satisfies the boolean? predicate.
                                                          • It is self-evaluating.

                                                          There is another object, #t.

                                                          • #t represents the canonical true value, but all objects other than #f are true.
                                                          • It is not a symbol or a list but satisfies the boolean? predicate.
                                                          • It is self-evaluating.

                                                          Some comments. - Scheme does not have such an elaborate type system as CL and, apart from numbers, doesn’t really have subtype relations the way CL does.

                                                          A summary

                                                          CL’s treatment of nil clearly makes some people very unhappy indeed. In particular they seem to think CL is somehow inconsistent, which it clearly is not. Generally this is either because they don’t understand how it works, because it doesn’t work the way they want it to work, or (usually) both. Scheme’s treatment is often cited by these people as being better. But CL requires precisely one implementationally-weird object, while Scheme requires two, or three if you count #t which you probably should. Both languages have idiosyncratic evaluation rules around these objects. Additionally it’s worth understanding that things like CL’s boolean type mean essentially nothing implementationally: boolean is just a name for a set of symbols. The only thing preventing you from defining a type like this yourself is the requirement for type-of to return the type.

                                                          Is one better than the other? No: they’re just not the same. Certainly the CL approach carries more historical baggage. Equally certainly it is perfectly consistent, and changing it would break essentially all CL programs that exist.

                                                          Thanks to Zyni for most of this: I’m really writing it up just so we can remember it. We’re pretty confident about the CL part, less so about the Scheme bit.

                                                          1. peculiar, adjective: having eccentric or individual variations in relation to the general or predicted pattern, as in peculiar motion or velocity. noun: a parish or church exempt from the jurisdiction of the ordinary or bishop in whose diocese it is placed; anything exempt from ordinary jurisdiction. 

                                                          Nicolas MartyanoffFixing unquote-splicing behaviour with Paredit

                                                          · 101 days ago

                                                          Paredit is an Emacs package for structural editing. It is particularly useful in Lisp languages to manipulate expressions instead of just characters.

                                                          One of the numerous little features of Paredit is the automatic insertion of a space character before a delimiting pair. For example, if you are typing (length, typing ( will have Paredit automatically insert a space character before the opening parenthesis, to produce the expected (length ( content.

                                                          Paredit is smart enough to avoid doing so after quote, backquote or comma characters, but not after an unquote-splicing sequence (,@) which is quite annoying in languages such as Scheme or Common Lisp. As almost always in Emacs, this behaviour can be customized.

                                                          Paredit decides whether to add a space or not using the paredit-space-for-delimiter-p function, ending up with applying a list of predicates from paredit-space-for-delimiter-predicates.

                                                          Let us add our own. For more flexibility, we will start by defining a list of prefixes which are not to be followed by a space:

                                                          (defvar g-paredit-no-space-prefixes (list ",@"))

                                                          We then write our predicate which simply checks if we are right after one of these prefixes:

                                                          (defun g-paredit-space-for-delimiter (endp delimiter)
                                                            (let ((point (point)))
                                                              (or endp
                                                                   (lambda (prefix)
                                                                     (and (> point (length prefix))
                                                                          (let ((start (- point (length prefix)))
                                                                                (end point))
                                                                            (not (string= (buffer-substring start end) prefix)))))

                                                          Finally we add a Paredit hook to append our predicate to the list:

                                                          (defun g-init-paredit-space-for-delimiter ()
                                                            (add-to-list 'paredit-space-for-delimiter-predicates
                                                          (add-hook 'paredit-mode-hook 'g-init-paredit-space-for-delimiter)

                                                          Not only does it fix the problem for unquote-slicing, but it makes it easy to add new prefixes. For example I immediately added #p (used for pathnames in Common Lisp, e.g. #p"/usr/bin/") to the list.

                                                          Nicolas MartyanoffSLIME compilation tips

                                                          · 102 days ago

                                                          I recently went back to Common Lisp to solve the daily problems of the Advent of Code. Of course it started with installing and configuring SLIME, the main major mode used for Common Lisp development in Emacs.

                                                          The most useful feature of SLIME is the ability to load sections of code into the Common Lisp implementation currently running. One can use C-c C-c to evaluate the current top-level form, and C-c C-k to reload the entire file, making incremental development incredibly convenient.

                                                          However I found the default configuration frustrating. Here are a few tips which made my life easier.

                                                          Removing the compilation error prompt

                                                          If the Common Lisp implementation fails to compile the file, SLIME will ask the user if they want to load the fasl file (i.e. the compiled form of the file) anyway.

                                                          I cannot find a reason why one would want to load the ouput of a file that failed to compile, and having to decline every time is quite annoying.

                                                          Disable the prompt by setting slime-load-failed-fasl to 'never:

                                                          (setq slime-load-failed-fasl 'never)

                                                          Removing the SLIME compilation buffer on success

                                                          When compilation fails, SLIME creates a new window containing the diagnostic reported by the Common Lisp implementation. I use display-buffer-alist to make sure the window is displayed on the right side of my three-column split, and fix my code in the middle column.

                                                          However if the next compilation succeeds, SLIME updates the buffer to indicate the absence of error, but keeps the window open even though it is not useful anymore, meaning that I have to switch to it and close it with q.

                                                          One can look at the slime-compilation-finished function to see that SLIME calls the function referenced by the slime-compilation-finished-hook variable right after the creation or update of the compilation buffer. The default value is slime-maybe-show-compilation-log which does not open a new window if there is no error, but does not close an existing one.

                                                          Let us write our own function and use it:

                                                          (defun g-slime-maybe-show-compilation-log (notes)
                                                            (with-struct (slime-compilation-result. notes successp)
                                                              (when successp
                                                                (let ((name (slime-buffer-name :compilation)))
                                                                  (when (get-buffer name)
                                                                    (kill-buffer name))))
                                                              (slime-maybe-show-compilation-log notes)))
                                                          (setq slime-compilation-finished-hook 'g-slime-maybe-show-compilation-log)`

                                                          Nothing crazy here, we obtain the compilation status (in a very SLIME-specific way, with-struct is not a standard Emacs Lisp macro) and kill the compilation buffer if there is one while compilation succeeded.

                                                          Making compilation less verbose

                                                          Common Lisp specifies two variables, *compile-verbose* and *load-verbose*, which control how much information is displayed during compilation and loading respectively.

                                                          My implementation of choice, SBCL, is quite chatty by default. So I always set both variables to nil in my $HOME/.sbclrc file.

                                                          However SLIME forces *compile-verbose*; this is done in SWANK, the Common Lisp part of SLIME. When compiling a file, SLIME instructs the running Common Lisp implementation to execute swank:compile-file-for-emacs which forces *compile-verbose* to t around the call of a list of functions susceptible to handle the file. The one we are interested about is swank::swank-compile-file*.

                                                          First, let us write some Common Lisp code to replace the function with a wrapper which sets *compile-verbose* to nil.

                                                          (let ((old-function #'swank::swank-compile-file*))
                                                            (setf (fdefinition 'swank::swank-compile-file*)
                                                                  (lambda (pathname load-p &rest options &key policy &allow-other-keys)
                                                                    (declare (ignore policy))
                                                                    (let ((*compile-verbose* nil))
                                                                      (apply old-function pathname load-p options)))))

                                                          We save it to a file in the Emacs directory.

                                                          In Emacs, we use the slime-connected-hook hook to load the code into the Common Lisp implementation as soon as Slime is connected to it:

                                                          (defun g-slime-patch-swank-compilation-function ()
                                                            (let* ((path (expand-file-name "swank-patch-compilation-function.lisp"
                                                                   (lisp-path (slime-to-lisp-filename path)))
                                                              (slime-eval-async `(swank:load-file ,lisp-path))))
                                                          (add-hook 'slime-connected-hook 'g-slime-patch-swank-compilation-function)

                                                          Quite a hack, but it works.

                                                          For older items, see the Planet Lisp Archives.

                                                          Last updated: 2023-03-17 18:00