Matthias Felleisen jested "Why are you still using CL when Scrbl/Racket is so much better :-)" ? My response was as follows:
you are right Racket is so much better in so many dimensions. I use Lisp because I just can't bear programming in a language without proper syntactic abstraction, and that is a dimension where Racket is far ahead of Common Lisp (CL), which sadly also remains far ahead of the rest of competition. Racket also has managed to grow a remarkable way to mix typed and untyped program fragments, which sets it ahead of most. But I am under the impression that there are still many dimensions in which Racket lags behind other languages and Common Lisp (CL) in particular.
- The Common Lisp Object System (CLOS) has multiple-inheritance, multi-methods, method combinations, introspection and extensibility via the MOP, generic functions that work on builtin classes, support for dynamic instance class change (change-class, update-instance-for-changed-class) and class redefinition (defclass, update-instance-for-redefined-class), a semi-decent story for combining parametric polymorphism and ad hoc polymorphism (my own lisp-interface-library), etc. Racket seems to still be playing catch-up with respect to ad hoc polymorphism, and is lacking a set of good data structure libraries that take advantage of both functional and object-oriented programming (a good target is Scala's scalaz and its rival cats).
- While the ubiquity of global side-effects in CL is very bad, the facts that all objects that matter are addressable by a path from some global namespace and that live redefinition is actively supported makes debugging and maintaining long-lived systems with in-image persistent data more doable (see again CLOS's update-instance-for-redefined-class). This is in contrast with the Racket IDE which drops live data when you recompile the code, which is fine for student exercises, but probably wrong for live systems. CL is one of the few languages that takes long-term data seriously (though not quite as seriously as Erlang).
- Libraries. CL seems to have much more libraries than Racket, and though the quality varies, these libraries seem to often have more feature coverage and more focus on production quality. From a cursory look, Racket libraries seem to often stop at "good enough for demo". An effort on curating libraries, homogeneizing namespaces, etc., could also help Racket (I consider CL rather bad in this respect, yet Racket seems worse). My recent experience with acmart, my first maintained Racket library, makes me think that writing libraries is even higher overhead in Racket than in CL, which is already mediocre.
- Speedwise, SBCL still produces code that runs noticeably faster than Racket (as long as you don't need full delimited control, which would requires a much slower CL-to-CL compiler). This difference may be reduced as Racket adopts the notoriously fast Chez Scheme as a backend (or not). Actually, the announcement of the new Racket backend really makes me eager to jump ship.
- As for startup latency, Common Lisp is also pretty good with its saved images (they start in tens of milliseconds on my laptop), making it practical to write trivial utilities for interactive use from the shell command-line with an "instantaneous" feel. Racket takes hundreds of milliseconds at startup which puts it (barely) in the "noticeable delay" category.
All these reasons, in addition to inertia (and a non-negligible code-base and mind-base), have made me stick to CL — for now. I think Racket is the future of Lisp (at least for me), I just haven't jumped ship right yet. If and when I do, I'll probably be working on some of these issues.
PS: Here are ways that Racket is indeed vastly superior to CL, that make me believe it's the future of Lisp:
- First and foremost, Racket keeps evolving, and not just "above" the base language, but importantly *below*. This alone makes it vastly superior to CL (that has evolved tremendously "above" its base abstractions, but hasn't evolved "below", except for FFI purpose, in the last 20 years), and superior to most languages (that tend to not evolve much "above", and not at all "below" their base abstractions).
- Racket is by far ahead of the pack in terms of Syntactic abstraction
- Racket has a decent module system, including build and phase separation, and symbol selection and renaming.
- Racket has typed modules, and a good interface between typed and untyped modules.
- Racket has lots of great teaching material.
- Racket has a one-stop-shop for documentation, though it isn't always easy to navigate and often lack examples. That still puts it far ahead of CL and a lot of languages.
- Racket provides purity by default a decent set of data structures.
- Racket has many primitives for concurrency, virtualization.
- Racket has standard primitives for laziness, pattern-matching, etc.
- Racket has a standard, portable, gui.
- Racket has a lively, healthy, user and developer community.
I probably forget more.
I owe you apologies for skipping reports in the meantime. Since January I'm not withdrawing money from our fundraiser thanks to other paid work, which means we have budget for more bounties. Keep in mind, that doesn't mean any work has ceased from my side.
I'm still working on a tutorial and demo application mentioned in the paper proposal. During that bugs and incompatibilities get fixed and pull requests are accepted. When questions arise on IRC or mailing list I try to answer them.
As a reminder: we have two active bounties at the moment:
Suggestions which other issues should have a bounty on them are appreciated and welcome.
If you have any questions, doubts or suggestions - please contact me
either by email (firstname.lastname@example.org) or on IRC (my nick is
If anyone wishes to inherit the domain from me and put it to better use, they should get in touch.
There are a bunch of new videos as of yesterday.
I have released new versions of nibbles (0.13) and ironclad (0.34). They are available from their respective tags in their github repositories; I have not created tarballs for them. Ironclad, in particular, has many new features; please see the NEWS files for both packages for some of the changes.
This is also an appropriate time to announce that I will no longer be maintaining nibbles, ironclad, nor any of my other Common Lisp packages. This has been the de facto state of affairs for several years now; we might as well make it official.
Today, I published an updated client with a new bundle feature: if :include-local-projects is true, everything in Quicklisp's ql:*local-project-directories* is copied into the bundle and made available when the bundle is loaded.
To get this update, use (ql:update-client). The new code will be loaded when Lisp is restarted.
This work was commissioned by Rigetti Computing.
If you have Quicklisp feature needs, feel free to get in touch with me!
We're looking for a developer to be a part of the core team for our Clojure and ClojureScript web application. Our stack includes reactive single-page web client code and a distributed backend to handle internal computations.
You will be responsible for designing, implementing, and testing functionality in our web application, collaborating closely with other developers to improve your and other's code, and working with our UI/UX team to make it slick. Most of all, we are looking for team members that are interested in learning and knowledge sharing.
I use and love the slime-selector. It’s a very quick way to jump around between various SLIME buffers.
In my .emacs,
C-c s is bound to
(global-set-key "\C-cs" 'slime-selector)
Then, in any buffer, I can use
\C-c s to pop up a minibuffer selector that takes a single additional key. Here’s the list of keys and what they do:
4: Select in other window ?: Selector help buffer. c: SLIME connections buffer. d: *sldb* buffer for the current connection. e: most recently visited emacs-lisp-mode buffer. i: *inferior-lisp* buffer. l: most recently visited lisp-mode buffer. n: Cycle to the next Lisp connection. p: Cycle to the previous Lisp connection. q: Abort. r: SLIME Read-Eval-Print-Loop. s: *slime-scratch* buffer. t: SLIME threads buffer. v: *slime-events* buffer.
r the most to instantly get into the repl. But I also use
l to jump to the last Lisp file I was working on, or
d to find a debugger buffer I might have buried.
It’s also quite helpful with
c, to bring up a list of active Lisp connections. For Quicklisp work, I sometimes want to switch between four or five active slime connections, and
\C-c s c makes it quick and easy to choose.
I'm happy to announce the release of Declt 2.1 "Jonathan Archer".
New in this release:
- Handle recent change in SBCL's
- Handle list of contacts (strings) in ASDF systems author and maintainer slots.
- Some backward-incompatible changes in the keyword arguments to the
- More hyperlinks between systems and source files.
- More robust system's packages collection (no more code walking).
- More robust handling of unavailable ASDF components.
- More robust naming of cross-references.
Find it at the usual place...
- agnostic-lizard — A portable code walker that makes a best effort to be correct in most cases — GPLv3+
- bit-ops — Optimized bit-vector operations — LLGPL
- cl-forest — Unofficial Common Lisp bindings to Rigetti Forest. — Apache 2.0
- cl-httpsqs — A client lib for accessing HTTPSQS written in Common Lisp — MIT
- cl-ledger — Double-entry accounting system. — BSD-3
- cl-password-store — Password management for Common Lisp (web) applications. — LLGPL
- cl-str — Modern, simple and consistent Common Lisp string manipulation library. — MIT
- cl-video — Video decoder implemented in Common Lisp — BSD
- cmu-infix — Mathematical infix notation for Common Lisp. — Custom (See LICENSE.txt)
- cserial-port — library for serial communication inspired by lispworks' serial-port — MIT
- cue-parser — A library for parsing audio CUE files — 2-clause BSD
- freebsd-sysctl — Sysctl kernel control mechanism for common lisp — 2-clause BSD
- jsonrpc — JSON-RPC 2.0 server/client implementation — BSD 2-Clause
- lifoo — a fresh take on Forth in the spirit of Common Lisp — MIT
- maiden — A modern and extensible chat bot framework. — Artistic
- named-read-macros — Make read macros more Lispy. Attach read macros to symbols. — BSD-3
- weblocks-examples — Example applications for Weblocks framework — Public Domain
To get this update, use (ql:update-dist "quicklisp").
As announced in my previous post, I've decided to do a write-up that illustrates my adventures in developing Portacle, a cross-platform, portable, installer-less, self-contained Common Lisp development environment.
The reason why I started with portacle in the first place is that, some day, probably in the distant future, I want to write a long series of long articles that should eventually accumulate into a book that introduces complete novices and interested people to programming in Common Lisp. As a prerequisite for that, I deem it absolutely necessary that the readers have an easy way to try out examples. Forcing them to perform a strange setup process that has absolutely nothing to do with the rest of the book and is likely to be error-prone in multiple ways is completely out of the question to me. They should be able to download an archive, extract it, and then just click on an icon to get a fully-featured IDE with which they can start out and play around with.
So, right off the bat this sets a few constraints that are rather hard to deal with.
- It needs to be cross-platform. I can't constrain the users to a certain operating system. Requiring them to set up a virtual machine would be madness.
- It needs to be cross-distribution. I can't know which distribution of Linux, or which version of it users are going to have. Requiring a specific one or a specific version would, too, be madness.
- It needs to be self-contained and should not poison the rest of the environment unless necessary or intended. This is necessary for it to be truly portable and work from a single folder.
- It needs to be reproducible and upgradable in order to be future-proof. This means I cannot simply assemble a package by hand once and call it a day once it works. It needs to be automatically buildable.
- It needs to be able to run on all platforms simultaneously as a single distribution. Otherwise, you would not be able to put it onto a USB stick and use it from any machine.
- It needs to be fully-featured enough to provide for both quick experiments and larger projects. As such, it will need to package a few different components and make them all work alongside, with the above constraints on top.
Since I want the eventual article series to be for absolute beginners as well, I also had some serious concerns about the usability aspects. It shouldn't be too confusing or hindering to use. Despite this, I still settled for Emacs+SLIME for the IDE, simply because there isn't really any alternative out there that is free and supports Lisp to a sufficient degree. I'm still not entirely set on the way the Emacs customisation is currently laid out, though. I might have to make more or less severe changes to make it easier for people to get started with. But, that's something for another time.
Now, considering the last constraint listed above, I decided on the following packages to be included in Portacle:
- Emacs, for the primary editor.
- Emacs customisation files, since the default is horrid.
- SBCL, for a capable Lisp implementation.
- ASDF, for building and system management.
- Quicklisp, for package management and library access.
- Git, for version control and project sharing.
ASDF was included specifically so that I would not have to rely on the implementation updating its internally shipped version and could instead deliver one that is possibly ahead of it, and thus less buggy/more feature-rich.
The first goal was to figure out how to build everything on OS X, Linux, and Windows. This alone took its fair share of experimentation and scouring as I had to look for the right combination of feature flags and build flags to make things work minimally.
In order to make this process work, I've developed a series of bash scripts that take care of downloading the source files of the various parts, configuring them, building them, and installing them in the proper locations in the resulting Portacle distribution tree. I settled on the following layout:
portacle -- Base Portacle directory build -- Build scripts and artefacts $package -- Build artefacts for the package config -- Configuration files and user files projects -- User projects directory all -- Cross-platform packages and resources emacsd -- Emacs configuration package quicklisp -- Quicklisp installation $platform -- Platform-specific packages and resources $package -- Installed package files lib -- Shared library files bin -- Shared executable files
The layout used to be different at first, namely the
$package directories used to be flipped around in the hierarchy. That proved to be less than ideal, however. I won't go into the details as to why. Suffice to say that complications that cropped up along the way ended up poisoning the hierarchy. Having a platform directory for all the specific files means you can create a Portacle distribution that works on all platforms simultaneously too.
Now, in order to properly illustrate the problems that cropped up, I'm going to talk about the development process for each platform separately. Pretty much every one of them had unique problems and complications that lead to a lot of wasted time and questions about the universe.
Linux was the first platform I started developing for, and ironically enough the last one I finished for good. The first phase of just getting everything built was comparatively painless. Things just worked.
However, as soon as I tried to run a complete installation on another Linux setup, things just segfaulted left and right. No wonder, too. After all, every component has shared library dependencies. Those are going to have different versions, and possibly even different names on different distributions. The first idea to deal with this mess was to simply copy every shared library the applications touched to a directory and set
LD_LIBRARY_PATH in a wrapper script before the respective application was launched.
In order to do this, I extended the build scripts with tools that would statically analyse all the executables that were shipped and recursively trace a full list of shared library dependencies. It would then copy them all over to the shared library files directory.
After doing this, things worked a bit better. But only a tiny bit. SBCL worked now, at least. Everything else still crashed. Unfortunately however, SBCL also only worked on my particular system. Much later I would find out that on others it would still fail. The root of the problem lay in the fact that
LD_LIBRARY_PATH does not cover everything. The system will still pick some shared libraries from elsewhere. Particularly,
libc and so forth.
So I scoured the web in search of a solution that would somehow let me constrain where linux looked for shared libraries. After a long while, I finally found something:
ld-linux.so. This file, which lives on every Linux system, is responsible for setting up the shared libraries an application depends on, and then running the application. It can be called directly, and it accepts a library-path argument, with which I can force it to look in a particular place first. So I changed the wrappers around to also start things with
ld-linux.so. Doing this allowed me to get Emacs working.
However, now SBCL didn't work anymore. For some reason that is still completely beyond me, if I try to run SBCL under
ld-linux.so, it acts as if the argument that is the SBCL binary path was simply not there. You can try it for yourself:
/lib64/ld-linux.so $(which sbcl) will show you the help text, as if that argument was just not there. If you pass it the path twice it does something, but it won't work right either. Fortunately, since SBCL doesn't have many dependencies to speak of, I've been able to get by without wrapping it in
ld-linux.so. I can't wait for the day for that to break as well, though.
Anyway, back to where I was. Emacs now worked. Kind of. Running other binaries with it did not, because the
LD_LIBRARY_PATH would poison their loading. Regular binaries on the system somewhere would not execute. So I had to do away with that. Thanks to the
ld-linux.so trick though, it wasn't really needed, at least for Emacs. For Git, the situation was worse. Git does some weird stuff where it has a multitude of
git-* binaries that each perform certain tasks. Some of those tasks call other Git binaries. Some of them call bash or things like that. I couldn't set
LD_LIBRARY_PATH because that would crash bash and others, and I couldn't use
ld-linux.so because that doesn't work when a new process is created. Back to Google, then.
After even more endless searching I finally found a solution. Using
LD_PRELOAD I could inject a shared library to be loaded before any other. This library could then replace the
exec* family of libc system functions and ensure that the process you wanted to call was actually called under
ld-linux.so. You can do this kind of symbol replacement by using
libdl, fetching the function locations of the functions you want to replace using
dlsym in the library's
init function, and then defining your own versions that call out to the original functions. You can find a few interesting blog articles out there that use
LD_PRELOAD and this kind of technique for malicious purposes. This culminated in something I called
ld-wrap.so. Getting all the
exec* calls to work again was its own adventure. On that path I discovered that
execv will actually call out to a shell if it finds a shell file, and, even worse than that,
excecvp* will actually call a shell any time if they can't seem to execute the file regularly. I think it's pretty insane for a "low-level system call" to potentially invoke a shell to run a script file, rather than "just" executing an ELF binary. In fact, there is no way to "just" execute a binary without implementing your own system calls directly. That's bonkers.
ld-wrap.so to work right involved multiple iterations, the last of which finally got Git working proper. I had to incorporate tests to check whether the binary was in the Portacle distribution, as well as checks to see whether the binary was static or not. Apparently launching a static binary under
ld-linux.so just results in a segfault. How very C-like. Anyway, it has not been the least bit of fun to go through this and pretty much every step of getting all of this worked out cost me weeks.
Aside from the SBCL problem mentioned above, there's an outstanding issue regarding the usage of fonts in Emacs. Since there's absolutely no guarantee for any particular font to exist on the system, and since I'd like to provide for some nice-looking default, I need to ship and install fonts automatically. I haven't gotten that to work quite yet, but I've been very disappointed to find that Emacs apparently has no support for loading a font from a File, and that Linux and OS X don't really support "temporarily" loading a font either. You have to install it into a user directory for it to be visible, so I have to touch things outside of the Portacle distribution folder.
Finally, apparently if you try to compile Emacs on some Linux systems, it will fail to run under
ld-linux.so and just crashes immediately with a "Memory Exhausted" warning. I have no clue what that is about and haven't received any feedback at all from the Emacs mailing lists either. So far this is not a problem, since the virtual system I build on works, but it is a major hassle because it means that building a distribution is out of the question for a lot of people. You can find out more about this bug on the issue tracker.
Windows has been an interesting system to work with. On one hand it has given me a lot of problems, but on the other it has also avoided a lot of them. The biggest pleasure was that shared library deployment "just worked". No need for complicated
ld-linux shenanigans, Windows is just consistent and faithful to loading the first library that it can find on its
PATH. Given that most of the libraries I need are already either provided by the Windows system or from the self-contained MSYS installation, there really haven't been any issues with getting things running on a deployed system.
However, it makes up for this in other areas.
While Emacs and SBCL have been very easy to get compiled and running, Git has once again proven to be a problem child. First, Git insists on keeping around
git-* prefixed hard-links to its binary because apparently a lot of things both internal and external still directly reference those. Hard links are difficult to ship because most archiving systems don't support them, making the resulting archive humongous. The current size of ~80 megabytes is already a lot by my measures, but the hard link issues exploded it to well over 100. Thus I had to go hunt for an archiver that allowed both self-extracting SFX executables -after all I couldn't ask people to install an archiver first- and was able to compress well and handle hard links. 7zip was up to the task, but it required complicating the deployment quite a bit in order to get the SFX working.
Next, Git is primarily a system written for Linux. As such, it has some "interesting" ideas about what the filesystem really looks like. I'm still not entirely sure how the path translation happens, but apparently Git interprets the root of the filesystem somewhere relative to its application location. This is fortunate for me, but took a long while to figure out. It is fortunate, because Git needs CURL to work, and CURL needs a certificate file to be able to do HTTPS. Naturally, Windows does not provide this by itself, so I have to do it. Git does allow you to set the path of the certificate file through a configuration variable, but it was one hell of a journey to get the entire setup working right. I'll try to explain why.
Because of - or rather thanks to Git's weird interpretation of the filesystem root, I can create a "fake" Git system configuration. Usually, Git looks up
/etc/gitconfig as part of the configuration files. Now, since the root is sort of relative to the Git binary, I can create an
etc subdirectory with the configuration file in the Git platform directory. This allows me to specify the certificate file path without having to disturb any of the other platforms. Then, thanks again to this root interpretation I can specify the path to the certificate file as an absolute path within the configuration file. Since the Git interpreted root moves with the Git binary, it becomes effectively relative. Naturally this would not work on Linux or OS X, but thankfully there I don't need to resort to such tricks.
Finally, if I try to compile Git without gettext on Windows, it fails to run properly and just exits with
vsnprintf is broken. I did find some issues related to
vsnprintf on Google, but nothing conclusive. Either way, if I do compile with gettext it seems to work fine. It doesn't work with gettext on OS X though, so I can't just enable it everywhere either.
Last but not least, Windows is primarily responsible for me not shipping a spell checker system. I really wanted to include that, as a spell checker is a really great thing to have when you're writing lots of comments and documentation, or just using Emacs for prose. However, I was simply not able to get
hunspell to compile and run no matter how much I tried. I was also unable to find any other compatible alternatives to those three that could be used as a replacement. If anyone else knows of a solution that I can compile successfully, and for which I can generate working dictionaries, I'd be very grateful.
Actually, I suppose it's also worth mentioning that for a while, before I wrote the launcher, I was trying to use batch scripts to launch things. Please, don't ever try to do that. Batch files are unbelievably cryptic to read and a huge pain to work with. There's no easy way to pass the arglist to another program for example, as it'll just reinterpret spaces within an argument as an argument separator. If you can, use PowerShell, which is supposedly much better, or just write a proper launcher application that does that kind of logic.
Mac OS X
Finally, OS X. This is a bit of a problematic system for me because Apple has, for reasons beyond me, decided to make it as difficult as possible to test for. Since I needed Portacle to work on multiple versions of OS X if possible, and even beyond that just ensure that it works outside of a development environment, I had to get my hands on virtual machines somehow. I first bought a MacBook Air just for this purpose -that's a thousand dollars wasted in the wind- only to realise that trying to run any kind of Virtual Machine on it is futile because it's just unbearably slow. Fortunately there are ways to get VMWare Workstation on Linux to run OS X as well if you use specially prepared images and some nefarious tools to unlock OS X as an option. However, only versions 10.11+ seem to run at any usable speed. 10.10 is so slow that you can pretty much forget trying to use it for anything.
Anyway, even just setting up a suitable build and test environment proved to be a major hassle. Thanks to Homebrew and MacPorts the building aspect wasn't too bad, though there too I've found weird inconsistencies like the aforementioned gettext problem. Aside from the compilation though, it really bothers me a lot that the OS X coreutils lack so many useful features. The build scripts have several special cases just for OS X because some Unix utility lacks some option that I need to get it done succinctly or efficiently.
Aside from the virtualisation thing, Apple also seems to try their hardest to prevent anyone from developing software for their system without also paying them a substantial fee every year. If you want to launch Portacle as a normal user, you just get a "security" error message that blocks the application from running. To get it to launch, you have to start the systems settings application, navigate to the security tab, unlock it, then click on "run this application" in order to finally run it. They completely removed the option to allow foreign apps by default, too, so there's no to me visible option to make it shut up. Windows has a security popup similar to that too, but at least you can immediately tell it to launch it, rather than having to waste multiple minutes in menus to do so.
In addition to the "security" popup, Apple has also recently decided to activate a feature that makes it virtually impossible for me to properly ship additional libraries, or versions of the libraries that Portacle requires, thus forever constraining the possible versions it can run on. OS X will now automatically clear out
DYLD_LIBRARY_PATH whenever you execute a new application. You can only disable this lunatic behaviour by booting into recovery mode and changing a security option- definitely not something I can ask people to do just to use Portacle. Thus, it seems it is impossible for me to, in the long term, cover a wider version scheme than a single one. This is very annoying, especially given that lots of people seem to stop upgrading now that Apple is screwing up ever more with every new version.
Now that most of the platform problems are finished, Portacle is almost done. There are a few more things left to do, however. The Emacs configuration as it is right now is somewhat passable, but I'd like to add and change a few more things to make it more comfortable, especially for beginners. I don't want to change too much either though, as that would make it hard for beginners to find help to a specific issue online.
Otherwise, I'd also like to work a bunch more on the included documentation and help file. As it stands it gives an overview over almost everything one needs to know to get started, but I haven't had it run by anyone yet, so I can't really give any estimates as to whether it's comprehensible, readable, or even useful in the first place.
I might write about Portacle again another time, hopefully when it has finally reached something I can call "version 1.0". Until then, I'll try to write some more articles about other subjects.
I’ve seen this idiom a few times when working with a binary stream:
(let ((elements-read (read-sequence buffer stream))) ...)
While the return value is the number of elements read in that situation, it’s a useful coincidence in a common case. The actual return value of read-sequence is:
the index of the first element of sequence that was not updated
Therefore, treating the return value as the number of elements read is completely wrong if you use the
start option to read-sequence.
For example, if
stream has one or more elements available to read:
(read-sequence buffer stream :start 42 :end 43) => 43
The actual number of elements read is 1, not 43.
It's been a good while since I last wrote an entry. I didn't write anything all January for a multitude of reasons, at the forefront being that it was exam season again. That's over with now, fortunately, so I do have some more time on my hands to goof off. Unfortunately though, I only have one more week left of this precious "free" time before university strikes me in the back again. I best use it wisely.
The first week of my holidays now I've spent on a heavy reworking of Portacle, a portable, multiplatform, upgradable, install-less Common Lisp development environment project. I've only just now finally got it to work properly on all three platforms and differing versions thereof. It has been an unbelievable struggle, as every platform posed unique problems and nasty edge-cases that were hard to catch without testing everything on a multitude of virtual machines all the time. If you look through the commit history you can find plentyful examples of the borderline insanity that I reached at points. I might write an in-depth article about my adventures another time.
During the exam session itself I was mostly occupied with working on Lichat, my own light-weight chat protocol. It has the nice properties that it is very uniform and rather simple to implement a client for. Among the nice aspects of the protocol are that everything goes over a channel, even private messages, that each update is associated with an ID and failures or responses can be tracked even if they happen to be out of order, and that connection multiplexing is provided directly by the protocol, so you can log into the same user account from multiple machines or instances simultaneously. I've written both a TCP and a WebSockets server, and a CL and JS client for it as well, so you can easily set up your own distributions of it. These CL and JS clients also have respective user interfaces as well, namely LionChat and Lichat-JS. Lionchat might turn into a generic chat client at some point in the future.
I've also continued work on Maiden, and finished a Lichat client for it as well. It's now running as a bot alongside the previous Colleen and I'm intending on switching over fully soon. The only thing that I haven't switched over yet is the chat logging, as I haven't had time to test that extensively yet. I've also been using its Text To Speech module to provide some fun interactivity for my video game streams. Before I can unleash Maiden onto Quicklisp I'll need to heavily document everything though, as it is quite a lot more complex -and capable- than Colleen was.
Then, I've also taken the time to switch over TyNET to the new Radiance 1.0. Radiance is now complete and ready for release as far as I'm aware. I've been holding off on releasing it for good as I've been waiting for some feedback on its tutorial from a couple of friends of mine. Once they get ahead with that I'll type up a good, long entry to announce everything. If my paper for ELS 2017 gets accepted, I will even be presenting it at the symposium in April. Speaking of the symposium- I've been working on a new website for it on and off, to which it will be switching in a while. You can get a preview peek at it here.
For Christmas I gave my father a Raspberry Pi 3 and installed Emacs and CCL onto it. He's been using it, and a DAQ board, to interface with a Korg synthesizer, since he's interested in making music on a more low, analog level. To do this I've been trying to teach Lisp to him since Christmas. While it's been a hard road so far, he's slowly getting into it and making advances. Given that he's been a Fortran programmer for almost all his life, with probably close to forty years of experience or more in it, I'm actually surprised at how well he's been doing with something that probably seems very different and alien to what he's used to. As part of this, though, I've had to write a few libraries, namely cl-gpio to interface with the Pi's GPIO pins, cl-spidev to use the serial port interface, cl-k8055 for the DAQ kit he's been using, and pi-plates for the pi-plates extension boards. The last library isn't done yet as I haven't had the time to test it yet, but the other three should work just fine.
So, I've been slowly, but surely, working off my long stack of projects that need to be completed someday. With Maiden out of the way soon too, I'll finally be able to get back to two of the remaining major projects: Trial, my game engine, and Parasol, my painting application. I'm still undecided as to which I'll tackle first, though. If you have a preference, let me know!
Finally, aside from programming work, I've gotten back into the swing of drawing. Before I was constantly feeling down and unmotivated about it. Everything just seemed to suck and I could never come up with ideas for things that I actually wanted to draw. I'm still not exploding with ideas now, but at least I've found something that inspires me a lot. Specifically, I was reading through Yokohama Shopping Trip and it was filled with exactly the kind of scenery-heavy, romantic drawings that I always wanted to make. "Romantic" referring to the Romanticism art movement. For the past few days then I've been trying to get a scenery drawing done every evening. You can see them on my tumblr. I still have a very long way to go and I'm painfully aware of all the things I do badly at or can't figure out how to draw properly, but I just have to keep on keeping on. It's too late to throw it all to the wind now.
In the near future I'd like to focus some more on writing things again, and perhaps also on actually reading some of the books that I've started accumulating over the years. But then, I'd also like to continue streaming video games, as I've been enjoying that quite a lot. Ah, there's just so much I'd like to do. If only someone were to figure out the age-old problem of finite capacity. Or in the very least: if only I didn't have to worry about earning any money to make a living, I could just merrily focus on being as productive as I can be instead of having to focus on what's marketable. Ah well, you can't have everything, eh?
So, in the more immediate future you can, if you want to, look forward to a full release announcement of Portacle, Radiance, and Maiden, more drawings, and hopefully more articles and stories as well. No promises, though. I'm not very good at keeping to deadlines for projects.
Here’s a message from Andrew Shalit to openmcl-devel:
To the CCL Community -
As many of you know, Gary Byers has been on a leave of absence from his Clozure CL work to attend to his health. I'm writing to let you know that, unfortunately, this leave will be permanent. Gary has resigned from his position at Clozure and will no longer be the lead developer of Clozure CL.
Over the last 30+ years Gary has made a tremendous contribution to the Common Lisp programming community, first through his work on Macintosh Common Lisp, and then through the creation of CCL. Gary has always been ready to share his deep knowledge of the Common Lisp standard and the issues attending its implementation. I don't know anyone whose understanding of the details of Common Lisp could match Gary's. His many contributions will be missed.
The work on CCL will continue, though. In Gary's absence, Matthew Emerson will be stepping up as the primary maintainer of CCL. Matthew is moving CCL to GitHub, and he hopes that others will pitch in to continue the development of this great software platform. Clozure Associates will also continue to support CCL, through direct contributions and through our consulting work. I'll be supporting those efforts, and I hope you'll consider supporting them as well.
To run it, CL-JPEG version 2.7 is required. Since it's not in Quicklisp yet as of time of writing, it must be cloned into your
local-projects. Then you can load
cl-video-player system and run
(cl-video::play <your filename.avi>).
The implementation is still very much naive, as expected from a weekend project. There is no support for indexing, and a number of edge cases of legit AVI encodings might fail. The library will decode video frames in the order they occur. No parsing of
BITMAPINFOHEADER structures; the assumption is they are MJPG DIB. No audio playback, although the stream is being read and adding at least PCM/WAV playback shouldn't be too big an effort.The decoder is factored out into independent implementation in
cl-video.lisp. A primitive CLX media player is included in
player.lisp. Each AVI instance contains one or more (audio or video) stream record objects, which hold ring buffers of frames with read and write cursors. The interaction between the decoder and player frontend runnning in separate threads is done via mutexed access to this ring buffer. The player thread can be kicked off a callback supplied to decode method: it will be called once the header part is parsed.
Since CL-JPEG doesn't use any SIMD extensions, the performance is modest. On my 6 year old 3GHz i5 (running one core of course) it decodes 480p 25fps file without hiccups, but going beyond that would require implementing multicore support in decode. Still it might be useful as is in applications not requiring real-time decoding performance.
- ahungry-fleece — A general utility library of convenience functions and features. — GPLv3
- asd-generator — Automatic directory scanner/generator for .asd project files. — GPLv3
- cl-cache-tables — A wrapper around native hash-tables to facilitate in-process caching of common lisp data structures. — MIT
- cl-feedparser — Common Lisp universal feed parser — LLGPL
- cl-gpio — A library for the Linux GPIO kernel module as used on hobby kits such as the Raspberry Pi — Artistic
- cl-k8055 — Bindings to the k8055 DAQ hobby board. — Artistic
- cl-online-learning — Online Machine Learning for Common Lisp — MIT Licence
- cl-spidev — A library for the Linux SPIDEV kernel module as used on hobby kits such as the Raspberry Pi — Artistic
- cl-xdg — freedesktop.org standards handling — GNU General Public License
- cl4l — esoteric CL essentials — MIT
- glisph — Glyph rendering engine using OpenGL shading language — MIT
- lichat-protocol — The independent protocol part of Lichat. — Artistic
- lichat-serverlib — Tools to help build a server using the lichat protocol. — Artistic
- lichat-tcp-client — A simple TCP client implementation for lichat — Artistic
- lichat-tcp-server — A simple TCP server implementation for lichat. — Artistic
- lichat-ws-server — A simple WebSocket server implementation for lichat. — Artistic
- lionchat — A GUI client for the Lichat protocol — Artistic
- lisp-binary — Declare binary formats as structs and then read and write them. — GPLv3
- omer-count — A script to assist in counting the time period between Pesach and Shavuot. — GPL V3
- ook — A CL compiler and enviroment for literate Orangutans. — Public Domain
- opticl-core — A library for representing and processing images — BSD
- rate-monotonic — A periodic thread scheduler inspired by RTEMS. — GPL-v3
- timer-wheel — A timer wheel implementation with BORDEAUX-THREADS backend. — MIT
- translate-client — A client to online web-server translators, currently only google translate — MIT
- trivial-escapes — C-style escape directives for Common Lisp. — Public Domain
Removed projects: cl-ledger, weblocks-examples.
To get this update, use: (ql:update-dist "quicklisp")
In course of last few months there were numerous small changes introduced to CL-JPEG. None substantial enough to warrant own announcement, but taken together perhaps it's due for an update. So here we go, as of version 2.6:
- Pre-allocated buffers in
DECODE-STREAMare now supported. This should help reduce consing in bulk-processing applications. The buffer can be pre-allocated based on dimensions via
- CMYK JPEG support. YCCK to CMYK conversion is now performed by decoder. To convert into 3-component RGB, use
- An option to disable colorspace conversion via
:COLORSPACE-CONVERSION NILsupplied to decode functions has been added. Can be useful e.g. if one needs the luminance component. Support of the corresponding option for encoder to improve performance in transcoding applications is in the plans.
- Small bugfixes and performance tweaks.
They’re looking for a “full stack Java programmer,” but must also have 3+ years hand-on experience in Lisp/Tk Programming on Linux.
Thank you, and best wishes for a happy 2017!
During this iteration I have continued working on the tutorial, fixing the issues and assuring CLIM II specification compatibility.
The most notable change is
refactor which simplifies the code and moves some computation to
creation time (until now we have searched through sheet hierarchy at
each iteration of command loop and at various other occasions at
The rest of the time was spent on peer review of the contributions, merging pull requests, development discussions, questions on IRC and other maintenance tasks.
Alessandro Serra has created a
Framebuffer Backend - a working proof
of concept that McCLIM may work on top of a frame buffer. Albeit a bit
slow you may find instructions how to run it here:
A detailed report is available at:
If you have any questions, doubts or suggestions - please contact me
either by email (email@example.com) or on IRC (my nick is
Happy new year!
docker run --tty easye/abcl /usr/local/bin/abcl
One may need to explicitly update the base openjdk8 container that easye/abcl is based on via:
docker pull easye/openjdk8
docker run easye/abcl /usr/local/bin/abcl
[Docker Engine]: https://www.docker.com/products/docker-engine
If you want a specific packaging to add your own Java/Lisp artifacts, you may easily rebuild the Docker image that the container starts with from cloning the ABCL source and then issuing:
docker pull easye/openjdk8
docker build -t easye/abcl .
See http://abcl.org/svn/trunk/abcl/Dockerfile for build instructions.
Orangutans grok Common Lisp
Please send bananas and DO NOT say "monkey"!
When I tried to find Nikodemus Siivola’s nice post about load-time-value and inline caches, I found his site, random-state.net, was no longer working.
At my request, he managed to get it up and running at a new address long enough for me to make a mirror of all his posts and articles.
Until the “real” location is fully restored, you can access the content of the site at random-state.xach.com, including the nice article about load-time-value and inline caches.
When random-state.net comes back, I’ll redirect everything back there.
Zach Beane — Make FFI callabacks work properly on 64 bit Windows OSes. · sharplispers/cormanlisp@2156983@2016-12-21 11:32 · 96 days ago
Making a project open source is no guarantee that anyone will care. But in the case of Corman Lisp, Artem Boldarev's hard work on fixing problems has not only revived the project, but revived Roger Corman's interest in it:
Artem, thanks so much for this fix!!!!! I believe this was the one the caused me to stop working on it for some time, and ultimately just open source it. I wasn't sure anyone else would be able (or willing) to track it down. I had dealt with and fixed many such issues during corman lisp development. However when I hit this issue specific to 64-bit OS (and those were fairly rare then, but of course now ubiquitous) I spent some evenings and just decided I couldn't do it any more. I got as far as noticing that the callback code broke and was getting a garbled address (I think an 8 byte address/4 byte address mismatch), and this probably happened when an access violation occurs (these are employed for garbage collection heap management). Now that I see it is working well on 64-bit systems I am inspired to get back into it.
Thank you, Roger, for sharing the source, and thank you, Artem, for diligently hacking on it!
I like the demo GIF!
After almost a year of development we are proud to present a new
release of ECL tagged with version
16.1.3. All changes are backward
compatible fixing bugs and other issues, implementing new interfaces
and cleaning up the code base.
ECL manual has been updated in a few places. Work on a new documentation is still pending. New documentation is still incomplete, but you may see it here: https://common-lisp.net/project/ecl/static/ecldoc/.
Before this release we have performed extensive tests on many platforms (Linux, FreeBSD, OpenBSD, NetBSD, OSX, Windows MSVC, Windows MinGW, Windows Cygwin, Android and Haiku). For details please consult https://gitlab.com/embeddable-common-lisp/ecl/issues/307. Extra attention has been paid to Windows testing to improve that platform support.
This release is available for download in form of a source archive:
ECL Development Team
Added better interface for package-locks.
ext:package-locked-p package ext:lock-package package ext:unlock-package package ext:without-package-locks &body body ext:with-unlocked-packages (&rest packages) &body body
To use these functions user has to require the module
defpackageaccepts new option
lockto allow locking package on creation:
(defpackage foo (:lock t))
mp:holding-lock-p: introduce new function for multiprocessing. Function verifies if lock is hold by the thread which calls the function. Usage:
make-random-state: fix problem with simple-vectors. The correct initialization types for
(OR RANDOM-STATE FIXNUM (MEMBER T NIL)).
Initializing a random state with an appropriate array (element type and arity dependent on platform) is also possible.
ext:random-state-array: new extension for random-states. Usage:
ext:terminate-process: new extension for external processes. Usage:
(ext:terminate-process process)with a second, optional boolean argument whenever termination should be forced or not.
CDR-5were already implemented, CDR-14 made us to list them in
if ECL is build with
:CXX-COREis present in
deprecated configure option
--with-local-gmphas been removed - use
--enable-gmp(defaults to auto).
configure options has been revised.
ASDF has been upgraded to version 220.127.116.11 (with a few patches scheduled for 3.2.0).
bundled CLX has been purged. Lately I've fixed ECL support on portable CLX maintained by sharplispers on https://github.com/sharplispers/clx (available via QuickLisp).
initial port for the Haiku platform. The port is done by Kacper Kasper's work, one of Haiku developers. Threads are not supported yet.
refactored ECL internal tests framework. Tests in
src/testsare now asdf-loadable (with
load-source-op) and divided into test suites.
make checktarget runs all regression and feature tests which aren't supposed to fail.
removed 15000 lines of obsolete code. Files not included in the buildsystem but lingering in the codebase or options failing to build. All info is added in the new documentation in the section "Removed interfaces".
improved man page and help output. Man page now contains up-to-date list of flags, as well as explanation of flag's behavior.
deprecated long flags with one dash, added two-dash version. Flags that aren't one-character, but start with one dash (e.g.
-eval) are now deprecated; long version
--evalwas added instead.
indented C/C++ code to follow emacs's gnu C style. This is a first step towards coding standards in the documentation. Additionally all in the src/c/ directory are listed in the appropraite documentation section (new-doc).
list_current_directory in unixfsys.d. Function was obfuscated with ifdefs with non-even pairs of
ECL signals floating point exceptions in top-level console.
mp:rwlockis treated as built-in class (previously process crashed if
class-ofwas called on such object).
ECL builds now succesfully with
ext:file-stream-fd: doesn't cause an internal-error if called with something not being a
stable-sort: bugfix and improvement in speed. Adapted from SBCL by Diogo Franco.
*type specifier as abbreviation of
Tas described in
2.4.3 Type Specifiersof the specification.
MOP: fix problemes when redefining non-standard and anonymous classes. Bugs identified and fixed by Pascal Costanza.
getcwd: fix issue with too long pathname. This fixes the regression, which crashed ECL at start when pathname exceeded 128 characters limit.
make-random-state: fix a problem with simple-vectors. Until now
#$reader macro accepted simple vectors as an argument, what lead to bugs if vector didn't match specific requirements like the element type or the arity. Now we sanitize this.
make-load-form: provide implementation for random-state objects.
thread fix on msvc: on windows importing thread was closing the thread handler so the thread wakeup wasn't working because the handler is not more valid.
import thread wasn't set upping a proper environment: on some case the thread was mistakenly thinking that the thread was already registered.
ECL_RESTART_CASEdidn't work as expected. Bug identified and fixed by Vadim Penzin.
For older items, see the Planet Lisp Archives.
Last updated: 2017-03-25 22:09