Planet Lisp

Vsevolod DyomkinThe Common Lisp Condition System Book

· 11 hours ago

Several months ago I had a pleasure to be one of the reviewers of the book The Common Lisp Condition System (Beyond Exception Handling with Control Flow Mechanisms) by Michał Herda. I doubt that I have contributed much to the book, but, at least, I can express my appreciation in the form of a reader review here.

My overall impression is that the book is very well-written and definitely worth reading. I always considered special variables, the condition system, and multiple returns values to be the most underappreciated features of Common Lisp, although I have never imagined that a whole book may be written on these topics (and even just two of them). So, I was pleasantly flabbergasted.

The book has a lot of things I value in good technical writing: a structured and logical exposition, detailed discussions of various nuances, a subtle sense of humor, and lots of Lisp. I should say that reading the stories of Tom, Kate, and Mark was so entertaining that I wished to learn more about their lives. I even daydreamt (to use the term often seen throughout the book) about a new semi-fiction genre: stories about people who behave like computer programs. I guess a book of short stories containing the two from this book and the story of Mac from "Practical Common Lisp" can already be initialized. "Anthropomorphic Lisp Tales"...

So, I can definitely recommend reading CLCS to anyone interested in expanding their Lisp knowledge and general understanding of programming concepts. And although I can call myself quite well versed with the CL condition system, I was also able to learn several new tricks and enrich my understanding. Actually, that is quite valuable as you never know when one of its features could become handy to save your programming day. In my own Lisp career, I had several such a-ha moments and continue appreciating them.

This book should also be relevant to those, who have a general understanding of Lisp, but are compelled to spend their careers programming in inferior languages: you can learn more about one of the foundations of interactive programming and appreciate its value. Perhaps, one day you'll have access to programming environments that focus on this dimension or you'll be able to add elements of interactivity to your own workflow.

As for those who are not familiar with Lisp, I'd first start with the classic Practical Common Lisp.

So, thanks to Michał for another great addition to my virtual Lisp books collection. The spice mush flow, as they say...

Michał HerdaDamn Fast Priority Queue: a speed-oriented priority queue implementation

· 7 days ago

I think I have accidentally outperformed all of the Quicklisp priority queue implementations. Enter Damn Fast Priority Queue.

Detailed description and benchmarks are available on the GitHub repository. It seems that my implementation is consistently an order of magnitude faster than most of the other priority heaps (with Pileup being the runner-up, only being about 3-4x slower than DFPQ).

Michał HerdaCafe Latte - a condition system in Java

· 8 days ago

I've more or less finished Cafe Latte - an implementation of Common Lisp dynamic variables, control flow operators, and condition system in plain Java.

It started out as a proof that a condition system can be implemented even on top of a language that has only automatic memory management and a primitive unwinding operator (throw), but does not have dynamic variables or non-local returns by default.

It should be possible to use it, or parts of it, in other projects, and its source code should be readable enough to understand the underlying mechanics of each Lisp control flow operator.

Nicolas HafnerClosing in on Production - November Kandria Update

· 16 days ago

header
October somehow flew by really quickly for me. It's already November, and we're nearing the end of the year, too. Just thinking about that is making me reminiscent, but I'll have to hold off on doing my yearly wrap-up for another two months! Who knows, a lot more can still happen in that time. Last month marked another release for Kandria, and this month marked the start of Kandria being an actual team effort!

I'm really glad that it's no longer just me working on things. Fred already introduced himself in the last monthly, and by now he has already started work and delivered some really great stuff:

new light attack player idle

As a result, the game already feels a lot more fun to play. The step up from the combat animations I had made early in the year is huge!

We're still not done with it though, there's a few more moves missing, and a lot more left to adjust and fine-tune of course. We'll also have to get started on some real enemy designs soon and implement those to have some interesting encounters to test things with.

I can now also finally announce the third team member, Tim White, who'll be working on characters, story, and dialogue for the game:

Hey there! I'm Tim, a games writer from the UK. I've been in the industry for ten years now (where did the time go?!), and have been lucky enough to work at Jagex on Transformers Universe, and most recently with Brightrock Games on War for the Overworld and an unannounced game.

Kandria jumped out the screen at me straight away, with its detailed world and story, custom-made dev tools, and strong creative and artistic direction. I also have a real soft spot for post-apocalyptic worlds, and the ethics surrounding artificial life. Applying was a no brainer, and I can't wait to start!

You can find Tim on Twitter at @TimAlanWhite, or on the official Kandria Discord.

Both Tim and Fred will be giving quick updates on what's happening in the weekly newsletter from now on. The newsletter has now also been moved away from Mailchimp to my own mailing list service called Courier. I'm glad to finally have made the switch, freeing me from Mailchimp's slow and clunky interface!

On the engine side, I reworked the lighting and background systems to allow changing the lighting and parallax background to fit the current environment. As part of this I also changed the shadow casting to work properly so that it no longer contains the weird corner case glitches it used to.

I also had to make some fixes to the animation system to make it more capable and to make it less of a hassle to use when animations are changed or added. Previously the tooling there would easily mess up your data.

Then, in order to prepare for Tim, I reworked the quest system to be much easier to manage and control, and added a couple of additional features that should be very useful to control branching. To test it I made some quick draft animations for Fi and jotted her down in the test level.

fi

She'll now comment on things you can find throughout the level.

I also wrote a bunch of documentation to help Tim and Fred get set up and running with the game, introduced some very useful tooling like hot-reloading to make it faster to iterate on animations and textures, and improved the editor, especially for the in-game animation properties.

With all of this now in, we are very, very close to ending post-production. There's a few not-so-small things that I still need to do, like an animation system for the UI that I started working on yesterday, and one very nasty bug that popped up on Windows systems with surround sound configured. Still, with all of this in mind, I think we're well on track for the vertical slice release in March.

I hope there'll be a 0.0.4 demo release by the end of this month, which will be the last public demo until the vertical slice 0.1.0 demo. After that- I don't know yet how things will go. A lot about the game is going to become much clearer in the coming months as we decide on stuff like the core plot and work out the first area of the game for the vertical slice.

Aside from putting out whatever fires Fred and Tim stumble across this month, I'll be focusing on two things: first, fix surround sound on Windows. This is important to me as having the game crash and burn because of something so... tangential, is really terrible. Second, implement a UI animation system. The UI toolkit I'm using, Alloy, does not currently have a way to animate things. This is fine for tools and other UI like that, but in games you really want to spruce things up by tweening and animating to make your UI more interesting to look at. That's the last major addition to Alloy that's needed to have everything we need.

If time permits, I'll also work on some more platforming challenge levels to give the 0.0.4 demo some more content.

Anyway, I'm really happy to have a team together now, and I'm very excited to see how quickly things develop from here! To be fair, I'm also quite a bit worried what with being, I suppose, my own boss now, and the responsibilities that brings. I suppose time will tell whether I can figure out a good schedule and manage things well. For now I'm cautiously optimistic.

Alright, back to thinking about the animation system now, and see you next month, or next week if you're on the mailing list!

Alexander Artemenkosphinxcontrib-cldomain

· 22 days ago

This is an add-on to the Sphinx documentation system which allows using of information about Common Lisp packages in the documentation.

Initially, Sphinx was created for Python's documentation and now it is widely used not only for python libraries but also for many other languages.

Sphinx uses reStructured text markup language which is extensible. You can write your own extensions in Python to introduce new building blocks, called "roles".

sphinxcontrib-cldomain consists of two parts. The first part is a python extension to the Sphinx which adds an ability to render documentation for CL functions, methods and classes. The second - a command-line docstring extractor, written in CL.

Initially, cldomain was created by Russell Sim, but at some moment I've forked the repository to port it to the newer Sphinx, Python3 and Roswell.

The coolest feature of the cldomain is its ability to mix handwritten documentation with docstring. The second - cross-referencing. You can link between different docstrings and chapters of the documentation.

Today I will not show you any code snippets. Instead, I've created an example repository with a simple Common Lisp system and documentation:

https://cl-doc-systems.github.io/sphinxcontrib-cldomain/

This example includes a GitHub workflow to update the documentation on a push to the main branch and can be used as a skeleton for you own libraries.

The main thing I dislike in Sphinx and cldomain is the Python :( Other cons are the complexity of the markup and toolchain setup.

In the next few posts, I'll review a few other documentation tools for Common Lisp and try to figure out if they can replace Sphinx for me.

I think we as CL community must concentrate our efforts to improve the documentation level of our software and choosing the best setup which can be recommended for everybody is the key.

ABCL DevABCL 1.8.0

· 24 days ago

Under the gathering storms of the Fall 2020, we are pleased to release ABCL 1.8.0 as the Ninth major revision of the implementation.

This Ninth Edition of the implementation now supports building and running on the recently released openjdk15 platform.  This release is intended as the last major release to support the openjdk6 openjdk7, and openjdk8 platforms, for with abcl-2.0.0 we intend to move the minimum platform to openjdk11 or better in order to efficiently implement atomic memory compare and swap operations.

With this release, the implementation of the EXT:JAR-PATHNAME and EXT:URL-PATHNAME subtypes of cl:PATHNAME has been overhauled to the point that arbitrary references to ZIP archives within archives now work for read-only stream operations (CL:PROBE-FILE CL:TRUENAME, CL:OPEN, CL:LOAD, CL:FILE-WRITE-DATE, CL:DIRECTORY, and CL:MERGE-PATHNAMES).  The previous versions of the implementation relied on the ability for java.net.URL to open streams of an archive within an archive, behavior that was silently dropped after Java 5, and consequently hasn't worked on common platforms supported by the Bear in a long time.  The overhaul of the implementation restores the feasibility of accessing fasls from within jar files.  Interested parties may examine the ASDF-JAR contrib for a recipe for packaging and accessing such artifacts.  Please consult the "Beyond ANSI: Pathnames" Section 4.2 of the User Manual for further details for how namestrings and components of PATHNAME objects have been revised.

A more comprehensive list of CHANGES is available with the source.




    


Alexander Artemenkocl-pdf

· 26 days ago

This is the library for PDF generation and parsing.

Today I'm too lazy to provided step by step examples, but I have a real task to do with this library.

Some time ago I've read the article about productivity which recommended to print a "life calendar". This calendar should remind you: "The life is limited and the time's price is high."

The calendar is a grid where every box is one week of you life. The article suggested to buy a poster with the calendar, but I don't want to wait for a parcel with the poster! I want to print it now!

And here is where cl-pdf comes on the scene!

I wrote this simple function to generate the poster of A1 format:

(defun render (&optional (filename "life.pdf"))
  (flet ((to-ppt (size-in-mm)
           (/ size-in-mm 1/72 25.4)))
    (let* ((width (to-ppt 594))       ;; This is A1 page size in mm
           (height (to-ppt 841))
           (margin-top (to-ppt 70))
           (margin-bottom (to-ppt 30))
           (span  (to-ppt 2))
           (year-weeks 52)
           (years 90)
           (box-size (/ (- (- height (+ margin-top margin-bottom))
                            (* span (1- years)))
                        years))
           (boxes-width (+ (* box-size year-weeks)
                            (* span (1- year-weeks))))
           (boxes-height (+ (* box-size years)
                             (* span (1- years))))
           ;; horizontal margin depends on box size,
           ;; because we need to center them
           (margin-h (/ (- width boxes-width)
                        2))
           (box-radius (/ box-size 3))
           (helvetica (pdf:get-font "Helvetica")))
      (pdf:with-document ()
        (pdf:with-page (:bounds (rutils:vec 0 0 width height))
          ;; For debug
          ;; (pdf:rectangle margin-h margin-bottom
          ;;                boxes-width
          ;;                boxes-height
          ;;                :radius box-radius)
          (loop for year from 0 below years
                do (loop for week from 0 below year-weeks
                         for x = (+ margin-h (* week (+ box-size span)))
                         for y = (+ margin-bottom (* year (+ box-size span)))
                         do (pdf:rectangle x y box-size box-size :radius box-radius)))
          
          ;; The title
          (pdf:draw-centered-text
           (/ width 2)
           (+ margin-bottom
               boxes-height
               ;; space between text and boxes in mm
               (to-ppt 15))
           "LIFE CALENDAR"
           helvetica
           ;; font-size in mm
           (to-ppt 30))

          ;; Labels for weeks
          (let ((font-size
                  ;; We want labels to be slightly smaller than boxes
                  (* box-size 2/3)))
            (pdf:draw-right-text
             (+ margin-h
                 (/ box-size 4))
             (+ margin-bottom
                 boxes-height
                 ;; space between text and boxes in mm
                 (to-ppt 10))
             "Weeks of the year"
             helvetica
             font-size)
            
            (loop for week below year-weeks
                  do (pdf:draw-centered-text
                      (+ margin-h
                          (/ box-size 2)
                          (* week (+ box-size span)))
                      (+ margin-bottom
                          boxes-height
                          ;; space between text and boxes in mm
                          (to-ppt 3))
                      (rutils:fmt "~A" (1+ week))
                      helvetica
                      font-size))

            ;; Labels for years
            (pdf:with-saved-state
              (pdf:translate
               (- margin-h
                   (to-ppt 10))
               (- (+ margin-bottom
                      boxes-height)
                   (/ box-size 4)))
              (pdf:rotate 90)
              (pdf:draw-left-text
               0 0
               "Years of your life"
               helvetica
               font-size))
            
            (loop for year below years
                  do (pdf:draw-left-text
                      (- margin-h
                          ;; space between text and boxes in mm
                          (to-ppt 3))
                      (+ margin-bottom
                          (/ box-size 4)
                          (* year (+ box-size span)))
                      (rutils:fmt "~A" (- years year))
                      helvetica
                      font-size))

            ;; The Question.
            (pdf:draw-left-text
             (- width margin-h)
             (- margin-bottom
                 (to-ppt 10))
             "Is this the End?"
             helvetica
             (* font-size 2))
            
            (pdf:close-and-stroke)))
        (pdf:write-document filename)))))

Here is how result will look like:

The PDF can be downloaded here.

This program demonstrates a few features of cl-pdf:

  • ability to set page size;
  • text drawing and rotation;
  • font manipulation.

There are a lot more features but all of them aren't documented, only several examples :(

GitHub shows 4 forks with some patches. And some of them are turned into a pull-request, but maintainer is inactive on the GitHub since 2019 :(

Alexander Artemenkocl-async-await

· 28 days ago

This library implements the async/await abstraction to make it easier to write parallel programs.

Now we'll turn "dexador" http library calls into async and will see if we can parallel 50 requests to the site which response in 5 seconds.

To create a function which can return a delayed result, a "promise", we have to use cl-async-await:defun-async:

POFTHEDAY> (cl-async-await:defun-async http-get (url &rest args)
             (apply #'dexador:get url args))

Now let's call this function. When called it returns a "promise" object not the real response from the site:

POFTHEDAY> (http-get "https://httpbin.org/delay/5")
#<CL-ASYNC-AWAIT:PROMISE Not awaited>

Now we can retrieve the real result, using cl-async-await:await function:

POFTHEDAY> (cl-async-await:await *)
"{
  \"args\": {}, 
  \"data\": \"\", 
  \"files\": {}, 
  \"form\": {}, 
  \"headers\": {
    \"Accept\": \"*/*\", 
    \"Content-Length\": \"0\", 
    \"Host\": \"httpbin.org\", 
    \"User-Agent\": \"Dexador/0.9.14 (SBCL 2.0.8); Darwin; 19.5.0\", 
    \"X-Amzn-Trace-Id\": \"Root=1-5f9732d6-148ee9a305fab66c26a2dbfd\"
  }, 
  \"origin\": \"188.170.77.131\", 
  \"url\": \"https://httpbin.org/delay/5\"
}
"
200 (8 bits, #xC8, #o310, #b11001000)
#<HASH-TABLE :TEST EQUAL :COUNT 7 {1002987DE3}>
#<QURI.URI.HTTP:URI-HTTPS https://httpbin.org/delay/5>
#<CL+SSL::SSL-STREAM for #<FD-STREAM for "socket 192.168.43.216:49762, peer: 35.170.225.136:443" {10085B0BF3}>>

If we look a the promise object again, we'll see it has a state now:

POFTHEDAY> **
#<CL-ASYNC-AWAIT:PROMISE (VALUES {
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "0", 
    "Host": "httpbin.org", 
    "User-Agent": "Dexador/0.9.14 (SBCL 2.0.8); Darwin; 19.5.0", 
    "X-Amzn-Trace-Id": "Root=1-5f9732d6-148ee9a305fab66c26a2dbfd"
  }, 
  "origin": "188.170.77.131", 
  "url": "https://httpbin.org/delay/5"
}

  200 #<HASH-TABLE :TEST EQUAL :COUNT 7 {1002987DE3}>
  https://httpbin.org/delay/5
  #<SSL-STREAM for #<FD-STREAM for "socket 192.168.43.216:49762, peer: 35.170.225.136:443" {10085B0BF3}>>) >

Ok, it is time to see if we can retrieve results from this site in parallel. To make it easier to test speed, I'll wrap all code into the separate function.

The function returns the total number of bytes in all 50 responses:

POFTHEDAY> (defun do-the-test ()
             (let ((promises
                     (loop repeat 50
                           collect (http-get "https://httpbin.org/delay/5"
                                             :use-connection-pool nil
                                             :keep-alive nil))))
               ;; Now we have to fetch results from our promises.
               (loop for promise in promises
                     for response = (cl-async-await:await
                                     promise)
                     summing (length response))))

POFTHEDAY> (time (do-the-test))
Evaluation took:
  6.509 seconds of real time
  2.496912 seconds of total run time (1.672766 user, 0.824146 system)
  38.36% CPU
  14,372,854,434 processor cycles
  1,519,664 bytes consed
  
18300

As you can see, the function returns in 6.5 seconds instead of 250 seconds! This means cl-async-await works!

The only problem I found was this concurrency issue:

https://github.com/j3pic/cl-async-await/issues/3

But probably it is only related to Dexador.

Alexander Artemenkoparseq

· 31 days ago

With this library, you can write parsers to process strings, lists and binary data!

Let's take a look at one of the examples. It is a parser for the dates from RFC 5322. This format is used in email messages:

Thu, 13 Jul 2017 13:28:03 +0200

Parser consist of rules, combined in different ways. We'll go through the parser's parts one by one.

This simple rule matches one space character:

POFTHEDAY> (parseq:defrule FWS ()
               #\space)

;; It matches if string contains one space
POFTHEDAY> (parseq:parseq 'FWS
                          " ")
#\ 
T

;; But not on string from many spaces:
POFTHEDAY> (parseq:parseq 'FWS
                          "   ")
NIL
NIL

;; And of cause not on some other string
POFTHEDAY> (parseq:parseq 'FWS
                          "foo")
NIL
NIL

The next rule we need is the rule to parse hours, minutes and seconds. These parts have two digits and we'll use rep expression to specify how many digits matches the rule:

POFTHEDAY> (parseq:defrule hour ()
               (rep 2 digit))

POFTHEDAY> (parseq:parseq 'hour
                          "15")
(#\1 #\5)
T

See, this rule returns digits as the list! But to make it useful, we need the integer. Parseq rules support different kinds of transformations. They can are optional and can be specified like this:

;; This transformation will return as the string instead of list:
POFTHEDAY> (parseq:defrule hour ()
               (rep 2 digit)
             (:string))

POFTHEDAY> (parseq:parseq 'hour
                          "15")
"15"
T

;; Now we'll add a transformation from string to integer:
POFTHEDAY> (parseq:defrule hour ()
               (rep 2 digit)
             (:string)
             (:function #'parse-integer))

POFTHEDAY> (parseq:parseq 'hour
                          "15")
15 (4 bits, #xF, #o17, #b1111)
T

We'll define the minute and second rules the same way.

The next rule matches the abbreviated day of the week. It combines other rules or terms using or expression:

POFTHEDAY> (parseq:defrule day-of-week ()
               (or "Mon" "Tue" "Wed"
                   "Thu" "Fri" "Sat"
                   "Sun"))

POFTHEDAY> (parseq:parseq 'day-of-week
                          "Friday")
NIL
NIL

POFTHEDAY> (parseq:parseq 'day-of-week
                          "Fri")
"Fri"
T

;; The same way we define a rule for month abbrefiation
POFTHEDAY> (parseq:defrule month ()
               (or "Jan" "Feb" "Mar" "Apr"
                   "May" "Jun" "Jul" "Aug"
                   "Sep" "Oct" "Nov" "Dec"))

A little bit complex rule is used for matching timezone. Timezone is a string from 4 digits prefixed by plus or minus sign. We'll combine this knowledge using or/and expressions and will use option :string to get results as a single string:

POFTHEDAY> (parseq:defrule zone ()
               (and (or "+" "-")
                    (rep 4 digit))
             (:string))

POFTHEDAY> (parseq:parseq 'zone
                          "0300")
NIL
NIL

POFTHEDAY> (parseq:parseq 'zone
                          "+0300")
"+0300"
T

POFTHEDAY> (parseq:parseq 'zone
                          "-0300")
"-0300"
T

Now let's return to the time of day parsing. According to the RFC, seconds part is optional. Parseq has an expression ? to match optional rules.

Here is how a rule matching the time of day will look like:

POFTHEDAY> (parseq:defrule time-of-day ()
               (and hour
                    ":"
                    minute
                    (? (and ":" second))))

POFTHEDAY> (parseq:parseq 'time-of-day
                          "10:31:05")
(10 ":" 31 (":" 5))
T

To make the rule return only digits we have to use :choose transform. Choose extracts from results by index. You can specify index as an integer or as a list if you need to extract the value from the nested list:

POFTHEDAY> (parseq:defrule time-of-day ()
               (and hour
                    ":"
                    minute
                    (? (and ":" second)))
             (:choose 0 2 '(3 1)))

POFTHEDAY> (parseq:parseq 'time-of-day
                          "10:31:05")
(10 31 5)

;; Seconds are optional because of ? expression:
POFTHEDAY> (parseq:parseq 'time-of-day
                          "10:31")
(10 31 NIL)
T

;; This (:choose 0 2 '(3 1)) is equivalent to:
POFTHEDAY> (let ((r '(10 ":" 31 (":" 5))))
             (list (elt r 0)
                   (elt r 2)
                   (elt (elt r 3)
                        1)))
(10 31 5)

Another interesting transformation rule is :flatten. It is used to "streamline" result having nested structure and used in this rule which matches both time of day and timezone:

;; Without flatten we'll get nested lists:
POFTHEDAY> (parseq:defrule time ()
               (and time-of-day FWS zone)
             (:choose 0 2))

POFTHEDAY> (parseq:parseq 'time
                          "10:31 +0300")
((10 31 NIL) "+0300")

POFTHEDAY> (parseq:defrule time ()
               (and time-of-day FWS zone)
             (:choose 0 2)
             (:flatten))

;; Pay attention, :flatten removes nils:
POFTHEDAY> (parseq:parseq 'time
                          "10:31 +0300")
(10 31 "+0300")
T

Now, knowing how rules are combined and data is transformed, you will be able to read rest rules yourself:

POFTHEDAY> (parseq:defrule day ()
               (and (? FWS)
                    (rep (1 2) digit)
                    FWS)
             (:choose 1)
             (:string)
             (:function #'parse-integer))

POFTHEDAY> (parseq:defrule year ()
               (and FWS
                    (rep 4 digit)
                    FWS)
             (:choose 1)
             (:string)
             (:function #'parse-integer))

POFTHEDAY> (parseq:defrule date ()
               (and day month year))

(parseq:defrule date-time ()
    (and (? (and day-of-week ","))
         date
         time)
  (:choose '(0 0) 1 2)
  (:flatten))

Another cool Parseq's feature is an ability to debug parser execution. Now I'll turn on this debug mode and parse a string:

POFTHEDAY> (parseq:trace-rule 'date-time :recursive t)

POFTHEDAY> (parseq:parseq 'date-time
                          "Thu, 13 Jul 2017 13:28:03 +0200")
1: DATE-TIME 0?
 2: DAY-OF-WEEK 0?
 2: DAY-OF-WEEK 0-3 -> "Thu"
 2: DATE 4?
  3: DAY 4?
   4: FWS 4?
   4: FWS 4-5 -> #\ 
   4: FWS 7?
   4: FWS 7-8 -> #\ 
  3: DAY 4-8 -> 13
  3: MONTH 8?
  3: MONTH 8-11 -> "Jul"
  3: YEAR 11?
   4: FWS 11?
   4: FWS 11-12 -> #\ 
   4: FWS 16?
   4: FWS 16-17 -> #\ 
  3: YEAR 11-17 -> 2017
 2: DATE 4-17 -> (13 "Jul" 2017)
 2: TIME 17?
  3: TIME-OF-DAY 17?
   4: HOUR 17?
   4: HOUR 17-19 -> 13
   4: MINUTE 20?
   4: MINUTE 20-22 -> 28
   4: SECOND 23?
   4: SECOND 23-25 -> 3
  3: TIME-OF-DAY 17-25 -> (13 28 3)
  3: FWS 25?
  3: FWS 25-26 -> #\ 
  3: ZONE 26?
  3: ZONE 26-31 -> "+0200"
 2: TIME 17-31 -> (13 28 3 "+0200")
1: DATE-TIME 0-31 -> ("Thu" 13 "Jul" 2017 13 28 3 "+0200")

("Thu" 13 "Jul" 2017 13 28 3 "+0200")
T

We can improve this parser by using :function transformation to return a local-time:timestamp. First, let's redefine rule for matching the month and make it return the month number:

POFTHEDAY> (parseq:defrule january  () "Jan" (:constant 1))
POFTHEDAY> (parseq:defrule february () "Feb" (:constant 2))
POFTHEDAY> (parseq:defrule march    () "Mar" (:constant 3))
POFTHEDAY> (parseq:defrule april    () "Apr" (:constant 4))
POFTHEDAY> (parseq:defrule may      () "May" (:constant 5))
POFTHEDAY> (parseq:defrule june     () "Jun" (:constant 6))
POFTHEDAY> (parseq:defrule july     () "Jul" (:constant 7))
POFTHEDAY> (parseq:defrule august   () "Aug" (:constant 8))
POFTHEDAY> (parseq:defrule september () "Sep" (:constant 9))
POFTHEDAY> (parseq:defrule october  () "Oct" (:constant 10))
POFTHEDAY> (parseq:defrule november () "Nov" (:constant 11))
POFTHEDAY> (parseq:defrule december () "Dec" (:constant 12))

POFTHEDAY> (parseq:defrule month ()
               (or january february march april
                   may june july august
                   september october november december))

POFTHEDAY> (parseq:parseq 'month "Sep")
9 (4 bits, #x9, #o11, #b1001)
T

Next, we need to reimplement the rule matching a timezone to make it return local-time:timezone.

We'll be using an advanced technique of binding variables to pass value from one rule to another, because I want to store the timezone as a string and to parse it's hour and minute parts simultaneously.

To accomplish this task, we have to divide or timezone matching rule into two. The first rule will match it as a string of sign and four digits. Then it will save the result into an external variable and exit with a nil result to give a chance to execute the second rule:

POFTHEDAY> (parseq:defrule zone-as-str ()
               (and (or #\+ #\-)
                    (rep 4 digit))
             (:string)
             (:external zone-as-str)
             ;; Save the value into a variable:
             (:lambda (z)
               (setf zone-as-str z))
             ;; and just exit:
             (:test (z)
               (declare (ignore z))
               nil))

Now we'll redefine our zone rule to call zone-as-str first and then to parse the same text again, this time as hours and minutes. As the final step, it creates a local-time:timezone object:

POFTHEDAY> (parseq:defrule zone ()
               (or zone-as-str
                   (and (or #\+ #\-)
                        hour
                        minute))
             (:let zone-as-str)
             (:lambda (sign hour minute)
               (local-time::%make-simple-timezone
                zone-as-str
                zone-as-str
                ;; This is an offset in seconds:
                (+ (* (ecase sign
                        (#\+ 1)
                        (#\- -1))
                      hour
                      3600)
                   (* minute 60)))))

;; Here is the execution trace:
POFTHEDAY> (parseq:parseq 'zone
                          "+0300")
1: ZONE 0?
 2: ZONE-AS-STR 0?
 2: ZONE-AS-STR -|
 2: HOUR 1?
 2: HOUR 1-3 -> 3
 2: MINUTE 3?
 2: MINUTE 3-5 -> 0
1: ZONE 0-5 -> #<LOCAL-TIME::TIMEZONE +0300>
#<LOCAL-TIME::TIMEZONE +0300>
T

Now we need to redefine the original date-time rule, to create local-time:timestamp as the result:

POFTHEDAY> (parseq:parseq 'date-time
                          "Thu, 13 Jul 2017 13:28:03 +0200")
("Thu" 13 7 2017 13 28 3 #<LOCAL-TIME::TIMEZONE +0200>)
T

POFTHEDAY> (parseq:defrule date-time ()
               (and (? (and day-of-week ","))
                    date
                    time)
             (:choose '(1 2)                       ; year
                      '(1 1)                       ; month
                      '(1 0)                       ; day
                      '(2 0)                       ; hour
                      '(2 1)                       ; minute
                      '(2 2)                       ; second
                      '(2 3))                      ; timezone
             
             (:lambda (year month day hour minute second timezone)
               (local-time:encode-timestamp
                0             ; nanoseconds
                (or second 0) ; secs are optional
                minute
                hour
                day
                month
                year
                :timezone (or timezone
                              local-time:*default-timezone*))))

POFTHEDAY> (parseq:parseq 'date-time
                          "Thu, 13 Jul 2017 13:28:03 +0200")
@2017-07-13T14:28:03.000000+03:00
T

I've got a different value for the time because local-time prints timestamp in my timezone which is UTC+3.

The cool feature of the Parseq is its ability to work with any data, including binary. This way it can be used to parse binary formats.

As an example of parsing binary data, Parseq includes this parser rules for working with PNG image format:

https://github.com/mrossini-ethz/parseq/blob/master/examples/png.lisp

There are other interesting features. Please, read the docs to learn more.

If you are aware of other parsing libraries which worth to be written about, let me know in the comments.

Michał HerdaThe Common Lisp Condition System is out now

· 32 days ago

tclcs-cover-s.png

After just a bit more than six months, my first programming book is out and generally available. I hope that it works well for everyone who wants to explore the condition system, how it differs from standard exception-throwing systems in other programming languages, how to implement it and how to leverage it in real-world scenarios.

Links:

  • Apress - for buying and general information
  • Amazon - for buying and general information
  • GitHub - includes the full source code from the book and the online-only Appendix E ("Discussing the Common Lisp Condition System")

Alexander Artemenkopzmq

· 35 days ago

ZeroMQ is a networking library. It is not a message broker and it will not run tasks for you. Instead, it provides simple primitives for different network patterns.

With ZeroMQ you can easily implement these patterns: Request-Response, Pub-Sub, Push-Pull.

I found 3 CL systems implementing bindings to ZeroMQ:

I know, names of the repositories, CL systems and packages are all different. That is the HELL :(

There is also at least two different versions of the zmq:

  • First one is referred by https://www.cliki.net/cl-zmq and included into Quicklisp. But examples from the ZeroMQ Guide not work with this zmq because msg-data-as-is function is absent.
  • The second one is https://github.com/tsbattman/cl-zmq and seems it is the version, used in ZeroMQ Guide. But it is not in the Quicklisp (yet or anymore).

Anyway, both of them are stale and didn't get updates 7-8 years. They are using the old 3.2 version of ZeroMQ. Today we'll talk about pzmq.

PZMQ has some activity in the repository and uses ZeroMQ 4. It does not have docs but it has some examples, ported from the ZeroMQ Guide.

I slightly modified the examples code, to make the output more readable when client and server are running from one REPL.

This snippet shows the server's code. It listens on the 5555 port and blocks until a message received, then responds and waits for another message:

POFTHEDAY> (defun hwserver (&optional (listen-address "tcp://*:5555"))
             (pzmq:with-context nil ; use *default-context*
               (pzmq:with-socket responder :rep
                 (pzmq:bind responder listen-address)
                 (loop
                   (write-line "SERVER: Waiting for a request... ")
                   (format t "SERVER: Received ~A~%"
                           (pzmq:recv-string responder))
                   (sleep 1)
                   (pzmq:send responder "World")))))

The client does the opposite - it sends some data and waits for the response. Depending on the pattern you use, you have to set socket types. For the server, we used :rep (reply) and for client we are using :req (request).

POFTHEDAY> (defun hwclient (&optional (server-address "tcp://localhost:5555"))
             (pzmq:with-context (ctx :max-sockets 10)
               (pzmq:with-socket (requester ctx) (:req :affinity 3 :linger 100)
                 ;; linger is important in case of (keyboard) interrupt;
                 ;; see http://api.zeromq.org/3-3:zmq-ctx-destroy
                 (write-line "CLIENT: Connecting to hello world server...")
                 (pzmq:connect requester server-address)
                 (dotimes (i 3)
                   (format t "CLIENT: Sending Hello ~d...~%" i)
                   (pzmq:send requester "Hello")
                   (write-string "CLIENT: Receiving... ")
                   (write-line (pzmq:recv-string requester))))))

Here is what we'll see when running the server in the background and starting the client in the REPL:

POFTHEDAY> (defparameter *server-thread*
             (bt:make-thread #'hwserver))

SERVER: Waiting for a request... 

POFTHEDAY> (hwclient)
CLIENT: Connecting to hello world server...
CLIENT: Sending Hello 0...
CLIENT: Receiving... Hello
SERVER: Waiting for a request... World
CLIENT: Sending Hello 1...
CLIENT: Receiving... Hello
SERVER: Waiting for a request... World
CLIENT: Sending Hello 2...
CLIENT: Receiving... Hello
SERVER: Waiting for a request... World
NIL

What is next?

Read about Pub-Sub and Push-Pull patterns at the ZeroMQ Guide and try to port them on pzmq.

Also, it would be cool to port all Common Lisp examples from the unsupported library to the pzmq and to send a pull-request.

By the way, there is at least one cool project, which already uses pzmq to connect parts written in Common Lisp and Python. It is recently reviewed common-lisp-jupyter library.

To conclude, this library definitely should be tried if you are going to implement a distributed application! Especially if it will interop with parts written in other languages than Common Lisp.

Alexander Artemenkoquickfork

· 38 days ago

This is an interesting system which provides information about other systems sources. Also, it is able to show commands, necessary to clone libraries into the local-projects dir.

This system is not in Quicklisp yet, but it can be installed from Ultralisp or by clone into some directory like /quicklisp/local-projects~.

Also, to make it work, you have to clone quicklisp-projects repository somewhere. This repository contains metadata about all projects in the Quicklisp:

POFTHEDAY> (uiop:run-program
            "git clone https://github.com/quicklisp/quicklisp-projects /tmp/projects")
POFTHEDAY> (setf quickfork::*projects-directory* "/tmp/projects/projects")

An interesting thing happens right after you load quickfork system. It installs a few hooks into Quicklisp and ASDF and begins tracking the systems which are installed during the ql:quickload:

POFTHEDAY> (ql:quickload :dexador)
To load "dexador":
  Load 14 ASDF systems:
    alexandria asdf babel bordeaux-threads cffi cffi-grovel
    cl-ppcre cl-utilities flexi-streams local-time
    split-sequence trivial-features trivial-gray-streams
    uiop
  Install 17 Quicklisp releases:
    chipz chunga cl+ssl cl-base64 cl-cookie cl-reexport
    dexador fast-http fast-io proc-parse quri smart-buffer
    static-vectors trivial-garbage trivial-mimes usocket
    xsubseq
; Fetching #<URL "http://beta.quicklisp.org/archive/usocket/2019-12-27/usocket-0.8.3.tgz">
; 83.84KB
...

; Loading "dexador"
...
[package cl+ssl]..................................
[package dexador].

Systems compiled by QL: 
 ("proc-parse"
  #P"/Users/art/poftheday/.qlot/dists/quicklisp/software/proc-parse-20190813-git/")
 ("xsubseq"
  #P"/Users/art/poftheday/.qlot/dists/quicklisp/software/xsubseq-20170830-git/")
  ...
 ("dexador"
  #P"/Users/art/poftheday/.qlot/dists/quicklisp/software/dexador-20200427-git/")
Systems loaded by QL: 
 ("proc-parse"
  #P"/Users/art/poftheday/.qlot/dists/quicklisp/software/proc-parse-20190813-git/")
 ("xsubseq"
  #P"/Users/art/poftheday/.qlot/dists/quicklisp/software/xsubseq-20170830-git/")
 ...
 ("dexador"
  #P"/Users/art/poftheday/.qlot/dists/quicklisp/software/dexador-20200427-git/")
Systems installed by QL: 
 "usocket"
 "trivial-mimes"
 ...
 "chipz"
 "dexador"
Inspect ql:*compiled-systems*, ql:*loaded-systems*, and ql:*installed-systems* for more info.
(:DEXADOR)

Also, there is a function quickfork::make-clone-commands which prints which commands should be executed in command-line to clone given system and all its dependencies.

Sadly, quickfork::make-clone-commands fails on dexador with some strange errors. You will need my fix, to make it work like this:

CL-USER> (quickfork::make-clone-commands :dexador)

git clone  "https://github.com/sharplispers/split-sequence.git"
git clone  "https://github.com/sionescu/static-vectors.git"
git clone  "https://github.com/sionescu/bordeaux-threads.git"
git clone  "https://github.com/fukamachi/dexador.git"
git clone  "https://github.com/fukamachi/fast-http.git"
git clone  "https://gitlab.common-lisp.net/alexandria/alexandria.git"
git clone  "https://github.com/fukamachi/proc-parse.git"
git clone  "https://github.com/cl-babel/babel.git"
git clone  "https://github.com/trivial-features/trivial-features.git"
git clone  "https://github.com/fukamachi/xsubseq.git"
git clone  "https://github.com/fukamachi/smart-buffer.git"
git clone  "https://github.com/trivial-gray-streams/trivial-gray-streams.git"
git clone  "https://github.com/fukamachi/quri.git"
git clone  "https://github.com/rpav/fast-io.git"
git clone  "https://github.com/fukamachi/cl-cookie.git"
git clone  "https://github.com/dlowe-net/local-time.git"
git clone  "https://github.com/Shinmera/trivial-mimes.git"
git clone  "https://github.com/sharplispers/chipz.git"
git clone  "https://github.com/takagi/cl-reexport.git"
git clone  "https://github.com/cl-plus-ssl/cl-plus-ssl.git"
git clone  "https://github.com/lmj/global-vars.git"
git clone  "https://github.com/trivial-garbage/trivial-garbage.git"

Non-git dependencies:
("cl-utilities" :HTTPS
 "https://common-lisp.net/project/cl-utilities/cl-utilities-latest.tar.gz")
NIL
("flexi-streams" :EDIWARE-HTTP "flexi-streams")
("uiop" :HTTPS "https://common-lisp.net/project/asdf/archives/uiop.tar.gz")
("cffi" :HTTPS
 "https://common-lisp.net/project/cffi/releases/cffi_latest.tar.gz")
("chunga" :EDIWARE-HTTP "chunga")
("cl-ppcre" :EDIWARE-HTTP "cl-ppcre")
("cl-base64" :KMR-GIT "cl-base64")
("usocket" :HTTPS
 "https://common-lisp.net/project/usocket/releases/usocket-latest.tar.gz")

Suddenly, I've remembered another similar project: ql-checkout.

Probably, yesterday we'll see how it works!

Quicklisp newsOctober 2020 Quicklisp dist update now available

· 38 days ago

 New projects

Updated projects: adopt, agnostic-lizard, algae, april, base-blobs, bdef, beast, binary-io, bobbin, bodge-blobs-support, bodge-chipmunk, bodge-glad, bodge-glfw, bodge-nuklear, bodge-ode, bodge-openal, bodge-sndfile, chancery, chanl, check-bnf, chipmunk-blob, chirp, ci-utils, cl-async-await, cl-base64, cl-buchberger, cl-capstone, cl-cffi-gtk, cl-collider, cl-covid19, cl-csv, cl-digraph, cl-flow, cl-forms, cl-gamepad, cl-grip, cl-ipfs-api2, cl-kaputt, cl-liballegro-nuklear, cl-markless, cl-marshal, cl-messagepack, cl-mixed, cl-muth, cl-naive-store, cl-netpbm, cl-patterns, cl-pcg, cl-portaudio, cl-pslib, cl-readline, cl-rss, cl-sdl2, cl-semver, cl-setlocale, cl-ssh-keys, cl-webkit, class-options, claw, clj, closer-mop, clsql, clsql-local-time, clunit2, com-on, common-lisp-jupyter, croatoan, crypto-shortcuts, dartscltools, data-lens, djula, eclector, femlisp, flexichain, font-discovery, gadgets, gendl, glad-blob, glfw-blob, glkit, golden-utils, gtirb, harmony, hu.dwim.def, hu.dwim.presentation, hu.dwim.quasi-quote, hu.dwim.rdbms, hu.dwim.web-server, hyperluminal-mem, hyperobject, jingoh, kmrcl, lack, literate-lisp, markup, mcclim, messagebox, meta-sexp, mgl-pax, millet, mito-attachment, mmap, mutility, nanovg-blob, nuklear-blob, ode-blob, openal-blob, origin, overlord, paren6, petalisp, pngload, postmodern, puri, py4cl2, reversi, ryeboy, sc-extensions, sel, serapeum, shasht, simple-flow-dispatcher, sly, sndfile-blob, stmx, stumpwm, ten, trivial-do, ucons.

Removed projects: cl-piglow, cl-proj, clutz, roan, scalpl.

To get this update, use (ql:update-dist "quicklisp").

Enjoy!

Joe MarshallApropos of Nothing

· 39 days ago
Lisp programmers are of the opinion that [] and {} are just () with delusions of grandeur.

Alexander Artemenkofont-discovery

· 40 days ago

This is a library by @Shinmera to find out which fonts are known to OS and where their files are located.

Here is how you can list all "Arial" fonts and find where the "bold" version is located:

POFTHEDAY> (org.shirakumo.font-discovery:list-fonts :family "Arial")
(#<ORG.SHIRAKUMO.FONT-DISCOVERY:FONT "Arial" ROMAN REGULAR NORMAL>
 #<ORG.SHIRAKUMO.FONT-DISCOVERY:FONT "Arial" ITALIC REGULAR NORMAL>
 #<ORG.SHIRAKUMO.FONT-DISCOVERY:FONT "Arial" ROMAN BOLD NORMAL>
 #<ORG.SHIRAKUMO.FONT-DISCOVERY:FONT "Arial" ITALIC BOLD NORMAL>)

POFTHEDAY> (third *)
#<ORG.SHIRAKUMO.FONT-DISCOVERY:FONT "Arial" ROMAN BOLD NORMAL>

POFTHEDAY> (org.shirakumo.font-discovery:file *)
#P"/System/Library/Fonts/Supplemental/Arial Bold.ttf"

It is also possible to find a single font filtering it by family, slant and other parameters:

POFTHEDAY> (org.shirakumo.font-discovery:find-font :family "PragmataPro")
#<ORG.SHIRAKUMO.FONT-DISCOVERY:FONT "PragmataPro" ROMAN REGULAR NORMAL>

POFTHEDAY> (describe *)
#<ORG.SHIRAKUMO.FONT-DISCOVERY:FONT "PragmataPro" ROMAN REGULAR NORMAL>
  [standard-object]

Slots with :INSTANCE allocation:
  FILE                           = #P"/Users/art/Library/Fonts/PragmataProR_0828.ttf"
  FAMILY                         = "PragmataPro"
  SLANT                          = :ROMAN
  WEIGHT                         = :REGULAR
  SPACING                        = NIL
  STRETCH                        = :NORMAL

However, I found this library is still unstable on OSX and sometimes crashes somewhere in the CFFI code. @Shinmera has fixed some of these errors but some of them are still uncaught.

Read the full documentation on it here:

https://shinmera.github.io/font-discovery/

Alexander Artemenkotesseract-capi

· 42 days ago

It has nothing in common with the magic artefact, but anyway is able to do pretty cool things!

This library is a wrapper around Tesseract OCR and is suitable for an image to text transformations.

To test this system, you need to install the Tesseract library first. On OSX just do:

brew install tesseract

The CL library is not in the Quicklisp but can be installed from the https://ultralisp.org Follow the instruction from the repository:

https://github.com/egao1980/tesseract-capi

To make it work, you'll need a trained data for Tesseract engine. Happily, the Homebrew package includes some files for the English language.

To find out a path, do:

[poftheday] echo $(brew --prefix tesseract)/share/tessdata/
/usr/local/opt/tesseract/share/tessdata/

Now we can test this library on this sample from the repository:

Here is how to translate this image into the text:

POFTHEDAY> (tesseract-capi::tessversion)
"4.1.1"

POFTHEDAY> (let ((tesseract-capi:*tessdata-directory*
                   "/usr/local/opt/tesseract/share/tessdata/"))
             (tesseract-capi:image-to-text
              (asdf:system-relative-pathname
               :tesseract-capi "tests/data/fox.png")))
"This is a lot of 12 point text to test the
ocr code and see if it works on all types
of fle format

The quick brown dog jumped over the
lazy fox. The quick brown dog jumped
over the lazy fox. The quick brown dog
jumped over the lazy fox. The quick
brown dog jumped over the lazy fox
"

Now let's try it on this XKCD comic:

POFTHEDAY> (let ((tesseract-capi:*tessdata-directory*
                   "/usr/local/opt/tesseract/share/tessdata/"))
             (tesseract-capi:image-to-text
              "docs/media/0203/xkcd-312.png"))
"A GODS LAMENT
SOME SAID THE WORLD SHOULD BE IN PERL;
SOME. SAID IN LISP
NOW, HAVING GIVEN BOTH A WHIRL,
I HELD WITH THOSE WHO FAVORED PERL.
BUT I FEAR WE PASSED TO MEN
A DISAPPOINTING FOUNDING MYTH,
AND SHOULD WE WRITE IT ALL AGAIN,
TO ENO IT WITH
A CLOSE -PAREN.
SY i
\\ ad an ran
cS co V
a, em oo
2 ee eee
@ P= : } 0.\" :
a OO
"

As you can see, it outputs some junk at the end, but main text is recognized almost perfectly!

I think, this is a pretty cool library! For example, you can use it plus Montezuma to build a personal search engine for your image and scans collection. It's a matter of one night.

Alexander Artemenkocl-cont

· 44 days ago

This is a pretty old system which implements Delimited Continuations for Common Lisp. Initially, it was part of the Weblocks web-framework.

Sadly, but cl-cont has no documentation. I found only one example on this page.

It always was hard to wrap my mind around continuations. Probably that is why I decided to remove their support from the core of the Weblocks when I did the refactoring.

Now it is time to dive into continuations and probably to return them to Weblocks as an additional library.

Let's see what continuation is and how they can be used in practice!

The first thing to note is that each piece of code which uses this magic should be wrapped into with-call/cc. The second thing to remember is that let/cc form allows you to capture the moment and to save the execution point somewhere.

The code below prints three lines. It prints "Begin", then captures the execution point, prints "Before returning" and returns the captured point:

POFTHEDAY> (cont:with-call/cc
             (format t "Begin~%")
             (cont:let/cc k
               (format t "Before returning k~%")
               k)
             (format t "End~%")
             :final-result)
Begin
Before returning k
#<FUNCTION (LAMBDA (&OPTIONAL #:G1455 &REST #:G1456)) {22A10A0B}>

What has happened to our third print "End"? It didn't have a chance to be executed yet. But we can continue execution, by calling the function we've received as the result on the previous code snippet:

POFTHEDAY> (funcall *)
End
:FINAL-RESULT

POFTHEDAY> (funcall **)
End
:FINAL-RESULT

That is why it is called "continuation"! Yeah! As you can see, we can call this captured function any amount of times.

Now, let's try to create a function which will interrupt its execution and return a continuation.

Our first attempt might be like this:

POFTHEDAY> (defun foo ()
             (cont:with-call/cc
               (format t "Begin foo~%")
               (cont:let/cc k
                 (format t "Before returning k from foo~%")
                 k)
               (format t "End foo~%")
               :final-result))

POFTHEDAY> (cont:with-call/cc
             (format t "Before foo~%")
             (foo)
             (format t "After foo~%"))
Before foo
Begin foo
Before returning k from foo
After foo ;; Ups! I've expected it will not output this
NIL       ;; and return a continuation function instead of NIL!

As you can see, only half of our function was executed and then control flow continued, printed "After foo" and finished without giving us any continuation to play with :(

To make this code work as expected, we need to move with-call/cc form and make it wrap the function definition:

POFTHEDAY> (cont:with-call/cc
             (defun foo-wrapped ()
               (format t "Begin foo~%")
               (cont:let/cc k
                 (format t "Before returning k from foo~%")
                 k)
               (format t "End foo~%")
               :final-result))


POFTHEDAY> (cont:with-call/cc
             (format t "Before foo~%")
             (foo-wrapped)
             (format t "After foo~%"))
Before foo
Begin foo
Before returning k from foo
#<CLOSURE (LAMBDA (&OPTIONAL #:G1561 &REST #:G1562)) {10067F637B}>

This version works exactly as I've expected. It halts execution inside the foo's call and returns this continuation.

Now we can call continuation to continue computation of the foo function and the rest of our top-level form:

POFTHEDAY> (funcall *)
End foo
After foo
NIL

The latter case works because cont:with-call/cc is smart enough and if it wraps the function foo-wrapped into a special funcallable object:

;; This function is usual:
POFTHEDAY> (fdefinition 'foo)
#<FUNCTION FOO>

;; But this one is not.
;; It supports nested continuations:
POFTHEDAY> (fdefinition 'foo-wrapped)
#<CL-CONT::FUNCALLABLE/CC {10063435FB}>

Now let's adapt some examples from this Wikipedia article about continuations. The first example shows how to save continuation into the global variable and what happens when you use the same function to create the second continuation:

POFTHEDAY> (defvar *the-continuation*)

POFTHEDAY> (cont:defun/cc test ()
             (let ((i 0))
               ;; let/cc binds to k symbol a variable representing
               ;; this point in the program as the argument to
               ;; that function.
               ;;
               ;; In this case, we assigns that
               ;; continuation to the variable *the-continuation*
               ;; and then return the incremented value of 'i'.
               ;;
               (cont:let/cc k
                 (setf *the-continuation* k)
                 (incf i))

               ;; The next time *the-continuation* is called,
               ;; we start here:
               (incf i)))

POFTHEDAY> (test)
1

POFTHEDAY> (funcall *the-continuation*)
2

POFTHEDAY> (funcall *the-continuation*)
3

;; Stores the current continuation (which will print 4 next) away
POFTHEDAY> (defparameter *another-continuation* *the-continuation*)

;; Resets *the-continuation*:
POFTHEDAY> (test)
1

POFTHEDAY> (funcall *the-continuation*)
2

;; Uses the previously stored continuation:
POFTHEDAY> (funcall *another-continuation*)
4

The second example is more interesting because it let us create a simple framework for running green threads.

First, we need to define such two primitives: fork and yield:

POFTHEDAY> (defparameter *queue* nil)

POFTHEDAY> (defun empty-queue? ()
             (null *queue*))

POFTHEDAY> (defun enqueue (func)
             (setf *queue*
                   (append *queue*
                           (list func))))

POFTHEDAY> (defun dequeue ()
             (pop *queue*))

;; This stops running the current thread by placing it into the queue
;; and starts running a (func).
POFTHEDAY> (cont:defun/cc fork (func &rest args)
             (cont:let/cc k
               (enqueue k)
               (apply func args)))

;; This stops running the current thread by placing it into the queue
;; and starts running the other thread from the queue if there is any:
POFTHEDAY> (cont:defun/cc yield ()
             (cont:let/cc k
               (enqueue k)
               (funcall (dequeue))))

How does fork function work?

;; This is the function we want to run in "parallel":
POFTHEDAY> (defun do-job ()
             (format t "Inside job~%"))

;; Initially, our queue is empty:
POFTHEDAY> *queue*
NIL

;; Now when we'll call the fork,
;; it will:
;;
;; - capture current continuation;
;; - put it into the queue;
;; - execute do-job function.
POFTHEDAY> (cont:with-call/cc
             (format t "Before fork~%")
             (fork #'do-job)
             (format t "After fork~%"))
Before fork
Inside job
NIL

;; Now queue has one function which is
;; the rest of our initial computation.
POFTHEDAY> *queue*
(#<FUNCTION (LAMBDA (&OPTIONAL #:G1655 &REST #:G1656)) {22A1719B}>)

;; When the rest of the computation gets called,
;; it prints "After fork" and exits:
POFTHEDAY> (funcall (dequeue))
After fork
NIL

Yield works similarly. It captures the current continuation, appends it to the queue, takes the next coroutine from the top of the queue and executes it.

To test how two coroutines will behave when running in parallel, let's create a function which will print its name in the loop. On each iteration a coroutine will call yield to give other coroutines a chance to get executed:

POFTHEDAY> (cont:defun/cc do-stuff-n-print (name)
             (loop for n from 1 upto 3
                   do (format t "~A ~A~%" name n)
                      (yield)
                      (sleep 1)))

;; We also need to add this primive to our framework
POFTHEDAY> (defun wait-for-threads ()
             (loop
               when (empty-queue?)
                 return nil
               do (funcall (dequeue))))

POFTHEDAY> (cont:with-call/cc
             (fork #'do-stuff-n-print "Foo")
             (fork #'do-stuff-n-print "Bar")
             (wait-for-threads))
Foo 1
Bar 2
Foo 3
Bar 1
Foo 2
Bar 3

The result we've got is the same as the result of the Wikipedia article. Messages from both coroutines are interleaving. That is great!

Now, cl-cont does not look so strange to me. It is time to reimplement continuation widgets for the Weblocks! :)

Nicolas HafnerExpanding Outwards - October Kandria Update

· 45 days ago

header
A new month, a new Kandria update! First of all, a new prototype version is now up for download. As always, you can get it here. A short list of major improvements:

  • Audio! There's now music and sound effects. It'll be placeholder stuff for a while, but at least the game won't be silent any longer.
  • A new level. The new level is larger, contains a proper platforming challenge, and several items to inspect to give you a bit of a preview of what the exploration will be like.
  • Mechanics changes. Climbing is now limited by stamina, and the behaviour for dashing, jumping, and general collision has been adjusted.
  • Ropes and water. Ropes allow unlimited climbing and water provides different physics.
  • Dialogue and quests. As mentioned, you can now inspect items, which will bring up a dialogue screen. This is integrated with a quest system that can track progress through tasks - there's no UI for that yet though, so you'll have to take my word for that one.
  • Feedback and diagnostics. There's now an integrated feedback system. You can hit F12 at any point and it'll pause the game and bring up a feedback form to allow you to submit a report. It'll also do that automatically if the game crashes. With F11 you can also bring up a diagnostics screen to debug performance issues.

There's more stuff that was done, but it's mostly bugfixes and improvements to the development tools and all that.

Most of the month however was not spent writing code, and instead evaluating and processing job applications. The long awaited team expansion is really finally happening, and I'm really excited for it! There'll be two new members, an artist, and a writer. I'll talk a bit about the application process and what I learned from it first, though.

Initially I thought I'd get a couple of applications at most, maybe ten per job offer. With that in mind I put up a simple page to announce the jobs asking people to email me, and linked to that on Twitter. I also asked a bunch of people from gamedev communities I'm a part of to help spread it.... and help that did. My offer tweet got over a hundred retweets, which is far more than I ever got on anything else. I know it's still not much by Twitter standards, but hell it was plenty.

I also got approached by the owner of https://remotegamejobs.com, who offered me free listings for the jobs on his site. That was a really nice gesture and I gladly accepted. This further boosted the exposure of the listing by a lot, so I'll definitely take up their service again in the future.

Applications started rolling in pretty quick after I posted on Twitter, and kept on coming in for the coming weeks. I got a total of 166 applications for both jobs. 46 of those were for the artist position, and the rest all for the writer position, which surprised me a bit. I guess writing is far more popular than I expected! The range of applications I got was also quite something. Some barely wrote anything in their emails, others wrote very in-depth personal letters that impressed me a lot. I also got a couple of applications from people that I guess only read the tweet and just DMed me instead of emailing me like I asked.

It was really interesting to see things from the position of a recruiter, rather than the more common position of being an applicant. I think I learned a lot about what to do and not to do when I have to apply for a position in the future. My most important point to take away was to take time writing the application email and really look into not only the job offer, but also the background of the company and the project, so that you can show that you understood it and are genuinely interested in being a part of the team. Looking for people that wrote personal letters and showed they looked at things already filtered out so many applicants, it was staggering.

Sorting through all of these applications, and especially figuring out how I felt about each of them took a long, long time. Way longer than I had anticipated and hoped it would. Finally deciding on who to interview, and ultimately, who to pick, also took a lot of deliberation. Still, I got two people that I'm happy with now and they're going to start soon. We've already worked out the contracts, so all that's left now is to wait for the official starting time!

The artist I can already announce, so I asked him to write a short introduction for himself:

Hey everyone! My name's Fred, a French artist residing in San Francisco. Been a lover of pixel art since I'm 10 and worked on a few projects before like Black Future 88 and Batbarian: Testament of the Primordials. I'm really excited to join the Kandria's art team and help develop this cool new universe!

You can find him on Twitter at @brobbeh, and he's now also on the official Discord. Fred will be starting on the 19th, so not much longer from now! The art he's done on BF88 and Batbarian is really slick and nice, so I'm really looking forward to what he can come up with for Kandria!

The writer will be starting on the 1st of November and has asked me to hold off on announcing him until then, so he'll have to stay a secret for now!

So, with all of this exciting stuff out of the way, let's take another look at the rough roadmap and where we stand by now. Everything on that list has been done with the exception of better combat. That's what I hope to figure out together with Fred. I'm sure his animation expertise will help a lot in making the combat more fluid and fun. Other than that, there's nothing on that roadmap that's in the way of the full production phase.

However, with that list being fairly coarse grained, I think it's worthwhile to draw up a new, more detailed roadmap for the future. This is the roadmap from now until the 1st of March 2021. That's a hard deadline, as that's the submission deadline to apply for the Pro Helvetia grant, which I want to apply for again. Getting the grant would extend my funds and ability to continue hiring my two new teammates significantly.

  • Write documentation for the existing systems to serve as introduction and reference for my new team members
  • Implement better tooling like a quest editor and an improved animation editor
  • Design better combat animations and a better combat system in general
  • Implement a couple of very simple enemy types
  • Design and write out the introduction and tutorial area of the game
  • Design and write out the base camp and first two quests
  • Flesh things out and polish them for a proper vertical slice

This should be doable until March. There's of course other, minor things to do and stuff that needs to be done for the application besides the vertical slice, but it is a good anchor point to set everything around. Currently I'm optimistic that we can get things done, but also quite anxious since I've never had a real team before, and this is now my full-time job, so there's a lot on the line. I'm worried that I'd fuck up the management and communication, or that my tools and development environment are too annoying and broken to efficiently work with. I've always been very nervous about things like that though, so I'm going to assume that's just me being overly concerned more likely than anything.

Anyway, things are moving faster and faster now, and I'm really excited to see where things are going from here. Next month we'll have the writer on the team as well, and hopefully already got some more art, animation, and prep-work done to go full steam ahead on production!

If you want to be kept up to date more regularly for development news and insights, subscribe to the mailing list!

Alexander Artemenkolmdb

· 46 days ago

This is a binding to the embedded database LMDB, mentioned in this tweet. LMDB is a fast key/value database which can be embedded into your app as a C library.

Documentation on LMDB says it is really fast. I found this performance benchmark which compares it to the BoltDB and Badger. According to it, LMDB is slightly faster than BoldDB, but both lose to Badger.

It would be interesting to make our own benchmarks, but to compare LMDB with LevelDB which also has a binding to Common Lisp. But that is a story for another day.

Here is a fixed and slightly modified example from the CL wrapper's README. It just writes a string by the key and reads it back:

POFTHEDAY> (let ((env (lmdb:make-environment #p"./the-database/")))
             (lmdb:with-environment (env)
               ;; Create a transaction
               (let ((txn (lmdb:make-transaction env)))
                 (lmdb:begin-transaction txn)
                 ;; Create a database access object
                 (let ((db (lmdb:make-database txn "db")))
                   (lmdb:with-database (db)
                     ;; Here is how we can write some data to the storage
                     (lmdb:put db "the key" "The string")
                     ;; and read it back:
                     (let ((vec (lmdb:get db "the key")))
                       (print vec)))))))

;; Pay attention, the data is returned as a vector and your
;; app have to interpret it:
#(84 104 101 32 115 116 114 105 110 103)

POFTHEDAY> (babel:octets-to-string *)
"The string"

What is interesting, I found this library was used in this Wiki software, written in Common Lisp: Antimer.

But LMDB backend was removed from Antimer at some moment and replaced with SQLite. Most probably because it needed the full power of SQL instead of simple key-value queries.

To finalize, this LMDB binding would be a good solution for small apps which makes simple queries and need a high-performance and low latency.

BTW, the LMDB's repository needs some love because there are some hanging pull requests and a few unanswered issues. Does somebody know how does Fernando Borretti feels himself? Maybe he needs some help?

Zach BeaneNon-symbols as keyword arguments

· 47 days ago

Sometimes I like to have a short function call to make a lookup table with arbitrary keys and values. Something like this:

(table "name" "Alice" "age" 42) => #<TABLE ...>

This is easy enough to define like so:

(defun table (&rest keys-and-values)
   ;;; loop over keys and values to construct a table
   ...)

But there’s another option that works:

(defun table (&rest keys-and-values &key &allow-other-keys)
  ...)

This version has the advantage that missing arguments (e.g. (table "name")) are caught at compile-time!

I’ve been using this construct for a few years, but recently found out about 3.5.1.5, which says:

It is not permitted to supply a keyword argument to a function using a name that is not a symbol.

Yikes! That seems simple and straightforward: what I’m doing is not permitted. However! Every implementation I’ve tried (only three, admittedly) actually allows my not-symbol keywords without complaint!

I’m too old to think that “it works everywhere today” means “it will continue to work in SBCL tomorrow”, so I’m trying to figure out where I stand. 3.5.1.5 also says this:

If this situation occurs in a safe call, an error of type program-error must be signaled unless keyword argument checking is suppressed as described in Section 3.4.1.4.1 (Suppressing Keyword Argument Checking); and in an unsafe call the situation has undefined consequences.

So does that mean my code is:

  • a safe call with suppressed keyword argument checking?
  • …and is that good to use now and forever?
  • an unsafe call with undefined consequences?
  • something else?

I know the same effect could be achieved with a compiler macro, but I’d like to know if I can use this simpler option safely.

Alexander Artemenkotrivial-thumbnail

· 49 days ago

This is the library by @Shinmera. It does one trivial thing - shrinks images, squeezes them into smaller thumbnails. Actually, it is a wrapper around ImageMagick.

The cool thing about this library is that it is able to preserve GIF animations. This way you can create animated avatars for users of your next-generation Facebook killer social network, written in Common Lisp!

For example, let's pretend the user loads this cute dog animation which has 800×800 pixels size, and weights 44KB:

If we want our avatars to be 64x64 pixels, then trivial-thumbnail will help us:

POFTHEDAY> (trivial-thumbnail:create "docs/media/0200/dog.gif"
                                     "docs/media/0200/small-dog.gif"
                                     :width 64)
#P"docs/media/0200/small-dog.gif"

Here is the result:

To make it work you have to install ImageMagick. On OSX the easiest way to do this is to use Homebrew:

brew install imagemagick

There are also other options. They are covered by documentation.

Seems this is a cool library to make avatars, preview and thumbnails.

Alexander Artemenkodynamic-mixins

· 52 days ago

This is an interesting library which allows to add and remove mixin classes to the CLOS objects on the fly!

Common Lisp allows to change object's class, but this library goes further. It keeps track which mixins were already added to the object and allows to add new or to remove existing!

To demonstrate, how this works, let's pretend we have a graphics system where each figure can be filled with color and/or can have rounded corners. And we can have generic methods behave differently depending on traits of the figure:

POFTHEDAY> (defclass figure () ())

POFTHEDAY> (defclass box (figure) ())

POFTHEDAY> (defclass filled ()
             ((fill-color :initarg :fill-color)))

POFTHEDAY> (defclass rounded ()
             ((border-radius :initarg :border-radius)))

POFTHEDAY> (defmethod describe-object ((obj box) stream)
             (format stream "This is the box.~%"))

POFTHEDAY> (defmethod describe-object :after ((obj rounded) stream)
             (format stream "It has round corners.~%"))

POFTHEDAY> (defmethod describe-object :after ((obj filled) stream)
             (format stream "It filled with color.~%"))

Now we can construct the box object and simulate how it evolves over time when the user decides to make it's corner smoother and to fill it with a color:

POFTHEDAY> (defparameter *obj* (make-instance 'box))

POFTHEDAY> *obj*
#<BOX {10016F64A3}>

POFTHEDAY> (describe *obj*)
This is the box.

;; Now we'll add a trait to our object:
POFTHEDAY> (dynamic-mixins:ensure-mix *obj* 'rounded)

POFTHEDAY> *obj*
#<#<DYNAMIC-MIXINS:MIXIN-CLASS (ROUNDED BOX) {100A46CDB3}> {10016F64A3}>

POFTHEDAY> (describe *obj*)
This is the box.
It has round corners.

;; And yet another trait!
POFTHEDAY> (dynamic-mixins:ensure-mix *obj* 'filled)

POFTHEDAY> (describe *obj*)
This is the box.
It has round corners.
It filled with color.

;; We also can remove a mixin:
POFTHEDAY> (dynamic-mixins:delete-from-mix *obj* 'rounded)

POFTHEDAY> (describe *obj*)
This is the box.
It filled with color.

The only problem I found is that it is impossible to pass initargs to the ensure-mix function. Because of that, slots which we added along with the mixin, remain unbound.

But I found the solution to this problem:

POFTHEDAY> (defun add-mixin (object mixin-class &rest initargs)
             (let ((new-class (dynamic-mixins::ensure-mixin
                               (funcall #'dynamic-mixins::%mix
                                        object mixin-class))))
               (apply #'change-class object new-class initargs)))

POFTHEDAY> (slot-boundp *obj* 'fill-color)
NIL

;; Now we'll remove and add this mixin again:
POFTHEDAY> (dynamic-mixins:delete-from-mix *obj* 'filled)

POFTHEDAY> (add-mixin *obj* 'filled
                      :fill-color "#FF7F00")

POFTHEDAY> (slot-boundp *obj* 'fill-color)
T

POFTHEDAY> (slot-value *obj* 'fill-color)
"#FF7F00"

Hope, Ryan Pavlik will incorporate my pull request with this additional function!

If you are found this post interesting, then you also might like a post about dynamic-classes system.

Alexander Artemenkoplace-modifiers

· 55 days ago

This is a library by @HexstreamSoft. It provides a shorthand macro to modify data-structures in place.

The library has comprehensive documentation so, I'll only show you one example to demonstrate how it works.

Let's pretend we have some data received from an API and "age" field should be converted into the integer in place.

In plain CL we'll do it like this:

POFTHEDAY> (let ((data
                   '#((:name "Bob" :params (:age "45"))
                      (:name "Alice" :params (:age "23")))))
             (loop for item across data
                   do (setf (getf (getf item :params)
                                  :age)
                            (parse-integer
                             (getf (getf item :params)
                                   :age))))
             data)
#((:NAME "Bob" :PARAMS (:AGE 45)) (:NAME "Alice" :PARAMS (:AGE 23)))

But place-modiifiers allows you to keep your code DRY:

POFTHEDAY> (let ((data
                   '#((:name "Bob" :params (:age "45"))
                      (:name "Alice" :params (:age "23")))))
             (loop for item across data
                   do (place-modifier:modify
                       (parse-integer
                        (getf (getf item :params)
                              :age))))
             data)
#((:NAME "Bob" :PARAMS (:AGE 45)) (:NAME "Alice" :PARAMS (:AGE 23)))

Here I've extracted forms responsible for the modification:

;; Plain Common Lisp
(setf (getf (getf item :params)
            :age)
      (parse-integer
       (getf (getf item :params)
             :age)))

;; Using Place Modifiers macro
(place-modifier:modify
 (parse-integer
  (getf (getf item :params)
        :age)))

To learn more about place-modifiers, read it's docs! There is a lot of examples.

You might also be interested in reading the post about access library.

Zach BeaneNew SBCL 2.0.9 behavior breaks some stuff

· 56 days ago

The latest SBCL handles slot :initform and :type options in a new way. It’s mentioned in the release notes.

minor incompatible change: the compiler signals a warning at compile-time when an initform of T, NIL or 0 does not match a STANDARD-CLASS slot’s declared type._

Sounds pretty benign, but it breaks dozens of projects in Quicklisp. (To be fair, most of the failures are caused by a small number of core systems on which many other systems depend.)

Here’s an example of the new behavior:

(defclass foo ()
  ((name :type string :initform nil)))

With the above defclass form, SBCL 2.0.9 will signal a warning at compile time:

; processing (DEFCLASS FOO ...)

; file: foo.lisp
; in: DEFCLASS FOO
;     (NAME :TYPE STRING :INITFORM NIL)
; ==>
;   (SB-KERNEL:THE* (STRING :SOURCE-FORM NIL :USE-ANNOTATIONS T) NIL)
; 
; caught WARNING:
;   Constant NIL conflicts with its asserted type STRING.
;   See also:
;     The SBCL Manual, Node "Handling of Types"
; 
; compilation unit finished
;   caught 1 WARNING condition

This compile-time warning means “failure” as far as loading with ASDF is concerned.

If you have both :type and :initform in your slot definitions, and you want to be compatible with the latest SBCL, make sure the initform type matches the slot type. If you want to use NIL as the initform, one easy option is to set the type to (or null <actual type>).

Alexander Artemenkocommon-lisp-jupyter

· 57 days ago

This library provides a Common Lisp kernel for Jupyter.

Jupyter is a scientific environment for experiments. It is good when you want to play with data, to plot graphics and provides some comments in markdown.

Jupyter saves your programming session along with results in one file allowing to share your results with other programmers or analytics.

Maybe you didn't know, but GitHub is able to render such notebooks. Here I found a large list of interesting notebooks. Take a look at this one, for example:

https://github.com/mqlaql/geospatial-data/blob/master/Geospatial-Data-with-Python.ipynb

Now, let's return to the Common Lisp. Jupyter is using a protocol allowing to write backends in different programming languages. They are called "kernels".

Here is how we can install Common Lisp Jupyter kernel on OSX. I'm using Homebrew and Roswell because they are making everything so easy!

[poftheday] brew install zeromq

[poftheday] brew install jupyterlab

[poftheday] ros install common-lisp-jupyter

Now we can start a notebook in console mode:

[poftheday] jupyter console --kernel=common-lisp
Jupyter console 6.2.0

common-lisp-jupyter: a Common Lisp Jupyter kernel
(C) 2019 Tarn Burton (MIT)
In [1]: (lisp-implementation-type)
Out[1]: "SBCL"

In [2]: (lisp-implementation-version)
Out[2]: "2.0.8"

In [3]: (values 1 2 3)
Out[3]: 1
Out[3]: 2
Out[3]: 3

In [4]: (jupyter:file "/Users/art/Desktop/Screenshot 2020-09-25 at 23.50.02.png")
Out[4]: /Users/art/Desktop/Screenshot 2020-09-25 at 23.50.02.png

And this command will start a webserver with full Jupyter Notebook:

# To start a web UI, run
[poftheday] jupyter notebook

When the browser will open Jupyter, choose this menu to start Common Lisp Jupyter kernel:

Now if you enter the same code as we did before in console, you'll see, that web version is able to render our "screenshot" file below the "code cell":

It is also very easy to render formulas and to request an input from the user:

Also, you can render any HTML along with styles:

Or you might define functions which will return HTML or files:

This way, libraries extending common-lisp-jupyter may be created. They can do plotting for example, or render graphs, etc.

Here how you can make you own classes renderable by Jupyter:

Though, it would be nice to make it possible to define render method for object not inherited from the jupyter:result.

The developer of this library did a very good job documenting it and providing examples. You will find all of them here.

This project is in active development phase. For example, right now support for Jupyter widgets is added.

Please, join this effort and make your pull requests to this repository, if you are interested in building CL environment for data science!

Alexander Artemenkowhich

· 59 days ago

This is a tiny library by Fernando Borretti. It implements analogue of the UNIX utility which:

POFTHEDAY> (which:which "which")
#P"/usr/bin/which"

POFTHEDAY> (which:which "sbcl")
#P"/Users/art/.bin/sbcl"

POFTHEDAY> (which:which "python3")
#P"/usr/bin/python3"

POFTHEDAY> (which:which "missing-binary")
NIL

That is it. No more, no less. What do you think, when this library can be useful?

By the way, there are many other trivial (but useful) libraries. All of them are marked with a trivial tag on #pofthedday site.

Quicklisp newsSeptember 2020 Quicklisp dist update now available

· 59 days ago

 New projects

  • cl-base16 — Common Lisp implementation of base16 — GPLv2
  • cl-bcrypt — Common Lisp system for generating and parsing of bcrypt password hashes — BSD 2-Clause
  • cl-getx — This is a naive, persisted, in memory (lazy loading) data store for Common Lisp. — MIT
  • cl-indentify — A code beautifier for Common Lisp. — MIT
  • cl-kaputt — A Simple Interactive Test Framework for Common Lisp — MIT
  • cl-mango — A minimalist CouchDB 2.x database client. — BSD3
  • cl-minify-css — To minify css with common lisp. — GPLv3
  • cl-rfc4251 — Common Lisp library for encoding and decoding data in RFC 4251 compliant format — BSD 2-Clause
  • cl-setlocale — FFI to setlocale and ncurses locale helper — 2-clause BSD
  • cl-ssh-keys — Common Lisp system for generating and parsing of OpenSSH keys — BSD 2-Clause
  • cl-wave-file-writer — A wave file writer — MIT
  • class-options — Provides easy access to the defining class and its options during initialization. — Unlicense
  • compatible-metaclasses — Validates superclasses according to a simple substitution model, thereby greatly simplifying the definition of class mixins. — Unlicense
  • enhanced-find-class — Provides a canonical way of converting class designators to classes. — Unlicense
  • evaled-when — Provides a way of extracting and replicating the compile-time side-effects of forms. — Unlicense
  • file-attributes — Access to file attributes (uid, gid, atime, mtime, mod) — zlib
  • gadgets — Ben McGunigle's utility collection — Apache License, version 2.0
  • gooptest — A microcontroller testing framework. — GPL-3.0
  • kekule-clj — A Kekule widget for Common Lisp Jupyter — MIT
  • magicffi — cffi interface to libmagic(3) — Simplified BSD License
  • math —это математическая библиотека, реализующая некоторые алгоритмы: - линейной алгебры; - операций работы с матрицами; - статистические функции; - линейной и билинейной интерполяции; - нахождения приближающих многочленов, реализованная на Common Lisp — GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 or later
  • messagebox — A library to show a native message box dialog. — zlib
  • metalock — A metaclass that makes building parallel systems easier by providing each slot within a class a lock which is grabbed automatically. — MIT
  • nbd — Network Block Device server library. — MIT
  • object-class — Ensures that special subclasses of standard-object cluster right in front of standard-object in the class precedence list. — Unlicense
  • picl — Python Itertools in Common Lisp — MIT
  • pkg-doc — View package documentation in a clim-treeview — BSD Simplified
  • py4cl2 — Some improvements over py4cl — MIT
  • shasht — JSON reading and writing for the Kzinti. — MIT
  • simple-guess — Defines a simple extensible protocol for computing a guess using advisors. — Unlicense
  • trivial-do — Looping extensions that follow the style of the core DO functions. — MIT
  • uncursed — Another TUI library, this time without curses. — BSD 3-Clause
  • xcat — XCAT mass LAN big file distributor — MIT
  • zippy — A fast zip archive library — zlib

Updated projects3b-hdr3bmdacclimationalexandriaalgaeanypoolaprilatomicsbabelbdefbstci-utilscity-hashcl-allcl-aristidcl-autowrapcl-base64cl-bnfcl-cffi-gtkcl-collidercl-conllucl-covid19cl-dotcl-erlang-termcl-fixcl-formscl-fusecl-gamepadcl-gservercl-html-parsecl-krakencl-liballegrocl-liballegro-nuklearcl-marklesscl-migratumcl-mixedcl-mpg123cl-naive-storecl-patternscl-pngcl-pslibcl-rabbitcl-readlinecl-rediscl-renderdoccl-rrtcl-rsvg2cl-sdl2-ttfcl-steamworkscl-stompcl-storecl-strcl-unificationcl-utilscl-webkitcl-zyreclack-pretendclastclawkclcs-codeclimacscljcloser-mopclunit2com-oncom.google.basecommon-lisp-jupytercommonqtcroatoandeploydiff-match-patchdjulaeasy-audioeasy-routeseazy-processeclectoreosexscribef2clfare-quasiquotefast-iofile-selectfiveamflareflexi-streamsflexichainfloat-featuresfont-discoveryfsetfunctional-treesgendlgeneric-clglacierglsl-toolkitgolden-utilsgtirbgtirb-capstoneharmonyhu.dwim.asdfhu.dwim.delicohu.dwim.walkerhunchentoot-multi-acceptorhyperluminal-memhyperobjectinferior-shellinner-conditionalironcladjingohjonathanjpeg-turbokmrcllazylinear-programminglisp-binarylisp-gflagslispcordliterate-lisplocal-timelog4clmaidenmarkupmcclimmethod-hooksmgl-paxmodfmutilitynamed-readtablesnibblesnodguinull-packageopticloriginosicatoverlordparen6parsepathname-utilsperceptual-hashespetalispphoe-toolboxpngloadportable-condition-systempostmodernprotobufpsychiqpy4clquilcquux-hunchentootrandom-stateread-as-stringreplicroanrpcqs-graphvizsanity-clausesc-extensionsscalplselserapeumshadowsheepleshellpoolsimple-actorsslimeslysnoozestumpwmsxqltootertrace-dbtriviatrivial-argumentstrivial-clipboardtrivial-custom-debuggertrivial-garbagetrivial-gray-streamstrivial-utf-8trucleruax-14umbraunix-optsvernacular.

Removed projects: unicly.

To get this update, use (ql:update-dist "quicklisp").

Enjoy!

Alexander Artemenkotrivial-timeout

· 61 days ago

Today I found that :read-timeout option of the Dexador does not work as expected and remembered about this small but useful library. It provides the only one macro which executes code and limits it's execution to a given number of seconds.

For illustration, I'll use https://httpbin.org This is a service which helps you to test HTTP libraries. If you didn't hear about it, I recommend to look at.

Let's retrieve an URL, which responds in 10 seconds. Even with :read-timeout option, dexador waits 10 seconds:

POFTHEDAY> (time
            (nth-value 1
              (dex:get "https://httpbin.org/delay/10"
                       :read-timeout 2)))
Evaluation took:
  10.692 seconds of real time
  
200

If the site is not responding, a request may hang and block your application. Here is where trivial-timeout comes to the rescue!

POFTHEDAY> (trivial-timeout:with-timeout (2)
             (time
              (nth-value 1
                (dex:get "https://httpbin.org/delay/10"))))
Evaluation took:
  2.003 seconds of real time
  before it was aborted by a non-local transfer of control.
  
; Debugger entered on #<COM.METABANG.TRIVIAL-TIMEOUT:TIMEOUT-ERROR {10055B5373}>

Internally, this library generates the implementation-specific code to interrupt the code execution. Here how our example will look like for SBCL:

(let ((seconds 2))
  (flet ((doti ()
           (progn
             (time (nth-value 1
                     (dexador:get "https://httpbin.org/delay/10"))))))
    (cond
      (seconds
       (handler-case
           (sb-ext:with-timeout seconds
             (doti))
         (sb-ext:timeout (com.metabang.trivial-timeout::c)
           (declare (ignore com.metabang.trivial-timeout::c))
           (error 'com.metabang.trivial-timeout:timeout-error))))
      (t (doti)))))

And this is the same code, expanded on ClozureCL:

(let ((seconds 2))
  (flet ((doit nil
           (progn (time (nth-value 1
                          (dexador:get "https://httpbin.org/delay/10"))))))
    (cond (seconds
           (let* ((semaphore (ccl:make-semaphore))
                  (result)
                  (process
                    (ccl:process-run-function
                     "Timed Process process"
                     (lambda nil
                       (setf result
                             (multiple-value-list (doit)))
                       (ccl:signal-semaphore semaphore)))))
             (cond ((ccl:timed-wait-on-semaphore
                     semaphore
                     seconds)
                    (values-list result))
                   (t
                    (ccl:process-kill process)
                    (error 'com.metabang.trivial-timeout:timeout-error)))))
          (t (doit)))))

Don't know if such running the code in the separate thread can have some side-effects. At least, library's README says that it might be dangerous :)))

Alexander Artemenkopersistent-variables

· 63 days ago

This is a simple library which allows to define global variables and save/restore their state to some persistent storage.

For example, we can define variables for database host and password:

;; In real application you should define these
;; variables in the lisp file:

POFTHEDAY> (persistent-variables:defpvar *password*)
POFTHEDAY> (persistent-variables:defpvar *db-host*)

;; Then in the REPL you can setup the app
POFTHEDAY> (setf *password* "Some $ecret")
POFTHEDAY> (setf *db-host* "some-host.internal-to.my-company.com")

;; And save it's state:
POFTHEDAY> (with-open-file (stream "/tmp/app.config"
                                   :if-does-not-exist :create
                                   :if-exists :supersede
                                   :direction :output)
             (persistent-variables:pv-save stream))

;; At startup your app might restore values for these variables:
POFTHEDAY> (with-open-file (stream "/tmp/app.config"
                                   :direction :input)
             (persistent-variables:pv-load stream))

What this system does - it saves all symbols, defined with defpvar into the hash-table. And pv-save/pv-load serializes and deserializes them as sexps:

POFTHEDAY> (rutils:print-ht persistent-variables::*persisted*)
#{
  :DEFAULT '(*DB-HOST* *PASSWORD*)
 } 

POFTHEDAY> (with-output-to-string (s)
             (persistent-variables:pv-save s))
"(\"POFTHEDAY\" \"*DB-HOST*\" \"\\\"some-host.internal-to.my-company.com\\\"\")
(\"POFTHEDAY\" \"*PASSWORD*\" \"\\\"Some $ecret\\\"\")
"

This library can be useful for interactive applications where user can change the settings and they should be restored on restart. You probably also be interested in ubiquitous library which I didn't review yet.

Leo ZovicThe Prisoners Part 2

· 64 days ago

Dawn of the second day.

According to the internet, the thing I intend to build is called a Roguelikelike, teetering on the very edge of being a Roguelike. So it goes; we'll see if I end up taking the title or not.

Last time, we laid out the basics of prisoners, their interactions and their strategies. This time, lets get some different scenarios and some player interaction going.

Scenarios

Payoff matrices involve deciding who gets what bonus or penalty as a result of an interaction. Given a pair of defect/cooperate choices, a payoff-matrix will return the scores to be delivered to each player in turn.

(defun payoff-matrix (cc-a cc-b cd-a cd-b dc-a dc-b dd-a dd-b)
  (let ((tbl {(cons :cooperate :cooperate) (list cc-a cc-b)
	      (cons :defect :cooperate) (list dc-a dc-b)
	      (cons :cooperate :defect) (list cd-a cd-b)
	      (cons :defect :defect) (list dd-a dd-b)}))
    (lambda (a b) (lookup tbl (cons a b)))))

Now we can define some basic scenarios. A dilemma is the name I'll pick for the situation where co-operating is better for the group, and both defecting is the worst thing for everyone, but a single defector will end out better off by defecting.

(defparameter dilemma
  (payoff-matrix
   3 3  1 5
   5 1  0 0))

A stag-hunt is a situation where a pair of players can pool their resources for a greater prize, and ignore each other for the lesser. If either player attempts to hunt the stag alone, they get nothing, while their defecting partner still gets a rabbit.

(defparameter stag-hunt
  (payoff-matrix
   3 3  0 1
   1 0  1 1))

A trade is one in which both parties benefit, but to which both parties must agree.

(defparameter trade
  (payoff-matrix
   3 3  0 0
   0 0  0 0))

A theft is one where a player takes from the other. But if both players cooperate, or both try to rob each other, they come to an impasse.

(defparameter theft
  (payoff-matrix
   0 0   -3 3
   3 -3   0 0))

A trap is a situation where cooperating leads to disaster, ignoring the situation leads to no gain, and defecting to make it clear to your partner that you don't intend to follow ends up benefiting both players.

(defparameter trap
  (payoff-matrix
   -3 -3  2 2
    2  2  0 0))

The last scenario I'll concern myself with is the mutual-prediction. Where guessing what your partner/opponent will choose benefits you, and failing to do so does nothing.

(defparameter mutual-prediction
  (payoff-matrix
   3 3  0 0
   0 0  3 3))

Adventure

In order to move through the world, our prisoners need a world to move through. Let us begin at the ending.

(defparameter ending
  {:description "You have come to the end of your long, perilous journey."})

There is nothing to do at the end other than display this fact.

(defun repl! (adventure)
  (format t "~%~%~a~%~%" (lookup adventure :description)))
THE-PRISONERS> (repl! ending)


You have come to the end of your long, perilous journey.

NIL
THE-PRISONERS>

But what led us here was a choice. An adventure is more than a description, it's also the options, a prisoner, the scenario, and a way to continue the action. continueing means making a choice and effectively playing the opposing/cooperating prisoner and abiding by the results.

(defun mk-adventure ()
  (let ((prisoner (polo)))
    {:description
     "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
     :cooperate "accept" :defect "refuse" :prisoner prisoner :scenario trade
     :continue (lambda (choice)
		 (let ((their-choice (play prisoner)))
		   (update! prisoner choice)
		   (funcall trade choice their-choice)
		   ending))}))

This sort of adventure also takes a bit more machinery to run from the repl. We need to present the description, but also get an appropriate choice from the user. Getting that choice is a bit more complicated than you might think at first.

(defun get-by-prefix (lst prefix)
  (let ((l (length prefix)))
    (loop for elem in lst
       when (and (>= (length elem) l)
		 (== (subseq elem 0 l) prefix))
       do (return elem))))

(defun get-repl-choice (adventure)
  (let* ((responses (mapcar #'string-downcase (list (lookup adventure :cooperate) (lookup adventure :defect))))
	 (r-map {(string-downcase (lookup adventure :cooperate)) :cooperate
		 (string-downcase (lookup adventure :defect)) :defect})
	 (by-pref nil)
	 (resp ""))
    (loop until (and (symbolp resp)
		     (setf by-pref
			   (get-by-prefix
			    responses
			    (string-downcase (symbol-name resp)))))
       do (format
	   t "~a/~a:"
	   (lookup adventure :cooperate)
	   (lookup adventure :defect))
       do (setf resp (read)))
    (lookup r-map by-pref)))

Well behaved players are easy to deal with, true...

THE-PRISONERS> (get-repl-choice (mk-adventure))
Accept/Refuse:acc

:COOPERATE
T
THE-PRISONERS> (get-repl-choice (mk-adventure))
Accept/Refuse:ref

:DEFECT
T
THE-PRISONERS> (get-repl-choice (mk-adventure))
Accept/Refuse:a

:COOPERATE
T

... but we want to be a bit more general than that.

THE-PRISONERS> (get-repl-choice (mk-adventure))
Accept/Refuse:fuck you
Accept/Refuse:Accept/Refuse:boo
Accept/Refuse: (error 'error)
Accept/Refuse: (quit)
Accept/Refuse:r

:DEFECT
T
THE-PRISONERS>

That's the only hard par though. Interacting with the game once we're sure we have valid input from our player is relatively simple.

(defun repl! (adventure)
  (format t "~%~%~a~%~%" (lookup adventure :description))
  (when (contains? adventure :continue)
    (let ((choice (get-repl-choice adventure)))
      (repl! (funcall (lookup adventure :continue) choice)))))
THE-PRISONERS> (repl! (mk-adventure))


A stranger approaches. "I see you have baubles. Would you like to trade, that we both may enrich ourselves?"

Accept/Refuse:acc


You have come to the end of your long, perilous journey.

NIL
THE-PRISONERS>

This is obviously not the perilous journey being spoken of. At least, not all of it. The simplest way to extend it into one is to wrap scenarios around our existing adventure.

(defun mk-adventure ()
  (let ((def (defector)))
    {:description "A muscled street thug approachs, knife drawn."
     :cooperate "surrender" :defect "run" :prisoner def :scenario theft
     :continue (lambda (choice)
		 (let ((their-choice (play def)))
		   (update! def choice)
		   (funcall theft choice their-choice))
		 (let ((prisoner (polo)))
		   {:description
		    "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
		    :cooperate "accept" :defect "refuse" :prisoner prisoner :scenario trade
		    :continue (lambda (choice)
				(let ((their-choice (play prisoner)))
				  (update! prisoner choice)
				  (funcall trade choice their-choice)
				  ending))}))}))
THE-PRISONERS> (repl! (mk-adventure))


A muscled street thug approachs, knife drawn.

Surrender/Run:run


A stranger approaches. "I see you have baubles. Would you like to trade, that we both may enrich ourselves?"

Accept/Refuse:acc


You have come to the end of your long, perilous journey.

NIL
THE-PRISONERS>

Of course, since we want it to be much longer and more perilous, we'll want that process automated to at least some degree.

(defun wrap-scenario (adventure scenario)
  (insert
   scenario
   (cons
    :continue
    (lambda (choice)
      (let* ((them (lookup scenario :prisoner))
	     (their-choice (play them)))
	(update! them choice)
	(funcall (lookup scenario :scenario) choice their-choice)
	adventure)))))

(defun mk-adventure ()
  (wrap-scenario
   (wrap-scenario
    ending
    {:description
     "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
     :cooperate "accept" :defect "refuse" :prisoner (polo) :scenario trade})
   {:description
    "A muscled street thug approachs, knife drawn. \"Yer money or yer life, fop!\""
    :cooperate "surrender" :defect "run" :prisoner (defector) :scenario theft}))

This isn't enough for the Roguelikelike title, and I don't think I'll get there today, but I do want the ability to make an arbitrarily long adventure. The dumbest way of doing this is to make a list of scenarios, and pick from them when the need arises.

(defun random-scenario ()
  (pick
   (list
    {:description
     "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
     :cooperate "accept" :defect "refuse" :prisoner (polo) :scenario trade}
    {:description
     "A muscled street thug approachs, knife drawn. \"Yer money or yer life, fop!\""
     :cooperate "surrender" :defect "run" :prisoner (defector) :scenario theft})))


(defun mk-adventure (&key (scenarios 5))
  (let ((adventure ending))
    (loop repeat scenarios
       do (setf adventure (wrap-scenario adventure (random-scenario))))
    adventure))

An adventure of even 5 scenarios will end up being repetitive since we currently only have a grand total of two. But we can do something about that...

(defun random-scenario ()
  (pick
   (list
    {:description
     "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
     :cooperate "accept" :defect "refuse" :prisoner (polo) :scenario trade}
    {:description
     "A muscled street thug approachs, knife drawn. \"Yer money or yer life, fop!\""
     :cooperate "surrender" :defect "run" :prisoner (defector) :scenario theft}
    {:description
     "As you walk through an expansive market square, a gambler motions you over. \"Fancy your chances at evens or odds?"
     :cooperate "Evens!" :defect "Odds!" :prisoner (gambler) :scenario mutual-prediction}
    {:description
     "A hunter approaches you in a forest clearing. \"Hallo there, young one. Would you help me hunt a deer? I've had enough hares for now, but I promise we'll eat well if we work together!\""
     :cooperate "<Nocks bow>" :defect "Rather go my own way" :prisoner (dantes) :scenario stag-hunt}
    {:description
     "\"Hey follow me into this bear trap!\""
     :cooperate "Sure; I've grown tired of living" :defect "No. No, I'd rather not."
     :prisoner (robin) :scenario trap}
    {:description
     "You see a merchant ahead of you, paying little attention to his overfull coin purse. You could cut it and run."
     :cooperate "It's too tempting" :defect "No; I hold strong"
     :prisoner (dantes) :scenario theft}
    {:description
     "At the end of your travails with your co-conspirator, you get to the treasure first and can pocket some if you want."
     :cooperate "Take it" :defect "No, we split fairly"
     :prisoner (gambler :defect 5) :scenario dilemma})))

This gives me some ideas about how to go about generating scenarios a lot more programmatically, but I'll leave that for later, when I'm in the right frame of mind to do cosmetic improvements.

Wanna play a game?

THE-PRISONERS> (repl! (mk-adventure))

At the end of your travails with your co-conspirator, you get to the treasure first and can pocket some if you want.

Take it/Split fairly:split


You see a merchant ahead of you, paying little attention to his overfull coin purse. You could cut it and run.

It's too tempting/No:it's


"Hey follow me into this bear trap!"

Sure; I've grown tired of living/No. No, I'd rather not.:no


You see a merchant ahead of you, paying little attention to his overfull coin purse. You could cut it and run.

It's too tempting/No:it's


A stranger approaches. "I see you have baubles. Would you like to trade, that we both may enrich ourselves?"

accept/refuse:accept


You have come to the end of your long, perilous journey.

NIL
THE-PRISONERS>

This is about as far as I'm going today, and I'm not entirely sure how far I'm going during my next session.

As always, I'll let you know.


For older items, see the Planet Lisp Archives.


Last updated: 2020-11-23 12:41