Sunday, July 11, 2010

URI Pathnames

Among other goodies, ABCL-0.20 includes good support for using URI resources as arguments to about any function you would specify a filepath. Any builtin scheme like "http" or "file" or "jar" works as a full-on Lisp Pathname, with the caveat that one may not write to an associated Stream. We should be generic enough in implementation that the JVM extension mechanism for extending the classloader works for us for the most common use case, OSGi.

We pick up the ability to refer to URI, associating an input Stream with its representation as a network (or local system) bytes easily as the following one-liner:


CL-USER> (with-open-file (stream #p"http://google.com/index.html")
(format nil "~A" (read-line stream nil)))

Underneath we have implemented a new subtype of PATHNAME, namely the type >URL-PATHNAME. We name this URL as opposed to URI as the underlying Java object is a java.net.URL with associated java.net.URLConnection that we'll be relying on for input sources.

Looking through an APROPOS for URL-PATHNAME shows the following related symbols:


CL-USER> (apropos (type-of #p"http://google.com/index.html"))
EXTENSIONS::SET-URL-PATHNAME-SCHEME (fbound)
EXTENSIONS::SET-URL-PATHNAME-AUTHORITY (fbound)
EXTENSIONS::SET-URL-PATHNAME-QUERY (fbound)
EXTENSIONS::SET-URL-PATHNAME-FRAGMENT (fbound)
URL-PATHNAME
URL-PATHNAME-AUTHORITY (fbound)
URL-PATHNAME-SCHEME (fbound)
URL-PATHNAME-FRAGMENT (fbound)
URL-PATHNAME-QUERY (fbound)
; No value
CL-USER>


which we see the defined functions to set the AUTHORITY, SCHEME, FRAGMENT, and QUERY portions of a URL-PATHNAME.

After these modifications, the ABCL PATHNAME class now has two "beyond ANSI" subtypes, namely URL-PATHNAME and JAR-PATHNAME. A short document outlining the design of URL-PATHNAME can be found in our documentation directory.

In implementing URL-PATHNAME, we actually had the ability to analyze what another CL did for their implementation. We analyzed the CLForJava technical paper, but decided not to use their abstractions as detailed on a lengthy armedbear-devel post.

Thursday, May 27, 2010

ABCL 0.20.0 released, including first funded feature

On behalf of the developers of ABCL (Armed Bear Common Lisp) I'm glad to be able to announce the 0.20.0 release.

ABCL is a Common Lisp implementation implemented in Java and running on the JVM, featuring both an interpreter and a compiler. The compiler targets the JVM directly meaning that its output is runnable JVM bytecode. The fact that ABCL is written in Java allows for relatively easy embedding in larger applications. For integration with existing applications ABCL implements Java Specification Request (JSR) 223: Java scripting API.

This release is the first to include a funded feature: funds were provided to implement the CLOS METACLASS feature and tests. Next to that, this release contains a large number of fixes and improvements, such
as the ability to use JARs and URLs as pathnames and the a new ASDF version (ASDF2). You can find the full release notes at:


and the list of changes at:


Latest and older binary and source distributions can be downloaded from

Thursday, May 6, 2010

Running SLIME under cygwin with ABCL

If ABCL may be said to have an IDE, it has one in the combination of Emacs using the SLIME to connect to running Lisp instance(s). In the parlance of SLIME, Emacs is said to be the superior process with the running Lisp instance(s) known as the inferior Lisp process. With SLIME, Emacs provides symbol completion in a REPL with edit history, Lisp syntax highlighting, finding the source location for a given form, the ability to run multiple instances of the inferior Lisp (or even different versions of Lisps), the ability to interactively inspect Lisp values, a nice wrapper for controlling ASDF, and much more.

When confronted with a Microsoft Windows environment that I need to productively produce code in, the first thing I do is install the cygwin package to get to a minimal UNIX-like environment. Unlike many Windows enhancements, the installation of cygwin does not require Administrator privileges so it is possible even in a somewhat restricted corporate environment. cygwin provides a UNIX compatibility layer that has enabled the porting of many standard GNU utilities to run under Windows, including an X11 server.

One of the first packages that I install for use under cygwin is the Emacs X11 package, which provides The One True editor with a lot of tools that make simple programmatic tasks easy, and the construction of complicated system possible. Historically there have been other efforts to bring Emacs to Windows such as NTEmacs, but now that GNU Emacs 23.1 runs fine in an X11 server with cygwin, I think the ease by which cygwin installs and updates outweighs putative advantages of Windows-specific Emacs ports. In truth, over time, GNU Emacs has absorbed most of the useful features pioneered by other, more native, versions.

People have reported that they have been able to get SLIME working with Emacs under Windows, but I never had much success, maybe because cygwin imposes a UNIX-like pathname structure that are naturally quite different from Windows pathnames. Not only is the '\' <--> '/' convention switch, Windows drive letters (like 'C:') are mapped to mounts unto the UNIX-like root filesystem, so 'C:\' becomes '/cygdrive/c'. Since ABCL under Windows runs on the JVM it expects to be dealing with Windows pathnames, it never worked for me. Until today. Or rather, yesterday. So, without further ado about pathnames, here's …

HOWTO install SLIME under Windows for ABCL

0. Install cygwin, selecting the 'emacs-x11' and 'cvs' packages in the interactive chooser.

1. Make a directory called 'c:\work' to contain ABCL and SLIME. You can of course use whatever directory you wish here, but the adjust the rest of these instructions to whatever you choose. Under cygwin, this directory will appear as '/cygdrive/c/work'.

2. Install ABCL under 'c:\work\abcl', so that 'c:\work\abcl\abcl.bat' invokes ABCL.

3. Use CVS to retrieve a copy of SLIME HEAD later than 2010-05-05. SLIME doesn't do releases which put me off for a bit at first, so even though there is a slime-2.0 floating around somewhere, it's ancient by this time, and a release process probably won't be repeated by the SLIME community in the foreseeable future.

From the bash prompt you installed in step 0, from the /cygdrive/c/work directory issue the following command:
cmd$ cvs -d  :pserver:anonymous:anonymous@common-lisp.net:/project/slime/cvsroot checkout slime
This will checkout a copy of SLIME including documentation. Alternatively one may download a CVS snapshot but then it will be harder to update SLIME later if you wish.

4. Use Emacs to edit '~/.emacs' to contain the following code
;;; .emacs for SLIME working with cygwin emacs

(add-to-list 'load-path "/cygdrive/c/work/slime")
(setq slime-lisp-implementations
'((abcl ("/cygdrive/c/work/abcl/abcl.bat"))))
(require 'slime)
(slime-setup '(slime-repl slime-asdf slime-fancy slime-banner))

(defun slime-to-lisp-translation (filename)
(replace-regexp-in-string
"\n" "" (shell-command-to-string
(format "cygpath.exe --windows %s" filename))))

(defun lisp-to-slime-translation (filename)
(replace-regexp-in-string
"\n" "" (shell-command-to-string
(format "cygpath.exe --unix %s filename"))))

(setq slime-to-lisp-filename-function #'slime-to-lisp-translation)
(setq lisp-to-slime-filename-function #'lisp-to-slime-translation)
This code sets up SLIME to be autoloaded by Emacs, using the 'cygpath.exe' to translate pathnames between the cygwin UNIX conventions (which Emacs expects), and the native conventions (which ABCL running on the JVM expects).

5. Upon evaluating the elisp code which you just used to configure SLIME (which can be affected by restarting Emacs), SLIME can be invoked via "M-x slime". ABCL will be invoked as a separate process, it will compile the elisp server known as swank that communicates between Emacs and the inferior Lisp process, it will load that code to initiate a connection, you'll see some "flying letters" coalesce into a REPL with a message in the mini-buffer welcoming you to SLIME

Happy hacking!

Monday, April 26, 2010

ABCL on N900

I decided to play a bit with my new mobile toy. The Nokia N900 should be capable of all kinds of things, including running a Java VM. After some googling, I found this page:

http://wiki.maemo.org/OpenJDK_6.0_0_%28Cambridge_Software_Labs%29_on_N900

The instructions there seemed simple and straightforward, so I did as advised, downloaded the jdk package and extracted it onto the device, and then just copied abcl.jar over and ran it with the incantation that's usually found in the abcl wrapper script. The results are pretty good:

~ $ /opt/OpenJDK-camswl/bin/java -cp /home/user/abcl.jar org.armedbear.lisp.Main
Armed Bear Common Lisp 0.20.0-dev
Java 1.6.0_0 Sun Microsystems Inc.
OpenJDK Zero VM
Low-level initialization completed in 4.436 seconds.
Startup completed in 16.765 seconds.
Type ":help" for a list of available commands.
CL-USER(1): (format t "abcl rules!")
abcl rules!
NIL
CL-USER(2):

It runs obviously much slower than on any desktop, the device is not as powerful, and its i/o is slower which hurts autoloading. But it runs, and it is much easier to setup than java ever was on N800 and such, and it also runs much faster than the non-zero VMs on N800 did.

Saturday, April 24, 2010

CGOL on ABCL

The other day I received an e-mail from an ABCL user, containing the following:

I took a brief look at cgol. Neat! Interestingly, I tried it in
clisp, sbcl, ecl, cmulisp, and abcl, and it only works in abcl!


For those wanting to play with CGOL too, it's available from the CMU Artificial Intelligence Repository.

Seems like ABCL is Common Lisp conformant enough to be able to run some of the older lisp code available: this source stems from 1993 and is unchanged since - judging by the timestamps in the download archive.

Saturday, April 3, 2010

Even better handling of low memory conditions

In previous research, ABCL came out nicely when tested for handling of Stack Overflow (SO) and Out Of Memory (OOM) conditions. See the blog item "out-of-memory: a sad case"; ABCL was classified as one of the few lisps that handled OOM conditions at all, by ending in the Lisp debugger even.

By looking more closely - which we did before the release of 0.18 - it turned out that handling of these conditions was only part of forms executed from REPL. If the same conditions occurred during program execution, the program was simply terminated, just like the other Lisps in the survey.

Of course, when you have a program running into low memory conditions regularly, it makes sense to make more memory available to it at startup. There are a few startup switches for java which allow to set the initial memory amount, most notably:
  • -Xmx: which sets the maximum amount of heap available
    examples: 512m, 1024k, 13m
  • -Xms: like -Xmx but specifies the initial amount
  • -Xss: specifies the thread stack size
These options are only available at startup though. These options can be added to ABCL's startup wrapper script, either after building, or by letting the build add them automatically. If you opt for the latter, you need to add a line like the following to your abcl.properties file (before abcl-0.19.1 this was named build.properties):

java.options=-Xmx512m -Xss6m

If your program keeps large caches, it may be important to be able to detect OOM conditions, clear some or all of the caches and retry the operation. Other situations where it's good to be able to handle OOM conditions can be thought of. That's why it's important to have your programming environment help you detect these situations.

One of the advantages of running in the JVM is that ABCL itself doesn't doesn't need heuristics to detect these situations: the JVM throws a StackOverflowError or an OutOfMemoryError as appropiate. The only thing ABCL needs to do is handle these errors in appropriate situations.

Starting with 0.18, ABCL has well defined points where it traps the SO and OOM conditions in any Lisp program: HANDLER-BIND (and as a consequence HANDLER-CASE) establishes a try/catch block around the form. This try/catch block makes sure OOM and SO get converted to Lisp errors, for HANDLER-BIND to handle.

The situation isn't perfect: wouldn't it be nice to have a 'low memory' signal which provided an environment with enough memory to allow user-lisp code to run, for example to free up memory held by caches. Even though it's not perfect, ABCL now provides the functionalities which the "sad case" article would lead you to believe it already did.

An option to pursue low memory condition notifications was pointed out by Douglas Miles, one of the people frequenting the #abcl irc channel on freenode.net, would be to install handlers for the JMX (java management extensions) low memory notifications.

Saturday, March 27, 2010

ABCL 0.19.1 released

On behalf of the developers of ABCL (Armed Bear Common Lisp) I'm glad to be able to announce the 0.19.1 release. Version 0.19.0 was never released.

ABCL is a Common Lisp implementation implemented in Java and running on the JVM, featuring both an interpreter and a compiler. The compiler targets the JVM directly meaning that its output is runnable JVM bytecode. The fact that ABCL is written in Java allows for relatively easy embedding in larger applications. For integration with existing applications ABCL implements Java Specification Request (JSR) 223: Java scripting API.


This release features - among lots of other things - a fix for unbinding PROGV bound variables upon exiting the PROGV scope and a much improved integration of access to filenames specified by URLs. You can find the full release notes at:


and the list of changes at:



If you have questions regarding use or licensing, or you find issues, please report back to the development list:

armedbear-devel at common-lisp dot net


Source distribution archives can be downloaded in ZIP or gzipped tar form:


Signatures are available under:



In addition, binaries are also available:


With associated signatures: