Saturday, December 18, 2010

Integrating ABCL with Ant - the second step

While working to extend the ABCL integration into the ABCL-web Ant-based build - the project that I started in my previous post - I spent a lot of time trying to find a way to calculate the output paths, based on the names of the input paths.

Given the lisp way of 1-to-1 correspondence between a lisp file and a fasl, I expected this to be a trivial task to be done at the build system level. So, I kept searching for a solution in Ant (using standard components only). Surely I found the pathconvert and mapper building blocks, however, neither really seemed to work as the latter turned out not to be a generic mechanism but to be supported only by specific tasks (such as apply) and the former converts its output to a string instead of a set of paths.

Realizing that the issue of mapping source paths to output paths was left to the individual tasks, I turned to abcl to solve the problem as part of the compilation script. The final script between the CDATA tags as presented in the previous post looks like this:

(let* ((the-project (jcall "getProject" self))
       (src-iterator (jcall (jmethod "org.apache.tools.ant.types.Path" "iterator")
                            (jcall "getReference" the-project "lisp-files")))
       (src-wildcard (pathname (format nil "~A**/*.lisp" (jcall "getProperty" the-project "build.src.dir"))))
       (dst-wildcard (pathname (format nil "~A**/*.abcl" (jcall "getProperty" the-project "build.dst.dir")))))
   (when (and (jcall "hasNext" src-iterator))
     (loop
        for src-path = (pathname (jcall (jmethod "java.lang.Object" "toString")
                                 (jcall "next" src-iterator)))
        for dst-path = (translate-pathname src-path src-wildcard dst-wildcard)
        do (compile-file src-path :output-file dst-path)
        while (and (jcall "hasNext" src-iterator)))))

It's based on 2 properties having been defined: "build.src.dir" and "build.dst.dir". In addition, it wants a set of paths (fileset or path set) to be defined called "lisp-files". Iterating over the set of source files, it calculates the output file by replacing a path prefix and the file extension.

In my NetBeans project settings, I had to exclude "**/*.lisp" from packaging in the destination JAR file.

Now the JAR contains the fasl instead of the source file!

Do you have useful or resourceful ABCL based scripts plugged into Ant? Let me know and post a reaction.

Sunday, December 12, 2010

Integrating ABCL with Ant

While working with ABCL embedded in Java applications, you typically have java and lisp source files. Whenever I'm in that situaition, I find that I want to be able to build all sources from the same build mechanism.

Last week, Alex Mizrahi and I started to work on reviving ABCL-web (a project to create Java Servlets backed by Common Lisp code running in ABCL). That's when I found myself in need of a solution to this problem again: the project would greatly benefit from having a way to build lisp files for inclusion in the resulting JAR file.

Looking around, I found there are three options to achieve this goal:
  1. Create a custom Ant task
  2. Implement a command line switch to create fasls from the ABCL command line
  3. Use our JSR-223 support to write in-line Lisp in build files
While option (1) looked very attractive at first, it means one of two things: (1) adding an additional dependency for creation of our jar for distribution, or (2) creation of a separate jar file with only this task in it. In all cases, it adds a dependency to our release build. Because our current release build doesn't require Ant (you can build our release using the lisp-based build), I wasn't ready to accept this solution yet.

Option (2) looked very attractive, because it would not only integrate with Ant, but with build systems in general.  This option became less attractive when I found that the standard Ant project doesn't have any helpers for iterating sets. The thing that I had been thinking about was to implement the basics first, ie working the way gcc is used in most projects: 1 invocation per source file. So, in order to support the scheme that Ant does allow, the parameters ABCL would have to support became a lot more complex, needing to support file-sets to be compiled.

Then, I found the 'script' tag, which serves to integrate JSR-223 (Java scripting languages) into Ant build files. As we support JSR-223, I thought I'd give it a try! (So far I hadn't used it yet, I embed ABCL directly in my applications.) This solution -if it works out- is really great: no additional dependencies and no hacks [in our own build system, we feed the lisp that we want evaluated to ABCL through standard input.]

So, I gave it a try and within minutes, I was in business! I'm using a NetBeans generated project, so there's a "-post-compile" target which gets invoked after the "compile" target completes succesfully. 

<target name="-post-compile" description="lisp file compilation">
<script language="ABCL" classpath="lib/abcl.jar">
<![CDATA[
(let* ((the-project (jcall "getProject" self))
       (the-fileset (jcall "getReference" the-project "lisp-files"))
       (files-iterator
        (jcall (jmethod "org.apache.tools.ant.types.Path"
                        "iterator")
               the-fileset)))
   (loop while (jcall "hasNext" files-iterator)
        do (print (jcall "next" files-iterator))))
]]>
</script>
</target>

The snippet basically retrieves a path-set with the ID "lisp-files" defined elsewhere in the project file. Then, it continues to print all the (Java) objects it retrieves from the iterator. The next step is to replace the (print ...) form with a (compile-file ) form, retrieving the path string from the Path object and making a lisp pathname of it. Want to signal failed build? Just do what a real Task implementation would have done: throw a BuildException.

Concluding: while adding command line options may still be a desirable path to pursue, Ant integration is here for anybody embedding ABCL in a broader Java context.


Thursday, November 25, 2010

ABCL 0.23.0 released

On behalf of the developers of ABCL (Armed Bear Common Lisp) I'm glad to be able to announce the 0.23.0 release.

ABCL is a Common Lisp implementation implemented in Java and running on the JVM, featuring both an interpreter and a compiler. The compiler targets the JVM directly meaning that its output is runnable JVM bytecode. The fact that ABCL is written in Java allows for relatively easy embedding in larger applications. For integration with existing applications ABCL implements Java Specification Request (JSR) 223: Java scripting API.


This release features - among lots of other things - CLOS thread safety fixes and concurrency improvements, support for Oracle's JRockit JVM, and a new (generic) class file writer. In addition, this is the first release to complete the Maxima test suite without failures, thanks to close cooperation between our projects. You can find the full release notes at:

 http://common-lisp.net/project/armedbear/release-notes-0.23.shtml

and the list of changes at:

 http://trac.common-lisp.net/armedbear/browser/trunk/abcl/CHANGES


If you have questions regarding use or licensing, or you find issues, please report back to the development list:

 armedbear-devel at common-lisp dot net


Source distribution archives can be downloaded in ZIP or gzipped tar form:

 http://common-lisp.net/project/armedbear/releases/0.23.0/abcl-src-0.23.0.tar.gz
 http://common-lisp.net/project/armedbear/releases/0.23.0/abcl-src-0.23.0.zip

Signatures are available under:
 http://common-lisp.net/project/armedbear/releases/0.23.0/abcl-src-0.23.0.tar.gz.asc
 http://common-lisp.net/project/armedbear/releases/0.23.0/abcl-src-0.23.0.zip.asc



In addition, binaries are also available:

 http://common-lisp.net/project/armedbear/releases/0.23.0/abcl-bin-0.23.0.tar.gz
 http://common-lisp.net/project/armedbear/releases/0.23.0/abcl-bin-0.23.0.zip

With associated signatures:
 http://common-lisp.net/project/armedbear/releases/0.23.0/abcl-bin-0.23.0.tar.gz.asc
 http://common-lisp.net/project/armedbear/releases/0.23.0/abcl-bin-0.23.0.zip.asc

Sunday, October 10, 2010

ABCL hash tables: threading and CLOS

More people are starting to use ABCL in threaded environments - I have been doing so myself since spring 2008. My own use revealed some threading issues in the compiler which have been long solved now.

Last April, David Kirkman mailed the list with some threading problems in our CLOS implementation. Because of other work - including implementation of METACLASS support - there wasn't much time to do much about the issue.

This week, he mailed having found the problem - accompanied with a patch with a solution. Because of our need to support the four Common Lisp equality operators, ABCL implements its own hash tables -four of them - with a shared ancestor class which requires implementation of a few primitive operations. Hash access from the Lisp world was being synchronized by the common ancestor. However, in some locations on the Java side the - unsynchronized - primitives were being called directly.

The solution was of course to add synchronization to the primitives as well. Evaluating the result, the new situation was rather unsatisfying: only a single thread could be reading or writing at the same time, meaning only a single CLOS effective method dispatch could be happening at the same time. You probably understand my reluctance to accept the status quo.

The quick solution to the CLOS dispatch problem was to use the java.util.concurrent.ConcurrentHashMap type: we were keying on symbols, which have EQ equility in the Java world (ie in terms of Object.equals()).

That still left the lisp world with rather unsatisfactory threading performance of our own hash tables and since those were repeating large chunks of code in their specialized primitives, I decided to refactor them completely.

The result is - admittedly by looking very closely at ConcurrentHashTable - a single hash table implementation with 4 Comparator classes, which have nearly-lockless reads from all threads. Inserts into the hash table are still synchronized from all threads, but that's expected to be a lesser problem: you'd primarily expect many reads per write in a hash table.

Concluding, the only (known) CLOS threading issue has been fixed - there are presumably many more, please report if you find them - and as a bonus we have a much more efficient hash table implementation.

Sunday, October 3, 2010

Maxima on ABCL: full pass on the test suite

Over the past two weeks Raymond Toy helped out to fix the last 9 failures in the Maxima test suite running: ABCL now passes the full 8.666 Maxima tests! We achieved a goal set for ABCL more than two years ago!

Thinking back to the days when Peter Graves had just handed over the project shows just how far we have come: back in September 2008 lots of tests (423, according to Hunter Monroe) would plainly terminate the test suite. That number had come down from over 1000 tests crashing the test suite in January of that year.

After manually disabling the crashing tests, Hunter Monroe writes on September 27 (2008) that he's been able to complete the tests with a total number of failures of 612 (out of 4454 tests) .

Many of the problems were caused by incorrect handling of special variables which was one of the first things to be addressed in ABCL - stabilizing over the first half of 2009. Other improvements which contributed to the stability of Maxima were fixes to the code generating non-local returns (THROW, GOTO stepping out of a closure, RETURN-FROM stepping out of a closure). Next to its reduction in specials, the Maxima team had to change lots of number comparisons from EQ to EQL, because in ABCL integers aren't (guaranteed) to be EQ - which is allowed by the spec, but not very customary in CL implementations.

At the same time evaluation time of the test suite has gone down considerably too - not due to increased processing power, but due to improvements on both sides. One example of an improvement which has greatly benefitted ABCL's performance is the gradual elimination of specials is Maxima - an on-going effort in their team. On the other side, binding and unwinding specials has become much faster in ABCL too, reducing the impact of remaining excessive specials.

The last few tests to be fixed required adjustments to the Maxima test suite as well as fixes to ABCL. Thanks to everybody who helped achieve this result, most notably Robert Dodier, Hunter Monroe and Raymond Toy from the Maxima team as well as Ville Voutilainen and Mark Evenson from the ABCL team.

Sunday, September 26, 2010

ABCL 0.22.0 released

On behalf of the developers of ABCL (Armed Bear Common Lisp) I'm glad to be able to announce the 0.22.0 release.

ABCL is a Common Lisp implementation implemented in Java and running on the JVM, featuring both an interpreter and a compiler. The compiler targets the JVM directly meaning that its output is runnable JVM bytecode. The fact that ABCL is written in Java allows for relatively easy embedding in larger applications. For integration with existing applications ABCL implements Java Specification Request (JSR) 223: Java scripting API.

The release is a small maintenance release: most efforts were focussed on work happening on branches some of which has already been merged to trunk for 0.23.



Latest and older binary and source distributions can be downloaded from http://common-lisp.net/project/armedbear/releases/

Sunday, September 19, 2010

Parsing latin1 data with an utf-8 stream

We recently noticed that ABCL isn't capable of reading files containing certain latin1 characters, like the author comments in albert. ABCL's DecodingReader uses java.nio.charset.CharsetDecoder, which defaults to throwing exceptions when it finds unmappable character or input it otherwise considers malformed. Therefore, reading albert files, which contain Norwegian/Danish o-slashes resulted in an exception being thrown. Same result occurred with files that contain o-umlauts.

The remedy that seemed obvious was changing the CharsetDecoder's error handling strategy to replace malformed input and unmappable characters. Alas, this didn't work, to much surprise, as it resulted in the DecodingReader to loop endlessly. Despite this document, which seems to suggest that an utf-8 byte sequence can only start with f0-f4, it seemed that o-umlaut (f6) and o-slash (f8) threw the decoder off-balance.

After debugging the problem, we noticed that in the case of these characters, the DecodingReader reported an overflow but didn't advance the buffers it uses. So, after some consideration, we introduced a hack where, in case of an overflow that didn't advance the buffers at all, we advance the buffers manually by one byte and insert a ? character into the resulting stream. This reeks of a hack, but so far we haven't been able to find a more elegant solution. This solution does seem to enable us to parse files with such latin1 chars like o-umlaut and o-slash properly, and the fix was committed into ABCL trunk as commit r12902, and will be part of ABCL 0.22.

Sunday, August 1, 2010

ABCL-0.21.0 released....

.... so long ago that it's time to release 0.22 in the week(s) to come.

Sunday, July 11, 2010

URI Pathnames

Among other goodies, ABCL-0.20 includes good support for using URI resources as arguments to about any function you would specify a filepath. Any builtin scheme like "http" or "file" or "jar" works as a full-on Lisp Pathname, with the caveat that one may not write to an associated Stream. We should be generic enough in implementation that the JVM extension mechanism for extending the classloader works for us for the most common use case, OSGi.

We pick up the ability to refer to URI, associating an input Stream with its representation as a network (or local system) bytes easily as the following one-liner:


CL-USER> (with-open-file (stream #p"http://google.com/index.html")
(format nil "~A" (read-line stream nil)))

Underneath we have implemented a new subtype of PATHNAME, namely the type >URL-PATHNAME. We name this URL as opposed to URI as the underlying Java object is a java.net.URL with associated java.net.URLConnection that we'll be relying on for input sources.

Looking through an APROPOS for URL-PATHNAME shows the following related symbols:


CL-USER> (apropos (type-of #p"http://google.com/index.html"))
EXTENSIONS::SET-URL-PATHNAME-SCHEME (fbound)
EXTENSIONS::SET-URL-PATHNAME-AUTHORITY (fbound)
EXTENSIONS::SET-URL-PATHNAME-QUERY (fbound)
EXTENSIONS::SET-URL-PATHNAME-FRAGMENT (fbound)
URL-PATHNAME
URL-PATHNAME-AUTHORITY (fbound)
URL-PATHNAME-SCHEME (fbound)
URL-PATHNAME-FRAGMENT (fbound)
URL-PATHNAME-QUERY (fbound)
; No value
CL-USER>


which we see the defined functions to set the AUTHORITY, SCHEME, FRAGMENT, and QUERY portions of a URL-PATHNAME.

After these modifications, the ABCL PATHNAME class now has two "beyond ANSI" subtypes, namely URL-PATHNAME and JAR-PATHNAME. A short document outlining the design of URL-PATHNAME can be found in our documentation directory.

In implementing URL-PATHNAME, we actually had the ability to analyze what another CL did for their implementation. We analyzed the CLForJava technical paper, but decided not to use their abstractions as detailed on a lengthy armedbear-devel post.

Thursday, May 27, 2010

ABCL 0.20.0 released, including first funded feature

On behalf of the developers of ABCL (Armed Bear Common Lisp) I'm glad to be able to announce the 0.20.0 release.

ABCL is a Common Lisp implementation implemented in Java and running on the JVM, featuring both an interpreter and a compiler. The compiler targets the JVM directly meaning that its output is runnable JVM bytecode. The fact that ABCL is written in Java allows for relatively easy embedding in larger applications. For integration with existing applications ABCL implements Java Specification Request (JSR) 223: Java scripting API.

This release is the first to include a funded feature: funds were provided to implement the CLOS METACLASS feature and tests. Next to that, this release contains a large number of fixes and improvements, such
as the ability to use JARs and URLs as pathnames and the a new ASDF version (ASDF2). You can find the full release notes at:


and the list of changes at:


Latest and older binary and source distributions can be downloaded from

Thursday, May 6, 2010

Running SLIME under cygwin with ABCL

If ABCL may be said to have an IDE, it has one in the combination of Emacs using the SLIME to connect to running Lisp instance(s). In the parlance of SLIME, Emacs is said to be the superior process with the running Lisp instance(s) known as the inferior Lisp process. With SLIME, Emacs provides symbol completion in a REPL with edit history, Lisp syntax highlighting, finding the source location for a given form, the ability to run multiple instances of the inferior Lisp (or even different versions of Lisps), the ability to interactively inspect Lisp values, a nice wrapper for controlling ASDF, and much more.

When confronted with a Microsoft Windows environment that I need to productively produce code in, the first thing I do is install the cygwin package to get to a minimal UNIX-like environment. Unlike many Windows enhancements, the installation of cygwin does not require Administrator privileges so it is possible even in a somewhat restricted corporate environment. cygwin provides a UNIX compatibility layer that has enabled the porting of many standard GNU utilities to run under Windows, including an X11 server.

One of the first packages that I install for use under cygwin is the Emacs X11 package, which provides The One True editor with a lot of tools that make simple programmatic tasks easy, and the construction of complicated system possible. Historically there have been other efforts to bring Emacs to Windows such as NTEmacs, but now that GNU Emacs 23.1 runs fine in an X11 server with cygwin, I think the ease by which cygwin installs and updates outweighs putative advantages of Windows-specific Emacs ports. In truth, over time, GNU Emacs has absorbed most of the useful features pioneered by other, more native, versions.

People have reported that they have been able to get SLIME working with Emacs under Windows, but I never had much success, maybe because cygwin imposes a UNIX-like pathname structure that are naturally quite different from Windows pathnames. Not only is the '\' <--> '/' convention switch, Windows drive letters (like 'C:') are mapped to mounts unto the UNIX-like root filesystem, so 'C:\' becomes '/cygdrive/c'. Since ABCL under Windows runs on the JVM it expects to be dealing with Windows pathnames, it never worked for me. Until today. Or rather, yesterday. So, without further ado about pathnames, here's …

HOWTO install SLIME under Windows for ABCL

0. Install cygwin, selecting the 'emacs-x11' and 'cvs' packages in the interactive chooser.

1. Make a directory called 'c:\work' to contain ABCL and SLIME. You can of course use whatever directory you wish here, but the adjust the rest of these instructions to whatever you choose. Under cygwin, this directory will appear as '/cygdrive/c/work'.

2. Install ABCL under 'c:\work\abcl', so that 'c:\work\abcl\abcl.bat' invokes ABCL.

3. Use CVS to retrieve a copy of SLIME HEAD later than 2010-05-05. SLIME doesn't do releases which put me off for a bit at first, so even though there is a slime-2.0 floating around somewhere, it's ancient by this time, and a release process probably won't be repeated by the SLIME community in the foreseeable future.

From the bash prompt you installed in step 0, from the /cygdrive/c/work directory issue the following command:
cmd$ cvs -d  :pserver:anonymous:anonymous@common-lisp.net:/project/slime/cvsroot checkout slime
This will checkout a copy of SLIME including documentation. Alternatively one may download a CVS snapshot but then it will be harder to update SLIME later if you wish.

4. Use Emacs to edit '~/.emacs' to contain the following code
;;; .emacs for SLIME working with cygwin emacs

(add-to-list 'load-path "/cygdrive/c/work/slime")
(setq slime-lisp-implementations
'((abcl ("/cygdrive/c/work/abcl/abcl.bat"))))
(require 'slime)
(slime-setup '(slime-repl slime-asdf slime-fancy slime-banner))

(defun slime-to-lisp-translation (filename)
(replace-regexp-in-string
"\n" "" (shell-command-to-string
(format "cygpath.exe --windows %s" filename))))

(defun lisp-to-slime-translation (filename)
(replace-regexp-in-string
"\n" "" (shell-command-to-string
(format "cygpath.exe --unix %s filename"))))

(setq slime-to-lisp-filename-function #'slime-to-lisp-translation)
(setq lisp-to-slime-filename-function #'lisp-to-slime-translation)
This code sets up SLIME to be autoloaded by Emacs, using the 'cygpath.exe' to translate pathnames between the cygwin UNIX conventions (which Emacs expects), and the native conventions (which ABCL running on the JVM expects).

5. Upon evaluating the elisp code which you just used to configure SLIME (which can be affected by restarting Emacs), SLIME can be invoked via "M-x slime". ABCL will be invoked as a separate process, it will compile the elisp server known as swank that communicates between Emacs and the inferior Lisp process, it will load that code to initiate a connection, you'll see some "flying letters" coalesce into a REPL with a message in the mini-buffer welcoming you to SLIME

Happy hacking!

Monday, April 26, 2010

ABCL on N900

I decided to play a bit with my new mobile toy. The Nokia N900 should be capable of all kinds of things, including running a Java VM. After some googling, I found this page:

http://wiki.maemo.org/OpenJDK_6.0_0_%28Cambridge_Software_Labs%29_on_N900

The instructions there seemed simple and straightforward, so I did as advised, downloaded the jdk package and extracted it onto the device, and then just copied abcl.jar over and ran it with the incantation that's usually found in the abcl wrapper script. The results are pretty good:

~ $ /opt/OpenJDK-camswl/bin/java -cp /home/user/abcl.jar org.armedbear.lisp.Main
Armed Bear Common Lisp 0.20.0-dev
Java 1.6.0_0 Sun Microsystems Inc.
OpenJDK Zero VM
Low-level initialization completed in 4.436 seconds.
Startup completed in 16.765 seconds.
Type ":help" for a list of available commands.
CL-USER(1): (format t "abcl rules!")
abcl rules!
NIL
CL-USER(2):

It runs obviously much slower than on any desktop, the device is not as powerful, and its i/o is slower which hurts autoloading. But it runs, and it is much easier to setup than java ever was on N800 and such, and it also runs much faster than the non-zero VMs on N800 did.

Saturday, April 24, 2010

CGOL on ABCL

The other day I received an e-mail from an ABCL user, containing the following:

I took a brief look at cgol. Neat! Interestingly, I tried it in
clisp, sbcl, ecl, cmulisp, and abcl, and it only works in abcl!


For those wanting to play with CGOL too, it's available from the CMU Artificial Intelligence Repository.

Seems like ABCL is Common Lisp conformant enough to be able to run some of the older lisp code available: this source stems from 1993 and is unchanged since - judging by the timestamps in the download archive.

Saturday, April 3, 2010

Even better handling of low memory conditions

In previous research, ABCL came out nicely when tested for handling of Stack Overflow (SO) and Out Of Memory (OOM) conditions. See the blog item "out-of-memory: a sad case"; ABCL was classified as one of the few lisps that handled OOM conditions at all, by ending in the Lisp debugger even.

By looking more closely - which we did before the release of 0.18 - it turned out that handling of these conditions was only part of forms executed from REPL. If the same conditions occurred during program execution, the program was simply terminated, just like the other Lisps in the survey.

Of course, when you have a program running into low memory conditions regularly, it makes sense to make more memory available to it at startup. There are a few startup switches for java which allow to set the initial memory amount, most notably:
  • -Xmx: which sets the maximum amount of heap available
    examples: 512m, 1024k, 13m
  • -Xms: like -Xmx but specifies the initial amount
  • -Xss: specifies the thread stack size
These options are only available at startup though. These options can be added to ABCL's startup wrapper script, either after building, or by letting the build add them automatically. If you opt for the latter, you need to add a line like the following to your abcl.properties file (before abcl-0.19.1 this was named build.properties):

java.options=-Xmx512m -Xss6m

If your program keeps large caches, it may be important to be able to detect OOM conditions, clear some or all of the caches and retry the operation. Other situations where it's good to be able to handle OOM conditions can be thought of. That's why it's important to have your programming environment help you detect these situations.

One of the advantages of running in the JVM is that ABCL itself doesn't doesn't need heuristics to detect these situations: the JVM throws a StackOverflowError or an OutOfMemoryError as appropiate. The only thing ABCL needs to do is handle these errors in appropriate situations.

Starting with 0.18, ABCL has well defined points where it traps the SO and OOM conditions in any Lisp program: HANDLER-BIND (and as a consequence HANDLER-CASE) establishes a try/catch block around the form. This try/catch block makes sure OOM and SO get converted to Lisp errors, for HANDLER-BIND to handle.

The situation isn't perfect: wouldn't it be nice to have a 'low memory' signal which provided an environment with enough memory to allow user-lisp code to run, for example to free up memory held by caches. Even though it's not perfect, ABCL now provides the functionalities which the "sad case" article would lead you to believe it already did.

An option to pursue low memory condition notifications was pointed out by Douglas Miles, one of the people frequenting the #abcl irc channel on freenode.net, would be to install handlers for the JMX (java management extensions) low memory notifications.

Saturday, March 27, 2010

ABCL 0.19.1 released

On behalf of the developers of ABCL (Armed Bear Common Lisp) I'm glad to be able to announce the 0.19.1 release. Version 0.19.0 was never released.

ABCL is a Common Lisp implementation implemented in Java and running on the JVM, featuring both an interpreter and a compiler. The compiler targets the JVM directly meaning that its output is runnable JVM bytecode. The fact that ABCL is written in Java allows for relatively easy embedding in larger applications. For integration with existing applications ABCL implements Java Specification Request (JSR) 223: Java scripting API.


This release features - among lots of other things - a fix for unbinding PROGV bound variables upon exiting the PROGV scope and a much improved integration of access to filenames specified by URLs. You can find the full release notes at:


and the list of changes at:



If you have questions regarding use or licensing, or you find issues, please report back to the development list:

armedbear-devel at common-lisp dot net


Source distribution archives can be downloaded in ZIP or gzipped tar form:


Signatures are available under:



In addition, binaries are also available:


With associated signatures:

Thursday, February 11, 2010

REQUIRE now works with ASDF systems

The ANSI *MODULES*/REQUIRE/PROVIDE interface is usually implemented in a fairly simple manner. When a piece of code wishes to depend on functionality of a given module, a statement of the form

(require :module)

first looks for whether the modules is currently in the list referenced by *MODULES*. If this isn't the case, a Lisp implementation simply looks in a few standard places for a file to load based on the STRING form of ":module", like "module.lisp", or possibly by a FASL. That file is then loaded, during which the form

(provide :module)

is evaluated which pushes the keyword form of its argument to the *MODULES* list.

This very simple dependency mechanism really doesn't provide a mechanism for searching file hierarchies, being more appropriate to manage dependencies in a large collection of files located in a single directory as is the case in the ABCL system directory. Presumably this simplicity and lack of extensibility led to the facility as being noted as deprecated in the HyperSpec. But it is part of the ANSI standard so until the New Implementation of Lisp is released, it forms the only mechanism that portable code can rely on.

In the contemporary Lisp era, ASDF forms the most common method to specify system dependencies. Since ANSI doesn't specify how REQUIRE searches for the dependencies, the clever folks behind SBCL decided to make REQUIRE invoke ASDF if the dependency cannot be resolved by the simple version of the mechanism.

Since implementation is the sincerest form of flattery, as of ABCL svn r12247 we have adopted this extension to REQUIRE. To use this mechanism, you first have to ensure that ASDF itself is loaded via

(require 'asdf)

Afterwards, a command like

(require :hunchentoot)

will search for an ASDF loadable system named "hunchentoot".

But how does ASDF find these systems? By successively searching the contents of ASDF:*CENTRAL-REGISTRY* which is a list of locations to search for files or symbolic links named "hunchentoot.asd". Since maintaining this list for a large number of ASDF definitions would be onerous, the usual practice is to have a single directory which contains symbolic links to the system definitions. It makes maintaining various versions a bit easier as one can simply manipulate the symbolic links.

I use the following piece of code in my "~/.abclrc" to collect all my ASDF pointers in the directory "~/.sbcl/systems/":

(require 'asdf)
(pushnew '(merge-pathnames ".sbcl/systems/" (user-homedir-pathname))
asdf:*central-registry*)

How does one install ASDF systems? One downloads the code and creates a link to its "*.asd" file in "~/.sbcl/systems/". But that can get onerous to do manually especially if one ASDF system depends on other systems. The ASDF-INSTALL package provides a semi-interactive automated mechanism to download, compile, and install ASDF definitions. For now, using SBCL to install ASDF systems that ABCL can share works pretty well, but we're currently working on incorporating a port of ASDF-INSTALL into the base ABCL system. Stay tuned!


Monday, January 18, 2010

ABCL 0.18.1 released

Due to some unfortunate regressions in 0.18.0, the ABCL project had to release an 0.18.1 release. It's readily available for download and is strongly advised for everybody who's now running 0.18.0: the .0 release can behave seemingly randomly after loading compiled code.

Wednesday, January 13, 2010

ABCL 0.18.0 released

On behalf of the developers of ABCL (Armed Bear Common Lisp) I'm glad to be able to announce the 0.18.0 release.

This release features - among lots of other things - faster initial startup, faster special variable lookup and portable fasl files. Please refer to the release notes for the full list.

If you have questions regarding use or licensing, or you find issues, please report back to the development list: armedbear-devel at common-lisp dot net


Source distribution archives can be downloaded in ZIP (zip signature file) or gzipped tar (tar signature file) format.