A Midsummer's Eve with ABCL 1.9.2

On the threshold of the Northern Hemisphere's Midsummer's Eve, we unveil the second revision of the Tenth Edition of the Armed Bear Common Lisp implementation, viz. abcl-1.9.2.

Most notably, we are pleased to present the fruits of Alejandro Zamora Fonseca's labors on an interpreted implementation of CL:STEP with the first release of the ABCL-STEPPER contrib.  See <https://abcl.org/svn/tags/1.9.2/contrib/abcl-stepper/README.markdown> for more details.  This implementation was the subject of his demonstration <https://zenodo.org/record/7815887> as this year's European Lisp Symposium.

The POSIX-SYSCALLS contrib now provides the ability to set environment variables under UNIX systems <https://abcl.org/trac/changeset/15700>, as well as providing an example of how best to add abstract additional functionality to the core implementation.

This release also features a substantially re-worked Gray Stream implementation which fixes enough our implementation to be a first class citizen of SLIME's usage.  With the next release of SLIME, one will have to use abcl-1.9.2 in order to use the slime-repl.  See <https://github.com/slime/slime/pull/765> for more details.

The ASDF-JAR contrib has been restored to a usable functionality, which includes re-packaging of ASDF systems from within jar files as well as better abstraction for finding non-source artifacts.  Please refer to <https://gitlab.common-lisp.net/abcl/abcl/-/blob/master/contrib/asdf-jar/README.markdown> for a refresher on what facilities ASDF-JAR provides for the aspiring ASDF packager.

A complete list of changes may be viewed at <https://abcl.org/svn/tags/1.9.2/CHANGES>.

The release itself is available at <https://abcl.org/releases/1.9.2/>.

Thanks to everyone who continue to support the Bear.  Enjoy yer evening…

ABCL 1.9.1 "never use a dot oh"

If one has been hesitating about using the latest ABCL because one "never uses a dot oh release", we have now sloughed off abcl-1.9.1 for  your appraisal from the depths of a Bear's long winter nap.  Now one can use the somewhat less buggy version of the Tenth Edition of Armed Bear Common Lisp, available, as usual, at <https://abcl.org/releases/1.9.1/> or (shortly) via Maven <https://search.maven.org/artifact/org.abcl/abcl/1.9.1/jar>.

Please note that when running on openjdk17 and later runtimes, there is quite a bit of necessary fiddling with command line arguments to"open" various modules to the Bear's introspective gaze. For example, see <https://abcl.org/svn/tags/1.9.1/ci/create-abcl-properties.bash> for the switches necessary to use CFFI to successfully run the CL+SSL test suite.

As a reward for your patience, we mention the following humble improvements:

CFFI compatibility

We improved the compiler to use wide indices for stack frame locals  for the ALOAD, ASTORE, ILOAD, ISTORE, LLOAD, and LSTORE opcodes, which  among other goodness, allows CFFI-TESTS to compile again.  In addition, we have updated the jar artifact used by CFFI to jna-5.12.1 which includes support for native linkage on the Apple Silicon and  other exotic architectures.

(Alan Ruttenberg) Ability to discriminate generic function execution on sub-types of MOP:SPECIALIZER

Following SBCL, CCL, and ECL, the Bear now accepts subtypes of MOP:SPECIALIZER as a generic function discriminator.

Overhauled relationship to later openjdk threading models

Going back to the future of the original green thread models, recent openjdk releases have started offering lightweight threading implementations.  For the Bear, the ability to spawn virtual threads is indicated by the presence of :VIRTUAL-THREADS in CL:*FEATURES*. Setting the special THREADS:*THREADING-MODEL* to :VIRTUAL results in THREADS:MAKE-THREADS spawning virtual threads.  On some openjdks one may have to pass a command-line switch the JVM to enable virtual threading.

(Uthar) Implement array types for JAVA:JNEW-RUNTIME-CLASS

Support for creating synthetic Java classes at runtime via JAVA:JNEW-RUNTIME-CLASS has been improved by introducing machinery for dealing with array types.

(Alejandrozf) Compiler uses of signals to fallback to interpreted form

We fall back to use an interpreted form for compilation results larger than 65535 bytes, which fixes the loading of the FirCAS computer algebra system.

(Alejandrozf) Further fixes to COMPILE-FILE-PATHNAME

We have restored the ability to compile Maxima by hopefully untanglingthe last kinks in the reworking of our COMPILE-FILE-PATHNAME logic.

(Tarn W. Burton) Avoid NIL in simple LOOP from CL:FORMAT directives

Simplify simple CL:LOOP requires only compound forms, in the place where returning NIL is neither permitted nor desired.

Broad testing and tweaks across Java Long Term Support (LTS) binaries

We have extensively tested against recent openjdk8, openjdk11,openjdk17, and openjdk19 binaries.  Unfortunately, for openjdk17+ runtimes one needs to pass additional runtime command line switches to the hosting JVM which "open" Java modules to reflective actions.

Fuller details

More details can be found in <https://abcl.org/svn/tags/1.9.1/CHANGES>.

Enjoy, and please CONS responsibly…

ABCL 1.8.0

Under the gathering storms of the Fall 2020, we are pleased to release ABCL 1.8.0 as the Ninth major revision of the implementation.

This Ninth Edition of the implementation now supports building and running on the recently released openjdk15 platform.  This release is intended as the last major release to support the openjdk6 openjdk7, and openjdk8 platforms, for with abcl-2.0.0 we intend to move the minimum platform to openjdk11 or better in order to efficiently implement atomic memory compare and swap operations.

With this release, the implementation of the EXT:JAR-PATHNAME and EXT:URL-PATHNAME subtypes of cl:PATHNAME has been overhauled to the point that arbitrary references to ZIP archives within archives now work for read-only stream operations (CL:PROBE-FILE CL:TRUENAME, CL:OPEN, CL:LOAD, CL:FILE-WRITE-DATE, CL:DIRECTORY, and CL:MERGE-PATHNAMES).  The previous versions of the implementation relied on the ability for java.net.URL to open streams of an archive within an archive, behavior that was silently dropped after Java 5, and consequently hasn't worked on common platforms supported by the Bear in a long time.  The overhaul of the implementation restores the feasibility of accessing fasls from within jar files.  Interested parties may examine the ASDF-JAR contrib for a recipe for packaging and accessing such artifacts.  Please consult the "Beyond ANSI: Pathnames" Section 4.2 of the User Manual for further details for how namestrings and components of PATHNAME objects have been revised.

A more comprehensive list of CHANGES is available with the source.

    

ABCL 1.7.1

With gentle prodding, the Bear has released ABCL 1.7.1, a decidedly minor release correcting a few bugs resulting from the overhaul of arrays specialized on unsigned byte types.

The brief list of CHANGES is a available for your perusal.

A Future History of Arming Bears

As part of the ongoing Online Lisp Meeting series, a recording of a talk about ABCL with the following precis is now available online:

With the recent releases of Armed Bear Common Lisp over the past six months, the future of extending the implementation has come into sharper focus.  The majority of this work has occurred within the head of one individual with little chance for public review and reflection. We believe that our externalized exposition of the reasoning behind these efforts will be of interest to those interested in the future history of Common Lisp implementations.

In the past few months, we released abcl-1.6.0 in which we extended the set of underlying Java Virtual Machines (JVM) that the implementation runs on to include openjdk11 and openjdk14 while maintaining compatibilty with openjdk6.  With the internal overhaul or arrays specialized on unsigned bytes in abcl-1.7.0, we made it possible to share such byte vectors with memory allocated outside of the hosting JVM via system interfaces such as malloc().  

We first present a brief prehistory on the Armed Bear Common Lisp Implementation.  Then, we first present the goals and challenges in affecting these changes within the ABCL codebase by showing examples from recent experience.  Then, we use this initial exposition to serve as a springboard to discuss outstanding needed changes in the ABCL 1 branch, and to outline some of the features intended to be present in ABCL 2, due to be released in the Fall of 2020.

A revised version of the notes for the presentation is available for deeper, asynchronous reflection. 

The Bear Arms for Sharing Byte Vectors with 1.7.0

We are pleased to announce the immediate availability of the ABCL
1.7.0 release.

After consuming a steady diet of java.nio.ByteBuffer objects over the
past month, the Bear has managed to incorporate the use of these
abstractions for arrays specialized on the commonly used unsigned-byte
types (or (unsigned-byte 8) (unsigned-byte 16) (unsigned-byte 32)).
This replacement of the use arrays of primitive bytes is denoted by
the presence of the :NIO keyword in CL:*FEATURES*.

With this :NIO overhaul, we have extended our implementation of ANSI
Common Lisp CL:MAKE-ARRAY with two additional keywords,
viz. :NIO-BUFFER and :NIO-DIRECT.

Now, the :NIO-BUFFER keyword argument to CL:MAKE-ARRAY allows one to construct a vector directly utilizing the contents of an already allocated
java.nio.ByteBuffer object.  When combined with the ability of JNA to
allocate memory on the heap via a malloc() system call, we implemented
shareable byte vectors in CFFI-SYS:MAKE-SHAREABLE-BYTE-VECTOR.

    (let* ((length 16)
           (byte-buffer (java:jstatic "allocate"
                                      "java.nio.ByteBuffer" length)))
      (make-array length :element-type ’(unsigned-byte 8) 
                         :nio-buffer byte-buffer))

When the :NIO-DIRECT keyword argument is called with a non-NIL value,
the implementation creates a byte vector with a "directly allocated"
java.nio.ByteBuffer object.  Such direct buffers typically have
somewhat higher allocation and deallocation costs than non-direct
buffers.  The contents of direct buffers may reside outside of the
normal garbage-collected heap, and so their impact upon the memory
footprint of an application might not be obvious. It is therefore
recommended that direct buffers be allocated primarily for large,
long-lived buffers that are subject to the underlying system’s native
I/O operations.  In general it is best to allocate direct buffers only
when they yield a measurable gain in program performance. In the near
future, we intend to explore the performance gains available CL:LOAD
by accessing direct buffers memory mapped to our on-disk fasl
representation.  Our fasls, as zipped archives, currently require a
new seek() from the beginning for each component they
contain.  With a memory mapped direct buffer we should be able to
simply read from the appropriate byte offset for each component.

A complete overview of the accumulated fixes and changes since the
previous release may be viewed in the file describing our CHANGES.

ABCL 1.6.1 Springs Forth

As augured, the Bear is pleased to join fellow open ANSI Common Lisp implementations CCL, ECL, and SBCL in publishing a new release around the online advent of the thirteenth European Lisp Symposium, ELS2020.

The ABCL 1.6.1 binaries and signatures are now available with their associated CHANGES.

Thanks to everyone involved in continuing to further the progress of our implementation.