Type checking (or: safety belt on with compiler code)

ABCL's compiler stores properties of blocks of code in a structure. Although it recognizes a number of different types of blocks (BLOCK, TAGBODY, etc.), there's only one structure type - called BLOCK-NODE.

While cleaning up this situation, separating the different block-node uses into different structures, I found ABCL didn't verify that the argument passed to the accessor functions for structure slots.

Cutting a long story short, we had to implement:

1. Structure type verification in the accessor functions
2. THE special operator type verification in the interpreter
3. THE special operator type verification for other policies than a *safety* value of 3 in the compiler

The first point being an issue with the accessor functions generated by ABCL: they didn't generate code to verify the argument passed. The effect being that a different structure with the same (or larger) number of slots could be passed in without an error occuring.

The second point being an issue that - even if there was a THE form - the interpreter would never verify the type specified as if it wasn't there. Talking to Peter Graves, I found that he had never intended the interpreter to be a full Common Lisp interpreter, meaning for ABCL to be a compiler-only system. The interpreter was merely there as a bootstrapping mechanism.
With all the energy spent last year to get it to the same level of CL conformance as the compiler, this point just had to be ironed out.

The third point being the issue that the compiler would treat THE as TRUELY-THE for any other *safety* value than 3. This is clearly not strict enough: it means no type verification takes place at all at any of these levels, while the user may expect some level of type verification for any level of *safety* other than zero.

Now, I can continue the reorganization of the compiler code with a safety-belt on: with the right *safety* setting, I know my structure types (and their changes) are being verified!

As a general benefit: this applies to all code running in ABCL, of course. Should you want to prevent type-verification (for example for speed reasons) in your code, just use a *safety* value of zero. In this case, the compiler simply assumes the type fits.

Efforts paying off

Well, we've had a nice break on our self-set applicability target. As explained before, a Common Lisp implementation isn't of much use if it's not able to run much of the already-existing software available for the language. As an indicator we use ABCL's ability to run a number of software packages. Maxima is one of them.

Last autumn, ABCL wasn't able to complete the test suite delivered with Maxima. This spring ABCL would run it, but with many errors. Due to continued efforts - from both sides - and mainly the fact that Maxima changed their number comparisons to be CL-compliant (EQL), there are only 3 failures remaining - out of over 4500.

The remaining 3 failure clearly ABCL issues, where the outcome returned is of a lesser precision than expected by Maxima's test suite. So, even though we're not completely there yet, I'd say we're definitely usable with Maxima.

The note to add would be that our performance is less than acceptable with Maxima: a lot slower than any other Lisp implementation. The performance issue will also be addressed from both sides: we'll research how to improve performance (generally) on our side. One thing we know to be a performance issue with Maxima is their over-use of special variables. This is - according to Robert Dodier - something being addressed on their side.

With the specials over-use fixed, ABCL can be over 40% faster, as proven with a local hack to compensate for part of the over-use. Unfortunately, the hack can't make it into the ABCL repository as a general solution.

Work is under way though to improve repeated special binding access. Such repetitions occur if a loop uses a special variable as a looping variable, or if a special variable is used to collect results. The former is a use-case in Maxima's code; the latter is a pattern regularly seen in ABCL's compiler.

In ABCL's current code, each time a special variable is accessed, be it for reading or writing, the binding is looked up. The work focuses on reusing a binding after it has been looked up once. While in most cases this won't make a performance difference: the binding will be used only once, in some applications it may help improve performance quite a bit.