Even better handling of low memory conditions

In previous research, ABCL came out nicely when tested for handling of Stack Overflow (SO) and Out Of Memory (OOM) conditions. See the blog item "out-of-memory: a sad case"; ABCL was classified as one of the few lisps that handled OOM conditions at all, by ending in the Lisp debugger even.

By looking more closely - which we did before the release of 0.18 - it turned out that handling of these conditions was only part of forms executed from REPL. If the same conditions occurred during program execution, the program was simply terminated, just like the other Lisps in the survey.

Of course, when you have a program running into low memory conditions regularly, it makes sense to make more memory available to it at startup. There are a few startup switches for java which allow to set the initial memory amount, most notably:

• -Xmx: which sets the maximum amount of heap available
  examples: 512m, 1024k, 13m
• -Xms: like -Xmx but specifies the initial amount
• -Xss: specifies the thread stack size

These options are only available at startup though. These options can be added to ABCL's startup wrapper script, either after building, or by letting the build add them automatically. If you opt for the latter, you need to add a line like the following to your abcl.properties file (before abcl-0.19.1 this was named build.properties):

java.options=-Xmx512m -Xss6m

If your program keeps large caches, it may be important to be able to detect OOM conditions, clear some or all of the caches and retry the operation. Other situations where it's good to be able to handle OOM conditions can be thought of. That's why it's important to have your programming environment help you detect these situations.

One of the advantages of running in the JVM is that ABCL itself doesn't doesn't need heuristics to detect these situations: the JVM throws a StackOverflowError or an OutOfMemoryError as appropiate. The only thing ABCL needs to do is handle these errors in appropriate situations.

Starting with 0.18, ABCL has well defined points where it traps the SO and OOM conditions in any Lisp program: HANDLER-BIND (and as a consequence HANDLER-CASE) establishes a try/catch block around the form. This try/catch block makes sure OOM and SO get converted to Lisp errors, for HANDLER-BIND to handle.

The situation isn't perfect: wouldn't it be nice to have a 'low memory' signal which provided an environment with enough memory to allow user-lisp code to run, for example to free up memory held by caches. Even though it's not perfect, ABCL now provides the functionalities which the "sad case" article would lead you to believe it already did.

An option to pursue low memory condition notifications was pointed out by Douglas Miles, one of the people frequenting the #abcl irc channel on freenode.net, would be to install handlers for the JMX (java management extensions) low memory notifications.