ABCL on Google App Engine

Triggered by the interest of one of our users, last week was mostly dedicated to finding out how (how well) ABCL runs on Google App Engine (GAE). This is what we found out:

To those readers who don't know: GAE is an environment for hosting web applications, backed by Google's storage and server clouds. It supports running applications written in Python and Java.

GAE's Java environment turns out to be a servlet environment. This means there is a single servlet instance per JVM which gets a chance to initialize itself in an init() method. The first request waits for this method to complete before it's being processed. Google makes no guarantees regarding the number of JVMs your application might be running on concurrently, or the lifetime of a single JVM: when GAE needs memory to run other apps, your JVM might get torn down when not serving any requests.

Knowing the above, getting ABCL to run on GAE involved several steps:

1. Getting the Java SDK for GAE (don't forget to get Java's SDK too!)
2. Implementing Java classes wrapping ABCL
3. Writing a minimal servlet in ABCL

Step (1) turns out to be rather easy: just get it from the GAE website, unzip it and -if you want your local paths to match the examples on their site- rename it to remove the version number at the end (appengine-java-sdk-1.2.6 becomes appengine-java-sdk).

Step (2) turned out to be a bit more involved, but after some twiddling, we found that we needed a minimum of 2 classes: at least one servlet class and a singleton class which loads a single ABCL into the JVM. [Note: a web application may contain any number of servlets with a minimum of 1.] The resulting application classes were committed to the ABCL repository.

From there, a minimal "Hello world" web app was easy, making step (3) a quick one. The end result was entirely committed to the examples directory in ABCL's repository.

GAE offers a performance dashboard to monitor your application through an administration web interface. From there, you can check the application logs, response times (called latency in the dashboard) and see how much CPU your application is using. For the latter, they use an indicative measure: the time which it would have taken to handle the request on an unloaded Core2 1.2Ghz processor. This compensates for many of the variances in the Google infrastructure which influence how long it actually takes to handle the request.

With a working application in place, the next step was performance. Most notably that of the first request: all subsequent requests are handled within miliseconds (7 to 15 miliseconds), so there's no issue there. This is the part that dominated last week: It turned out that although the latency was around 12 seconds, the CPU consumption was around 19secs [1], both very high and said to be close to some upper limit which remains unspecified.

We're striving to get these figures down: even though they would not really impact operation of a servlet in a regular hosting situation, GAE's regular servlet restarting makes these times more important. The best way to reduce figures like these is to get the figures the application scores on your local system down first. One of the first things which comes to mind is ABCL's "long" startup time: on my local Core2 - 1GB - 1.6Ghz machine it takes 1.7 seconds.

More on the steps we took to optimize this startup time in a next blog post.


Conclusion: ABCL - if you accept the long initial request response - is definitely an option for writing your web applications in Common Lisp on a Java/JVM based infrastructure. It'll even run on Google App Engine. We'll keep you posted on how we fare on supporting that even better!


[1] In comparison: On the Clojure mailing list, 5.5 seconds is mentioned for Clojure and on #appengine (on irc.freenode.net), 7 to 10 seconds are said to be normal for JVM based apps.