clbuild FAQ

Since bloggers started covering clbuild (1,2), there has been a noticable surge of newbie questions.

To record answers to those questions, there is now a longer FAQ for clbuild, covering these topics:

• How does clbuild differ from asdf-install?
• My favourite application is not supported. How can I add it?
• Why did you get rid of all tarball-only downloads?
• It doesn't load my ~/.sbclrc!
• "clbuild lisp" doesn't seem to use my monster.core!
• Can I start the lisp with swank already preloaded?
• Can I run emacs and slime without going through "clbuild slime"?
• Is the "dependencies" file autogenerated?
• The "dependencies" file is broken!
• Why is clbuild written in shell? Lisp is so much better.
• Does it work on cygwin?

Note the slime-related changes, which allow swank startup the way Bill is explaining it in his post, but without the workarounds.

comp.lang.lisp needs to stop complaining about overcommit

or: How I got sucked back into SBCL hacking on Christmas.

: david@radon:~; ps -o pid,vsz,rss,comm -p `pidof sbcl`
  PID    VSZ   RSS COMMAND
 1019 570424  4424 sbcl
: david@radon:~; ps -o pid,vsz,rss,comm -p `pidof sbcl`
  PID    VSZ   RSS COMMAND
 1019 963320 404204 sbcl
: david@radon:~; ps -o pid,vsz,rss,comm -p `pidof sbcl`
  PID    VSZ   RSS COMMAND
 1019 3988328 825804 sbcl

Ten years of Closure HTML

More than a decade ago, Gilbert Baumann started writing the Closure web browser. It includes a great HTML parser, written all in Lisp.

Released today, Closure HTML is a stand-alone version of the parser.

It supports HTML 4, understands malformed HTML, and can (optionally) be used in conjunction with Closure XML and its data structures.

An easy way to get started with Closure HTML itself is with its LHTML builder, which represents HTML elements as simple lisp lists.

Together, the two parsers can be used to turn HTML into XHTML or vice versa, and in particular to parse HTML into DOM or STP. Even for users who only parse and work with XHTML internally, the new code can be useful to emit normal HTML 4 as the last step of processing.

Forking SBCL for Dummies

http://repo.or.cz/ hosts a git repository for SBCL and makes it easy to publish your own fork of SBCL.

Here is a quick step-by-step guide for anyone planning to have his repository hosted there. (All of this will be painfully obvious to the git experts.)

• Register a user account. All you need is an SSH public key, no questions asked.
• Create the fork. Find the SBCL project and go to "fork". Enter a project name for the fork and an admin password.

Done. Now you have a fork, but you need to initialize it first.

• Go to your "Project Settings" page and add yourself as a user.

Otherwise you cannot push to your own project.

• Push into the fork. One way to do this is to clone the normal SBCL repository, then use

  git push --all ssh://yourusername@repo.or.cz/srv/git/sbcl/yourprojectname.git

Don't forget the --all, which instructs git to push all refs. Whatever a ref is, anyway.

cloak

It is unfinished, slow, buggy, unmaintained, in need of a rewrite -- and now you can hack it yourself!

Doesn't that sound exciting?  Of course it does.

Prerequisites. Only Linux/x86 is supported¹. You will need several hours of spare CPU time, about 1 GB of RAM, and lots of disk space. Compilation involves building SBCL and classpath first, so make sure to install all required dependencies first. Debian users can run

# apt-get install sbcl svn cvs wget jikes
# apt-get build-dep classpath

to do so.

Build script. Grab cloak using git [edit: needs git 1.5, no idea why]:

$ git clone http://www.lichteblau.com/git/cloakbuild.git

and compile it using clbuild-like commands:

$ ./build update
$ ./build world

Usage. The bin directory contains scripts called java, javac (courtesy of ecj), javap, and javah that run Lisp with the right arguments.

$ ./bin/java -version
CLOAK Virtual Machine, running on SBCL 0.9.8.6 (Linux 2.6.22 X86)

Copyright (C) 2003-2007 David Lichteblau

Technically it is a precompiler, and to avoid unpleasant surprises at run time, you might want to run

./bin/precompile foo.jar

before starting anything non-trivial.

Finally, read cloak/TODO and start hacking.

What's new? Compared to the big binary tarball available previously, this one comes with sources only, has been updated for current SBCL, and for Classpath 0.91 (which is still ancient, but a little step forward). The scripts in bin/ are also new.

¹ No AMD 64 support yet. For now, use an x86 chroot instead.

A new data structure for XML

As mentioned earlier, I set out to define an alternative to the W3C's Document Object Model, inspired heavily by Java's XOM, but made for Common Lisp.

The result is STP, a data structure for XML that is full-featured and uses CLOS, but is more natural than DOM and gets namespaces right. Its implementation cxml-stp is available as an add-on library for Closure XML.

(For most purposes, it should be preferable to other alternatives, but DOM fans -- in case there are any -- can be assured that DOM support in cxml will not go away either.)

Read more about STP in the tutorial.

Macros for XSLT

Gary King is confused because [XSLT] seems so ridiculously verbose. Others have already suggested mad higher-order function tricks using XSLT 2.0.

My solution: Macros. If XSLT lacks an element to do what you want (creating a text node with a newline, in Gary's case), just invent the feature you need and send your XSLT stylesheet through another XSLT stylesheet to implement it.

BR

Say you have demo.xsl which wants to use

  <x:br/>

to emit a newline. (In this example, `x' is simply the namespace for our extensions.) Write an additional stylesheet macros.xsl and send the original demo.xsl through the macro stylesheet to generate the actual XSLT source code. A macro template for <x:br> would be as simple as:

  <xsl:template match="x:br">
    <_xsl:text><xsl:text>&#10;</xsl:text></_xsl:text>
  </xsl:template>

In the macro stylesheet, xsl is the namespace of the "macro definition" and _xsl is the namespace of the "macro expansion". (If you care about details, the trick is to use xsl:namespace-alias to make the XSLT processor believe they are different namespaces.)

DOTIMES

For a more interesting example of macro use, suppose we want to repeat our code count times. Doing this kind of iteration involves a recursive template call, which we want to hide. We will define a macro <x:dotimes> that can be used like this:

  <x:dotimes var="i" count="3">
    <xsl:value-of select="$i"/>
  </x:dotimes>

Our macro stylesheet replaces each use of <x:dotimes> with a template call, and adds a recursive template as a top-level element:

  <xsl:template match="xsl:stylesheet">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
      <xsl:for-each select="//x:dotimes">
        <_xsl:template name="x:dotimes_{generate-id()}">
          ... recursive template definition here ...
        </_xsl:template>
      </xsl:for-each>
    </xsl:copy>
  </xsl:template>
  
  <xsl:template match="x:dotimes">
    <_xsl:call-template name="x:dotimes_{generate-id()}">
      ... parameters elided for brevity ...
    </_xsl:call-template>
  </xsl:template>

Download the full macros.xsl and demo.xsl to try the example. To run it with xsltproc, use the Makefile in the same directory.