sábado, 2 de mayo de 2015

Erudite: a tool for Literate Programming in Common Lisp


Literate programming is an approach to programming introduced by Donald Knuth in which a program is given as an explanation of the program logic in a natural language, such as English, interspersed with snippets of macros and traditional source code, from which a compilable source code can be generated [1].

Literate programming is certainly controversial, but it can make sense for some projects, at least for long-living projects [2] with not an obvious design and fairly complex algorithms.

LP in Common Lisp

There are some interesting tools if you want to do LP on CL.

Org-mode is an Emacs mode for "keeping notes, maintaining TODO lists, planning projects, and authoring documents with a fast and effective plain-text system". But it can be used for LP via Babel contrib. Its distinctive features are the possibility to you show live code examples and integrate different programming languages; but it is tied to Emacs and it is not particularly thought for Lisp.

Noweb is one of the most popular LP tools. It is language independent and is known by its simplicity (at least compared to WEB, the first LP system).

CLWEB is apparently good if you want to use traditional LP in your CL project.

All these systems suffer from a required tangling phase in which code is extracted from the documents, which then has to be compiled and run. One of the distinctive features of Lisp is the possibility to develop incrementally and on a live system from a Lisp listener. This is not possible any more if a code tangling phase is present.

LP/Lisp is a LP system which solves that. LP directives and text are put in standard Lisp comments. That means there is no tangling phase; the source code is a completely normal Lisp program and thus working from a lisp listener is perfectly possible. The downside of this system is that it is not portable (it compiles on Allegro Common Lisp only). There was a version 2 planned where that and other things would be improved, but it hasn't happened yet.


Erudite is my implementation of a Literate Programming tool for Common Lisp. It turned out to be similar to LP/Lisp, but more extensible, with support for multiple input syntaxes, multiple outputs generation, and it is also portable (compiles in several of the open source Lisps).

Similarly to LP/Lisp, there's no code generation involved, so interactive development is possible; and LP directives and text reside in Lisp comments.

Erudite provides it's own LP syntax, which then can be converted to the desired output. Erudite syntax looks like this: @operation{parameters}. So for instance, @emph{this is emphasized}, is transformed to \emph{this is emphasized} in the LaTeX output, and to **this is emphasized** in the Markdown output.

If you want to use raw LaTeX syntax, then you can write LaTeX in comments and then indicate you are using LaTeX as syntax for the parser.

The supported input types are Erudite, LaTeX, Restructured Text (Sphinx flavour), and Markdown.

The supported outputs (for when using Erudite syntax) are LaTeX, Restructured Text (Sphinx flavour), and Markdown.

This is the PDF generated from Erudite source code, and these are the PDF and Markdown outputs of an output test.


[1] Knuth, Donald E. (1984). "Literate Programming" (PDF). The Computer Journal (British Computer Society) 27 (2): 97–111. doi:10.1093/comjnl/27.2.97. Retrieved January 4, 2009.
[2] Literate Programming in the Large