Why code-as-data?

Question

What is code-as-data? I've heard it's superior to "code-as-ascii-characters" but why? I personally find the code-as-data philosophy a bit confusing actually.

I've dabbled in Scheme, but I never really got the whole code-as-data thing and wondered what exactly does it mean?

Solution 1

It means that your program code you write is also data which can be manipulated by a program. Take a simple Scheme expression like

(+ 3 (* 6 7))

You can regard it as a mathematical expression which when evaluated yields a value. But it is also a list containing three elements, namely +, 3 and (* 6 7). By quoting the list,

 '(+ 3 (* 6 7))

You tell scheme to regard it as the latter, namely just a list containing three elements. Thus, you can manipulate this list with a program and then evaluate it. The power it gives you is tremendous, and when you "get" the idea, there are some very cool tricks to be played.

Solution 2

Code-as-data is actually only one side of the coin. The other is data-as-code.

The possibility to embed arbitrary data in Lisp code and load and reload it on the fly makes it (the data) very convenient to handle because it can eliminate any potential impedance mismatch between the way the data is represented and the way the code works.

Let me give you an example.

Let's say you want to write some kind of computer game with various monster classes. You have basically two choices: model the monster classes within your programming language or use a data-driven approach where the class descriptions are read from, say, an XML file.

Doing the modelling within the programming language has the benefits of ease of use and simplicity (which is always a good thing). It's also easy to specify custom behaviour depending on the monster class as needed. Finally, the implementation is probably pretty optimised.

On the other hand, loading everything from data files is much more flexible. You can do multiple inheritance where the language doesn't support it; you can do dynamic typing; you can load and reload things at run-time; you can use simple, to-the-point, domain-specific syntax, and much more. But now you need to write some kind of runtime environment for the whole thing, and specifying behaviour means either splitting the data up between the data files and the game code or embedding a scripting language, which is yet another layer of incidental complexity.

Or you can do it the Lisp way: specify your own sublanguage, translate that into code, and execute it. If the programming language you're using is sufficiently dynamic and syntactically flexible, you get all the benefits from using a data-driven approach (since code is data) combined with the simplicity of keeping everything in the code (since data is code).

This isn't specific to Lisp, by the way. There are various shades of code-data-equivalence gray in between Lisp and, say, C++. Ruby, for example, makes embedding data within the application easier than Python does, and Python makes it easier than Java does. Both data-as-code and code-as-data are more of a continuum than they are either-or questions.

Solution 3

As a Lisp programmer you learn to think of a program source as data. It is no longer static text, but data. In some forms of Lisp the program itself is that data structure, which gets executed.

Then all the tools are oriented that way. Instead of a textual macro processor Lisp has a macro system which works over programs as data. The transformation of programs to and from text has also its tools.

Let's think about adding two elements of a vector:

(let ((v (vector 1 2 3)))
   (+ (aref v 0)
      (aref v 1)))

There is nothing unusual about it. You can compile and run it.

But you could also do this:

(let ((v (vector 1 2 3)))
   (list '+
         (list 'aref v 0)
         (list 'aref v 1)))

That returns a list with a plus symbol and two sublists. These sublists have the symbol aref, then the array value of v and the index value.

That means that the constructed program contains actually symbols, but also data. The array is really a part of the sublists. So you can construct programs and these programs are data and can contain arbitrary data.

EVAL then evaluates the program as data.

CL-USER 17 > (setf *print-circle* t)
=>  T

Above tells us that the printer should print circular data structures such that the identities are preserved when read back.

CL-USER 18 > (let ((v (vector 1 2 3)))
               (list '+
                     (list 'aref v 0)
                     (list 'aref v 1)))
=>  (+ (AREF #1=#(1 2 3) 0) (AREF #1# 1))

Now let's eval the data as a Lisp program:

CL-USER 19 > (EVAL (let ((v (vector 1 2 3)))
                     (list '+
                           (list 'aref v 0)
                           (list 'aref v 1))))

=>  3

If your compiler expects text as source one can construct these texts, but they can never reference data directly. For these text based source construction many tools have been developed, but many of these tend to work in stages. In Lisp the functionality of manipulating data can be directly applied to manipulate programs and this functionality is directly built-in and part of the evaluation process.

So Lisp gives you an additional degree of freedom and new ways to think.

Solution 4

In Scheme (or any Lisp) you can declare list literals like this:

> '(1 2 3)
=> (1 2 3)

This is similar to many other high-level languages, except for slight differences in notation. For instance, this is how some other languages represent list literals:

[1, 2, 3] # Python
#(1 2 3) "Smalltalk. This is in fact an array in Smalltalk. Let us ignore that for now."

Lists can contain any type of values. As functions are first-class objects, a list can contain functions as well. Let us replace the first element in the above list with a function:

> '(+ 2 3)
=> (+ 2 3)

The single-quote (') identifies the list literal. (Just like the # in Smalltalk). What will happen if we remove the quote? Then the Scheme interpreter will treat the list specially. It will consider the first element as a function (or procedure) and the rest of the elements as arguments to that function. The function is executed (or evaluated):

> (+ 2 3)
=> 5

The ability to represent executable code using a data structure in the language opens a new possibility - we can write programs that write programs. That means, extensions that require changes to the compiler and the runtime system in other languages could be implemented in Lisp, as a few lines of Lisp itself. Imagine you need a new control structure in your language called when. It is similar to if but makes reading code a little more natural in some situations:

 (when this-is-true do-this)

You can extend your Lisp system to support when by writing a short macro:

 (defmacro when (condition &rest body)
    `(if ,condition (progn ,@body)))

A macro is nothing but a list, which gets expanded at compile time. More complex language structures or even entire paradigms could be added to the core language using such lists. For example, CLOS, the Common Lisp Object Systems is basically a collection of macros written in Common Lisp itself.

Solution 5

Code-as-data refers to the fact that your code is expressed in terms of the language's data structures. I wouldn't try to argue here that it's the best way to program, but I find it to be a beautiful way to express the ideas in the code.

One of the benefits is that metaprogramming is very nearly the same as regular programming. With code-as-ascii-characters, you often end up having to do some serious parsing to do anything meta, and you skip those nasty bits with Lisp.

View more solutions

Comments

[Jim Ferrans]

Your wording is "subjective and argumentative", but this is still an interesting question. Can you soften your language?

[Peter C]

Yeah, sorry if I'm a little blunt. Will change.

[Peter C]

Nifty. I'll have to look into that.

[Quibblesome]

nice comment but I just gotta say: BEST NAME EVER! :)

[jcolebrand]

I just need to say this: oh my head. signed, procedural programmer

[J S]

@drachenstern: And you have in fact seen nothing yet. The true power of code as data comes with macros.

[jcolebrand]

@JS I'm quite sure it does. But ow. That blew my mind as it was.

[Uberto]

thanks this answer just snapped something in my mind and now I make sense of many things I read about Lisp

[lukas.pukenis]

Good answer but I would give +10 for tremendous examples ;)

[Muhammad Atif]

Isnt it like eval function in many language? (I dont know lisp. so sorry if my question donot sound reasonable.)

[MeatFlavourDev]

This. ...is the explanation I've been looking for to really grok the "what/why" of Lisp.