Lisp and Erlang Atoms, Ruby and Scheme Symbols. How useful are they?

Question

How useful is the feature of having an atom data type in a programming language?

A few programming languages have the concept of atom or symbol to represent a constant of sorts. There are a few differences among the languages I have come across (Lisp, Ruby and Erlang), but it seems to me that the general concept is the same. I am interested in programming language design, and I was wondering what value does having an atom type provide in real life. Other languages such as Python, Java, C# seem to be doing quite well without it.

I have no real experience of Lisp or Ruby (I know the syntaxes, but haven't used either in a real project). I have used Erlang enough to be used to the concept there.

Solution 1

A short example that shows how the ability to manipulate symbols leads to cleaner code: (Code is in Scheme, a dialect of Lisp).

(define men '(socrates plato aristotle))

(define (man? x) 
    (contains? men x))

(define (mortal? x) 
    (man? x))

;; test

> (mortal? 'socrates)
=> #t

You can write this program using character strings or integer constants. But the symbolic version has certain advantages. A symbol is guaranteed to be unique in the system. This makes comparing two symbols as fast as comparing two pointers. This is obviously faster than comparing two strings. Using integer constants allows people to write meaningless code like:

(define SOCRATES 1)
;; ...

(mortal? SOCRATES)
(mortal? -1) ;; ??

Probably a detailed answer to this question could be found in the book Common Lisp: A Gentle Introduction to Symbolic Computation.

Solution 2

Atoms are literals, constants with their own name for value. What you see is what you get and don't expect more. The atom cat means "cat" and that's it. You can't play with it, you can't change it, you can't smash it to pieces; it's cat. Deal with it.

I compared atoms to constants having their name as their values. You may have worked with code that used constants before: as an example, let's say I have values for eye colors: BLUE -> 1, BROWN -> 2, GREEN -> 3, OTHER -> 4. You need to match the name of the constant to some underlying value. Atoms let you forget about the underlying values: my eye colors can simply be 'blue', 'brown', 'green' and 'other'. These colors can be used anywhere in any piece of code: the underlying values will never clash and it is impossible for such a constant to be undefined!

^{taken from http://learnyousomeerlang.com/starting-out-for-real#atoms}

With this being said, atoms end up being a better semantic fit to describing data in your code in places other languages would be forced to use either strings, enums or defines. They're safer and friendlier to use for similar intended results.

Solution 3

Atoms (in Erlang or Prolog, etc.) or symbols (in Lisp or Ruby, etc.)—from herein only called atoms—are very useful when you have a semantic value that has no natural underlying "native" representation. They take the space of C-style enums like this:

enum days { MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, SUNDAY }

The difference is that atoms don't typically have to be declared and they have NO underlying representation to worry about. The atom monday in Erlang or Prolog has the value of "the atom monday" and nothing more or less.

While it is true that you could get much of the same use out of string types as you would out of atoms, there are some advantages to the latter. First, because atoms are guaranteed to be unique (behind the scenes their string representations are converted into some form of easily-tested ID) it is far quicker to compare them than it is to compare equivalent strings. Second, they are indivisible. The atom monday cannot be tested to see if it ends in day for example. It is a pure, indivisible semantic unit. You have less conceptual overloading than you would in a string representation in other words.

You could also get much of the same benefit with C-style enumerations. The comparison speed in particular is, if anything, faster. But... it's an integer. And you can do weird things like have SATURDAY and SUNDAY translate to the same value:

enum days { SATURDAY, SUNDAY = 0, MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY }

This means you can't trust different "symbols" (enumerations) to be different things and thus makes reasoning about code a lot more difficult. Too, sending enumerated types through a wire protocol is problematical because there's no way to distinguish between them and regular integers. Atoms do not have this problem. An atom is not an integer and will never look like one behind the scenes.

Solution 4

As a C programmer I had a problem with understanding what Ruby symbols really are. I was enlightened after I saw how symbols are implemented in the source code.

Inside Ruby code, there is a global hash table, strings mapped to integers. All ruby symbols are kept there. Ruby interpreter, during source code parse stage, uses that hash table to convert all symbols to integers. Then internally all symbols are treated as integers. This means that one symbol occupies only 4 bytes of memory and all comparisons are very fast.

So basically you can treat Ruby symbols as strings which are implemented in a very clever way. They look like strings but perform almost like integers.

When a new string is created, then in Ruby a new C structure is allocated to keep that object. For two Ruby strings, there are two pointers to two different memory locations (which may contain the same string). However a symbol is immediately converted to C int type. Therefore there is no way to distinguish two symbols as two different Ruby objects. This is a side effect of the implementation. Just keep this in mind when coding and that's all.

Solution 5

In Lisp symbol and atom are two different and unrelated concepts.

Usually in Lisp an ATOM is not a specific data type. It is a short hand for NOT CONS.

(defun atom (item)
  (not (consp item)))

Also the type ATOM is the same as the type (NOT CONS).

Anything that is not a cons cell is an atom in Common Lisp.

A SYMBOL is a specific datatype.

A symbol is an object with a name and identity. A symbol can be interned in a package. A symbol can have a value, a function and a property list.

CL-USER 49 > (describe 'FOO)

FOO is a SYMBOL
NAME          "FOO"
VALUE         #<unbound value>
FUNCTION      #<unbound function>
PLIST         NIL
PACKAGE       #<The COMMON-LISP-USER package, 91/256 internal, 0/4 external>

In Lisp source code the identifiers for variables, functions, classes and so on are written as symbols. If a Lisp s-expression is read by the reader, it does create new symbols if they are not known (available in the current package) or reuses an existing symbol (if it is available in the current package. If the Lisp reader reads a list like

(snow snow)

then it creates a list of two cons cells. The CAR of each cons cell point to the same symbol snow. There is only one symbol for it in the Lisp memory.

Also note that the plist (the property list) of a symbol can store additional meta information for a symbol. This could be the author, a source location, etc. The user can also use this feature in his/her programs.

View more solutions

Comments

[Rainer Joswig]

In Lisp the symbols don't cost much for the compiler, since the lookup is done already by the 'reader'.

[paulfischer]

Upvote for Touretsky's book! It's one of my favorite Lisp texts.

[I GIVE TERRIBLE ADVICE]

O(NumberOfAtoms) is not necessarily right -- All you need is to have a sane unique id generation scheme (Erlang uses references, which are incrementing values bound to the VM's lifetime) making new atoms is mostly a free operation that needs not to be considered. In the case of Erlang, atoms are not GC'ed though, so it's usually a bad idea to generate them dynamically anyway.

[JUST MY correct OPINION]

Wouldn't you be using O(NumberOfUniqueStrings) in a string-based alternative to atoms/symbols? And I'd guess that it's more O(1) than O(n) since, as I GIVE TERRIBLE ADVICE noted, you just need a sane ID generation system.

[JUST MY correct OPINION]

All very interesting and true, but not answering the question. The question is talking about the "atom data type" which, given the OP's comment about knowing Erlang, would be referring to what Erlang calls an atom and what Lisp calls a symbol (as does Ruby if memory serves). The clue is contained in "A few programming languages have the concept of atom or symbol to represent a constant of sorts. There are a few differences among the languages I have come across (Lisp, Ruby and Erlang), but it seems to me that the general concept is the same."

[I GIVE TERRIBLE ADVICE]

Having re-read the comment better, in Erlang's case, you do need O(LengthOfAllStrings+NUniqueIDs) in terms of storage. However, each active use of the atom in the code doesn't require to know the string itself and only the ID can be used. Different implementations (i.e. Prolog) will have garbage collection of atoms, and you can bet that depending on the actual application, different tradeoffs will be done: using the same string 5000 times vs. using 5 atoms a thousand times give different memory usage results

[Rainer Joswig]

@JUST MY correct OPINION: The OP was talking about 'Atom' in Lisp and Erlang. Also about Symbols in Ruby and Scheme. I explained that ATOM and Symbols are not related, so his question makes limited sense. I explained then the difference between ATOMs and Symbols in Lisp, and what is offered by Symbols.

[Rainer Joswig]

@JUST MY correct OPINION: Naming constants is only one use case for symbols in Lisp. Symbols are mostly used as identifiers for some concept (function, variable, class) with possibly added metadata. In Ruby a symbol is comparable to what Lisp calls a keyword symbol. But that has limited use. It has not the attributes a Lisp symbol has. A keyword symbol in Lisp evaluates always to itself and is in the keyword package.

[YasirA]

+1 But don't forget, for example, erlang:atom_to_list/1 and its opposite erlang:list_to_atom/1. They allow you to convert between atoms and strings (lists). It's discouraged though :-)

[JUST MY correct OPINION]

Yasir: But a conversion, by definition, means it's no longer an atom (or a list, depending on direction).

[YasirA]

I was commenting your "The atom monday cannot be tested to see if it ends in day for example." part WRT Erlang. Also, you forgot to put @ in front of my name, I wouldn't notice your comment :-)

[YasirA]

All I just said can be true for Erlang. Not sure about other languages, mentioned in the question.

[JUST MY correct OPINION]

@Yasir Arsanukaev: I know what you were commenting on. I was pointing out that if you convert the atom to a list, you're not comparing part of an atom any longer. You're comparing a list (as a string). Just like I can compare if the bottom end of an integer is "1671" by converting to a string -- it's not comparing integers any longer.

[Muhammad Alkarouri]

So a symbol is a global efficient constant with some sort of type checking, right? And thanks for the book.

[Muhammad Alkarouri]

An aside: are they unique across Erlang VM invocations? Can I store an atom and read it later?

[Muhammad Alkarouri]

Thanks. I mixed up the terminology in Lisp. I was thinking of alphanumeric atoms, which are properly symbols in Lisp. While my question was about Erlang symbols, your answer was definitely useful in removing my confusion.

[I GIVE TERRIBLE ADVICE]

@Muhammad Alkarouri: All erlang terms are serializable to a binary format with functions such as term_to_binary(Atom). A serialized atom in Erlang will have a specific tag at the beginning of the binary saying it is indeed an atom, and will then have a textual representation of itself within the binary value. When unpacking the atom (using functions like binary_to_term(Bin)), the VM looks it up into its current atom table. If it's there, it gets the existing unique ID. If it's not there, a new one is attributed. This allows for safe distribution and storage of atoms.

[I GIVE TERRIBLE ADVICE]

I think more interesting than the serialization/deserialization of the atoms is the options list accepted by file:open/2! You don't have to handle a bunch of constants or binary OR them or anything. Just give them as they are or as in a list and it'll work. Want to add an option? simply write the code for it. No need for defines and special cases. Equality testing works it fine.

[YasirA]

I second @I GIVE TERRIBLE ADVICE, and there's a full External Term Format specification. There's also BERT-RPC specification, which is being developed and used in production within the infrastructure of GitHub and play a part in serving nearly every page of the site. I've developed BERT and BERT-RPC client libraries for some Scheme implementations, and terms and atoms in particular are identical on either sides in spite they're being sent over the wire.

[damg]

Muhammad, an atom is a string constant same way as an integer value is. When you see 1 in the code, it simply means 1; if you see 1.3f, then it means 1.3f. Same way an atom foo means foo.

[Damien Pollet]

I was thinking of Smalltalk symbols, where the system has a collection of all instances but ensures it reuses those instead of allocating a new one with the same name. Also that is compatible with garbage collection because the system-wide set of symbols would typically use weak references. // PS. What do you mean by "a sane ID generation system"? (In Smalltalk a Symbol is a kind of String and the ID is its identity, e.g. its pointer)

[YasirA]

I had to emphasize the word unique: after all, we're talking about atoms! :-)

[user2508301]

The same implementation is en Erlang for atoms, more or less.

[Egor Pavlikhin]

In C# strings are also guaranteed to point to the same address if they have identical values.

[dhara tcrails]

@HeavyWave, that is not strictly correct, there is no "guarantee" of string interning. String Interning is possible, but is not required. String that are stored directly in the executable are interned by default, but any time you call the string constructor, you are creating a new instance.

[keymone]

@EgorPavlikhin do those strings point to same address in different processes? or on different network machines? no they don't.

[Dima Fomin]

as written above lispy symbols have property list, so you can put there as many metadata as you with. including "class"

[enigmaticPhysicist]

Oh yeah. I missed that. So, not actually an improvement on lisp. I'll nuke that last part.