113 KiB
Packages in Common Lisp, a tutorial
- Introduction.
- Main sets of objects.
- Packages
- Tracking a symbol.
- Examining a symbol.
- But what the hell is a package?
- Internal symbols.
- Importing a symbol.
- Name conflicts.
- The function
intern. - Moving a symbol between two packages.
- External symbols and used packages.
- First toy examples.
- Importing an external symbol.
- A symbol can shadow another symbol.
- A mathematical definition of the packaging system.
Introduction.
It is not possible to write a serious piece of code without the use of some kind of identifiers or names. Such identifiers are convenient means for the programmer to reference, process and modify data using more or less descriptive names, which represent the data and depend in some way on the context. The standard modularization technology encourages the division of large programs in modules. Such modules can be relatively independent of each other, but conversely, they must be able to communicate. Most programming languages offer methods for such an organization. For example, the set of identifiers of a module could be divided into two parts, the private identifiers, known only inside this module, and the public identifiers, known everywhere.
In this domain as in many others, Common Lisp is by far the most powerful and flexible programming language. At the expense of considerable complexity, Common Lisp's modular technology requires a good grasp of the corresponding definitions in the language standard. If the ANSI definition of the language is the only available documentation, these definitions are not easy to understand. An identifier in Lisp is a symbol which has a rich internal structure and which has lifetime and visibility during a Lisp session. The modular structure of a Lisp program is defined through the packaging system, which allows a programmer to divide a large program into several files, where each file is aware of all the symbols of a certain package, among them the current default package, and only the external (public) symbols of some other packages. A symbol which is not external in a package is internal (private). Also, a package may import (that is, "see") an arbitrary collection of symbols, internal or external to other packages. In the general spirit of Lisp, the whole structure of such an organization, if necessary, can be modified dynamically, even though this is not usually the best course of action.
In this tutorial, a mathematical definition of this system is provided. This should elucidate the process for many a Lisp programmer and enable her to use it more effectively. A basic but inexact description of the packaging system is the following:
- A package is a Lisp object in the same way that numbers, lists, and
character strings are Lisp objects. In particular, the type of a package
object is the system class
package. In Common Lisp, packages are first class citizens, which means they can be the value of a symbol, can be passed as an argument to a function, can be returned by a function, etc. - A symbol is a Lisp object whose type is the system class
symbol. The comments made about packages apply for symbols. In particular, a symbol can be the value of another symbol, or even of itself. - Symbols are divided into packages.
- At any time during a Lisp session, some package is the default package and, as a first approximation, only the symbols of this package are accessible. The default package can be modified at any time by the user or a program.
- If necessary, there are several methods to access the symbols of packages other than the default package. Conversely, a package may make its symbols more or less easily accessible from the other packages.
This first approximation of the packaging system conveys a general idea of its organization, but general ideas can be deceiving. For example, the idea that the symbols are divided into packages is essentially false. More precisely, this division can be made to appear essential (!), via the so-called home package of a symbol, but it plays only a minor role and can be ignored by the programmer risk-free. This notion of a home package is terribly misleading for beginners when they have to design non-trivial packaging structures. In a Lisp environment, one has to deal with two realms, the space of symbols and the space of packages. It is only inside of packages that something can be read, which, as an organization, looks a little like a division of symbols into packages, but the symbols themselves essentially ignore this division. Our subject is to clarify and explain this organization.
The ANSI definition of the packaging system is a complete description of this organization, and, in a sense, nothing is missing. However, experience shows that it is easily misunderstood, especially, in non-trivial situations, where it can lead to erroneous programming choices and mysterious bugs. The ANSI definition of packaging is complete and it is the perfect Reference Manual. However, for a topic of this complexity a reference guide is not sufficient and a User Guide might be useful. And for that matter, an elementary mathematical description of the Common Lisp packaging system might help programmers to understand it, and then to use it with greater ease and also to greater effect. In every domain, when precise mathematical language can be employed and is understood, the state-of-affairs improves.
We begin with a review of the ANSI definition of the packaging system while adding in parallel a mathematical description of the main concepts. Finally, we give a complete mathematical definition of this system as a set of mathematical formulas.
Main sets of objects.
The first part of the packaging system is made of three disjoint sets:
- The set $St$ of strings.
- The set $P$ of packages.
- The set $Sm$ of symbols. In our description of the packaging system, the statuses of these sets are different. Think of the (infinite) set $St$ of strings as made of all the character strings that can be used by a user or a program at any time. Any string can be used at any time, whatever the history of the Lisp session in which we are working may be. In the packaging system, most often, a particular string is used only for a short time, mainly during the read-eval-print loop, when the Lisp reader works. A string is just an intermediary connecting the user and the Lisp session, more precisely the current environment. In contrast, in a given environment, at any time during a Lisp session, some finite set $P$ of specific packages is installed in this environment, and also a finite set $Sm$ of specific symbols is available. Those packages and symbols can then be used, and they are the historical consequence of the current Lisp session. Various Lisp functions allow the user to modify the sets $P$ and $Sm$, and the way they are combined and related to each other. In other words, consider the set of strings $St$ as constant, vast and independent of the environment, while, on the contrary, the sets $P$ and $Sm$ are restricted, they change during a Lisp session, and they are an essential component of the current environment.
Packages
There is a simple, almost bijective (one-to-one), correspondence between
strings and packages, and a convenient place to start our subject. First, at
any time during a Lisp session, a default package is defined and it can be
discovered via the global symbol *package*:
*package*
This is to be read as: the value of the symbol *package* is the
package whose name is the string "COMMON-LISP-USER". The types symbol
and package are system-defined, disjoint sets of machine objects. A symbol
is frequently used as a convenient intermediate object, which allows the
user to reach some object of arbitrary nature, its value. Here,
*package* is a symbol, the value of which is the package named
"COMMON-LISP-USER". Technically, a symbol contains a pointer to its
(possible) value, here a package which happens to be the package used
ordinarily in a simple Lisp session. There are no facilities to refer to a
package object directly. However, one can use its name and the function
find-package to obtain a reference:
(find-package "COMMON-LISP-USER")
Conversely, given a package, the function package-name returns its name:
(package-name (find-package "COMMON-LISP-USER"))
The example was given to illustrate that the functions find-package and
package-name are the inverse of each other, defining a 1-1 correspondence
between the packages currently defined in the environment and their
respective names, some character strings. Because of rules not of interest
here, it is common and convenient to use only uppercase character strings to
name packages. If a name does not correspond to any current package, the
find-package function returns the symbol nil, usually displayed
uppercase:
(find-package "COMMON-LISP-user")
which illustrates that a package name is case-sensitive. Frequently, only a
few packages are in use, and because of their importance, descriptive names
are used. These descriptive names can be a little long, and it is possible
to define and use various nicknames for these names. For example, the
"COMMON-LISP-USER" package usually has the nicknames "CL-USER" and
"USER". The function package-nicknames can be used to retrieve the list
of nicknames (strings) for a package, which doesn't include the name of the
package.
(find-package "COMMON-LISP-USER")(package-nicknames (find-package "COMMON-LISP-USER"))If a package is to be unambiguously identified by its name or one of its nicknames, two different packages must have disjoint sets of name + nicknames.
Tracking a symbol.
The Lisp symbols are very particular when compared to the identifiers of other programming languages. They persist throughout the runtime of an environment and are an important component of the power of Lisp. A package is referenced via its name; the same goes for a symbol, but the process is much more complex. The first and most common method to create or "to allocate a symbol" consists in just writing it down and passing it to the Lisp reader using the standard Lisp conventions. In a Lisp environment, there is a set of "pre-defined" symbols ready to be used. To caution the reader about the complexity of the subject, we begin with a seemingly strange example:
(multiple-value-list (find-symbol "X" "COMMON-LISP-USER"))(multiple-value-list (ignore-errors x))(multiple-value-list (find-symbol "X" "COMMON-LISP-USER"))
The function find-symbol is analogous to the function find-package:
given some name, a character string, it looks for a symbol with this name.
Remember the correspondence between packages and names? A similar
correspondence exists between symbols and names, but it's more involved. In
particular, a name is never sufficient to determine a symbol, a
package is also necessary. This is why the find-symbol function requires
two arguments, the name of the symbol and the package with respect to
which the search is to be done.
At the beginning of our story, no symbol with the name "X" existed in the
default package "COMMON-LISP-USER", and the return value of find-symbol
is negative, nil. (We will explain the second nil value in a moment.)
Then we pass the Lisp reader the one character expression 'x'. Lisp
evaluates it according to its read-eval-print cycle. First, Lisp
reads the expression and notices that the symbol x is used. It examines
the current package "COMMON-LISP-USER", and does not find such a symbol.
Before doing anything else, Lisp allocates (creates) this symbol, the
right terminology being: "Lisp interns a symbol x of name "X" in the
package "COMMON-LISP-USER"." To refer to this symbol, the pair made of
its name "X" and the (not its!!) package "COMMON-LISP-USER" are
necessary, but it would be cumbersome to have to write the expression
(find-symbol "X" "COMMON-LISP-USER") every time this symbol is used. To
simplify the process of referring to a symbol the designers of Lisp have
organized the workspace as follows: if a symbol is read, its character
string is capitalized, giving its name, and the default package is
assumed. In other words, the text 'x' is roughly equivalent to the text
(find-symbol "X" "COMMON-LISP-USER"). How convenient! ('Roughly', because
'x' may trigger the allocation of a symbol. In contrast, find-symbol
never allocates a symbol; but see the function intern later.) In the
second expression of our example, once Lisp has "understood" that the user
intends to use the symbol x, and observing that such a symbol is not
present in the package "COMMON-LISP-USER", it allocates a symbol of name
"X" and interns it in the package "COMMON-LISP-USER". More
specifically, a pointer referring to the just allocated symbol is added to
the list $InS(p)$ of the internal symbols of the package $p$.
Note also that it is common in Lisp to input the symbols in lowercase
letters, and Lisp prints them in uppercase. This might be convenient for
distinguishing input from output in an interactive session, but requires
some clarification. In particular, there is no difference between the
symbols nil and NIL, whereas the strings "nil" and "NIL" are
different.
(multiple-value-list (find-symbol "nil" "COMMON-LISP-USER"))(multiple-value-list (find-symbol "NIL" "COMMON-LISP-USER"))
The fruitless search for a symbol with the lower-case name "nil" (a
string) returns a double negative answer, but it is the second NIL that
is meaningful! Compare that with the second expression which has the same
first return value, but a different second one. The symbol NIL also plays
in Lisp the role of the boolean false, which creates an ambiguity: if the
answer is the symbol NIL is that a negative answer or is it a positive
answer made of the symbol NIL? The ambiguity is resolved by the second
return value. In the first case, the second NIL confirms that the first
NIL is to be interpreted as the boolean false. In the second expression,
the second return value :INHERITED, not a negative, indicates that the
symbol NIL has been found, and that its accessibility via the package
"COMMON-LISP-USER" is inherited, a point to be explained later. In other
words, the function find-symbol returns a genuine symbol and its
accessibility via the package argument, or a pair of nil's if no symbol is
found. The logician can deduce that an accessibility cannot be the symbol
nil!
Let us return to the one character expression 'x'. The read part of the
read-eval-print loop cycle reads the symbol x and interns it in the
package "COMMON-LISP-USER". The expression is read and must now be
evaluated. The evaluation of a symbol consists in looking for a value
of this symbol. To this end a value pointer in the symbol's internal
representation is examined. In this case it is found that, at this time,
there is no value for the freshly allocated symbol x. Therefore the eval
step of the read-eval-print cycle generates an error, terminating the
cycle with an informative error message. In any case, a symbol of name
"X" is now present in the package "COMMON-LISP-USER", as confirmed by
the value of the last expression. The symbol's accessibility via the
"COMMON-LISP-USER" package is internal, which is to be explained later.
Examining a symbol.
A symbol is a Lisp object with a rich internal structure and there are
several functions for exploring symbols and their properties. Let us
examine in detail the symbol x allocated in the previous section.
(symbol-name 'x)
The symbol-name function returns the name of a symbol; it is analogous
to the package-name function for a package. The Lisp function
symbol-name is crucial for our subject, and we denote it as a mathematical
function $sn: Sm \longrightarrow St$, a function which in general is not
injective: several symbols may have the same name, this is our main subject.
Note that in the previous expression the symbol x has been quoted 'x to
prevent its evaluation. More precisely, 'x is an abbreviation for the
expression (quote x), where quote is a special function which does
not evaluate its argument, and it returns this argument as is. Hence, the
previous expression is equivalent to:
(symbol-name (quote x))
Here, the non-quoted x does not generate an error. If instead you do not
quote the symbol x (implicitly via ', or explicitly via quote) it will
be evaluated before being used and generate an error:
(multiple-value-list
(ignore-errors
(symbol-name x)))
The package which owns a symbol, its home package, can be determined
with the function symbol-package.
(symbol-package 'x)(multiple-value-list (find-symbol "X" "COMMON-LISP-USER"))
As we have just seen, a symbol produces a name and a package (with the help
of symbol-name and symbol-package). Conversely, given a name and a
package, find-symbol produces the symbol with this name via this
package. The reader might think that there is a correspondence between
symbols and pairs of strings and packages:
$$Sm \longleftrightarrow St\times P$$
However, the situation is more involved: for the beginner, this might be a
convenient point of view, but in fact such a point of view is wrong. If you
intend to fully understand the complex relationship between strings,
packages and symbols, please forget this tempting but terribly misleading
correspondence. The goal of the next sections is to establish the correct
point of view. For completeness we finish this section with a brief glance
at the other data possibly stored in a symbol. A symbol can have a value.
You may remember that in our Lisp session the symbol x does not (yet) have
a value. This can be established by the predicate boundp:
(boundp 'x)
The main method to give a symbol a value is the setf function (a macro).
In most Common Lisp implementations, the setf macro may be used directly,
in which case the default status of the argument symbol is variable. In
strict ANSI implementations such as SBCL here, a defvar statement is
required to explicitly define this status. Other possible statuses are
constant (via defconstant) and parameter (via defparameter).
(defvar x)(setf x '(4 5 6))(boundp 'x)
Note that the first argument of setf, the symbol x, is not evaluated.
Otherwise an error would be generated, since the symbol x does not yet
have a value. After the (setf ...) expression has been evaluated, the
symbol x does have a value, in this case, the list (4 5 6). This value
would now be the result of the evaluation of the expression x. It can also
be obtained with the function symbol-value. (In this tutorial, we do not
study the possible difference between, on the one hand, the simple
evaluation of the symbol x asked for by the expression 'x' and the
evaluation of (symbol-value 'x) on the other. In our elementary examples
there is no difference between the two. Possible differences might come from
different variable scopes, such as lexical or dynamic, but this subject
is not studied here.)
x(symbol-value 'x)Don't forget to quote the symbol! See the following error:
(multiple-value-list
(ignore-errors
(symbol-value x)))A symbol can also have a functional value. This does not mean that the value we just discussed could be a function (which it could be!). It means that besides the previous value, a symbol can also have an additional value, a function, which can be used independently of the "ordinary" value of this symbol. For example the function $n \mapsto 10 - n$ could be recorded as the functional value of the same symbol, and this function could then be used as shown below.
(defun x (n) (- 10 n))(x 3)Both values coexist in the symbol without any interference:
(symbol-value 'x)(symbol-function 'x)and can be used jointly and independently:
(x (first x))
You might already suspect that the position of x just after a left
parenthesis (x ...) makes Lisp extract the functional value and not the
"ordinary" value. This is correct, but not the subject of this tutorial.
Finally, we mention only in passing the following:
- A symbol can have a functional value while not having an ordinary value.
- An ordinary value can be a functional object, usually then called via
funcallorapply. -
A symbol also contains a property list or
plist, with many possible uses, but not at all studied here.The internal structure of the symbol
xmight be visualized thus:+-----------------+ | Name o-|-----> "x" +-----------------+ | Value o-|-----> (4 5 6) +-----------------+ | Function o-|-----> (lambda (n) (- 10 n)) +-----------------+ | Package o-|-----> #<PACKAGE "COMMON-LISP-USER"> + ----------------+ | Property list o-|-----> NIL +-----------------+
But what the hell is a package?
Now we attack the heart of our subject.\\
Fact 1: A package $p\in P$ maintains four fundamental lists:
- A list $InS(p)$ of its internal symbols.
- A list $ExS(p)$ of its external symbols.
- A list $ShgS(p)$ of its shadowing symbols.
- A list $U(p)$ of its used packages.\\
The first important fact toward understanding the subject is this: the first
three lists are lists of symbols or, more precisely, lists of pointers
(machine addresses) aimed at the symbols. Every symbol has a unique
"existence": a symbol is a unique machine object in the environment, but
several packages may see the same symbol, that is, include the machine
address of this symbol. In particular these lists are not lists of symbol
names (strings). There is another mechanism that allows Lisp to reach a
symbol given its name. Every symbol can be accessed directly via a suitable
package, and several packages can in general play this role for the same
symbol at a given moment in a Lisp session. For example, all the
standard Lisp symbols in the "COMMON-LISP" package typically are
accessible via any package.
No duplicate symbol (more precisely, no duplicate pointer) is to be found in each of these lists. Also the relations $InS(p)\cap ExS(p)=\emptyset$ and $ShgS(p)\subset InS(p)\cup ExS(p)$ must be satisfied, for reasons to be studied later.\\
Fact 2: Given the current set of packages, assumed coherent, and given
the current state of all the fundamental lists of these packages, the
function find-symbol can unambiguously determine whether a symbol of a
given name (some string) is accessible via some package; if so, this
symbol is unique.\\
The importance of the package argument of find-symbol could make the
reader believe the function symbol-package will be useful for various
tests. This is not the case: All this function returns is the "birth place"
of the symbol, technically called its home package, which says almost
nothing about the current status of this symbol with respect to any package.
All it says is that at least it is interned in this package. In
particular, a symbol can be accessible via several packages, possibly
different from its home package. Here is a simple illustration:
(symbol-package 'nil)(multiple-value-list (find-symbol "NIL" "COMMON-LISP"))(multiple-value-list (find-symbol "NIL" "COMMON-LISP-USER"))(eq (find-symbol "NIL" "COMMON-LISP")
(find-symbol "NIL" "COMMON-LISP-USER"))
The home package of nil is "COMMON-LISP", and this symbol of name
"NIL" is accessible via "COMMON-LISP", but also via
"COMMON-LISP-USER", though not with the same status. It is external in
the first case and inherited in the second case, points to be examined
later. The value (T) of the last expression proves that both symbols,
determined by find-symbol via different packages, in fact are the same.
Fact 2 can be made a little more precise: At any given time during a Lisp
session, a set $P$ of packages and a set $Sm$ of symbols are defined. The
various lists of symbols $InS(p)$, $ExS(p)$ and $ShgS(p)$, and the list
$U(p)$ of packages used by a package $p\in P$ together define without any
ambiguity which symbols are accessible via the package $p$. In other words,
there is a well defined function $\rho$:
$$
\rho: P \longrightarrow [St \longrightarrow Sm \cup \{\mathtt{nil}\}]
$$
We chose the letter $\rho$ for reader, because this function is mainly used
by the reader. Our "mathematical" function $\rho$ is nothing but an avatar
of the Lisp function find-symbol. Let $p\in P$ be a package and
$st\in St$ be a string. Then:
$$
\rho(p)(st) = (\mathtt{find-symbol}\; st\; p)
$$
The left-hand side uses the standard mathematical functional notation, and
the right-hand one uses the Lisp notation. If no symbol is found for a pair
$(st, p)$, the symbol (!) nil is returned and the second value of
find-symbol, the symbol nil again, tells the user that the symbol search
did not return anything. A package $p$ is nothing but a way of defining the
function $\rho(p):St\longrightarrow Sm\cup\{\mathtt{nil}\}$. In other words,
the four fundamental lists of a package determine what character strings
appear as the names of the symbols accessible via this package.
This approach is described in detail now.
Internal symbols.
The first fundamental list of a package is the list of its internal symbols. It is a pity that no standard Lisp function can produce this list, but here is a simple solution to the problem:
(defun package-internal-symbols (package)
(let ((rslt))
(do-symbols (s package)
(when (eq (second
(multiple-value-list
(find-symbol (symbol-name s) package)))
:internal)
(push s rslt)))
rslt))We can list now all the internal symbols of a package:
(package-internal-symbols "COMMON-LISP-USER")
In fact, only the first ten internal symbols are displayed. We see in
particular the symbols package-internal-symbols, s and rslt used in
the definition of our function. Some are rather esoteric, strictly speaking
illegal: at the beginning of a Lisp session, the "COMMON-LISP-USER"
package should in principle be void of symbols. To make our experiments more
understandable it is better to allocate and use toy packages. The
defpackage function (a macro) allocates a package of a given name.
(defpackage "P1" (:use "CL"))
A package "P1" is now present in our environment, without any internal
symbols.
(package-internal-symbols "P1")
(The symbol nil also represents the empty list.) Let's allocate a few
symbols and make them internal in the package "P1"! The current default
package is "COMMON-LISP-USER" and we switch it to "P1" like this:
(in-package "P1")*package*
From now on, the default package is "P1". If we write down a symbol then it
will be understood by the Lisp reader as accessed via the package "P1",
that is, unless we use a special syntax. Let us create a list of three
symbols and assign it to the symbol my-list, and then examine the internal
symbols of the package "P1":
(setf (symbol-value 'my-list) '(symb1 symb2 symb3))
The reader probably would prefer (defvar list) followed by
(setf list ...), which is rejected by some Lisp implementations, in
particular SBCL, because in principle the symbol list, owned by the package
"COMMON-LISP", is "locked". We use the symbol my-list instead.
(multiple-value-list
(ignore-errors
(package-internal-symbols "P1")))
Oops! What happened? We've successfully used the function
package-internal-symbols earlier and suddenly it seems to have disappeared
from the environment. The point is that it was defined when
"COMMON-LISP-USER" was the default package. Let us examine the situation!
(multiple-value-list (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "COMMON-LISP-USER"))(multiple-value-list (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1"))
A symbol with the given name is found in both packages. Before encountering
the undefined functional value error, Lisp ( more precisely, the Lisp reader)
had interned a new symbol with the name "PACKAGE-INTERNAL-SYMBOLS" in the
current default package, "P1", for the sole reason that it was present in
the source text. Let us verify that both symbols are different!
(eq (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "COMMON-LISP-USER")
(find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1"))
The symbols are different. Compare that with the status of nil, where the
same symbol nil was accessible via "COMMON-LISP" and via
"COMMON-LISP-USER". In our present situation both symbols are different,
and so are their respective properties. For example, the first symbol does
have a functional value and the second does not, which can be tested with the
function fboundpx:
(fboundp (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "COMMON-LISP-USER"))(fboundp (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1"))
This is the first time we have seen the double-colon ‘::’ notation: When
the symbol package-internal-symbols was searched via "COMMON-LISP-USER",
the answer was positive but the returned symbol was a little lengthy:
COMMON-LISP-USER::PACKAGE-INTERNAL-SYMBOLS. The current default package is
"P1", via which this symbol is not accessible. However, we may access it
using the double-colon notation:
package-name::symbol-name
This is one of the methods which allows the user or Lisp to refer to a symbol
not accessible via the default package. The Lisp reader is told: "We mean the
symbol symbol-name via the package package-name." In contrast, for the
symbol with the same name but interned in the different "P1" package,
because "P1" is the current default package, the double colon notation is
not necessary. The longer notation lets us access a symbol interned in the
"COMMON-LISP-USER" package and its functional value as follows:
(cl-user::package-internal-symbols "P1")
The cl-user:: prefix tells Lisp to look for the symbol via the "CL-USER"
package (Remember that "CL-USER" is a nickname for "COMMON-LISP-USER"?)
and, because of its position just after a left parenthesis, to use its
functional value. Currently, there are four internal symbols in the package
"P1" and they are returned in an arbitrary order. By the way, what about
the symbol list, to which we assigned a list (!) of symbols?
(multiple-value-list (find-symbol "LIST" "P1"))
The symbol is accessible via the package "P1" but with a different status,
the status inherited. We will examine this situation later. In fact, the
symbol list, one of the most important symbols in Lisp, is already present
in the package "COMMON-LISP" at the beginning of a session, and by default
accessible from any package.
We must mention also that our function $\rho$, or the function find-symbol
if you prefer, is (almost) injective with respect to $st$. Let $p_1$ and
$p_2$ be two packages, $st_1$ and $st_2$ two strings. Then if
$\rho(p_1, st_1) = \rho(p_2, st_2)\neq\mathtt{nil}$, then $st_1 = st_2$. Pay
attention to the fact that the equality is an equality between symbols.
This is a consequence of $\mathtt{symbol-name}(\rho(p, st)) = st$; in other
words, there is no possibility of nicknames for a symbol name. In particular,
there is a 1-1 correspondence between the internal symbols of a package and
their respective names.
Importing a symbol.
Instead of using the lengthy notation cl-user::xxx to make an internal
symbol in "CL-USER" accessible from "P1", we may opt to import this
symbol in the package "P1". However, in doing so there is another problem
due to a name conflict, a point which will be examined in detail later:
(multiple-value-list
(ignore-errors
(import 'cl-user::package-internal-symbols "P1")))
The problem is a "collision" between two different symbols, which cannot
coexist as internal symbols in the same package "P1" with the same
name. Importing the symbol internal in "CL-USER" into the package "P1" is
not compatible with the current presence of another symbol in "P1" with
the same symbol-name. We must first unintern the accidental, pre-existing
symbol in "P1" to "make room" for the symbol from "CL-USER".
(unintern 'package-internal-symbols "P1")(import 'cl-user::package-internal-symbols "P1")(package-internal-symbols "P1")The Lisp terminology is rather misleading: in fact, it is not at all the symbol which is imported, but the connection string $\mapsto$ symbol. More precisely, let $p_1$ and $p_2$ be two packages, $st$ a string and $sm$ a symbol. If $\rho(p_1)(st) = sm$ holds before the importation, then, after the importation, both relations $\rho(p_1)(st) = sm$ and $\rho(p_2)(st) = sm$ are satisfied. In particular, the first relation remains valid. It is not really an importation; it is a copy in the package $p_2$ of the connection $st \mapsto sm$, available in the package $p_1$.
Furthermore, nothing is modified about the symbol itself, except possibly, but rarely, its home package, as will be examined later. It is tempting for a beginner to think the home-package of the symbol is being modified. Not at all! Another erroneous interpretation would be to think a copy of the symbol to be imported is made "inside" the target package, with all the corresponding ingredients of the symbol, value, functional value, and so on. Wrong again! A copy is made, but only of a pointer toward our symbol found in $InS(p_1)$ and copied in $InS(p_2)$.
The import function has two arguments, the symbol to be imported and the
target package. The symbol must be well defined, that is, taking account of
the current state of the environment. Here, before the import, the symbol is
accessible only via "CL-USER", and it is mandatory to use the notation
cl-user::... (and the symbol must also be quoted to preclude its
evaluation). After the import, as can be seen in the example, the symbol can
be used functionally from the package "P1" without the prefix notation.
Although the imported symbol is in the list of the internal symbols of the
package "P1", it is not quite the same as before. The next mandatory
exercise consists in comparing the original symbol and the imported one.
(eq 'cl-user::package-internal-symbols
'package-internal-symbols)
The function eq proves that both notations name the same symbol. Here's
another proof:
(eq (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "CL-USER")
(find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1"))
If you remember the symbol-package function, the next test is revealing:
(symbol-package (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "CL-USER"))(symbol-package (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1"))
The unique symbol accessed by two different paths has of course a unique
home package, namely its birth place, the "CL-USER" package. Please, do
not pay too much attention to this symbol package; it is just an artifact of
the session's history and does not have any real interest. You must focus on
the chain
$\mathit{package}\rightarrow[\mathit{string}\rightarrow \mathit{symbol}]$ and
not much else. There is no real reason to be concerned about the map
symbol-package: $\mathit{symbol}\rightarrow\mathit{package}$.
Name conflicts.
If poorly understood, the introduction of Lisp packages to a Lisp project will sooner or later lead to name conflicts. There is nothing mysterious about them and there are standard "home remedies" for dealing with them.\\
Warning. A name conflict can only be generated if two different symbols
$sm$, $sm' \in Sm$ end up with the same name: $sn(sm) = sn(sm')$. A name
conflict occurs if these symbols $sm$ and $sm'$ become accessible via the
same package $p$, which is a forbidden situation, since it would introduce
ambiguity to the find-symbol function. On the other hand, the fact that
some symbol (singular) is accessible via different packages never generates
a name conflict by itself.
Lisp constantly is on the lookout for possible name conflicts and stops with
an error if the evaluation of an expression causes a name conflict. Some
errors are continuable, which means that Lisp is able to propose a conflict
resolution via unintern or shadowing-import. Remember the use of
unintern in the previous section?
What happens if an imported symbol has the same name as a symbol already
accessible in the target package? In mathematical language, let $p_1$ and
$p_2$ be two different packages, $st$ some string, and $sm_1$ and $sm_2$ two
different symbols, internal in the respective packages $p_1$ and $p_2$, with
same name $st$. In other words, assume $\rho(p_1)(st) = sm_1$ and
$\rho(p_2)(st) = sm_2$, $sm_1 \in InS(p_1)$ and $sm_2 \in InS(p_2)$, and
$sm_1\neq sm_2$. A situation of this sort could be the following (Remember
the current default package is "P1"!):
(defvar cl-user::symb4)(setf cl-user::symb4 111)(defvar symb4)(setf symb4 111)(eq 'cl-user::symb4 'symb4)
The evaluation of the first expression interns a symbol of name "SYMB4" in
the package "CL-USER" and gives it the value 111, while the second
expression interns a different symbol, again of name "SYMB4", in the
package "P1", the current default package, and also gives it the value 111.
As a result two symbols have been allocated, installed somewhere in the
environment, with the same value of 111. Also the connections
$\rho(\mathtt{"CL-USER"}): \mathtt{"SYMB4"}\mapsto \mathtt{cl-user::symb4}$
and $\rho(\mathtt{"P1"}): \mathtt{"SYMB4"}\mapsto\mathtt{p1::symb4}$ have
been installed, that is, the addresses of both symbols have been pushed to
the respective lists of internal symbols of the packages "CL-USER" and
"P1". The eq comparison confirms that the symbols just allocated are
different. Two symbols with the same name "SYMB4" are now present in our
environment. This is possible because these symbols are interned in different
packages. Each one is accessible via the appropriate package. Both symbols
are live and can be freely used:
(setf cl-user::symb4 (+ symb4 cl-user::symb4))
What happens if we import cl-user::symb4 into the package "P1"? Let's
try!
(multiple-value-list
(ignore-errors
(import 'cl-user::symb4 "P1")))
The importation is rejected, for it would not be possible anymore to
coherently define $\rho(\mathtt{"P1"})(\mathtt{"SYMB4"})$. We will see later
that another process called shadowing allows the user to suppress the
symbol symb4 already present in "P1" and to authorize the importation of
the symbol present in "CL-USER".
The function intern.
Fact 3: Let $p$ be a package and $(sm_1,...,sm_n)$ the list of its
internal symbols. If $p$ is the default package, any of these symbols can be
directly used without the double colon notation '::'. If $p$ is not the
default package, these symbols can be used thanks to the notation $p::sm_i$.
All these internal symbols have different names. Nothing prevents several
packages from having the same symbols in their lists of internal symbols.
This is possible thanks to the import function which can work only if no
name conflict is generated.\\
To be complete on this matter, we now to consider the Lisp function
intern. It is used almost exactly as the function find-symbol with just a
difference: if no symbol is found, then a symbol is allocated and interned in
the package referred to in its second argument. Compare the following:
(multiple-value-list (find-symbol "SYMB5" "P1"))(multiple-value-list (intern "SYMB5" "P1"))(multiple-value-list (intern "SYMB5" "P1"))
We start without any symb5 symbol in "P1", which is confirmed by the
double negative answer of find-symbol. If instead we use the function
intern, Lisp notices that such a symbol is absent from "P1" and therefore
allocates a symbol of name "SYMB5" and interns it in the package "P1"
as an internal symbol. The second value nil is an indicator of the
allocation just made. If we repeat the same expression, which is not an
error, then Lisp signals that such a symbol is already present and returns
its status, which also informs us of its previous presence. A little weird!
This definition of the function intern guarantees the coherence of the set
of internal symbols of packages. A symbol can be allocated only through this
function intern. It requires two arguments, a string $st$ and a package
$p$. In a sense, the function intern first calls (find-symbol st p) with
the same arguments. If a symbol is found it is returned without modification
and no allocation of a new symbol takes place, which otherwise would create
a name conflict. On the other hand, if a corresponding symbol is not found,
it is allocated and interned in the package explicitly or implicitly quoted,
and this new symbol certainly does not generate a name conflict. In this way,
the uniqueness of the names of internal symbols of a package is enforced.
This does not prevent two different symbols of the environment from having
the same name, but the packaging system is organized such that they cannot be
accessed via the same package.
(package-internal-symbols "P1")
In practice, the Lisp reader constantly uses the intern function, for
example when reading an expression. If it detects the presence of a symbol,
the colon notation allows it to decide if a package is explicitly given,
otherwise it must use the default package. The name of the symbol, alone or
package-qualified using the colon notation, then unambiguously determines the
symbol. If present, it is used, otherwise it is allocated and then used. This
is exactly the role of the intern function.
Moving a symbol between two packages.
Up until now, we have seen no means of modifying the home package of a
symbol. In fact, it is possible but may seem a little contorted. It is almost
without any practical interest except as an opportunity to use this as a
vehicle to discover other facts. First, it is possible to unintern a
symbol. For example, we can unintern the symbol "SYMB1" from the package
"P1". What happens is precisely that this symbol is removed from the list
of the internal symbols of "P1".
(unintern 'symb1 "P1")(package-internal-symbols "P1")
What is modified is the list of internal symbols in the package "P1". The
symbol symb1 itself has not been modified at all, except possibly its home
package. In particular the symbol is not deallocated or destroyed, and
therefore always present somewhere in the environment.
But it's become inaccessible as such, for a symbol most often is accessible
only via a package, the default one or some other explicitly given. The
symbol symb1 was interned only in the package "P1", so that it appears
now to be unreachable! If a symbol is truly unreachable, the Lisp garbage
collector, always silently working in the background, will establish that
fact and free the corresponding part of memory for reuse. Except that our
symbol remains reachable, for it was the first element of the list assigned
to the symbol my-list. Let's have a look at its value!
my-listOur symbol is still present in the list but in a little different form. To be sure, it is still a symbol:
(type-of (first my-list))(symbol-package (first my-list))
No home package, i.e., our symbol is "homeless"! The prefix '#:' indicates
that this symbol is homeless. The unintern function had this effect because
the package argument of unintern was the package which owned the symbol.
Otherwise the home package would not be modified. Although our symbol has
become homeless, it continues to be an ordinary symbol, for example, we can
assign it a value:
(set (first my-list) 444)(symbol-value (first my-list))(eval (first my-list))
Unlike setf (macro), the set function evaluates its arguments, so that
before the assignment, the expression (first my-list) is evaluated, which
returns our mysterious symbol #:symb1. Once that's done, the set function
assigns (the value of) the second argument to it, as is confirmed by the
other expressions. It is even possible to import a homeless symbol in a
package, but then the target package becomes the home package of the imported
symbol. The importing package can be different from the original home
package. As a result our symbol is "moved" into another package. This
explains the new form of the value of my-list.
(import (first my-list) "CL-USER")(symbol-package 'cl-user::symb1)my-listOur symbol could also have been imported in other packages before being uninterned from its home package. The status of our symbol with respect to other packages would not have been modified.
(import 'symb3 "CL-USER")(eq 'cl-user::symb3 'symb3)(unintern 'symb3 "P1")(eq 'cl-user::symb3 (third my-list))(import 'cl-user::symb3 "CL-USER")(symbol-package 'cl-user::symb3)
Here we import symb3 in the package "CL-USER" before uninterning it from
the package "P1". The symbol is then accessible via the package "CL-USER"
or via the list my-list. The evaluation of the expression (eq ...) proves
that both paths lead to the same address. When the first (import ...)
expression is evaluated, the symbol does have a home package and that it is
not changed. When it is uninterned from the package "P1", it becomes
homeless. Lisp does not take account of the fact it is internal in other
packages. In theory, this would allow it to select one of these packages as
an "emergency home" package, but the choice would be quite arbitrary?
Furthermore this would force Lisp to examine the list of all other packages
and all their accessible symbols, which would be very expensive! Here,
despite the symbol being internal and therefore interned in the package
"CL-USER", it becomes homeless after evaluating the (unintern ...)
expression. Finally we import it in the package "CL-USER", where, in fact,
it is already present. According to the ANSI standard (see the following
note), in such a situation, its status remains unchanged. In particular its
home package remains nil. A little esoteric, huh? Don’t worry: the home
package does not have any real role for the Lisp user.
Note: The passage in the Common Lisp HyperSpec reads like this: "If the
symbol is already present in the importing package, import has no effect."
It appears that the behavior of SBCL and some other compilers is different:
The output is #<PACKAGE "COMMON-LISP-USER"> not NIL.
We will prove later that if a symbol $sm$ has a home package $p$, then $sm$ is certainly present in $p$, that is, internal or external in this package. The "converse" is false: a symbol may be present in several packages $p_1,\ldots , p_k$ while not having a home package. This might sound a little weird, but you shouldn't lose much sleep over it: except for the internal efficiency of the Lisp interpeter or compiler, the notion of a home-package is almost useless for the programmer.
External symbols and used packages.
The analysis of this notion of an internal symbol was a little lengthy, but, in this way, the general structure of the packaging system is now very clear, and it will be a lot easier to understand the other concepts. Two fundamental lists maintained in the packages play a "dual" role in the packaging system, the list of external symbols and the list of used packages. The following function gives the list of the external symbols of a package.
*package*(defun package-external-symbols (package)
(let ((rslt nil))
(do-external-symbols (s package)
(push s rslt))
rslt))
The evaluation of the first expression is just to confirm that the default
package remains "P1". Let us see what the lists of external symbols of the
packages "P1", "CL-USER" and "COMMON-LISP" are:
(package-external-symbols "P1")(package-external-symbols "CL-USER")(subseq (package-external-symbols "COMMON-LISP") 0 10)(length (package-external-symbols "COMMON-LISP"))
There are no external symbols in "P1" and "CL-USER", but 978 such symbols
in the package "COMMON-LISP". (Only the first ten are being displayed.) A
point of concern might be the following: these symbols are accessible via the
package "COMMON-LISP" and yet the double-colon notation
common-lisp::simple-error was not used. The explanation has two "dual"
parts:
- The symbol
simple-erroris external in the package"COMMON-LISP". - The package
"COMMON-LISP"is in thepackage-use-listof the package"P1":
(multiple-value-list (find-symbol "SIMPLE-ERROR" "COMMON-LISP"))(package-use-list "P1")(package-nicknames (find-package "COMMON-LISP"))
"CL" is the nickname of the package whose official name is "COMMON-LISP",
and which is not to be confused with "COMMON-LISP-USER".
(eq (find-package "CL")
(find-package "COMMON-LISP"))(eq (print (find-package "CL"))
(print (find-package "CL-USER")))
Applied to a package $p$, the Lisp function package-use-list returns the
list of all the packages used by $p$. If $p_1$ uses $p_2$, then it is said
that $p_2$ is used by $p_1$. Whether this relation is reflexive can be
doubted (What does "$p$ uses $p$" mean?), and it is neither symmetric
("$p_1$ uses $p_2$" does not imply "$p_2$ uses $p_1$") nor transitive
("$p_1$ uses $p_2$" and "$p_2$ uses $p_3$” does not imply
"$p_1$ uses $p_3$"). The oriented graph of the use relation is totally
arbitrary.
First toy examples.
Let's start over with two "fresh" packages "P1" and "P2".
(in-package "CL-USER")(delete-package (find-package "P1"))(defpackage "P1" (:use "CL"))(defpackage "P2" (:use "CL"))
We switch the default package to "P1":
(in-package "P1")
and we assume that, in this environment, we define the functions
package-internal-symbols and package-external-symbols as explained
before.
(defun package-internal-symbols (package)
(let ((rslt))
(do-symbols (s package)
(when (eq (second
(multiple-value-list
(find-symbol (symbol-name s) package)))
:internal)
(push s rslt)))
rslt))(defun package-external-symbols (package)
(let ((rslt))
(do-external-symbols (s package)
(push s rslt))
rslt))
What about the internal symbols of "P1"?
(package-internal-symbols "P1")
We see the symbols naming our functions and also the symbols rslt and s
used to define them. All the other symbols used in their definitions are in
fact external in "COMMON-LISP" and absent here. We move now to the package
"P2" from where we cannot access the symbol package-internal-symbols
directly.
(in-package "P2")(multiple-value-list (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P2"))
We have already seen the use of the '::' notation to make reference to the
symbol at once. We can also import the symbol, in which case it becomes
directly accessible from "P2" permanently:
(import 'p1::package-internal-symbols "P2")(package-internal-symbols "P2")Another solution requires two steps:
- Make the symbol
externalin some package, which is the role of the Lisp functionexport. - Declare that the current package uses the package where the symbol is external.
Let us use this method to make the symbol package-external-symbols directly
visible from "P2".
(export 'p1::package-external-symbols "P1")(use-package "P1" "P2")(package-external-symbols "P2")(package-external-symbols "P1")
An export expression has the same form as an import one:
(export $sm$ $st$ ), where $sm$ is a symbol and $st$ is a string naming a
package. The symbol $sm$ must be readable from the current default package,
for us "P2", which explains the (required) prefix 'p1::'. The argument
$st$ is a string naming the package where the symbol $sm$ is to become
external. Again, the terminology is somewhat misleading: the function
export does not export anything. All it does is to make some connection
$st \rightarrow sm$ "exportable" to other packages via use-package
expressions. The 978 external symbols of "COMMON-LISP" contain all the
standard Lisp constants, variables and functions. Because an allocated
package is always initialized with the package "COMMON-LISP" in its
package-use-list, these basic Lisp objects will be accessible from any
package. If we remove "COMMON-LISP" from the use-package-list of the
current default package, via unuse-package, the situation becomes a little
tricky.
Note: Before setting the saw to the branch on which we are sitting,
we have to take one precaution: We can't let go of LET. Otherwise the
interaction between Emacs and SLIME will cease to function.
(cl::import 'cl::let "P2")(unuse-package "COMMON-LISP" "P2")(cl::multiple-value-list
(cl::ignore-errors
(+ 2 2)))
The notation for the external symbol t in the package "COMMON-LISP" has
become COMMON-LISP:T, for "COMMON-LISP" is no longer used by "P2".
Since the symbol T is external in "COMMON-LISP", the simple colon
notation is sufficient. However, the most elementary operations are not
easily accessible anymore, for example, the addition function '+'. This is
not a good position to be in. Let us go back to the previous environment!
(cl::multiple-value-list
(cl::ignore-errors
(use-package "COMMON-LISP" "P2")))
This does not work since the symbol use-package can't be reached easily
either, and we must use the prefix notation.
(cl::multiple-value-list
(cl::ignore-errors
(cl::use-package "COMMON-LISP" "P2")))
But here is a new problem: earlier, when when we tried to use the symbol
'+' from "P2", Lisp detected that this symbol is not accessible from
"P2" and therefore interned a fresh symbol of name "+" in "P2". This
was legitimate then, but it is now incompatible with "COMMON-LISP" having a
different symbol with the same name "+", if "P2" were to use
"COMMON-LISP". The same goes for use-package. We must unintern these
symbols from "P2" to resolve the conflict. Let's not forget to use the
colon notation for unintern, which is otherwise unreachable!
(cl::unintern '+ "P2")(cl::unintern 'use-package "P2")Note: The following is necessary to keep the interaction between Emacs and SLIME going. Otherwise, this would be another conflicting symbol.
(cl::unintern '*default-pathname-defaults* "P2")(cl::use-package "COMMON-LISP" "P2")Phew, we are back in a state where Lisp is usable!
Importing an external symbol.
Recall that a symbol $sm$ is accessible from a package $p$ if
$\rho(p)(st) = sm$ for $st = \mathtt{symbol-name}(sm)$. In that case, the
possible return values from find-symbol are as follows:
- If $sm$ is one of the internal symbols of $p$ the second value returned by
find-symbolis:internal. - If $sm$ is one of the external symbols of $p$ the second value returned by
find-symbolis:external. - If $sm$ is an external symbol of a package $p'$ used by the package $p$
the second value returned by
find-symbolis:inherited.
We have seen how to import a symbol from a package $p'$ into a package $p$. The example given used a symbol $sm$ internal in the package $p'$, which then became also an internal symbol of the package $p$ . This also works if $sm$ is external in the package $p'$, regardless of $p$ using or not using $p'$. (… assuming that no name conflict is generated.)
Let's continue our journey in "P1"!
(in-package "P1")(export (intern "SYMB1" "P2") "P2")(multiple-value-list (find-symbol "SYMB1" "P2"))(multiple-value-list (find-symbol "SYMB1" "P1"))
We intern a symbol p2::symb1 in "P2" and make it external immediately. As
it stands this symbol is not accessible via "P1". The fact that it is
external in "P2" does not prevent us from importing it in "P1".
(import 'p2:symb1 "P1")(multiple-value-list (find-symbol "SYMB1" "P1"))(multiple-value-list (find-symbol "SYMB1" "P2"))
Once the importation is done, the symbol is accessible via "P1" with the
status internal. The external status with respect to "P2" is unchanged,
which underlines that the status of a symbol does not depend on anything in
the symbol itself. It is the package which must be examined to determine
the status of a symbol with respect to this package.
Curiously, the last find-symbol returns the symbol symb1, without the
prefix 'p2:'. Lisp prints a symbol according to the following rule:
First, Lisp determines whether a symbol is accessible via the current default package, in which case it is printed without using the colon notation. Otherwise, Lisp examines the home package, where the symbol is certainly internal or external, and it is printed with the double or simple colon notation. The only exception to this rule appears to be a keyword, that is, a symbol of the package
"KEYWORD". (Though the ANSI definition is somewhat incomplete for the"KEYWORD"package, we can assume that it is forbidden to change anything about the scope of keywords.) Keywords are printed simply as':xxx', without any prefix. Finally, if the symbol is homeless, it is printed as#:xxx.
The take-home message here is that an importation does not change anything for the relevant symbol; it is just a matter of updating a list of internal symbols in some package, that is, a list of respective pointers toward these symbols. Installing a new signpost in San Francisco that reads "New York" does not change anything in New York. Once more:
(eq (find-symbol "SYMB1" "P1")
(find-symbol "SYMB1" "P2"))
A novice could believe that adding "P2" to the package-use-list of "P1"
could generate a name conflict, because the symbols p1::symb1 and
p2::symb1 would become both accessible via "P1". This is not the case:
both notations represent one and the same symbol, and there is no ambiguity.
(use-package "P2" "P1")(multiple-value-list (find-symbol "SYMB1" "P1"))(multiple-value-list (find-symbol "SYMB1" "P2"))
Note that because our symbol is internal in "P1", the function
find-symbol reports its status as :internal. This is in a sense preferred
to the status :inherited, which after all would be possible. A name
conflict occurs only if two different symbols with the same name could be
accessible via the same package, possibly via another package used by the
first one. Two symmetrical illustrations:
(export (intern "SYMB2" "P2") "P2")(multiple-value-list (intern "SYMB2" "P1") )
This is not an error: The second intern does not allocate a new symbol, a
symbol of name "SYMB2" being already accessible via "P1", which uses
"P2". In particular the second intern does not import the pre-existing
symbol.
We have to come up with something else to really allocate a new symbol.
(multiple-value-list
(ignore-errors
(import (make-symbol "SYMB2") "P1")))
The function make-symbol allocates a homeless symbol of a given name,
that's its definition. A homeless symbol cannot cause a name conflict,
because it is not accessible via any package. Nothing prevents us from
trying to import this symbol in "P1", but this generates a name conflict
between two different symbols: the freshly allocated #:symb2 and the symbol
external in "P2" accessible via "P1" which uses "P2".
Here's another way to cause a name conflict:
(multiple-value-list (intern "SYMB3" "P1"))(multiple-value-list (intern "SYMB3" "P2"))(multiple-value-list
(ignore-errors
(export 'p2::symb3 "P2")))
The package "P1" uses the package "P2", but this does not prevent them
from having two different internal symbols with the same name "SYMB3", for
"P1" does not see the symbol internal in "P2". If we try to make this
symbol external in "P2", a name conflict arises.
Another sort of name conflict can occur between two used packages:
(export (intern "SYMB4" "P2") "P2")(export (intern "SYMB4" (defpackage :p3)) "P3")(multiple-value-list
(ignore-errors
(use-package "P3" "P1")))Compare that with:
(export (intern "SYMB5" "P2") "P2")(export (import 'p2:symb5 "P3"))(unexport 'p3:symb4 "P3")(use-package "P3" "P1")
Two symbols of name "SYMB5" are external in "P2" and "P3", but the
second one is in fact the same as the first one: it has been imported and
immediately made external in "P3".
This time the use-package, which makes the external symbols of "P2" and
"P3" visible from "P1", does not generate a name conflict, since we are
dealing with two different paths which lead to the same symbol. This works
as long as we do not forget to first unexport the symbol "SYMB4" of
"P3", i.e., to downgrade its status from external to internal.
A symbol can shadow another symbol.
In the matter of name conflicts, the Lisp creators could have adopted another
strategy and decided that the symbols present in a package, that is,
internal or external in this package, have precedence over inherited,
possibly different, symbols. However, they reasoned that this could easily
generate true programming errors because of possible perverse collisions of
identifiers, and turn into a source of obscure bugs. Nevertheless, in line
with the general spirit of Lisp, if a programmer really wants to use such a
strategy, she can do it on a case-by-case basis and explicitly describe the
intended exemptions. This is the role of the package-shadowing-symbols
list, which is maintained in every package $p$ and denoted by $ShgS(p)$.
This list is empty initially. $ShgS(p)$ can contain symbols present in the
concerned package, internal or external. These symbols will automatically
shadow any other competing symbol, such as symbols exported in other used
packages and with the same symbol-name.
The function shadow takes a symbol name (or a list of symbol names) and a
package name, allocates a symbol with this name if it is not present in
this package, and records it in the shadowing list of the package.
(shadow "SYMB6" "P1")(package-shadowing-symbols "P1")(export (intern "SYMB6" "P2") "P2")(eq 'symb6 'p2:symb6)
Once the symbol p1::symb6 is on the shadowing-list of "P1" the existence
of a different external symbol with the same name in the used package "P2"
does not generate a name conflict. The last eq-test proves both symbols are
really different. Along the same lines, the function shadowing-import, as
clearly indicated in its name, imports a symbol and immediately adds it to
the shadowing list, which could resolve a possible name conflict. For example
the different symbols p2:symb4 (external in "P2") and p3::symb4
(internal in "P3") are in our environment; the package "P1" uses both
packages "P2" and "P3", but p3::symb4 is internal in "P3" and
therefore not in conflict with p2::symb4. Remember also the current default
package is "P1". A simple importation of p3::symb4 in "P1" generates a
conflict, while a shadowing-import adds the imported symbol to the
shadowing list, shadowing the symbol p2:symb4 and resolving the name
conflict.
(multiple-value-list (find-symbol "SYMB4" "P2"))(multiple-value-list (find-symbol "SYMB4" "P3"))(multiple-value-list
(ignore-errors
(import 'p3::symb4 "P1")))(shadowing-import 'p3::symb4 "P1")(package-shadowing-symbols "P1")
If necessary, p2:symb4, shadowed by the internal symb4 in "P1", remains
accessible via the colon notation. The evaluations of the following
expressions assign the same symbol value to both symbols as is confirmed by
the subsequent '' form. Evaluating the next =eq expression proves that
both symbols are different. In contrast, because p1::symb4 was imported
from "P3", the last eq-test is affirmative.
(defvar symb4)(defvar p2:symb4)(setf symb4 (setf p2:symb4 111))(= symb4 p2:symb4)(eq 'symb4 'p2:symb4)(eq 'symb4 'p3::symb4)
It is even possible to add an inherited symbol to the shadowing list, maybe
to declare a precedence with respect to another future competing symbol in
another package intended to be used later. This experiment assumes three
different packages, where one of them is not yet used. We first "detach" for
example the package "P3" from "P1", leaving "P2" used by "P1".
(Remember "P1" is the current default package.)
(unuse-package "P3" "P1")
We then install two new external symbols with the same name in "P2" and
"P3":
(export (intern "SYMB7" "P2") "P2")(export (intern "SYMB7" "P3") "P3")
As we have seen already with essentially the same example, if "P1" now
decides to use "P3", the two "SYMB7" symbols collide:
(multiple-value-list
(ignore-errors
(use-package "P3" "P1")))
In some context, it could happen that "P1" has no reason to use p3:symb7
directly, and instead would like to give precedence to p2:symb7. This can
be achieved with a shadowing import.
(multiple-value-list (find-symbol "SYMB7" "P1") )(shadowing-import 'symb7 "P1")(use-package "P3" "P1")
The shadowing-import refers to p2:symb7 implicitly, which is external in
a package used by "P1" and therefore inherited by "P1". We see how the
use of shadowing-import solves the name conflict, which otherwise would be
raised by use-package. Let us review our symbols as seen now from our three
packages:
(multiple-value-list (find-symbol "SYMB7" "P1"))(multiple-value-list (find-symbol "SYMB7" "P2"))(eq 'symb7 'p2:symb7)(multiple-value-list (find-symbol "SYMB7" "P3"))(package-use-list "P1")
The statuses of symb7 have not changed in "P2" and "P3", but seen from
"P1", the symbol is now internal. This is to satisfy the rule that any
shadowing symbol in a package must be internal or external in this package,
and also a consequence of the implied importation.
A mathematical definition of the packaging system.
We recall that there are three general sets we must study:
- The set $St$ of strings
- The set $P$ of packages
- The set $Sm$ of symbols
There is one important difference: the set $St$ is the infinite set of all the possible strings, while $P$ and $Sm$ are finite sets made of the packages and symbols previously allocated in the current Lisp session.
We use the notations $pn$ to denote the function package-name
$pn: P \rightarrow St$ and $sn$ to denote the function symbol-name
$sn: Sm\rightarrow St$.
The map $pn$ is injective for the packages; the function $pn: P \rightarrow pn(P)$ defines a 1-1 correspondence between the packages and their respective names. Here, we ignore the potential nicknames of a package, which is not essential for our discussion.
Every package $P$ contains four fundamental lists:
- The list $InS(P)$ of its internal symbols
- The list $ExS(P)$ of its external symbols
- The list $ShgS(P)$ of its shadowing symbols
- The list $U(P)$ of its used packages
More precisely, these lists are lists of pointers to the mentioned objects, or, if you prefer, of the machine addresses of these objects. It is important to understand these lists are made of (pointers to) symbols, not of their names. Each of these lists is without any duplicates.
The fundamental lists defined in a package are the source of a relatively extensive terminology, which is detailed now. The set of symbols present in a package $p$, denoted by $PrS(p)$, is defined as the union $PrS(p) := InS(p) \cup ExS(p)$: a symbol is present in a package if it is internal or external in this package. This union is disjoint, the intersection $InS(p) \cap ExS(p)$ must be empty. Also the relation $ShgS(P) \subset Pr(P)$ is required. A shadowing symbol of the package $p$ must be internal or external in this package, i.e., a shadowing symbol is present.
Let $q$ be a package used by the package $p$. The symbols of $q$ shadowed by a symbol of $p$ are: $$ ShdS(q, p) = \{sm \in ExS(q)\;\mathbf{\underline{st}}\;\exists sm' \in ShgS(p)\;\mathbf{\underline{st}}\; sn(sm) = sn(sm')\} $$
In other words, a precedence relation is defined by the list $ShgS(p)$: if ever a symbol $sm' \in ShgS(p)$ and a symbol $sm\in ExS(q)$ have the same name, then, with respect to the package $p$, $sm'$ takes precedence over $sm$, which means $sm$ is shadowed. If $sm'$ is present in $p$ but not shadowing, with $sm'\neq sm$ and $sn(sm') = sn(sm)$, then a name conflict would be raised, which is forbidden.
The inherited symbols of a package $p$, denoted by $InhS(p)$, are defined as: $$ InhS(p) := \Big\{ \bigcup_{q\in U(P)}(ExS(q) − ShdS(q,p)) \Big\} − PrS(P) $$
A symbol is inherited in a package $p$ if it is external in a used package $q$, not shadowed by a shadowing symbol of $p$, and not present in $p$. The union in the preceding formula is not necessarily disjoint: a symbol can be external in several packages simultaneously, and without causing a name conflict. A symbol could be simultaneously present in $p$ and external in a package $q$ used by $p$. That is why the the last term, $−PrS(P)$, is necessary to define unambiguously what an inherited symbol is.
Finally the accessible symbols $AccS(p) := PrS(p) \cup InhS(p)$ are the
symbols which are present or inherited. The very definition of $InhS(p)$
implies that this union is disjoint. In this way, the status of a symbol
with respect to a package $p$ via which it is accessible is unambiguously
defined as a keyword :internal, :external or :inherited. For a package
$q\neq p$ there is no relation whatsoever between the status of some symbol
$sm$ with respect to $p$ and its status with respect to $q$.
Exercise: Show that the sixteen pairs of possible status combinations can
be realized!
It is important to be clear about possible intersections (overlaps) between the lists defined above. Let $p$ be some package and $q$ another package used by $p.$ If $sm\in ExS(q)$ is an external symbol of $q$, the symbol $sm$ may also be present in $p$, in which case its status is internal or external, but not inherited. In particular, this does not generate a name conflict, because it is the same symbol which is visible in two different packages via the same name, and a symbol has only one name! Nevertheless its status with respect to $p$ is not inherited. The status of a symbol is so defined deterministically, even if it is present in two interacting packages by one using the other. Of course the same symbol has status external in $q$.
Following the same logic, if $q,q'\in U(p)$, the intersection $ExS(q)\cap ExS(q')$ is not necessarily empty.
The Lisp system maintains the coherence of these lists, which is defined as follows:
Definition. A set of packages is coherent if, for every package $p$, the
function symbol-name defines a 1-1 correspondence
$sn: AccS(p) \rightarrow sn(AccS(p))$.
This map is by definition surjective; it is injective if two different
symbols accessible via $p$ have different names. In this way the function
find-symbol is free of ambiguity. It searches first in the present symbols,
and finally in the external symbols of the used packages. It is not necessary
to compute the list of shadowed symbols since a shadow/ing/ symbol, which is
also a present symbol, is found first. The main purpose of the coherence rule
is to prevent users from designing too contorted situations in name scopes,
wich are frequently the source of subtle and difficult to find bugs.
Several symbols with the same name may coexist in an environment. The most common way to access a symbol consists in using its name, but different symbols may have the same name. In fact, its symbol name is not sufficient to access a symbol, a package must also be specified. This can be the default package, in which case the direct notation without any colons is possible, or the package name can be explicitly used to prefix the symbol name using the one- or two-colon notation. Of course, the symbol so accessed must be defined in a deterministic manner. The definition just given says that a unique symbol of some name is accessible from some package.
The main function in the packaging organization is a map:
$$
\rho:P \rightarrow [St \rightarrow ((Sm\cup\mathtt{nil})\times Stt)]
$$
where $Stt$ is a constant set of four symbols, the Lisp symbol nil, and
three keywords, the possible statuses of an accessible symbol:
$$
Stt = \{\mathtt{nil}, \mathtt{:internal}, \mathtt{:external}, \mathtt{:inherited}\}
$$
Because of the assumed coherence of the package configuration, the function
$\rho$, an avatar of the function find-symbol of a package $p$ and a string
$st$, determines a unique symbol named $st$ and accessible via $p$, and
also its status. It is possible that the search fails, in which case the
function returns the pair (nil,nil). (More precisely (values nil nil).)
The second nil is a kind of a null status: "There is nothing accessible via
$p$ under the name $st$." The relation
$\rho(p)(s) = \mathtt{(find-symbol}\;p\;st\mathtt{)}$ is always satisfied.
It is now relatively simple to understand the various Lisp functions for handling packages and symbols, in particular what name conflicts might be generated.
(intern $st\;p$)
This function first evaluates (find-symbol $st\; p$=)=. If a symbol is
found, both values of (find-symbol ...) are returned.
If no symbol is found, then Lisp allocates (creates in memory) a new
symbol without any value or function value. The symbol-package is
initialized to $p$ and this symbol is added to $InS(p)$. In this case,
(values $sm$ nil) is returned, the second value nil being a code for
saying a new symbol has just been allocated.
No name conflict can be created this way.
This function is used implicitly by the Lisp reader when it encounters a
symbol in an expression. If the colon syntax is not used, the default
package is used for $p.$ Otherwise the (capitalized) package name of $p$
preceding the colon(s) is read, followed by the (capitalized) symbol-name
$st$ after the colon(s). Finally, (intern $st\; p$=)= is evaluated, which
yields the symbol so read by the Lisp reader. In this way, the symbol can
be either a pre-existing one or one freshly allocated as a consequence of
this reading.
(unintern $sm\;p$)
A symbol $sm$ is interned in a package $p$ if it is present in this package. It is present in package $p$ if $sm\in PrS(p) = InS(p) \cup ExS(p).$ Uninterning a symbol asks for this membership to be "revoked."
If $sm\notin PrS(p)$, nothing is changed and nil is returned. Otherwise
$sm$ is removed from $InS(p)$ or $ExS(p)$, and also from $Shg(p)$ if $sm$
was shadowing with respect to $p$; in this case, unintern returns t. In
the last case, when uninterning a shadowing symbol, a name conflict can be
generated: it is possible for symbols $sm'\in ExS(q)$ and $sm''\in ExS(q')$
to have the same name while being different symbols in the respective
packages $q$ and $q'$ used by $p$, and that this collision was resolved by
the symbol $sm$ shadowing both. If this happens, the unintern request is
rejected.
Also, if a symbol $sm$ is uninterned from a package $p$ and $p$ is the home
package of $sm$, then the home package of $sm$ is modified and becomes
nil, that is, the symbol becomes homeless. This is the only case where the
symbol itself is modified by unintern. The same symbol could be external
in another package used by $p$, in which case the symbol remains
accessible via $p$. Otherwise, if the home package was nil or another
package, it is not modified.
(import $sm\; p$)
This is a request for the symbol $sm$ to be added to $InS(p)$. Lisp first looks for a possible name conflict. This happens only if a different symbol $sm'\neq sm$ with the same name ($sn(sm) = sn(sm')$) is already accessible via $p$. If so, the importation is rejected.
If no name conflict is detected, Lisp proceeds as follows: If the symbol is already present, it is left unchanged in $InS(p)$ or $ExS(p)$. Else, if the symbol is inherited because of a package $q$ used by $p$, $ExS(q)$ is left unchanged and $sm$ is added to $InS(p)$ (not $ExS(p)$!). If $sm$ is not accessible via $p$, it is simply added to $InS(p)$.
In the specific case of an imported homeless symbol, the symbol-package of
the symbol is updated to $p$. This is the only case where a symbol object is
modified by an importation. If the imported symbol is already owned by a
package, whatever its home package, it is not modified.
If the importation succeeds, the symbol will always be present in the target package, even if it was previously accessible only via a used package.
(export $sm\; p$)
The terminology "export" is not really appropriate, "externalize" would be
more precise. The user of an export expression asks Lisp to record the
symbol $sm$ in $ExS(p).$ This is allowed only if $sm$ is accessible from
$p.$ In other words, the export function cannot add an accessible symbol
to $p.$ In view of the rule $InS(p)\cap ExS(p) = \emptyset$, if present in
$InS(p)$, the symbol $sm$ is also erased from that list. If $sm$ is external
in a package $q$ used by $p$, its status in $q$ remains unchanged, but $sm$
is nevertheless added to $ExS(p)$.
Remember that the use-package relation is not transitive. The fact that
$sm\in ExS(q)$ makes $sm$ accessible via $p$ using $q$ but not via $p'$
using $p$. If it is desired that $sm$ be accessible via $p'$, it is possible
to make it external in $p$ by an export expression. Another solution would
be to import $sm$ in $p'$, an operation which doesn't involve $p$.
After exporting it, the symbol $sm$ becomes accessible from any package $q$ using $p$, that is, if $p\in U(q)$. This can generate a name conflict with a possible different symbol $sm'$ of the same name, which is present in $q$ or accessible via $q$ in another package $p'\in U(p)$, that is, if $sm'\in ExS(p')$. If this happens, the export request is rejected.
(unexport $sm\; p$)
The symbol $sm$ must be accesible via the package $p.$ If the status of $sm$ is external, it is downgraded to internal. Otherwise it remains unchanged, even if it is inherited.
(shadow {$sm$ or $st$} $p$)
It is intended to import the symbol $sm$ in $p$, to record it in the shadowing list $Shg(p)$ of the package $p$, and also to resolve any name conflict possibly raised by the importation.
This time the first argument must be a symbol, a pre-existing one or a
symbol just allocated because of its presence in the source statement. The
symbol is imported (See the function import.) and added to the shadowing
list $Shg(p)$ of $p$ if not yet present in this list. This addition to the
shadowing list solves the possible name conflicts with symbols inherited
from packages used by $p.$ It is possible that a different symbol sm'
with the same name was present in $p$ beforehand. If so, such a symbol is
uninterned from $p$, that is, erased from the lists $InS(p)$, $ExS(p)$ and
$Shg(p)$.
The difference between shadow and shadowing-import is clarified by the
following toy example. We start a new Lisp session, allocate a package
"P" and experiment with symbols named "A" and "B" in the packages
"CL-USER" and "P".
(in-package "CL-USER")(defpackage "P" (:use :cl))(setf my-list '(a b p::a p::b))(shadow 'p::a "CL-USER")(shadowing-import 'p::b "CL-USER")(package-shadowing-symbols "CL-USER")(eq 'a 'p::a)(eq 'b 'p::b)my-list
If we shadow the symbol p::a in "CL-USER", the symbol a remains
different from the symbol p::a. In contrast, if we shadowing-import the
symbol p::b in "CL-USER", then the symbols b and p::b are equal. In
a sense, the expression (shadow ’p::a "CL-USER") is cheating because the
presence of 'p::' has no role whatsoever, for only the name of the symbol
p::a is used and this name is the same as the name "A" of a, that is,
because of the default package, cl-user::a. The symbol defined by "A" via
the package "CL-USER" remains unchanged, in particular, it remains
different from the symbol p::a.
In contrast, if shadowing-import is used, an import operation is induced,
so that the symbol p::b becomes directly accessible via "CL-USER". The
previous symbol $\rho:($="CL-USER"$)($"B"$)$ has therefore been
/uninterned/ and is now unreachable by =find-symbol. However, it is still
visible in the list named list as the homeless symbol #:B.
The conclusion of our test is this:
- The symbol
cl-user::ais unchanged but added to $ShgS($"USER"$)$. - The symbol
cl-user::bis almost unchanged, but no longer accessible via"CL-USER". The only change is itssymbol-packagebecomesnil, because the symbol has been uninterned from its home-package. - The symbol
p::aremains unchanged; only its name has been used for something which does not really concern it. - The symbol
p::bremains unchanged, except that it is now directly accessible via"CL-USER"and now printed asBin the value oflist.
(use-package $q$ $p$)
Unless $q$ is already in $U(p)$, the package $q$ is added to the
package-use-list of $p.$
Lisp checks beforehand for possible name conflicts. A symbol
$sm \in PrS(p)$ could collide with a different symbol $sm'\in ExS(q)$ if
they have the same name $sn(sm) =sn(sm')$. If $sm = sm'$, no name conflict
is generated. One cannot resolve such a name conflict by adding $sm'$ to
$ShgS(q)$, hoping to give the precedence to the new inherited symbol:
shadowing operates in a uni-directional manner, from a using package toward
a used package. If one does so by a shadowing-import of $sm'$ into $p$,
the symbol $sm$ would be coherently uninterned, erased from $PrS(p)$, and
in a sense replaced by $sm'$. One can prepare the precedence of $sm$ over
$sm'$ before the use-package by previously =shadow=ing $sm$ in $p.$ Also
Lisp on its own can propose to unintern the symbol $sm$ in a continuable
error in order to resolve such an observed name conflict.
Another type of name conflict could be raised if another package $q'$ is
already used by $p.$ It is then possible that two different symbols
$sm \neq sm'$ with the same name $st$ are external, respectively in $q$ and
$q'$, in which case $\rho(p)(st)$ is ambiguous. If so, evaluating the
use-package expression would generate an error. On the other hand, the
intersection $ExS(q) \cap ExS(q')$ may be non-empty: a symbol external in
both packages does not generate any ambiguity via $p$. As usual, it is
always important to remember the lists such as $ExS(q)$ and $ExS(q')$ are
lists of symbols, not strings. A name conflict can be generated only by
different symbols having the same symbol-name.
If no name conflict is detected, the package $q$ is added to $U(p)$ so that the new set of accessible symbols via $p$ is:
$$ \mathrm{new-}AccS(p) = \mathrm{old-}AccS(p) \cup (ExS(q) − ShdS(q, p)) $$
(unuse-package $q$ $p$)
If the package $q$ is in $U(p)$, it is removed from this list.
A bug in package-internal-symbols?
While preparing these notes, the author thought for a moment to have encountered a mysterious bug. The minimal model for this bug goes as follows:
(unuse-package "P2" "P1")(unuse-package "P1" "P2")(delete-package "P1")(delete-package "P2")(defpackage "P1" (:use :cl))(defpackage "P2" (:use :cl))(use-package "P2" "P1")(export 'p2::a "P2")(import 'p2::a "P1")(package-internal-symbols "P1")
The point is that the standard Lisp function do-symbols runs through all
the symbols accessible via "P1", but it does not check for possible
duplicates. Really? It seems possible that a symbol can appear more than once
in $AccS($="P1"=$)$!? This does not seem compatible with the exclusion of
duplicates in general in $InS(p)$ and $ExS(p)$, and also the rule
$InS(p) \cap ExS(p) = \emptyset$. Yes, but a – one !– symbol can be
simultaneously in $InS(p)$ and $ExS(q)$ for a package $q$ used by $p$, which
is the case here for our symbol p1::a = p2:a.
The evaluation of (do-symbols (s p) ...) makes s run through all the
internal symbols of $p$, then through all the external symbols of $p$, and
finally through all the external symbols of $q$ for every package $q$ used
by $p$. It happens that our symbol is in $InS($="P1"$)$ because it was
imported there, and also in $ExS($"P2"$)$ where it was initially defined.
Didn't the programmer of =package-internal-symbols check the status of the
symbol, so that the (fake) second symbol should have been rejected? No, not
really, and it is interesting to see the path followed by Lisp: only the
symbol-name and the examined package are used, so that when Lisp computes
(find-symbol (symbol-name s) package), it obtains the symbol, in fact
the only one in this story, together with its status with respect to the
package argument. In our case that is the package "P1" and the symbol's
status (with respect to "P1") is :internal, even if at the same time
$s \in ExS($ "P2" $)$.
This illustrates some of the hidden traps and pitfalls that one might encounter as a result of different packages seeing the same symbol in various ways. Remember: no name conflict can be caused by one symbol alone. A name conflict happens only if two different symbols with the same name are "visible" by the same package. Nothing of this sort happened here, where one symbol is visible by several packages.
To overcome this weakness in the implementation of
package-internal-symbols, using the Lisp function delete-duplicates is
sufficient.
(defun package-internal-symbols-dedup (package)
(let ((rslt))
(do-symbols (s package)
(when (eq (second
(multiple-value-list
(find-symbol (symbol-name s) package)))
:internal)
(push s rslt)))
(delete-duplicates rslt)))(package-internal-symbols-dedup "P1")Symbol-package revisited.
Throughout this tutorial, we kept insisting that this function is not really
interesting. To be complete, we prove that a symbol
$sm \in PrS($ (symbol-package = $sm$ =) $).$ In other words, if the symbol
-package of the symbol $sm$ is $p$, then $sm$ is present in $p$, either
internal or external. When a symbol is allocated (necessarily via intern),
it is in particular interned in the package which is the second argument of
intern so that it becomes a member of $InS(p)$. It may be exported in this
package, which would make it a member of $ExS(p)$, but it remains present.
The symbol can be added to $InS(q)$ for other packages $q$ via import or
shadowing-import. It can also then be "upgraded" to $ExS(q)$ in those
packages, but the membership in the initial package $PrS(p)$ remains
unaffected.
The only way to change this situation is using unintern with the package
argument being $p.$ Then the symbol becomes homeless.
In turn, this can be changed with import and shadowing-import, in which
case the symbol is interned in a possibly new package, and the story
continues along similar lines. Finally, unless a symbol is homeless, it is
always present in its home-package.
This also shows that, as was mentioned in the introduction, the
symbol-package qualifier of a symbol does divide the set of interned
symbols into packages. It is impossible to have two different interned
symbols with the same symbol-name and also the same symbol-package, for they
would be interned in the same package, with the same name, and then the set
of packages would not be coherent.
An interned symbol is internal or external in its symbol-package, and
certainly visible via this package. However, it may be visible from many
other packages, and this information is not stored in the internal
representation of the symbol. If it is external, it is accessible via the
packages which use its package. The list of these packages can be determined
by the standard Lisp function package-used-by-list. But a symbol can also
be imported by other packages, which in turn may decide to make (the pointer
to) this symbol external, and so visible from other packages.
Exercise: Try to define a function symbol-accessible-from with one
argument, a symbol, that returns the list of all packages where this symbol
is accessible. Do not forget the contorted particular case of
common-lisp:nil! Hint: Have a look at the standard Lisp function
list-all-packages.
An arbitrary number of homeless symbols may have the same name, as illustrated by:
(eq (print (make-symbol "Z"))
(print (make-symbol "Z")))
When make-symbol is called, Lisp allocates a new homeless symbol with the
prescribed name, even if another one with the same name has been already
allocated. You see both homeless symbols allocated above are printed in the
same way, yet they are different.
Practical uses of the packaging system.
Many of the somewhat esoteric technicalities detailed in this tutorial have no direct use in concrete programming. On the other hand, they can help to understand and demystify the behaviour of your Lisp system.
A particularly irritating fact exhibited by some systems is the following:
When you merely install, not load, a source file in the editor of your Lisp
IDE, these systems immediately scan for symbols appearing in these files and
intern them in a separate package. Sometimes this is the package indicated
in an inital (in-package "XXX") of the file, or still more
inappropriately, the standard "CL-USER" package, or also the current
default package, even if the initial in-package references another
package. Often this later leads to symbol collisions which are sometimes
hard to identify. A developer who meets this kind of mysterious symbol
collisions is advised to search first in this direction. In this case, you
should make independent micro-experiments to study the actual behavior of
your IDE when you install a file in your development environment. Probably
in the early days of Lisp, this was done to improve the efficiency, but on
modern systems, it is difficult to see compelling reasons to justify such a
strategy.
Most symbols of a Lisp file in fact are symbols with lexical scope, a scope
limited to a small part of the code, typically inside a function, a let
form, a do form and so on. These references are converted by the compiler
into raw machine addresses, and these symbols have no real life at runtime.
The most important symbols that have a real life at runtime are those that
name functions, those defined through a defun form and the like. If a
large program is divided in several files, then some decisions must be taken
about the status of such a symbol.
Usually, a correspondence "file $\leftrightarrow$ package" is maintained.
Several Lisp systems require that a file begins with an appropriate
(in-package ...) expression, declaring that all the symbols used in this
file are to be understood as present in this package, except when something
else is a consequence of decisions taken outside this file or even in this
file.
In the ANSI definition of in-package, the referenced package must already
exist in the environment. Beginning a file by:
(defpackage "MY-PACKAGE") (in-package "MY-PACKAGE")
is tempting but does not obey the requirement about the first instruction of
a file being the appropriate (in-package ...).
The solution is to have a separate file in the project which is called aptly
packages.lisp and which contains all the necessary (defpackage ...)
expressions for the project. The defpackage form is flexible and can
contain also all the main information about a package, in particular the
list of its external symbols and its used packages. (See the ANSI definition
of defpackage.) This file is a good foundation for structuring the
"public" documentation, explaining the role of these packages and their
external symbols.
Along the same lines, it might be worthwhile to have an "EXTERNAL" package
which contains all the external symbols of the application. In this way, the
other packages may use this package systematically, giving to this package a
status similar to the status of the "COMMON-LISP" package. This does not
prevent another package to import some symbols of other packages, symbols
whose role is too specific to justify to make them "globally" external.
Shortcuts.
There are several ways for the developer to shorten the Lisp expressions dealing with the packaging system.
To be rigorous in our explanations and close to the deep structure of Lisp,
in this tutorial, we used a string identifier everywhere for a package. A
symbol could be used instead, its name then being used as the intended
string. For example, (defpackage :my-package) is equivalent to
(defpackage "MY-PACKAGE"), for the name of the keyword :my-package is
used, and because of the capitalization rule of the symbols applied by the
Lisp reader. The name of :my-package is "MY-PACKAGE", which through the
defpackage expression becomes a package-name by following the standard
rule which asks for an uppercase name.
Also, each time a package argument is needed, frequently the second one, it can be omitted if this package is the current default package, which justifies this terminology. When using these shortcuts, do not forget about maintaining maximum readability of the source code!