#+TITLE: Packages in Common Lisp, a tutorial #+AUTHOR: Francis Sergeraert #+DATE: January 2014 * Introduction. It is not possible to write a serious piece of code without the use of some kind of /identifiers/ or names. Such identifiers are convenient means for the programmer to reference, process and modify data using more or less descriptive names, which represent the data and depend in some way on the context. The standard /modularization/ technology encourages the division of large programs in /modules/. Such modules can be relatively independent of each other, but conversely, they must be able to /communicate/. Most programming languages offer methods for such an organization. For example, the set of identifiers of a module could be divided into two parts, the /private/ identifiers, known only inside this module, and the /public/ identifiers, known everywhere. In this domain as in many others, Common Lisp is by far the most powerful and flexible programming language. At the expense of considerable complexity, Common Lisp's modular technology requires a good grasp of the corresponding /definitions/ in the language standard. If the ANSI definition of the language is the only available documentation, these definitions are not easy to understand. An identifier in Lisp is a /symbol/ which has a rich internal structure and which has lifetime and visibility during a Lisp session. The modular structure of a Lisp program is defined through the /packaging/ system, which allows a programmer to divide a large program into several files, where each file is aware of all the symbols of a certain package, among them the current /default/ package, and only the /external/ (public) symbols of some other packages. A symbol which is not external in a package is /internal/ (private). Also, a package may /import/ (that is, "see") an arbitrary collection of symbols, internal or external to other packages. In the general spirit of Lisp, the whole structure of such an organization, if necessary, can be modified dynamically, even though this is not usually the best course of action. In this tutorial, a /mathematical/ definition of this system is provided. This should elucidate the process for many a Lisp programmer and enable her to use it more effectively. A basic but /inexact/ description of the packaging system is the following: - A /package/ is a Lisp object in the same way that numbers, lists, and character strings are Lisp objects. In particular, the /type/ of a package object is the system class [[http://clhs.lisp.se/Body/t_pkg.htm][=package=]]. In Common Lisp, packages are first class citizens, which means they can be the value of a symbol, can be passed as an argument to a function, can be returned by a function, etc. - A /symbol/ is a Lisp object whose /type/ is the system class [[http://clhs.lisp.se/Body/t_symbol.htm][=symbol=]]. The comments made about packages apply for symbols. In particular, a symbol can be the value of another symbol, or even of itself. - Symbols are divided into packages. - At any time during a Lisp session, some package is the /default package/ and, as a first approximation, only the symbols of this package are /accessible/. The default package can be modified at any time by the user or a program. - If necessary, there are several methods to access the symbols of packages other than the default package. Conversely, a package may make its symbols more or less easily accessible from the other packages. This first approximation of the packaging system conveys a general idea of its organization, but general ideas can be deceiving. For example, the idea that the symbols are divided into packages is essentially /false/. More precisely, this division can be made to appear essential (!), via the so-called home package of a symbol, but it plays only a /minor/ role and can be ignored by the programmer risk-free. This notion of a home package is terribly misleading for beginners when they have to design non-trivial packaging structures. In a Lisp environment, one has to deal with two realms, the space of symbols and the space of packages. It is only /inside/ of packages that something can be read, which, as an organization, looks a /little/ like a division of symbols into packages, but the symbols themselves essentially /ignore/ this division. Our subject is to clarify and explain this organization. The ANSI definition of the packaging system is a /complete/ description of this organization, and, in a sense, nothing is missing. However, experience shows that it is easily misunderstood, especially, in non-trivial situations, where it can lead to erroneous programming choices and mysterious bugs. The ANSI definition of packaging is complete and it is the perfect /Reference Manual/. However, for a topic of this complexity a reference guide is not sufficient and a /User Guide/ might be useful. And for that matter, an elementary /mathematical/ description of the Common Lisp packaging system might help programmers to understand it, and then to use it with greater ease and also to greater effect. In every domain, when precise mathematical language can be employed and is understood, the state-of-affairs improves. We begin with a review of the ANSI definition of the packaging system while adding in parallel a mathematical description of the main concepts. Finally, we give a complete mathematical definition of this system as a set of mathematical formulas. * Main sets of objects. The first part of the packaging system is made of three disjoint sets: - The set $St$ of *strings*. - The set $P$ of *packages*. - The set $Sm$ of *symbols*. In our description of the packaging system, the statuses of these sets are different. Think of the (infinite) set $St$ of /strings/ as made of /all/ the character strings that /can/ be used by a user or a program at any time. Any string can be used at any time, whatever the history of the Lisp session in which we are working may be. In the packaging system, most often, a particular string is used only for a short time, mainly during the read-eval-print loop, when the Lisp reader works. A string is just an intermediary connecting the user and the Lisp session, more precisely the current environment. In contrast, in a given environment, at any time during a Lisp session, some finite set $P$ of /specific/ packages is installed in this environment, and also a finite set $Sm$ of /specific/ symbols is available. Those packages and symbols can /then/ be used, and they are the historical consequence of the current Lisp session. Various Lisp functions allow the user to modify the sets $P$ and $Sm$, and the way they are combined and related to each other. In other words, consider the set of strings $St$ as constant, vast and independent of the environment, while, on the contrary, the sets $P$ and $Sm$ are restricted, they change during a Lisp session, and they are an essential component of the /current/ environment. * Packages There is a simple, almost bijective (one-to-one), correspondence between strings and packages, and a convenient place to start our subject. First, at any time during a Lisp session, a /default package/ is defined and it can be discovered via the global symbol [[http://clhs.lisp.se/Body/v_pkg.htm][=*package*=]]: #+BEGIN_SRC lisp :session pcl :exports both *package* #+END_SRC This is to be read as: the /value/ of the /symbol/ =*package*= is the package whose /name/ is the string ="COMMON-LISP-USER"=. The types =symbol= and =package= are system-defined, disjoint sets of machine objects. A symbol is frequently used as a convenient intermediate object, which allows the user to reach some object of arbitrary nature, its /value/. Here, =*package*= is a symbol, the value of which is the package named ="COMMON-LISP-USER"=. Technically, a symbol contains a pointer to its (possible) value, here a package which happens to be the package used ordinarily in a simple Lisp session. There are no facilities to refer to a package object /directly/. However, one can use its /name/ and the function [[http://clhs.lisp.se/Body/f_find_p.htm][=find-package=]] to obtain a reference: #+BEGIN_SRC lisp :session pcl :exports both (find-package "COMMON-LISP-USER") #+END_SRC Conversely, given a package, the function [[http://clhs.lisp.se/Body/f_pkg_na.htm][=package-name=]] returns its name: #+BEGIN_SRC lisp :session pcl :exports both (package-name (find-package "COMMON-LISP-USER")) #+END_SRC The example was given to illustrate that the functions =find-package= and =package-name= are the inverse of each other, defining a 1-1 correspondence between the packages /currently/ defined in the environment and their respective names, some character strings. Because of rules not of interest here, it is common and convenient to use only uppercase character strings to name packages. If a name does not correspond to any current package, the =find-package= function returns the symbol =nil=, usually displayed uppercase: #+BEGIN_SRC lisp :session pcl :exports both (find-package "COMMON-LISP-user") #+END_SRC which illustrates that a package name is case-sensitive. Frequently, only a few packages are in use, and because of their importance, descriptive names are used. These descriptive names can be a little long, and it is possible to define and use various /nicknames/ for these names. For example, the ="COMMON-LISP-USER"= package usually has the nicknames ="CL-USER"= and ="USER"=. The function [[http://clhs.lisp.se/Body/f_pkg_ni.htm][=package-nicknames=]] can be used to retrieve the list of nicknames (strings) for a package, which doesn't include the name of the package. #+BEGIN_SRC lisp :session pcl :exports both (find-package "COMMON-LISP-USER") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (package-nicknames (find-package "COMMON-LISP-USER")) #+END_SRC If a package is to be unambiguously identified by its name or one of its nicknames, two different packages must have disjoint sets of /name + nicknames/. * Tracking a symbol. The Lisp symbols are very particular when compared to the identifiers of other programming languages. They persist throughout the runtime of an environment and are an important component of the power of Lisp. A package is referenced via its name; the same goes for a symbol, but the process is much more complex. The first and most common method to create or "to /allocate/ a symbol" consists in just writing it down and passing it to the Lisp reader using the standard Lisp conventions. In a Lisp environment, there is a set of "pre-defined" symbols ready to be used. To caution the reader about the complexity of the subject, we begin with a seemingly strange example: #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (find-symbol "X" "COMMON-LISP-USER")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (ignore-errors x)) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (find-symbol "X" "COMMON-LISP-USER")) #+END_SRC The function [[http://clhs.lisp.se/Body/f_find_s.htm][=find-symbol=]] is analogous to the function =find-package=: given some /name/, a character string, it looks for a symbol with this name. Remember the correspondence between packages and names? A similar correspondence exists between symbols and names, but it's more involved. In particular, a name is /never sufficient/ to determine a symbol, a /package/ is also necessary. This is why the =find-symbol= function requires /two/ arguments, the name of the symbol and the package with respect to which the search is to be done. At the beginning of our story, no symbol with the name ="X"= existed in the default package ="COMMON-LISP-USER"=, and the return value of =find-symbol= is negative, =nil=. (We will explain the second =nil= value in a moment.) Then we pass the Lisp reader the one character expression '=x='. Lisp evaluates it according to its /read-eval-print/ cycle. First, Lisp /reads/ the expression and notices that the symbol =x= is used. It examines the current package ="COMMON-LISP-USER"=, and does /not/ find such a symbol. Before doing anything else, Lisp /allocates/ (creates) this symbol, the right terminology being: "Lisp /interns/ a symbol =x= of name ="X"= in the package ="COMMON-LISP-USER"=." To refer to this symbol, the pair made of /its/ name ="X"= and the (not /its/!!) package ="COMMON-LISP-USER"= are necessary, but it would be cumbersome to have to write the expression =(find-symbol "X" "COMMON-LISP-USER")= every time this symbol is used. To simplify the process of referring to a symbol the designers of Lisp have organized the workspace as follows: if a symbol is read, its character string is capitalized, giving its /name/, and the default package is assumed. In other words, the text ='x'= is roughly equivalent to the text =(find-symbol "X" "COMMON-LISP-USER")=. How convenient! ('Roughly', because '=x=' may trigger the allocation of a symbol. In contrast, =find-symbol= never allocates a symbol; but see the function =intern= later.) In the second expression of our example, once Lisp has "understood" that the user intends to use the symbol =x=, and observing that such a symbol is not present in the package ="COMMON-LISP-USER"=, it allocates a symbol of name ="X"= and /interns/ it in the package ="COMMON-LISP-USER"=. More specifically, a pointer referring to the just allocated symbol is added to the list $InS(p)$ of the internal symbols of the package $p$. Note also that it is common in Lisp to input the symbols in lowercase letters, and Lisp prints them in uppercase. This might be convenient for distinguishing input from output in an interactive session, but requires some clarification. In particular, there is no difference between the symbols =nil= and =NIL=, whereas the strings ="nil"= and ="NIL"= are different. #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (find-symbol "nil" "COMMON-LISP-USER")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (find-symbol "NIL" "COMMON-LISP-USER")) #+END_SRC The fruitless search for a /symbol/ with the lower-case name ="nil"= (a /string/) returns a double negative answer, but it is the second =NIL= that is meaningful! Compare that with the second expression which has the same first return value, but a different second one. The symbol =NIL= also plays in Lisp the role of the boolean /false/, which creates an ambiguity: if the answer is the symbol =NIL= is that a negative answer or is it a positive answer made of the symbol =NIL=? The ambiguity is resolved by the /second/ return value. In the first case, the second =NIL= /confirms/ that the first =NIL= is to be interpreted as the boolean /false/. In the second expression, the second return value =:INHERITED=, not a negative, indicates that the symbol =NIL= has been found, and that its /accessibility/ via the package ="COMMON-LISP-USER"= is /inherited/, a point to be explained later. In other words, the function =find-symbol= returns a genuine symbol /and/ its accessibility via the package argument, or a pair of =nil='s if no symbol is found. The logician can deduce that an accessibility cannot be the symbol =nil=! Let us return to the one character expression '=x='. The read part of the read-eval-print loop cycle reads the symbol =x= and interns it in the package ="COMMON-LISP-USER"=. The expression is /read/ and must now be /evaluated/. The evaluation of a symbol consists in looking for a /value/ of this symbol. To this end a value pointer in the symbol's internal representation is examined. In this case it is found that, at this time, there is no value for the freshly allocated symbol =x=. Therefore the /eval/ step of the /read-eval-print/ cycle generates an error, terminating the cycle with an informative error message. In any case, a symbol of name ="X"= is now present in the package ="COMMON-LISP-USER"=, as confirmed by the value of the last expression. The symbol's accessibility via the ="COMMON-LISP-USER"= package is /internal/, which is to be explained later. * Examining a symbol. A symbol is a Lisp object with a rich internal structure and there are several functions for exploring symbols and their properties. Let us examine in detail the symbol =x= allocated in the previous section. #+BEGIN_SRC lisp :session pcl :exports both (symbol-name 'x) #+END_SRC The [[http://clhs.lisp.se/Body/f_symb_2.htm][=symbol-name=]] function returns the /name/ of a symbol; it is analogous to the =package-name= function for a package. The Lisp function =symbol-name= is crucial for our subject, and we denote it as a mathematical function $sn: Sm \longrightarrow St$, a function which in general is not injective: several symbols may have the same name, this is our main subject. Note that in the previous expression the symbol =x= has been quoted ='x= to prevent its evaluation. More precisely, ='x= is an abbreviation for the expression =(quote x)=, where [[http://clhs.lisp.se/Body/s_quote.htm][=quote=]] is a /special/ function which /does not/ evaluate its argument, and it returns this argument as is. Hence, the previous expression is equivalent to: #+BEGIN_SRC lisp :session pcl :exports both (symbol-name (quote x)) #+END_SRC Here, the non-quoted =x= does not generate an error. If instead you do not quote the symbol =x= (implicitly via ='=, or explicitly via =quote=) it will be evaluated before being used and generate an error: #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (ignore-errors (symbol-name x))) #+END_SRC The package which /owns/ a symbol, its /home package/, can be determined with the function [[http://clhs.lisp.se/Body/f_symb_3.htm][=symbol-package=]]. #+BEGIN_SRC lisp :session pcl :exports both (symbol-package 'x) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (find-symbol "X" "COMMON-LISP-USER")) #+END_SRC As we have just seen, a symbol produces a name and a package (with the help of =symbol-name= and =symbol-package=). Conversely, given a name and a package, =find-symbol= produces the symbol with this name /via/ this package. The reader might think that there is a correspondence between symbols and pairs of strings and packages: $$Sm \longleftrightarrow St\times P$$ However, the situation is more involved: for the beginner, this might be a convenient point of view, but in fact such a point of view is wrong. If you intend to fully understand the complex relationship between strings, packages and symbols, please forget this tempting but terribly misleading correspondence. The goal of the next sections is to establish the correct point of view. For completeness we finish this section with a brief glance at the other data possibly stored in a symbol. A symbol can have a value. You may remember that in our Lisp session the symbol =x= does not (yet) have a value. This can be established by the predicate [[http://clhs.lisp.se/Body/f_boundp.htm][=boundp=]]: #+BEGIN_SRC lisp :session pcl :exports both (boundp 'x) #+END_SRC The main method to give a symbol a value is the [[http://clhs.lisp.se/Body/m_setf_.htm][=setf=]] function (a macro). In most Common Lisp implementations, the =setf= macro may be used directly, in which case the default status of the argument symbol is /variable/. In strict ANSI implementations such as SBCL here, a =defvar= statement is required to explicitly define this status. Other possible statuses are /constant/ (via =defconstant=) and /parameter/ (via =defparameter=). #+BEGIN_SRC lisp :session pcl :exports both (defvar x) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (setf x '(4 5 6)) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both (boundp 'x) #+END_SRC Note that the first argument of =setf=, the symbol =x=, is not evaluated. Otherwise an error would be generated, since the symbol =x= does not yet have a value. After the =(setf ...)= expression has been evaluated, the symbol =x= does have a value, in this case, the list =(4 5 6)=. This value would now be the result of the evaluation of the expression =x=. It can also be obtained with the function [[http://clhs.lisp.se/Body/f_symb_5.htm][=symbol-value=]]. (In this tutorial, we do not study the possible difference between, on the one hand, the simple /evaluation/ of the symbol =x= asked for by the expression ='x'= and the /evaluation/ of =(symbol-value 'x)= on the other. In our elementary examples there is no difference between the two. Possible differences might come from different variable scopes, such as /lexical/ or /dynamic/, but this subject is not studied here.) #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim x #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (symbol-value 'x) #+END_SRC Don't forget to quote the symbol! See the following error: #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (ignore-errors (symbol-value x))) #+END_SRC A symbol can also have a /functional value/. This does not mean that the value we just discussed could be a function (which it could be!). It means that /besides/ the previous value, a symbol can also have an /additional/ value, a function, which can be used independently of the "ordinary" value of this symbol. For example the function $n \mapsto 10 - n$ could be recorded as the /functional/ value of the same symbol, and this function could then be used as shown below. #+BEGIN_SRC lisp :session pcl :exports both (defun x (n) (- 10 n)) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both (x 3) #+END_SRC Both values coexist in the symbol without any interference: #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (symbol-value 'x) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (symbol-function 'x) #+END_SRC and can be used jointly and independently: #+BEGIN_SRC lisp :session pcl :exports both (x (first x)) #+END_SRC You might already suspect that the position of =x= just after a left parenthesis =(x ...)= makes Lisp extract the /functional/ value and not the "ordinary" value. This is correct, but not the subject of this tutorial. Finally, we mention only in passing the following: 1. A symbol can have a functional value while not having an ordinary value. 2. An ordinary value can be a functional object, usually then called via =funcall= or =apply=. 3. A symbol also contains a property list or =plist=, with many possible uses, but not at all studied here. The internal structure of the [[http://clhs.lisp.se/Body/t_symbol.htm#symbol][symbol]] =x= might be visualized thus: #+BEGIN_EXAMPLE +-----------------+ | Name o-|-----> "x" +-----------------+ | Value o-|-----> (4 5 6) +-----------------+ | Function o-|-----> (lambda (n) (- 10 n)) +-----------------+ | Package o-|-----> # + ----------------+ | Property list o-|-----> NIL +-----------------+ #+END_EXAMPLE * But what the hell is a package? Now we attack the heart of our subject.\\ **Fact 1:** A package $p\in P$ maintains four fundamental lists: - A list $InS(p)$ of its internal symbols. - A list $ExS(p)$ of its external symbols. - A list $ShgS(p)$ of its shadowing symbols. - A list $U(p)$ of its used packages.\\ The first important fact toward understanding the subject is this: the first three lists are lists of /symbols/ or, more precisely, lists of pointers (machine addresses) aimed at the symbols. Every symbol has a /unique/ "existence": a symbol is a unique machine object in the environment, but /several/ packages may /see/ the same symbol, that is, include the machine address of this symbol. In particular these lists are not lists of symbol names (strings). There is another mechanism that allows Lisp to reach a symbol given its name. Every symbol can be accessed directly via a suitable package, and several packages can in general play this role for the same symbol at a given moment in a Lisp session. For example, all the [[http://clhs.lisp.se/Front/X_Symbol.htm][standard Lisp symbols]] in the ="COMMON-LISP"= package typically are accessible via any package. No duplicate symbol (more precisely, no duplicate pointer) is to be found in each of these lists. Also the relations $InS(p)\cap ExS(p)=\emptyset$ and $ShgS(p)\subset InS(p)\cup ExS(p)$ must be satisfied, for reasons to be studied later.\\ **Fact 2:** Given the current set of packages, assumed /coherent/, and given the current state of all the fundamental lists of these packages, the function =find-symbol= can unambiguously determine /whether/ a symbol of a given name (some string) is /accessible/ via some package; if so, this symbol is unique.\\ The importance of the package argument of =find-symbol= could make the reader believe the function =symbol-package= will be useful for various tests. This is not the case: All this function returns is the "birth place" of the symbol, technically called its /home package/, which says almost nothing about the current status of this symbol with respect to any package. All it says is that at least it is /interned/ in this package. In particular, a symbol can be accessible via several packages, possibly different from its home package. Here is a simple illustration: #+BEGIN_SRC lisp :session pcl :exports both (symbol-package 'nil) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (find-symbol "NIL" "COMMON-LISP")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (multiple-value-list (find-symbol "NIL" "COMMON-LISP-USER")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (eq (find-symbol "NIL" "COMMON-LISP") (find-symbol "NIL" "COMMON-LISP-USER")) #+END_SRC The /home package/ of =nil= is ="COMMON-LISP"=, and this symbol of name ="NIL"= is accessible via ="COMMON-LISP"=, but also via ="COMMON-LISP-USER"=, though not with the same status. It is /external/ in the first case and /inherited/ in the second case, points to be examined later. The value (=T=) of the last expression /proves/ that both symbols, determined by =find-symbol= via different packages, in fact are the same. Fact 2 can be made a little more precise: At any given time during a Lisp session, a set $P$ of packages and a set $Sm$ of symbols are defined. The various lists of symbols $InS(p)$, $ExS(p)$ and $ShgS(p)$, and the list $U(p)$ of packages used by a package $p\in P$ together define without any ambiguity which symbols are accessible via the package $p$. In other words, there is a well defined function $\rho$: $$ \rho: P \longrightarrow [St \longrightarrow Sm \cup \{\mathtt{nil}\}] $$ We chose the letter $\rho$ for reader, because this function is mainly used by the reader. Our "mathematical" function $\rho$ is nothing but an avatar of the Lisp function =find-symbol=. Let $p\in P$ be a package and $st\in St$ be a string. Then: $$ \rho(p)(st) = (\mathtt{find-symbol}\; st\; p) $$ The left-hand side uses the standard mathematical functional notation, and the right-hand one uses the Lisp notation. If no symbol is found for a pair $(st, p)$, the symbol (!) =nil= is returned and the second value of =find-symbol=, the symbol =nil= again, tells the user that the symbol search did not return anything. A package $p$ is nothing but a way of defining the function $\rho(p):St\longrightarrow Sm\cup\{\mathtt{nil}\}$. In other words, the four fundamental lists of a package determine what character strings appear as the names of the symbols accessible via this package. This approach is described in detail now. * Internal symbols. The first fundamental list of a package is the list of its /internal symbols/. It is a pity that no standard Lisp function can produce this list, but here is a simple solution to the problem: #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (defun package-internal-symbols (package) (let ((rslt)) (do-symbols (s package) (when (eq (second (multiple-value-list (find-symbol (symbol-name s) package))) :internal) (push s rslt))) rslt)) #+END_SRC We can list now all the internal symbols of a package: #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (package-internal-symbols "COMMON-LISP-USER") #+END_SRC In fact, only the first ten internal symbols are displayed. We see in particular the symbols =package-internal-symbols=, =s= and =rslt= used in the definition of our function. Some are rather esoteric, strictly speaking illegal: at the beginning of a Lisp session, the ="COMMON-LISP-USER"= package should in principle be void of symbols. To make our experiments more understandable it is better to allocate and use toy packages. The [[http://clhs.lisp.se/Body/m_defpkg.htm][=defpackage=]] function (a macro) allocates a package of a given name. #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (defpackage "P1" (:use "CL")) #+END_SRC A package ="P1"= is now present in our environment, without any internal symbols. #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (package-internal-symbols "P1") #+END_SRC (The symbol =nil= also represents the empty list.) Let's allocate a few symbols and make them internal in the package ="P1"=! The current default package is ="COMMON-LISP-USER"= and we switch it to ="P1"= like this: #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (in-package "P1") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" *package* #+END_SRC From now on, the default package is ="P1"=. If we write down a symbol then it will be understood by the Lisp reader as accessed via the package ="P1"=, that is, unless we use a special syntax. Let us create a list of three symbols and assign it to the symbol =my-list=, and then examine the internal symbols of the package ="P1"=: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (setf (symbol-value 'my-list) '(symb1 symb2 symb3)) #+END_SRC The reader probably would prefer =(defvar list)= followed by =(setf list ...)=, which is rejected by some Lisp implementations, in particular SBCL, because in principle the symbol =list=, owned by the package ="COMMON-LISP"=, is "locked". We use the symbol =my-list= instead. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (ignore-errors (package-internal-symbols "P1"))) #+END_SRC Oops! What happened? We've successfully used the function =package-internal-symbols= earlier and suddenly it seems to have disappeared from the environment. The point is that it was /defined when/ ="COMMON-LISP-USER"= was the default package. Let us examine the situation! #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "COMMON-LISP-USER")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1")) #+END_SRC A symbol with the given name is found in both packages. Before encountering the undefined functional value error, Lisp ( more precisely, the Lisp reader) had interned a /new/ symbol with the name ="PACKAGE-INTERNAL-SYMBOLS"= in the current default package, ="P1"=, for the sole reason that it was present in the source text. Let us verify that both symbols are different! #+BEGIN_SRC lisp :session pcl :exports both :package "P1" (eq (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "COMMON-LISP-USER") (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1")) #+END_SRC The symbols are different. Compare that with the status of =nil=, where the same symbol =nil= was accessible via ="COMMON-LISP"= and via ="COMMON-LISP-USER"=. In our present situation both symbols are /different/, and so are their respective properties. For example, the first symbol does have a functional value and the second does not, which can be tested with the function [[http://clhs.lisp.se/Body/f_fbound.htm][=fboundp=]]x: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" (fboundp (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "COMMON-LISP-USER")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" (fboundp (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1")) #+END_SRC This is the first time we have seen the double-colon =‘::’= notation: When the symbol =package-internal-symbols= was searched via ="COMMON-LISP-USER"=, the answer was positive but the returned symbol was a little lengthy: =COMMON-LISP-USER::PACKAGE-INTERNAL-SYMBOLS=. The current default package is ="P1"=, via which /this/ symbol is not accessible. However, we may access it using the double-colon notation: #+BEGIN_CENTER =package-name::symbol-name= #+END_CENTER This is one of the methods which allows the user or Lisp to refer to a symbol not accessible via the default package. The Lisp reader is told: "We mean the symbol =symbol-name= via the package =package-name=." In contrast, for the symbol with the same name but interned in the different ="P1"= package, because ="P1"= is the current default package, the double colon notation is not necessary. The longer notation lets us access a symbol interned in the ="COMMON-LISP-USER"= package and its functional value as follows: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (cl-user::package-internal-symbols "P1") #+END_SRC The =cl-user::= prefix tells Lisp to look for the symbol via the ="CL-USER"= package (Remember that ="CL-USER"= is a nickname for ="COMMON-LISP-USER"=?) and, because of its position just after a left parenthesis, to use its functional value. Currently, there are four internal symbols in the package ="P1"= and they are returned in an arbitrary order. By the way, what about the symbol =list=, to which we assigned a list (!) of symbols? #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (find-symbol "LIST" "P1")) #+END_SRC The symbol is accessible via the package ="P1"= but with a different status, the status /inherited/. We will examine this situation later. In fact, the symbol =list=, one of the most important symbols in Lisp, is already present in the package ="COMMON-LISP"= at the beginning of a session, and by default accessible from any package. We must mention also that our function $\rho$, or the function =find-symbol= if you prefer, is (almost) injective /with respect to/ $st$. Let $p_1$ and $p_2$ be two packages, $st_1$ and $st_2$ two strings. Then if $\rho(p_1, st_1) = \rho(p_2, st_2)\neq\mathtt{nil}$, then $st_1 = st_2$. Pay attention to the fact that the equality is an equality between /symbols/. This is a consequence of $\mathtt{symbol-name}(\rho(p, st)) = st$; in other words, there is no possibility of nicknames for a symbol name. In particular, there is a 1-1 correspondence between the internal symbols of a package and their respective names. * Importing a symbol. Instead of using the lengthy notation =cl-user::xxx= to make an internal symbol in ="CL-USER"= accessible from ="P1"=, we may opt to /import/ this symbol in the package ="P1"=. However, in doing so there is another problem due to a name conflict, a point which will be examined in detail later: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (ignore-errors (import 'cl-user::package-internal-symbols "P1"))) #+END_SRC The problem is a "collision" between two /different/ symbols, which cannot coexist as internal symbols in the /same/ package ="P1"= with the /same/ name. Importing the symbol internal in ="CL-USER"= into the package ="P1"= is not compatible with the current presence of /another/ symbol in ="P1"= with the same =symbol-name=. We must first [[http://clhs.lisp.se/Body/f_uninte.htm][=unintern=]] the accidental, pre-existing symbol in ="P1"= to "make room" for the symbol from ="CL-USER"=. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (unintern 'package-internal-symbols "P1") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (import 'cl-user::package-internal-symbols "P1") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (package-internal-symbols "P1") #+END_SRC The Lisp terminology is rather misleading: in fact, it is not at all the symbol which is imported, but the connection /string/ $\mapsto$ /symbol/. More precisely, let $p_1$ and $p_2$ be two packages, $st$ a string and $sm$ a symbol. If $\rho(p_1)(st) = sm$ holds before the importation, then, after the importation, both relations $\rho(p_1)(st) = sm$ /and/ $\rho(p_2)(st) = sm$ are satisfied. In particular, the first relation remains valid. It is not really an importation; it is a /copy/ in the package $p_2$ of the connection $st \mapsto sm$, available in the package $p_1$. Furthermore, nothing is modified about the symbol itself, except /possibly/, but rarely, its home package, as will be examined later. It is tempting for a beginner to think the home-package of the symbol is being modified. Not at all! Another erroneous interpretation would be to think a /copy/ of the symbol to be imported is made "inside" the target package, with all the corresponding ingredients of the symbol, value, functional value, and so on. Wrong again! A copy is made, but only of a /pointer/ toward our symbol found in $InS(p_1)$ and copied in $InS(p_2)$. The [[http://clhs.lisp.se/Body/f_import.htm][=import=]] function has two arguments, the /symbol/ to be imported and the /target package/. The symbol must be well defined, that is, taking account of the current state of the environment. Here, before the import, the symbol is accessible only via ="CL-USER"=, and it is mandatory to use the notation =cl-user::...= (and the symbol must also be quoted to preclude its evaluation). After the import, as can be seen in the example, the symbol can be used functionally from the package ="P1"= without the prefix notation. Although the imported symbol is in the list of the internal symbols of the package ="P1"=, it is not quite the same as before. The next mandatory exercise consists in comparing the original symbol and the imported one. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (eq 'cl-user::package-internal-symbols 'package-internal-symbols) #+END_SRC The function =eq= proves that both notations name the same symbol. Here's another proof: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (eq (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "CL-USER") (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1")) #+END_SRC If you remember the =symbol-package= function, the next test is revealing: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (symbol-package (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "CL-USER")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (symbol-package (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P1")) #+END_SRC The /unique/ symbol accessed by two different paths has of course a unique home package, namely its /birth place/, the ="CL-USER"= package. Please, do not pay too much attention to this symbol package; it is just an artifact of the session's history and does not have any real interest. You must focus on the chain $\mathit{package}\rightarrow[\mathit{string}\rightarrow \mathit{symbol}]$ and not much else. There is no real reason to be concerned about the map =symbol-package=: $\mathit{symbol}\rightarrow\mathit{package}$. * Name conflicts. If poorly understood, the introduction of Lisp packages to a Lisp project will sooner or later lead to /name conflicts/. There is nothing mysterious about them and there are standard "home remedies" for dealing with them.\\ *Warning.* A name conflict can only be generated if two /different/ symbols $sm$, $sm' \in Sm$ end up with the same name: $sn(sm) = sn(sm')$. A name conflict occurs if these symbols $sm$ and $sm'$ become accessible via the same package $p$, which is a forbidden situation, since it would introduce ambiguity to the =find-symbol= function. On the other hand, the fact that some symbol (singular) is accessible via /different/ packages never generates a name conflict by itself. Lisp constantly is on the lookout for possible name conflicts and stops with an error if the evaluation of an expression causes a name conflict. Some errors are /continuable/, which means that Lisp is able to propose a conflict resolution via =unintern= or =shadowing-import=. Remember the use of =unintern= in the previous section? What happens if an imported symbol has the same /name/ as a symbol already accessible in the target package? In mathematical language, let $p_1$ and $p_2$ be two different packages, $st$ some string, and $sm_1$ and $sm_2$ two different symbols, internal in the respective packages $p_1$ and $p_2$, with same name $st$. In other words, assume $\rho(p_1)(st) = sm_1$ and $\rho(p_2)(st) = sm_2$, $sm_1 \in InS(p_1)$ and $sm_2 \in InS(p_2)$, and $sm_1\neq sm_2$. A situation of this sort could be the following (Remember the current default package is ="P1"=!): #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (defvar cl-user::symb4) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (setf cl-user::symb4 111) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (defvar symb4) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (setf symb4 111) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (eq 'cl-user::symb4 'symb4) #+END_SRC The evaluation of the first expression interns a symbol of name ="SYMB4"= in the package ="CL-USER"= and gives it the value 111, while the second expression interns a /different/ symbol, again of name ="SYMB4"=, in the package ="P1"=, the current default package, and also gives it the value 111. As a result two symbols have been allocated, installed somewhere in the environment, with the same value of 111. Also the connections $\rho(\mathtt{"CL-USER"}): \mathtt{"SYMB4"}\mapsto \mathtt{cl-user::symb4}$ and $\rho(\mathtt{"P1"}): \mathtt{"SYMB4"}\mapsto\mathtt{p1::symb4}$ have been installed, that is, the addresses of both symbols have been pushed to the /respective/ lists of /internal/ symbols of the packages ="CL-USER"= and ="P1"=. The =eq= comparison confirms that the symbols just allocated are different. Two symbols with the same name ="SYMB4"= are now present in our environment. This is possible because these symbols are interned in different packages. Each one is accessible via the appropriate package. Both symbols are live and can be freely used: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (setf cl-user::symb4 (+ symb4 cl-user::symb4)) #+END_SRC What happens if we import =cl-user::symb4= into the package ="P1"=? Let's try! #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (ignore-errors (import 'cl-user::symb4 "P1"))) #+END_SRC The importation is rejected, for it would not be possible anymore to coherently define $\rho(\mathtt{"P1"})(\mathtt{"SYMB4"})$. We will see later that another process called /shadowing/ allows the user to suppress the symbol =symb4= already present in ="P1"= and to authorize the importation of the symbol present in ="CL-USER"=. * The function =intern=. **Fact 3:** Let $p$ be a package and $(sm_1,...,sm_n)$ the list of its internal symbols. If $p$ is the default package, any of these symbols can be directly used without the double colon notation ='::'=. If $p$ is not the default package, these symbols can be used thanks to the notation $p::sm_i$. All these internal symbols have different names. Nothing prevents several packages from having the same symbols in their lists of internal symbols. This is possible thanks to the =import= function which can work only if no name conflict is generated.\\ To be complete on this matter, we now to consider the Lisp function [[http://clhs.lisp.se/Body/f_intern.htm][=intern=]]. It is used almost exactly as the function =find-symbol= with just a difference: if no symbol is found, then a symbol is allocated and interned in the package referred to in its second argument. Compare the following: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (find-symbol "SYMB5" "P1")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (intern "SYMB5" "P1")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (intern "SYMB5" "P1")) #+END_SRC We start without any =symb5= symbol in ="P1"=, which is confirmed by the double negative answer of =find-symbol=. If instead we use the function =intern=, Lisp notices that such a symbol is absent from ="P1"= and therefore allocates a symbol of name ="SYMB5"= and /interns/ it in the package ="P1"= as an internal symbol. The second value =nil= is an indicator of the allocation just made. If we repeat the same expression, which is not an error, then Lisp signals that such a symbol is already present and returns its status, which also informs us of its previous presence. A little weird! This definition of the function =intern= guarantees the coherence of the set of internal symbols of packages. A symbol can be allocated only through this function =intern=. It requires two arguments, a string $st$ and a package $p$. In a sense, the function =intern= first calls =(find-symbol st p)= with the same arguments. If a symbol is found it is returned without modification and no allocation of a new symbol takes place, which otherwise would create a name conflict. On the other hand, if a corresponding symbol is not found, it is allocated and interned in the package explicitly or implicitly quoted, and this new symbol certainly does not generate a name conflict. In this way, the uniqueness of the names of internal symbols of a package is enforced. This does not prevent two different symbols of the environment from having the same name, but the packaging system is organized such that they cannot be accessed via the same package. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (package-internal-symbols "P1") #+END_SRC In practice, the Lisp reader constantly uses the =intern= function, for example when reading an expression. If it detects the presence of a symbol, the colon notation allows it to decide if a package is explicitly given, otherwise it must use the default package. The name of the symbol, alone or package-qualified using the colon notation, then unambiguously determines the symbol. If present, it is used, otherwise it is allocated and then used. This is exactly the role of the =intern= function. * Moving a symbol between two packages. Up until now, we have seen no means of modifying the home package of a symbol. In fact, it is possible but may seem a little contorted. It is almost without any practical interest except as an opportunity to use this as a vehicle to discover other facts. First, it is possible to /unintern/ a symbol. For example, we can unintern the symbol ="SYMB1"= from the package ="P1"=. What happens is precisely that this symbol is removed from the list of the internal symbols of ="P1"=. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (unintern 'symb1 "P1") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (package-internal-symbols "P1") #+END_SRC What is modified is the /list/ of internal symbols in the package ="P1"=. The symbol =symb1= itself has not been modified at all, except possibly its home package. In particular the symbol is not deallocated or destroyed, and therefore always present somewhere in the environment. But it's become inaccessible as such, for a symbol most often is accessible only via a package, the default one or some other explicitly given. The symbol =symb1= was interned only in the package ="P1"=, so that it appears now to be unreachable! If a symbol is truly unreachable, the Lisp garbage collector, always silently working in the background, will establish that fact and free the corresponding part of memory for reuse. Except that our symbol remains reachable, for it was the first element of the list assigned to the symbol =my-list=. Let's have a look at its value! #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim my-list #+END_SRC Our symbol is still present in the list but in a little different form. To be sure, it is still a symbol: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (type-of (first my-list)) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (symbol-package (first my-list)) #+END_SRC No home package, i.e., our symbol is "homeless"! The prefix ='#:'= indicates that this symbol is homeless. The =unintern= function had this effect because the package argument of =unintern= was the package which /owned/ the symbol. Otherwise the home package would not be modified. Although our symbol has become homeless, it continues to be an ordinary symbol, for example, we can assign it a value: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (set (first my-list) 444) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (symbol-value (first my-list)) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (eval (first my-list)) #+END_SRC Unlike =setf= (macro), the =set= function evaluates its arguments, so that before the assignment, the expression =(first my-list)= is evaluated, which returns our mysterious symbol =#:symb1=. Once that's done, the =set= function assigns (the value of) the second argument to it, as is confirmed by the other expressions. It is even possible to /import/ a homeless symbol in a package, but then the target package becomes the home package of the imported symbol. The importing package can be different from the original home package. As a result our symbol is "moved" into another package. This explains the new form of the value of =my-list=. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (import (first my-list) "CL-USER") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (symbol-package 'cl-user::symb1) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim my-list #+END_SRC Our symbol could also have been imported in other packages before being uninterned from its home package. The status of our symbol with respect to other packages would not have been modified. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (import 'symb3 "CL-USER") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (eq 'cl-user::symb3 'symb3) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (unintern 'symb3 "P1") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (eq 'cl-user::symb3 (third my-list)) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (import 'cl-user::symb3 "CL-USER") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (symbol-package 'cl-user::symb3) #+END_SRC Here we import =symb3= in the package ="CL-USER"= before uninterning it from the package ="P1"=. The symbol is then accessible via the package ="CL-USER"= or via the list =my-list=. The evaluation of the expression =(eq ...)= proves that both paths lead to the same address. When the first =(import ...)= expression is evaluated, the symbol does have a home package and that it is not changed. When it is uninterned from the package ="P1"=, it becomes homeless. Lisp does not take account of the fact it is internal in other packages. In theory, this would allow it to select one of these packages as an "emergency home" package, but the choice would be quite arbitrary? Furthermore this would force Lisp to examine the list of all other packages and all their accessible symbols, which would be very expensive! Here, despite the symbol being internal and therefore interned in the package ="CL-USER"=, it becomes homeless after evaluating the =(unintern ...)= expression. Finally we import it in the package ="CL-USER"=, where, in fact, it is already present. According to the ANSI standard (see the following note), in such a situation, its status remains unchanged. In particular its home package remains =nil=. A little esoteric, huh? Don’t worry: the home package does not have any real role for the Lisp user. ----- *Note:* The [[http://clhs.lisp.se/Body/f_import.htm#import][passage]] in the [[http://clhs.lisp.se/Front/index.htm][Common Lisp HyperSpec]] reads like this: /"If the symbol is already present in the importing package, import has no effect."/ It appears that the behavior of SBCL and some other compilers is different: The output is =#= not =NIL=. ----- We will prove later that if a symbol $sm$ has a home package $p$, then $sm$ is certainly present in $p$, that is, internal or external in this package. The "converse" is false: a symbol may be present in several packages $p_1,\ldots , p_k$ while not having a home package. This might sound a little weird, but you shouldn't lose much sleep over it: except for the internal efficiency of the Lisp interpeter or compiler, the notion of a home-package is almost useless for the programmer. * External symbols and used packages. The analysis of this notion of an internal symbol was a little lengthy, but, in this way, the general structure of the packaging system is now very clear, and it will be a lot easier to understand the other concepts. Two fundamental lists maintained in the packages play a "dual" role in the packaging system, the list of /external/ symbols and the list of /used/ packages. The following function gives the list of the external symbols of a package. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim *package* #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (defun package-external-symbols (package) (let ((rslt nil)) (do-external-symbols (s package) (push s rslt)) rslt)) #+END_SRC The evaluation of the first expression is just to confirm that the default package remains ="P1"=. Let us see what the lists of external symbols of the packages ="P1"=, ="CL-USER"= and ="COMMON-LISP"= are: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (package-external-symbols "P1") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (package-external-symbols "CL-USER") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (subseq (package-external-symbols "COMMON-LISP") 0 10) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (length (package-external-symbols "COMMON-LISP")) #+END_SRC There are no external symbols in ="P1"= and ="CL-USER"=, but 978 such symbols in the package ="COMMON-LISP"=. (Only the first ten are being displayed.) A point of concern might be the following: these symbols are accessible via the package ="COMMON-LISP"= and yet the double-colon notation =common-lisp::simple-error= was not used. The explanation has two "dual" parts: 1. The symbol =simple-error= is external in the package ="COMMON-LISP"=. 2. The package ="COMMON-LISP"= is in the =package-use-list= of the package ="P1"=: #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (multiple-value-list (find-symbol "SIMPLE-ERROR" "COMMON-LISP")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (package-use-list "P1") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (package-nicknames (find-package "COMMON-LISP")) #+END_SRC ="CL"= is the nickname of the package whose official name is ="COMMON-LISP"=, and which is not to be confused with ="COMMON-LISP-USER"=. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (eq (find-package "CL") (find-package "COMMON-LISP")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (eq (print (find-package "CL")) (print (find-package "CL-USER"))) #+END_SRC Applied to a package $p$, the Lisp function [[http://clhs.lisp.se/Body/f_pkg_us.htm][=package-use-list=]] returns the list of all the packages used by $p$. If $p_1$ uses $p_2$, then it is said that $p_2$ is used by $p_1$. Whether this relation is reflexive can be doubted (What does "$p$ uses $p$" mean?), and it is neither symmetric ("$p_1$ uses $p_2$" does not imply "$p_2$ uses $p_1$") nor transitive ("$p_1$ uses $p_2$" and "$p_2$ uses $p_3$” does not imply "$p_1$ uses $p_3$"). The oriented graph of the /use/ relation is totally arbitrary. * First toy examples. Let's start over with two "fresh" packages ="P1"= and ="P2"=. #+BEGIN_SRC lisp :session pcl :exports both :package "P1" :results value verbatim (in-package "CL-USER") #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (delete-package (find-package "P1")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (defpackage "P1" (:use "CL")) #+END_SRC #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (defpackage "P2" (:use "CL")) #+END_SRC We switch the default package to ="P1"=: #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (in-package "P1") #+END_SRC and we assume that, /in this environment/, we define the functions =package-internal-symbols= and =package-external-symbols= as explained before. #+BEGIN_SRC lisp :session pcl :package "P1" :exports both :results value verbatim (defun package-internal-symbols (package) (let ((rslt)) (do-symbols (s package) (when (eq (second (multiple-value-list (find-symbol (symbol-name s) package))) :internal) (push s rslt))) rslt)) #+END_SRC #+BEGIN_SRC lisp :session pcl :package "P1" :exports both :results value verbatim (defun package-external-symbols (package) (let ((rslt)) (do-external-symbols (s package) (push s rslt)) rslt)) #+END_SRC What about the internal symbols of ="P1"=? #+BEGIN_SRC lisp :session pcl :package "P1" :exports both :results value verbatim (package-internal-symbols "P1") #+END_SRC We see the symbols naming our functions and also the symbols =rslt= and =s= used to define them. All the other symbols used in their definitions are in fact external in ="COMMON-LISP"= and absent here. We move now to the package ="P2"= from where we cannot access the symbol =package-internal-symbols= directly. #+BEGIN_SRC lisp :session pcl :package "P1" :exports both :results value verbatim (in-package "P2") #+END_SRC #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (multiple-value-list (find-symbol "PACKAGE-INTERNAL-SYMBOLS" "P2")) #+END_SRC We have already seen the use of the ='::'= notation to make reference to the symbol at once. We can also /import/ the symbol, in which case it becomes directly accessible from ="P2"= permanently: #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (import 'p1::package-internal-symbols "P2") #+END_SRC #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (package-internal-symbols "P2") #+END_SRC Another solution requires /two/ steps: 1. Make the symbol =external= in some package, which is the role of the Lisp function [[http://clhs.lisp.se/Body/f_export.htm][=export=]]. 2. Declare that the current package /uses/ the package where the symbol is external. Let us use this method to make the symbol =package-external-symbols= directly visible from ="P2"=. #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (export 'p1::package-external-symbols "P1") #+END_SRC #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (use-package "P1" "P2") #+END_SRC #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (package-external-symbols "P2") #+END_SRC #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (package-external-symbols "P1") #+END_SRC An =export= expression has the same form as an =import= one: =(export= $sm$ $st$ =)=, where $sm$ is a symbol and $st$ is a string naming a package. The symbol $sm$ must be readable from the current default package, for us ="P2"=, which explains the (required) prefix ='p1::'=. The argument $st$ is a string naming the package where the symbol $sm$ is to become external. Again, the terminology is somewhat misleading: the function =export= does not export anything. All it does is to make some connection $st \rightarrow sm$ "exportable" to other packages via =use-package= expressions. The 978 external symbols of ="COMMON-LISP"= contain all the standard Lisp constants, variables and functions. Because an allocated package is always initialized with the package ="COMMON-LISP"= in its =package-use-list=, these basic Lisp objects will be accessible from any package. If we remove ="COMMON-LISP"= from the =use-package-list= of the current default package, via =unuse-package=, the situation becomes a little tricky. ----- *Note:* Before setting the saw to the branch on which we are sitting, we have to take one precaution: We can't let go of =LET=. Otherwise the interaction between Emacs and SLIME will cease to function. #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (cl::import 'cl::let "P2") #+END_SRC ----- #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (unuse-package "COMMON-LISP" "P2") #+END_SRC #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (cl::multiple-value-list (cl::ignore-errors (+ 2 2))) #+END_SRC The notation for the external symbol =t= in the package ="COMMON-LISP"= has become =COMMON-LISP:T=, for ="COMMON-LISP"= is no longer used by ="P2"=. Since the symbol =T= is external in ="COMMON-LISP"=, the simple colon notation is sufficient. However, the most elementary operations are not easily accessible anymore, for example, the addition function ='+'=. This is not a good position to be in. Let us go back to the previous environment! #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (cl::multiple-value-list (cl::ignore-errors (use-package "COMMON-LISP" "P2"))) #+END_SRC This does not work since the symbol =use-package= can't be reached easily either, and we /must/ use the prefix notation. #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (cl::multiple-value-list (cl::ignore-errors (cl::use-package "COMMON-LISP" "P2"))) #+END_SRC But here is a new problem: earlier, when when we tried to use the symbol ='+'= from ="P2"=, Lisp detected that this symbol is not accessible from ="P2"= and therefore interned a fresh symbol of name ="+"= in ="P2"=. This was legitimate then, but it is now incompatible with ="COMMON-LISP"= having a different symbol with the same name ="+"=, if ="P2"= were to use ="COMMON-LISP"=. The same goes for =use-package=. We must unintern these symbols from ="P2"= to resolve the conflict. Let's not forget to use the colon notation for =unintern=, which is otherwise unreachable! #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (cl::unintern '+ "P2") #+END_SRC #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (cl::unintern 'use-package "P2") #+END_SRC ----- *Note:* The following is necessary to keep the interaction between Emacs and SLIME going. Otherwise, this would be another conflicting symbol. #+BEGIN_SRC lisp :session pcl :package "P2" :exports both :results value verbatim (cl::unintern '*default-pathname-defaults* "P2") #+END_SRC ----- #+BEGIN_SRC lisp :session pcl :exports both :results value verbatim (cl::use-package "COMMON-LISP" "P2") #+END_SRC Phew, we are back in a state where Lisp is usable! * Importing an external symbol. Recall that a symbol $sm$ is /accessible/ from a package $p$ if $\rho(p)(st) = sm$ for $st = \mathtt{symbol-name}(sm)$. In that case, the possible return values from =find-symbol= are as follows: 1. If $sm$ is one of the internal symbols of $p$ the second value returned by =find-symbol= is =:internal=. 2. If $sm$ is one of the external symbols of $p$ the second value returned by =find-symbol= is =:external=. 3. If $sm$ is an external symbol of a package $p'$ used by the package $p$ the second value returned by =find-symbol= is =:inherited=. We have seen how to import a symbol from a package $p'$ into a package $p$. The example given used a symbol $sm$ internal in the package $p'$, which then became also an internal symbol of the package $p$ . This also works if $sm$ is external in the package $p'$, regardless of $p$ using or not using $p'$. (... assuming that no name conflict is generated.) Let's continue our journey in ="P1"=!