diff --git a/doc/asdf/asdf.texinfo b/doc/asdf/asdf.texinfo index 5ae6161e9..36e978f81 100644 --- a/doc/asdf/asdf.texinfo +++ b/doc/asdf/asdf.texinfo @@ -1,8 +1,8 @@ -\input texinfo @c -*- texinfo -*- +\input texinfo @c -*- mode: texinfo; coding: utf-8 -*- @c %**start of header +@documentencoding UTF-8 @setfilename asdf.info @settitle ASDF Manual -@syncodeindex tp fn @c %**end of header @c We use @&key, etc to escape & from TeX in lambda lists -- @@ -36,11 +36,11 @@ for Common Lisp programs and libraries. You can find the latest version of this manual at @url{https://common-lisp.net/project/asdf/asdf.html}. -ASDF Copyright @copyright{} 2001-2016 Daniel Barlow and contributors. +ASDF Copyright @copyright{} 2001-2019 Daniel Barlow and contributors. -This manual Copyright @copyright{} 2001-2016 Daniel Barlow and contributors. +This manual Copyright @copyright{} 2001-2019 Daniel Barlow and contributors. -This manual revised @copyright{} 2009-2016 Robert P. Goldman and Francois-Rene Rideau. +This manual revised @copyright{} 2009-2019 Robert P. Goldman and Francois-Rene Rideau. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the @@ -65,7 +65,7 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. @titlepage @title ASDF: Another System Definition Facility -@subtitle Manual for Version 3.3.2 +@subtitle Manual for Version 3.3.3 @c The following two commands start the copyright page. @page @vskip 0pt plus 1filll @@ -82,7 +82,7 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. @node Top, Introduction, (dir), (dir) @top ASDF: Another System Definition Facility @ifnottex -Manual for Version 3.3.2 +Manual for Version 3.3.3 @end ifnottex @@ -105,9 +105,9 @@ Manual for Version 3.3.2 * Ongoing Work:: * Bibliography:: * Concept Index:: -* Function and Class Index:: +* Function and Macro Index:: * Variable Index:: @c @detailmenu -@c +* Class and Type Index:: @c @detailmenu --- The Detailed Node Listing --- @@ -281,7 +281,7 @@ ASDF development FAQs @cindex :asdf2 @cindex :asdf3 -ASDF is Another System Definition Facility: +ASDF, or Another System Definition Facility, is a @emph{build system}: a tool for specifying how systems of Common Lisp software are made up of components (sub-systems and files), and how to operate on these components in the right order @@ -308,8 +308,8 @@ it plays a role like @code{make} or @code{ant}, not like a package manager. In particular, ASDF should not to be confused with Quicklisp or ASDF-Install, that attempt to find and download ASDF systems for you. Despite what the name might suggest, -ASDF-Install was never a part of ASDF, it was a separate piece of software. -ASDF-Install is also unmaintained and obsolete. +ASDF-Install was never a part of ASDF; it was always a separate piece of software. +ASDF-Install has also been unmaintained and obsolete for a very long time. We recommend you use Quicklisp (@uref{http://www.quicklisp.org/}) instead, a Common Lisp package manager which works well and is being actively maintained. @@ -934,15 +934,17 @@ ASDF hooks into the @code{cl:require} facility and you can just use: (require :@var{foo}) @end example -Note that the canonical name of a system is a string, conventionally lowercase. -A system name can also be specified as a symbol (including a keyword), -in which case its @code{symbol-name} is taken and lowercased. +Note that the canonical name of a system is a string, in @emph{lowercase}. +System names can also be specified as symbols (including keyword +symbols). +If a symbol is given as argument, its package is ignored, +its @code{symbol-name} is taken, and converted to lowercase. The name must be a suitable value for the @code{:name} initarg to @code{make-pathname} in whatever filesystem the system is to be found. -The lower-casing-symbols behaviour is unconventional, +Using lowercase as canonical is unconventional, but was selected after some consideration. -The type of systems we want to support +The type of file systems we support either have lowercase as customary case (Unix, Mac, Windows) or silently convert lowercase to uppercase (lpns). @c so this makes more sense than attempting to use @code{:case :common}, @@ -1001,18 +1003,13 @@ compiling, loading and testing. @c FIXME: the following is too complicated for here, especially since @c :force hasn't been defined yet. Move it. [2014/02/27:rpg] -@vindex *load-system-operation* @findex already-loaded-systems @defun load-system system @Arest{} keys @Akey{} force force-not verbose version @AallowOtherKeys{} -Apply @code{operate} with the operation from -@code{*load-system-operation*} -the @var{system}, and any provided keyword arguments. -@code{*load-system-operation*} by default is @code{load-op}; -it would be @code{load-bundle-op} by default on ECL, -if only an implementation bug were fixed. -Calling @code{load-system} is the regular, recommended way -to load a system into the current image. +Apply @code{operate} with the operation @code{load-op}, the +@var{system}, and any provided keyword arguments. Calling +@code{load-system} is the regular, recommended way to load a system +into the current image. @end defun @defun compile-system system @Arest{} keys @Akey{} force force-not verbose version @AallowOtherKeys{} @@ -1039,7 +1036,7 @@ The default behaviour is to load the system as if by @code{load-system}; but system authors can override this default in their system definition they may specify an alternate operation as the intended use of their system, with a @code{:build-operation} option in the @code{defsystem} form -(@pxref{The defsystem grammar, build-operation}), +(@pxref{Build-operation}), and an intended output pathname for that operation with @code{:build-pathname}. @c Document :build-operation in the defsystem section. @@ -1192,7 +1189,7 @@ especially if you intend your software to be eventually included in Quicklisp. @item Make sure you know how the @code{:version} numbers will be parsed! Only period-separated non-negative integers are accepted at present. -See below Version specifiers in @ref{The defsystem grammar}. +@xref{Version specifiers}. @item This file contains a single form, the @code{defsystem} declaration. @@ -1273,7 +1270,7 @@ see the discussion of the @code{:pathname} option in @ref{The defsystem grammar} @item The @code{:serial t} says that each sub-component of @code{mod} depends on the previous components, -so that @file{cooker.lisp} depends-on @file{utils.lisp}, which depends-on @file{reader.lisp}. +so that @file{cooker.lisp} depends-on @file{reader.lisp}, which depends-on @file{utils.lisp}. Also @file{data.raw} depends on all of them, but that doesn't matter since it's a static file; on the other hand, if it appeared first, then all the Lisp files would be recompiled when the data is modified, which is probably not what is desired in this case. @@ -1424,11 +1421,19 @@ Presumably, the 4th form looks like @code{(defparameter *foo-version* "5.6.7")}. @example -system-definition := ( defsystem system-designator @var{system-option}* ) +system-definition := ( defsystem system-designator @var{system-option}*) + +system-designator := simple-component-name | complex-component-name + +simple-component-name := lower-case string | symbol + NOTE: Underscores are not permitted. See below. + +complex-component-name := string | symbol + See below. system-option := :defsystem-depends-on system-list | :weakly-depends-on @var{system-list} - | :class class-name (see discussion below) + | :class class-name # @pxref{System class names} | :build-operation @var{operation-name} | system-option | module-option @@ -1441,6 +1446,7 @@ system-option := :homepage string | :long-name string | :source-control source-control | :version version-specifier + | :entry-point object # @pxref{Entry point} source-control := (keyword string) @@ -1468,13 +1474,13 @@ component-def := ( component-type simple-component-name @var{option}* ) component-type := :module | :file | :static-file | other-component-type other-component-type := symbol-by-name - (@pxref{The defsystem grammar,,Component types}) + # @pxref{Component types} # This is used in :depends-on, as opposed to ``dependency,'' # which is used in :in-order-to dependency-def := simple-component-name | ( :feature @var{feature-expression} dependency-def ) - # (@pxref{The defsystem grammar,,Feature dependencies}) + # @pxref{Feature dependencies} | ( :version simple-component-name version-specifier ) | ( :require module-name ) @@ -1489,6 +1495,10 @@ simple-component-name := string | symbol pathname-specifier := pathname | string | symbol + NOTE: pathnames should be all lower case, and have no + underscores, although hyphens are permitted. + + version-specifier := string | (:read-file-form ?) @@ -1497,7 +1507,8 @@ line-specifier := :at integer # base zero form-specifier := :at [ integer | ( integer+ )] method-form := (operation-name qual lambda-list @Arest{} body) -qual := method qualifier? +qual := method-qualifier? +method-qualifier = :before | :after | :around feature-expression := keyword | (:and @var{feature-expression}*) @@ -1505,13 +1516,53 @@ feature-expression := keyword | (:not @var{feature-expression}) @end example +@subsection System designators + +System designators are either simple component names, or +complex (``slashy'') component names. + +@subsection Simple component names (@code{simple-component-name}) + +Simple component names may be written as either strings or symbols. + +When using strings, use lower case exclusively. + +Symbols will be interpreted as convenient shorthand for the string +that is their @code{symbol-name}, converted to lower case. Put +differently, a symbol may be a simple component name @emph{designator}, +but the simple component name itself is the string. -@subsection Component names +@strong{Never} use underscores in component names, whether written as +strings or symbols. -Component names (@code{simple-component-name}) -may be either strings or symbols. +@strong{Never} use slashes (``/'') in simple component names. A slash +indicates a @emph{complex} component name; see below. Using a slash +improperly will cause ASDF to issue a warning. + +Violating these constraints by mixing case, or including underscores in +component names, may lead to systems or components being impossible to +find, because component names are interpreted as file names. These +problems will @emph{definitely} occur for users who have configured ASDF +using logical pathnames. + +@subsection Complex component names + +A complex component name is a master name followed by a slash, followed +by a subsidiary name. This allows programmers to put multiple system +definitions in a single @code{.asd} file, while still allowing ASDF to +find these systems. + +The master name of a complex system @strong{must} be the same as the +name of the @code{.asd} file. + +The file @code{foo.asd} will contain the definition of the system +@code{"foo"}. However, it may @emph{also} contain definitions of +subsidiary systems, such as @code{"foo/test"}, @code{"foo/docs"}, and so +forth. ASDF will ``know'' that if you ask it to load system +@code{"foo/test"} it should look for that system's definition in @code{foo.asd}. @subsection Component types +@anchor{Component types} Component type names, even if expressed as keywords, will be looked up by name in the current package and in the asdf package, if not found in @@ -1523,6 +1574,7 @@ the current package @code{my-system-asd} can be specified as allowed as component types for such children components. @subsection System class names +@anchor {System class names} A system class name will be looked up in the same way as a Component type (see above), @@ -1554,6 +1606,7 @@ system definition. @subsection Build-operation @cindex :build-operation +@anchor{Build-operation} The @code{:build-operation} option to @code{defsystem} allows the programmer to specify an operation that will be applied, in place of @@ -1601,6 +1654,7 @@ this anomalous behaviour may be removed without warning. @subsection Pathname specifiers @cindex pathname specifiers +@anchor{Pathname specifiers} A pathname specifier (@code{pathname-specifier}) may be a pathname, a string or a symbol. @@ -1641,13 +1695,17 @@ If a symbol is given, it will be translated into a string, and downcased in the process. The downcasing of symbols is unconventional, but was selected after some consideration. -Observations suggest that the type of systems we want to support -either have lowercase as customary case (Unix, Mac, windows) +The file systems we support +either have lowercase as customary case (Unix, Mac, Windows) or silently convert lowercase to uppercase (lpns), so this makes more sense than attempting to use @code{:case :common} as argument to @code{make-pathname}, which is reported not to work on some implementations. +Please avoid using underscores in system names, or component (module or +file) names, since underscores are not +compatible with logical pathnames (@pxref{Using logical pathnames}). + Pathname objects may be given to override the path for a component. Such objects are typically specified using reader macros such as @code{#p} or @code{#.(make-pathname ...)}. @@ -1656,7 +1714,7 @@ a shorthand for @code{#.(parse-namestring ...)} and that the behaviour of @code{parse-namestring} is completely non-portable, unless you are using Common Lisp @code{logical-pathname}s, which themselves involve other non-portable behaviour -(@pxref{The defsystem grammar,,Using logical pathnames}, below). +(@pxref{Using logical pathnames}). Pathnames made with @code{#.(make-pathname ...)} can usually be done more easily with the string syntax above. The only case that you really need a pathname object is to override @@ -1681,6 +1739,7 @@ be forced upon you if you were specifying a string. @subsection Version specifiers @cindex version specifiers @cindex :version +@anchor{Version specifiers} Version specifiers are strings to be parsed as period-separated lists of integers. I.e., in the example, @code{"0.2.1"} is to be interpreted, @@ -1695,9 +1754,8 @@ Instead of a string representing the version, the @code{:version} argument can be an expression that is resolved to such a string using the following trivial domain-specific language: in addition to being a literal string, it can be an expression of the form -@code{(:read-file-form [:at )}, -or @code{(:read-file-line [:at -)}. +@code{(:read-file-form [:at ])}, +or @code{(:read-file-line [:at ])}. As the name suggests, the former will be resolved by reading a form in the specified pathname (read as a subpathname of the current system if relative or a unix-namestring), and the latter by reading a line. @@ -1722,13 +1780,14 @@ where significant API incompatibilities are signaled by an increased major numbe Use the implementation's own @code{require} to load the @var{module-name}. -It is good taste to use @code{:if-feature @emph{:implementation-name}} -rather than @code{#+@emph{implementation-name}} +It is good taste to use @code{(:feature @emph{:implementation-name} (:require @var{module-name}))} +rather than @code{#+@emph{implementation-name} (:require @var{module-name})} to only depend on the specified module on the specific implementation that provides it. -@xref{if-feature-option}. +@xref{Feature dependencies}. @subsection Feature dependencies @cindex :feature dependencies +@anchor{Feature dependencies} A feature dependency is of the form @code{(:feature @var{feature-expression} @var{dependency})} @@ -1744,11 +1803,12 @@ definition to be loaded. E.g., one cannot use @code{(:feature :sbcl)} to require that a system only be used on SBCL. Feature dependencies are not to be confused with the obsolete -feature requirement (@pxref{The defsystem grammar,,feature requirement}), or +feature requirement (@pxref{feature requirement}), or with @code{if-feature}. @subsection Using logical pathnames @cindex logical pathnames +@anchor{Using logical pathnames} We do not generally recommend the use of logical pathnames, especially not so to newcomers to Common Lisp. @@ -1880,7 +1940,7 @@ from within an editor without clobbering its source location) @subsection if-feature option @cindex :if-feature component option -@anchor{if-feature-option} @c redo if this ever becomes a node in +@anchor{if-feature option} @c redo if this ever becomes a node in @c its own right... This option allows you to specify a feature expression to be evaluated @@ -1905,9 +1965,17 @@ been performed. This option was added in ASDF 3. For more information, @xref{required-features, Required features}. +@subsection Entry point +@cindex :entry-point +@anchor{Entry point} +The @code{:entry-point} option allows a developer to specify the entry point of an executable program created by @code{program-op}. + +When @code{program-op} is invoked, the form passed to this option is converted to a function by @code{uiop:ensure-function} and bound to @code{uiop:*image-entry-point*}. Typically one will specify a string, e.g. @code{"foo:main"}, so that the executable starts with the @code{foo:main} function. Note that using the symbol @code{foo:main} instead might not work because the @code{foo} package doesn't necessarily exist when ASDF reads the @code{defsystem} form. For more information on @code{program-op}, @pxref{Predefined operations of ASDF}. + @subsection feature requirement +@anchor{feature requirement} This requirement was removed in ASDF 3.1. Please do not use it. In -most cases, @code{:if-feature} (@pxref{if-feature-option}) will provide +most cases, @code{:if-feature} (@pxref{if-feature option}) will provide an adequate substitute. The @code{feature} requirement used to ensure that a chain of component @@ -1947,14 +2015,20 @@ of output from ASDF operations. @node The package-inferred-system extension, , Other code in .asd files, Defining systems with defsystem @section The package-inferred-system extension +@codequoteundirected on +@cindex Package inferred systems +@cindex Packages, inferring dependencies from +@findex package-inferred-system +@cindex One package per file systems Starting with release 3.1.2, ASDF supports a one-package-per-file style of programming, -whereby each file is its own system, +in which each file is its own system, and dependencies are deduced from the @code{defpackage} form -(or its variant @code{uiop:define-package}). +or its variant, @code{uiop:define-package}. -In this style, packages refer to a system with the same name (downcased); +In this style of system definition, package names map to systems with +the same name (in lower case letters), and if a system is defined with @code{:class package-inferred-system}, then system names that start with that name (using the slash @code{/} separator) @@ -1963,23 +2037,26 @@ For instance, if system @code{my-lib} is defined in @file{/foo/bar/my-lib/my-lib.asd}, then system @code{my-lib/src/utility} will be found in file @file{/foo/bar/my-lib/src/utility.lisp}. -This style was made popular by @code{faslpath} and @code{quick-build} before, -and at the cost of a stricter package discipline, -seems to make for more maintainable code. -It is used by ASDF itself (starting with ASDF 3), by @code{lisp-interface-library}, +One package per file style was made popular by @code{faslpath} and @code{quick-build}, +and at the cost of stricter package discipline, +may yield more maintainable code. +This style is used in ASDF itself (starting with ASDF 3), by @code{lisp-interface-library}, and a few other libraries. To use this style, choose a toplevel system name, e.g. @code{my-lib}, -and create a file @file{my-lib.asd} -with the @code{:class :package-inferred-system} option in its @code{defsystem}. +and create a file @file{my-lib.asd}. +Define @code{my-lib} +using the @code{:class :package-inferred-system} option in its @code{defsystem}. For instance: @example -#-asdf3.1 (error "my-lib requires ASDF 3.1") +;; This example is based on lil.asd of LISP-INTERFACE-LIBRARY. + +#-asdf3.1 (error "MY-LIB requires ASDF 3.1 or later.") (defsystem "my-lib" :class :package-inferred-system :depends-on ("my-lib/interface/all" - "my-lib/src/all" - "my-lib/extras/all") + "my-lib/src/all" + "my-lib/extras/all") :in-order-to ((test-op (load-op "my-lib/test/all"))) :perform (test-op (o c) (symbol-call :my-lib/test/all :test-suite))) @@ -1991,43 +2068,84 @@ For instance: (register-system-packages "closer-mop" - '(:c2mop :closer-common-lisp :c2cl :closer-common-lisp-user :c2cl-user)) + '(:c2mop + :closer-common-lisp + :c2cl + :closer-common-lisp-user + :c2cl-user)) @end example -In the code above, the first line checks that we are using ASDF 3.1, -which provides @code{package-inferred-system}. - -The function @code{register-system-packages} has to be called to register -packages used or provided by your system and its components -where the name of the system that provides the package -is not the downcase of the package name. - -Then, file @file{interface/order.lisp} under the @code{lil} hierarchy, -that defines abstract interfaces for order comparisons, -starts with the following form, -dependencies being trivially computed from the @code{:use} and @code{:mix} clauses: +@noindent +In the code above, the first form checks that we are using ASDF 3.1 or +later, which provides @code{package-inferred-system}. This is probably +no longer necessary, since none of the major lisp implementations +provides an older version of ASDF. + +@findex register-system-packages +The function @code{register-system-packages} must be called to register +packages used or provided by your system +when the name of the system/file that provides the package +is not the same as the package name (converted to lower case). + +Each file under the @code{my-lib} hierarchy will start with a +package definition. +@findex define-package +@findex uiop:define-package +The form @code{uiop:define-package} is supported as well as +@code{defpackage}. +ASDF will compute dependencies from the +@code{:use}, @code{:mix}, and other importation clauses of this package definition. Take the file +@file{interface/order.lisp} as an example: @example -(uiop:define-package :lil/interface/order +(uiop:define-package :my-lib/interface/order (:use :closer-common-lisp - :lil/interface/definition - :lil/interface/base - :lil/interface/eq :lil/interface/group) + :my-lib/interface/definition + :my-lib/interface/base) (:mix :fare-utils :uiop :alexandria) (:export ...)) @end example -ASDF can tell that this file depends on system @code{closer-mop} (registered above), -@code{lil/interface/definition}, @code{lil/interface/base}, -@code{lil/interface/eq}, and @code{lil/interface/group} -(package and system names match, and they will be looked up hierarchically). +ASDF can tell that this file/system depends on system @code{closer-mop} (registered above), +@code{my-lib/interface/definition}, and @code{my-lib/interface/base}. + +How can ASDF find the file @file{interface/order.lisp} from the +toplevel system @code{my-lib}, however? In the example above, +@file{interface/all.lisp} (and other @file{all.lisp}) reexport +all the symbols exported from the packages at the same or lower levels +of the hierarchy. This can be easily done with +@code{uiop:define-package}, which has many options that prove useful in this +context. For example: + +@example +(uiop:define-package :my-lib/interface/all + (:nicknames :my-lib-interface) + (:use :closer-common-lisp) + (:mix :fare-utils :uiop :alexandria) + (:use-reexport + :my-lib/interface/definition + :my-lib/interface/base + :my-lib/interface/order + :my-lib/interface/monad/continuation)) +@end example + +Thus the top level system need only depend on the @code{my-lib/.../all} systems +because ASDF detects +@file{interface/order.lisp} and all other dependencies from @code{all} +systems' @code{:use-reexport} clauses, which effectively +allow for ``inheritance'' of symbols being exported. ASDF also detects dependencies from @code{:import-from} clauses. -You may thus import a well-defined set of symbols from an existing package -as loaded from suitably named system; -or if you prefer to use any such symbol fully qualified by a package prefix, +You may thus import a well-defined set of symbols from an existing +package, and ASDF will know to load the system that provides that +package. In the following example, ASDF will infer that the current +system depends on @code{foo/baz} from the first @code{:import-from}. +If you prefer to use any such symbol fully qualified by a package prefix, you may declare a dependency on such a package and its corresponding system -via an @code{:import-from} clause with an empty list of symbols, as in: +via an @code{:import-from} clause with an empty list of symbols. For +example, if we preferred to use the name `foo/quux:bletch`, the second, +empty, @code{:import-from} form would cause ASDF to load +@code{foo/quux}. @example (defpackage :foo/bar @@ -2037,19 +2155,15 @@ via an @code{:import-from} clause with an empty list of symbols, as in: (:export ...)) @end example -The form @code{uiop:define-package} is supported as well as @code{defpackage}, -and has many options that prove useful in this context, -such as @code{:use-reexport} and @code{:mix-reexport} -that allow for ``inheritance'' of symbols being exported. - Note that starting with ASDF 3.1.5.6 only, ASDF will look for source files under -the @code{component-pathname} as specified via the @code{:pathname} option, +the @code{component-pathname} (specified via the @code{:pathname} option), whereas earlier versions ignore this option and use the @code{system-source-directory} where the @file{.asd} file resides. @c See this blog post about it: @c @url{http://davazp.net/2014/11/26/modern-library-with-asdf-and-package-inferred-system.html} +@codequoteundirected off @node The object model of ASDF, Controlling where ASDF searches for systems, Defining systems with defsystem, Top @comment node-name, next, previous, up @@ -2157,8 +2271,8 @@ discuss these in turn below. Operations are invoked on systems via @code{operate}. @anchor{operate} -@deffn {Generic function} @code{operate} @var{operation} @var{component} @Arest{} @var{initargs} @Akey{} @code{force} @code{force-not} @code{verbose} @AallowOtherKeys -@deffnx {Generic function} @code{oos} @var{operation} @var{component} @Arest{} @var{initargs} @Akey{} @AallowOtherKeys{} +@deffn {Generic function} operate @var{operation} @var{component} @Arest{} @var{initargs} @Akey{} @code{force} @code{force-not} @code{verbose} @AallowOtherKeys +@deffnx {Generic function} oos @var{operation} @var{component} @Arest{} @var{initargs} @Akey{} @AallowOtherKeys{} @code{operate} invokes @var{operation} on @var{system}. @code{oos} is a synonym for @code{operate} (it stands for operate-on-system). @@ -2210,7 +2324,7 @@ To see what @code{operate} would do, you can use: @end deffn -@defun @code{make-operation} @var{operation-class} @Arest{} @var{initargs} +@defun make-operation @var{operation-class} @Arest{} @var{initargs} @anchor{make-operation} The @var{initargs} are passed to @code{make-instance} call @@ -2229,8 +2343,6 @@ instances will cause a run-time error. @node Predefined operations of ASDF, Creating new operations, Operations, Operations @comment node-name, next, previous, up @subsection Predefined operations of ASDF -@c FIXME: All these deffn's should be replaced with deftyp. Also, we -@c should set up an appropriate index. All the operations described in this section are in the @code{asdf} package. They are invoked via the @code{operate} generic function. @@ -2239,7 +2351,7 @@ They are invoked via the @code{operate} generic function. (asdf:operate 'asdf:@var{operation-name} :@var{system-name} @{@var{operation-options ...}@}) @end lisp -@deffn Operation @code{compile-op} +@deftp Operation compile-op This operation compiles the specified component. A @code{cl-source-file} will be @code{compile-file}'d. @@ -2254,9 +2366,9 @@ all these dependencies will be loaded as well as compiled; yet, some parts of the system main remain unloaded, because nothing depends on them. Use @code{load-op} to load a system. -@end deffn +@end deftp -@deffn Operation @code{load-op} +@deftp Operation load-op This operation loads the compiled code for a specified component. A @code{cl-source-file} will have its compiled fasl @code{load}ed, @@ -2267,27 +2379,28 @@ which fasl is the output of @code{compile-op} that @code{load-op} depends on. @code{load-op} also depends on @code{prepare-op} which itself depends on a @code{load-op} of all of a component's dependencies, as well as of its parent's dependencies. -@end deffn +@end deftp -@deffn Operation @code{prepare-op} +@deftp Operation prepare-op This operation ensures that the dependencies of a component and its recursive parents are loaded (as per @code{load-op}), as a prerequisite before @code{compile-op} and @code{load-op} operations may be performed on a given component. -@end deffn +@end deftp -@deffn Operation @code{load-source-op}, @code{prepare-source-op} +@deftp Operation load-source-op +@deftpx Operation prepare-source-op @code{load-source-op} will load the source for the files in a module rather than the compiled fasl output. It has a @code{prepare-source-op} analog to @code{prepare-op}, that ensures the dependencies are themselves loaded via @code{load-source-op}. -@end deffn +@end deftp @anchor{test-op} -@deffn Operation @code{test-op} +@deftp Operation test-op This operation will perform some tests on the module. The default method will do nothing. @@ -2335,11 +2448,22 @@ Then one defines @code{perform} methods on ...) @end lisp -@end deffn +@end deftp -@deffn Operation @code{compile-bundle-op}, @code{monolithic-compile-bundle-op}, @code{load-bundle-op}, @code{monolithic-load-bundle-op}, @code{deliver-asd-op}, @code{monolithic-deliver-asd-op}, @code{lib-op}, @code{monolithic-lib-op}, @code{dll-op}, @code{monolithic-dll-op}, @code{image-op}, @code{program-op} +@deftp Operation compile-bundle-op +@deftpx Operation monolithic-compile-bundle-op +@deftpx Operation load-bundle-op +@deftpx Operation monolithic-load-bundle-op +@deftpx Operation deliver-asd-op +@deftpx Operation monolithic-deliver-asd-op +@deftpx Operation lib-op +@deftpx Operation monolithic-lib-op +@deftpx Operation dll-op +@deftpx Operation monolithic-dll-op +@deftpx Operation image-op +@deftpx Operation program-op These are ``bundle'' operations, that can create a single-file ``bundle'' for all the contents of each system in an application, @@ -2347,12 +2471,12 @@ or for the entire application. @code{compile-bundle-op} will create a single fasl file for each of the systems needed, grouping all its many fasls in one, -so you can deliver each system as a single fasl +so you can deliver each system as a single fasl. @code{monolithic-compile-bundle-op} will create a single fasl file for the target system and all its dependencies, so you can deliver your entire application as a single fasl. @code{load-bundle-op} will load the output of @code{compile-bundle-op}. -Note that if it the output is not up-to-date, +Note that if the output is not up-to-date, @code{compile-bundle-op} may load the intermediate fasls as a side-effect. Bundling fasls together matters a lot on ECL, where the dynamic linking involved in loading tens of individual fasls @@ -2408,7 +2532,7 @@ for building a linkable @file{.a} file (Windows: @file{.lib}) from all linkable object dependencies (FFI files, and on ECL, Lisp files too), and its monolithic equivalent @code{monolithic-lib-op}. And there is also @code{dll-op} -(respectively its monolithic equivalent @code{monolithic-lib-op}) +(respectively its monolithic equivalent @code{monolithic-dll-op}) for building a linkable @file{.so} file (Windows: @file{.dll}, MacOS X: @file{.dynlib}) to create a single dynamic library @@ -2428,9 +2552,16 @@ unless the operation is equal to the @code{:build-operation} argument to @code{defsystem}. This behaviour is not very satisfactory and may change in the future. Maybe you have suggestions on how to better configure it? -@end deffn +@end deftp -@deffn Operation @code{concatenate-source-op}, @code{monolithic-concatenate-source-op}, @code{load-concatenated-source-op}, @code{compile-concatenated-source-op}, @code{load-compiled-concatenated-source-op}, @code{monolithic-load-concatenated-source-op}, @code{monolithic-compile-concatenated-source-op}, @code{monolithic-load-compiled-concatenated-source-op} +@deftp Operation concatenate-source-op +@deftpx Operation monolithic-concatenate-source-op +@deftpx Operation load-concatenated-source-op +@deftpx Operation compile-concatenated-source-op +@deftpx Operation load-compiled-concatenated-source-op +@deftpx Operation monolithic-load-concatenated-source-op +@deftpx Operation monolithic-compile-concatenated-source-op +@deftpx Operation monolithic-load-compiled-concatenated-source-op These operations, as their respective names indicate, will concatenate all the @code{cl-source-file} source files in a system @@ -2446,7 +2577,7 @@ ASDF itself is delivered as a single source file this way, using @code{monolithic-concatenate-source-op}, prepending a prelude and the @code{uiop} library before the @code{asdf/defsystem} system itself. -@end deffn +@end deftp @node Creating new operations, , Predefined operations of ASDF, Operations @@ -2833,7 +2964,7 @@ performed by the exported function @code{coerce-name}. Unless overridden by a @code{:pathname} attribute, the name will be interpreted as a pathname specifier according to a Unix-style syntax. -@xref{The defsystem grammar,,Pathname specifiers}. +@xref{Pathname specifiers}. @subsubsection Version identifier @findex version-satisfies @@ -2841,8 +2972,7 @@ to a Unix-style syntax. This optional attribute specifies a version for the current component. The version should typically be a string of integers separated by dots, -for example @samp{1.0.11}. -For more information on version specifiers, see @ref{The defsystem grammar}. +for example @samp{1.0.11}. @xref{Version specifiers}. A version may then be queried by the generic function @code{version-satisfies}, to see if @code{:version} dependencies are satisfied, @@ -2888,7 +3018,7 @@ when the implementation is SBCL, CMUCL or Scieneer CL on an x86 machine. You cannot write it as @code{:if-feature (and x86 (or sbcl cmu scl))} since the symbols would not be read as keywords. -@xref{if-feature-option}. +@xref{if-feature option}. @subsubsection Dependencies @@ -2987,7 +3117,7 @@ in the defsystem grammar. This attribute is optional and if absent (which is the usual case), the component name will be used. -@xref{The defsystem grammar,,Pathname specifiers}, +@xref{Pathname specifiers}, for an explanation of how this attribute is interpreted. Note that the @code{defsystem} macro (used to create a ``top-level'' system) @@ -3045,7 +3175,7 @@ so use symbols or keywords or something. @comment node-name, next, previous, up @subsection Pre-defined subclasses of component -@deffn Component source-file +@deftp Component source-file A source file is any file that the system does not know how to generate from other components of the system. @@ -3062,9 +3192,9 @@ these do count as source files for our purposes. Subclasses of source-file exist for various languages. @emph{FIXME: describe these.} -@end deffn +@end deftp -@deffn Component module +@deftp Component module A module is a collection of sub-components. @@ -3082,7 +3212,7 @@ are inferred to be of this type. @item @code{:if-component-dep-fails} This attribute was removed in ASDF 3. Do not use it. -Use @code{:if-feature} instead (@pxref{required-features}, and @pxref{if-feature-option}). +Use @code{:if-feature} instead (@pxref{required-features}, and @pxref{if-feature option}). @item @code{:serial} When this attribute is set, @@ -3097,9 +3227,9 @@ most operations will not need to provide methods specialised on modules. @code{module} may be subclassed to represent components such as foreign-language linked libraries or archive files. -@end deffn +@end deftp -@deffn Component system +@deftp Component system @code{system} is a subclass of @code{module}. @@ -3109,7 +3239,7 @@ purposes; these are given elsewhere. Users can create new classes for their systems: the default @code{defsystem} macro takes a @code{:class} keyword argument. -@end deffn +@end deftp @node Creating new component types, , Pre-defined subclasses of component, Components @comment node-name, next, previous, up @@ -3209,7 +3339,7 @@ Therefore @code{"1.9.1"}, @code{"1.9.2"} and @code{"1.10"} all satisfy @code{"1. but @code{"1.8.4"} or @code{"1.9"} do not. For more information about how @code{version-satisfies} parses and interprets version strings and specifications, -@pxref{The defsystem grammar} (version specifiers) and +@pxref{Version specifiers} and @ref{Common attributes of components}. Note that in versions of ASDF prior to 3.0.1, @@ -3651,19 +3781,20 @@ if it begins with a paren @code{(}, otherwise it will be interpreted much like @code{TEXINPUTS}, as a list of paths, where - * paths are separated - by a @code{:} (colon) on Unix platforms (including cygwin), - by a @code{;} (semicolon) on other platforms (mainly, Windows). +@itemize +@item paths are separated by a @code{:} (colon) on Unix platforms + (including cygwin), by a @code{;} (semicolon) on other platforms + (mainly, Windows). - * each entry is a directory to add to the search path. +@item each entry is a directory to add to the search path. - * if the entry ends with a double slash @code{//} - then it instead indicates a tree in the subdirectories - of which to recurse. +@item if the entry ends with a double slash @code{//} then it instead + indicates a tree in the subdirectories of which to recurse. - * if the entry is the empty string (which may only appear once), - then it indicates that the inherited configuration should be - spliced there. +@item if the entry is the empty string (which may only appear once), + then it indicates that the inherited configuration should be spliced + there. +@end itemize @node Search Algorithm, Caching Results, Shell-friendly syntax for configuration, Controlling where ASDF searches for systems @section Search Algorithm @@ -4236,7 +4367,7 @@ An @code{:inherit-configuration} statement causes the search to recurse with the specifications from the next configuration in the bulleted list. @xref{Controlling where ASDF saves compiled files,,Configurations}, above. -@vindex @code{asdf::*user-cache*} +@vindex asdf::*user-cache* @itemize @item @code{:enable-user-cache} is the same as @code{(t :user-cache)}. @@ -4454,17 +4585,17 @@ But so are good design ideas and elegant implementation tricks. @node Error handling, Miscellaneous additional functionality, Controlling where ASDF saves compiled files, Top @comment node-name, next, previous, up @chapter Error handling -@findex SYSTEM-DEFINITION-ERROR -@findex OPERATION-ERROR +@tindex system-definition-error +@tindex operation-error @section ASDF errors If ASDF detects an incorrect system definition, it will signal a generalised instance of -@code{SYSTEM-DEFINITION-ERROR}. +@code{system-definition-error}. Operations may go wrong (for example when source files contain errors). These are signalled using generalised instances of -@code{OPERATION-ERROR}. +@code{operation-error}. @section Compilation error and warning handling @vindex *compile-file-warnings-behaviour* @@ -4718,7 +4849,7 @@ It takes two mandatory arguments @var{system} and @var{name} and a keyword argument @var{type}: @var{system} is name of a system, whereas @var{name} and optionally @var{type} specify a relative pathname, interpreted like a component pathname specifier -by @code{coerce-pathname}. @xref{The defsystem grammar,,Pathname specifiers}. +by @code{coerce-pathname}. @xref{Pathname specifiers}. It returns a pathname built from the location of the system's source directory and the relative pathname. For example: @@ -5262,7 +5393,7 @@ where you must now use an explicit pathname object: must now be written with the @code{#p} syntax: @code{(defsystem ... :pathname #p"LOGICAL-HOST:PATH;TO;SYSTEM;" ...)} -@xref{The defsystem grammar,,Pathname specifiers}. +@xref{Pathname specifiers}. @node Output translations, Source Registry Configuration, ASDF can portably name files in subdirectories, What has changed between ASDF 1 ASDF 2 and ASDF 3? @@ -6289,6 +6420,29 @@ Also, bugs are currently tracked on launchpad: @unnumbered Bibliography @itemize +@item Andrey Mokhov, Neil Mitchell and Simon Peyton Jones: +``Build Systems à la Carte'', 2018. + @url{https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems-final.pdf} + This article provides axes along which to describe build systems in general; + ASDF, in addition to being in-image (an axis not considered by these authors), + has the following characteristics: + ASDF's persistent build information is file modification times + (the way ASDF is written, it should be easy enough to write an extension that modifies it + to use a ``cloud cache'' à la Bazel, + but that would involve using some database, network and cryptographic libraries, + which cannot reasonably be included in the base ASDF, + that must remain a minimal bootstrappable system with no external dependencies). + The object model of ASDF was initially designed for ``static'' dependencies + with a ``topological'' scheduler, but its @code{defsystem-depends-on} mechanism + (and more generally, the ability to call ASDF from within an @code{.asd} file) + allows for multiple @emph{phases} of execution resulting + in ``dynamic'' dependencies with a ``suspending'' scheduler. + The rebuilder essentially uses a ``dirty bit'', except that the in-image model + and the multiple phase support mean that's actually more than a bit: + instead three bits, the timestamp and a phase depth level. + The build is guaranteed ``minimal'' in number of steps computed. + It is local; it assumes but does not enforce determinism; + it does not assume early cutoff of the build when rebuild dependencies didn't change. @item Francois-Rene Rideau: ``ASDF 3, or Why Lisp is Now an Acceptable Scripting Language'', 2014. This article describes the innovations in ASDF 3 and 3.1, @@ -6304,15 +6458,16 @@ Also, bugs are currently tracked on launchpad: modulo a different setup for toplevel hierarchies. @item Zach Beane: ``Quicklisp'', 2011. - The Quicklisp blog and Xach's livejournal contain information on Quicklisp. + The Quicklisp blog and Xach's personal blogs contain information on Quicklisp. @url{http://blog.quicklisp.org/} - @url{http://xach.livejournal.com/} + @url{http://lispblog.xach.com/} (new) + @url{http://xach.livejournal.com/} (old) @item Francois-Rene Rideau and Robert Goldman: ``Evolving ASDF: More Cooperation, Less Coordination'', 2010. This article describes the main issues solved by ASDF 2, and exposes its design principles. - @url{https://common-lisp.net/project/asdf/doc/ilc2010draft.pdf} - @url{https://gitlab.common-lisp.org/asdf/ilc2010} + @url{https://common-lisp.net/project/asdf/ilc2010draft.pdf} + @url{http://rpgoldman.goldman-tribe.org/papers/ilc2010-asdf.pdf} @item Francois-Rene Rideau and Spencer Brody: ``XCVB: an eXtensible Component Verifier and Builder for Common Lisp'', 2009. This article describes XCVB, a proposed competitor for ASDF; @@ -6342,7 +6497,7 @@ Also, bugs are currently tracked on launchpad: are available in the CLOCC repository as @code{defsystem-3.x}. @item Richard Elliot Robbins: ``BUILD: A Tool for Maintaining Consistency in Modular Systems'', MIT AI TR 874, 1985. - @url{ftp://publications.ai.mit.edu/ai-publications/pdf/AITR-874.pdf} + @url{http://www.dtic.mil/dtic/tr/fulltext/u2/a162744.pdf} @item Kent M. Pitman (kmp): ``The Description of Large Systems'', MIT AI Memo 801, 1984. Available in updated-for-CL form on the web at @url{http://nhplace.com/kent/Papers/Large-Systems.html} @@ -6354,21 +6509,26 @@ Also, bugs are currently tracked on launchpad: @end itemize -@node Concept Index, Function and Class Index, Bibliography, Top +@node Concept Index, Function and Macro Index, Bibliography, Top @unnumbered Concept Index @printindex cp -@node Function and Class Index, Variable Index, Concept Index, Top -@unnumbered Function and Class Index +@node Function and Macro Index, Variable Index, Concept Index, Top +@unnumbered Function and Macro Index @printindex fn -@node Variable Index, , Function and Class Index, Top +@node Variable Index, Class and Type Index, Function and Macro Index, Top @unnumbered Variable Index @printindex vr +@node Class and Type Index, , Variable Index, Top +@unnumbered Class and Type Index + +@printindex tp + @bye @c LocalWords: clbuild tarballs defsystem Quicklisp initarg uiop fasl diff --git a/src/org/armedbear/lisp/asdf.lisp b/src/org/armedbear/lisp/asdf.lisp index 19067bec0..f5f73eb7f 100644 --- a/src/org/armedbear/lisp/asdf.lisp +++ b/src/org/armedbear/lisp/asdf.lisp @@ -1,5 +1,5 @@ ;;; -*- mode: Lisp; Base: 10 ; Syntax: ANSI-Common-Lisp ; buffer-read-only: t; -*- -;;; This is ASDF 3.3.2: Another System Definition Facility. +;;; This is ASDF 3.3.3: Another System Definition Facility. ;;; ;;; Feedback, bug reports, and patches are all welcome: ;;; please mail to . @@ -19,7 +19,7 @@ ;;; http://www.opensource.org/licenses/mit-license.html on or about ;;; Monday; July 13, 2009) ;;; -;;; Copyright (c) 2001-2016 Daniel Barlow and contributors +;;; Copyright (c) 2001-2019 Daniel Barlow and contributors ;;; ;;; Permission is hereby granted, free of charge, to any person obtaining ;;; a copy of this software and associated documentation files (the @@ -45,6 +45,17 @@ ;;; The problem with writing a defsystem replacement is bootstrapping: ;;; we can't use defsystem to compile it. Hence, all in one file. +#+genera +(eval-when (:compile-toplevel :load-toplevel :execute) + (multiple-value-bind (system-major system-minor) + (sct:get-system-version) + (multiple-value-bind (is-major is-minor) + (sct:get-system-version "Intel-Support") + (unless (or (> system-major 452) + (and is-major + (or (> is-major 3) + (and (= is-major 3) (> is-minor 86))))) + (error "ASDF requires either System 453 or later or Intel Support 3.87 or later"))))) ;;;; --------------------------------------------------------------------------- ;;;; Handle ASDF package upgrade, including implementation-dependent magic. ;; @@ -818,10 +829,10 @@ UNINTERN -- Remove symbols here from PACKAGE." ;;;; Early meta-level tweaks -#+(or allegro clasp clisp clozure cmucl ecl mezzano mkcl sbcl) +#+(or allegro clasp clisp clozure cmucl ecl lispworks mezzano mkcl sbcl) (eval-when (:load-toplevel :compile-toplevel :execute) (when (and #+allegro (member :ics *features*) - #+(or clasp clisp cmucl ecl mkcl) (member :unicode *features*) + #+(or clasp clisp cmucl ecl lispworks mkcl) (member :unicode *features*) #+clozure (member :openmcl-unicode-strings *features*) #+sbcl (member :sb-unicode *features*)) ;; Check for unicode at runtime, so that a hypothetical FASL compiled with unicode @@ -1043,7 +1054,9 @@ Return a string made of the parts not omitted or emitted by FROB." #:simple-style-warning #:style-warn ;; simple style warnings #:match-condition-p #:match-any-condition-p ;; conditions #:call-with-muffled-conditions #:with-muffled-conditions - #:not-implemented-error #:parameter-error)) + #:not-implemented-error #:parameter-error + #:symbol-test-to-feature-expression + #:boolean-to-feature-expression)) (in-package :uiop/utility) ;;;; Defining functions in a way compatible with hot-upgrade: @@ -1089,17 +1102,17 @@ to supersede any previous definition." ;;; Magic debugging help. See contrib/debug.lisp (with-upgradability () (defvar *uiop-debug-utility* - '(or (ignore-errors - (probe-file (symbol-call :asdf :system-relative-pathname :uiop "contrib/debug.lisp"))) - (probe-file (symbol-call :uiop/pathname :subpathname - (user-homedir-pathname) "common-lisp/asdf/uiop/contrib/debug.lisp"))) + '(symbol-call :uiop :subpathname (symbol-call :uiop :uiop-directory) "contrib/debug.lisp") "form that evaluates to the pathname to your favorite debugging utilities") (defmacro uiop-debug (&rest keys) + "Load the UIOP debug utility at compile-time as well as runtime" `(eval-when (:compile-toplevel :load-toplevel :execute) (load-uiop-debug-utility ,@keys))) (defun load-uiop-debug-utility (&key package utility-file) + "Load the UIOP debug utility in given PACKAGE (default *PACKAGE*). +Beware: The utility is located by EVAL'uating the UTILITY-FILE form (default *UIOP-DEBUG-UTILITY*)." (let* ((*package* (if package (find-package package) *package*)) (keyword (read-from-string (format nil ":DBG-~:@(~A~)" (package-name *package*))))) @@ -1658,6 +1671,18 @@ message, that takes the functionality as its first argument (that can be skipped :format-control format-control :format-arguments format-arguments))) +(with-upgradability () + (defun boolean-to-feature-expression (value) + "Converts a boolean VALUE to a form suitable for testing with #+." + (if value + '(:and) + '(:or))) + + (defun symbol-test-to-feature-expression (name package) + "Check if a symbol with a given NAME exists in PACKAGE and returns a +form suitable for testing with #+." + (boolean-to-feature-expression + (find-symbol* name package nil)))) (uiop/package:define-package :uiop/version (:recycle :uiop/version :uiop/utility :asdf) (:use :uiop/common-lisp :uiop/package :uiop/utility) @@ -1672,7 +1697,7 @@ message, that takes the functionality as its first argument (that can be skipped (in-package :uiop/version) (with-upgradability () - (defparameter *uiop-version* "3.3.2") + (defparameter *uiop-version* "3.3.3") (defun unparse-version (version-list) "From a parsed version (a list of natural numbers), compute the version string" @@ -2335,8 +2360,8 @@ by the underlying implementation's MAKE-PATHNAME and other primitives" ;; See CLHS make-pathname and 19.2.2.2.3. ;; This will be :unspecific if supported, or NIL if not. (defparameter *unspecific-pathname-type* - #+(or abcl allegro clozure cmucl genera lispworks sbcl scl) :unspecific - #+(or clasp clisp ecl mkcl gcl xcl #|These haven't been tested:|# cormanlisp mcl mezzano) nil + #+(or abcl allegro clozure cmucl lispworks sbcl scl) :unspecific + #+(or genera clasp clisp ecl mkcl gcl xcl #|These haven't been tested:|# cormanlisp mcl mezzano) nil "Unspecific type component to use with the underlying implementation's MAKE-PATHNAME") (defun make-pathname* (&rest keys &key directory host device name type version defaults @@ -2574,7 +2599,14 @@ actually-existing directory." (make-pathname :directory (append (or (normalize-pathname-directory-component (pathname-directory pathspec)) (list :relative)) - (list (file-namestring pathspec))) + (list #-genera (file-namestring pathspec) + ;; On Genera's native filesystem (LMFS), + ;; directories have a type and version + ;; which must be ignored when converting + ;; to a directory pathname + #+genera (if (typep pathspec 'fs:lmfs-pathname) + (pathname-name pathspec) + (file-namestring pathspec)))) :name nil :type nil :version nil :defaults pathspec) (error (c) (call-function on-error (compatfmt "~@") pathspec c))))))) @@ -3056,7 +3088,13 @@ a CL pathname satisfying all the specified constraints as per ENSURE-PATHNAME" (or (ignore-errors (truename p)) ;; this is here because trying to find the truename of a directory pathname WITHOUT supplying ;; a trailing directory separator, causes an error on some lisps. - #+(or clisp gcl) (if-let (d (ensure-directory-pathname p nil)) (ignore-errors (truename d))))))) + #+(or clisp gcl) (if-let (d (ensure-directory-pathname p nil)) (ignore-errors (truename d))) + ;; On Genera, truename of a directory pathname will probably fail as Genera + ;; will merge in a filename/type/version from *default-pathname-defaults* and + ;; will try to get the truename of a file that probably doesn't exist. + #+genera (when (directory-pathname-p p) + (let ((d (scl:send p :directory-pathname-as-file))) + (ensure-directory-pathname (ignore-errors (truename d)) nil))))))) (defun safe-file-write-date (pathname) "Safe variant of FILE-WRITE-DATE that may return NIL rather than raise an error." @@ -4832,7 +4870,6 @@ or COMPRESSION on SBCL, and APPLICATION-TYPE on SBCL/Windows." (shell-boolean-exit (restore-image)))))))) (when forms `(progn ,@forms)))))) - #+(or clasp ecl mkcl) (check-type kind (member :dll :shared-library :lib :static-library :fasl :fasb :program)) (apply #+clasp 'cmp:builder #+clasp kind @@ -5209,12 +5246,28 @@ Simple means made of symbols, numbers, characters, simple-strings, pathnames, co (sb-c::undefined-warning-kind warning) (sb-c::undefined-warning-name warning) (sb-c::undefined-warning-count warning) + ;; the COMPILER-ERROR-CONTEXT struct has changed in SBCL, which means how we + ;; handle deferred warnings must change... TODO: when enough time has + ;; gone by, just assume all versions of SBCL are adequately + ;; up-to-date, and cut this material.[2018/05/30:rpg] (mapcar #'(lambda (frob) ;; the lexenv slot can be ignored for reporting purposes - `(:enclosing-source ,(sb-c::compiler-error-context-enclosing-source frob) - :source ,(sb-c::compiler-error-context-source frob) - :original-source ,(sb-c::compiler-error-context-original-source frob) + `( + #- #.(uiop/utility:symbol-test-to-feature-expression '#:compiler-error-context-%source '#:sb-c) + ,@`(:enclosing-source + ,(sb-c::compiler-error-context-enclosing-source frob) + :source + ,(sb-c::compiler-error-context-source frob) + :original-source + ,(sb-c::compiler-error-context-original-source frob)) + #+ #.(uiop/utility:symbol-test-to-feature-expression '#:compiler-error-context-%source '#:sb-c) + ,@ `(:%enclosing-source + ,(sb-c::compiler-error-context-enclosing-source frob) + :%source + ,(sb-c::compiler-error-context-source frob) + :original-form + ,(sb-c::compiler-error-context-original-form frob)) :context ,(sb-c::compiler-error-context-context frob) :file-name ,(sb-c::compiler-error-context-file-name frob) ; a pathname :file-position ,(sb-c::compiler-error-context-file-position frob) ; an integer @@ -5565,9 +5618,10 @@ it will filter them appropriately." (with-saved-deferred-warnings (warnings-file :source-namestring (namestring input-file)) (with-muffled-compiler-conditions () (or #-(or clasp ecl mkcl) - (apply 'compile-file input-file :output-file tmp-file - #+sbcl (if emit-cfasl (list* :emit-cfasl tmp-cfasl keywords) keywords) - #-sbcl keywords) + (let (#+genera (si:*common-lisp-syntax-is-ansi-common-lisp* t)) + (apply 'compile-file input-file :output-file tmp-file + #+sbcl (if emit-cfasl (list* :emit-cfasl tmp-cfasl keywords) keywords) + #-sbcl keywords)) #+ecl (apply 'compile-file input-file :output-file (if object-file (list* object-file :system-p t keywords) @@ -5619,19 +5673,20 @@ it will filter them appropriately." (defun load* (x &rest keys &key &allow-other-keys) "Portable wrapper around LOAD that properly handles loading from a stream." (with-muffled-loader-conditions () - (etypecase x - ((or pathname string #-(or allegro clozure genera) stream #+clozure file-stream) - (apply 'load x keys)) - ;; Genera can't load from a string-input-stream - ;; ClozureCL 1.6 can only load from file input stream - ;; Allegro 5, I don't remember but it must have been broken when I tested. - #+(or allegro clozure genera) - (stream ;; make do this way - (let ((*package* *package*) - (*readtable* *readtable*) - (*load-pathname* nil) - (*load-truename* nil)) - (eval-input x)))))) + (let (#+genera (si:*common-lisp-syntax-is-ansi-common-lisp* t)) + (etypecase x + ((or pathname string #-(or allegro clozure genera) stream #+clozure file-stream) + (apply 'load x keys)) + ;; Genera can't load from a string-input-stream + ;; ClozureCL 1.6 can only load from file input stream + ;; Allegro 5, I don't remember but it must have been broken when I tested. + #+(or allegro clozure genera) + (stream ;; make do this way + (let ((*package* *package*) + (*readtable* *readtable*) + (*load-pathname* nil) + (*load-truename* nil)) + (eval-input x))))))) (defun load-from-string (string) "Portably read and evaluate forms from a STRING." @@ -6930,7 +6985,7 @@ or an indication of failure via the EXIT-CODE of the process" (uiop/package:define-package :uiop/configuration (:recycle :uiop/configuration :asdf/configuration) ;; necessary to upgrade from 2.27. - (:use :uiop/common-lisp :uiop/utility + (:use :uiop/package :uiop/common-lisp :uiop/utility :uiop/os :uiop/pathname :uiop/filesystem :uiop/stream :uiop/image :uiop/lisp-build) (:export #:user-configuration-directories #:system-configuration-directories ;; implemented in backward-driver @@ -6945,7 +7000,8 @@ or an indication of failure via the EXIT-CODE of the process" #:report-invalid-form #:invalid-configuration #:*ignored-configuration-form* #:*user-cache* #:*clear-configuration-hook* #:clear-configuration #:register-clear-configuration-hook #:resolve-location #:location-designator-p #:location-function-p #:*here-directory* - #:resolve-relative-location #:resolve-absolute-location #:upgrade-configuration)) + #:resolve-relative-location #:resolve-absolute-location #:upgrade-configuration + #:uiop-directory)) (in-package :uiop/configuration) (with-upgradability () @@ -7337,7 +7393,28 @@ or just the first one (for direction :output or :io). "Compute (and return) the location of the default user-cache for translate-output objects. Side-effects for cached file location computation." (setf *user-cache* (xdg-cache-home "common-lisp" :implementation))) - (register-image-restore-hook 'compute-user-cache)) + (register-image-restore-hook 'compute-user-cache) + + (defun uiop-directory () + "Try to locate the UIOP source directory at runtime" + (labels ((pf (x) (ignore-errors (probe-file* x))) + (sub (x y) (pf (subpathname x y))) + (ssd (x) (ignore-errors (symbol-call :asdf :system-source-directory x)))) + ;; NB: conspicuously *not* including searches based on #.(current-lisp-pathname) + (or + ;; Look under uiop if available as source override, under asdf if avaiable as source + (ssd "uiop") + (sub (ssd "asdf") "uiop/") + ;; Look in recommended path for user-visible source installation + (sub (user-homedir-pathname) "common-lisp/asdf/uiop/") + ;; Look in XDG paths under known package names for user-invisible source installation + (xdg-data-pathname "common-lisp/source/asdf/uiop/") + (xdg-data-pathname "common-lisp/source/cl-asdf/uiop/") ; traditional Debian location + ;; The last one below is useful for Fare, primary (sole?) known user + (sub (user-homedir-pathname) "cl/asdf/uiop/") + (cerror "Configure source registry to include UIOP source directory and retry." + "Unable to find UIOP directory") + (uiop-directory))))) ;;; ------------------------------------------------------------------------- ;;; Hacks for backward-compatibility with older versions of UIOP @@ -7372,7 +7449,8 @@ DEPRECATED. Use UIOP:XDG-CONFIG-PATHNAMES instead." (xdg-config-pathnames "common-lisp")) (defun system-configuration-directories () "Return the list of system configuration directories for common-lisp. -DEPRECATED. Use UIOP:CONFIG-SYSTEM-PATHNAMES instead." +DEPRECATED. Use UIOP:SYSTEM-CONFIG-PATHNAMES (with argument \"common-lisp\"), +instead." (system-config-pathnames "common-lisp")) (defun in-first-directory (dirs x &key (direction :input)) "Finds the first appropriate file named X in the list of DIRS for I/O @@ -7521,7 +7599,7 @@ previously-loaded version of ASDF." ;; "3.4.5.67" would be a development version in the official branch, on top of 3.4.5. ;; "3.4.5.0.8" would be your eighth local modification of official release 3.4.5 ;; "3.4.5.67.8" would be your eighth local modification of development version 3.4.5.67 - (asdf-version "3.3.2") + (asdf-version "3.3.3") (existing-version (asdf-version))) (setf *asdf-version* asdf-version) (when (and existing-version (not (equal asdf-version existing-version))) @@ -7534,7 +7612,7 @@ previously-loaded version of ASDF." ;;; Upon upgrade, specially frob some functions and classes that are being incompatibly redefined (when-upgrading () (let* ((previous-version (first *previous-asdf-versions*)) - (redefined-functions ;; List of functions that changes incompatibly since 2.27: + (redefined-functions ;; List of functions that changed incompatibly since 2.27: ;; gf signature changed (should NOT happen), defun that became a generic function, ;; method removed that will mess up with new ones (especially :around :before :after, ;; more specific or call-next-method'ed method) and/or semantics otherwise modified. Oops. @@ -7545,8 +7623,8 @@ previously-loaded version of ASDF." ;; Also note that we don't include the defgeneric=>defun, because they are ;; done directly with defun* and need not trigger a punt on data. ;; See discussion at https://gitlab.common-lisp.net/asdf/asdf/merge_requests/36 - `(,@(when (version<= previous-version "3.1.2") '(#:component-depends-on #:input-files)) ;; crucial methods *removed* before 3.1.2 - ,@(when (version<= previous-version "3.1.7.20") '(#:find-component)))) + `(,@(when (version< previous-version "3.1.2") '(#:component-depends-on #:input-files)) ;; crucial methods *removed* before 3.1.2 + ,@(when (version< previous-version "3.1.7.20") '(#:find-component)))) (redefined-classes ;; redefining the classes causes interim circularities ;; with the old ASDF during upgrade, and many implementations bork @@ -7883,9 +7961,9 @@ or NIL for top-level components (a.k.a. systems)")) (defmethod component-parent ((component null)) nil) ;; Deprecated: Backward compatible way of computing the FILE-TYPE of a component. - ;; TODO: find users, have them stop using that, remove it for ASDF4. - (defgeneric source-file-type (component system) - (:documentation "DEPRECATED. Use the FILE-TYPE of a COMPONENT instead.")) + (with-asdf-deprecation (:style-warning "3.4") + (defgeneric source-file-type (component system) + (:documentation "DEPRECATED. Use the FILE-TYPE of a COMPONENT instead."))) (define-condition duplicate-names (system-definition-error) ((name :initarg :name :reader duplicate-names-name)) @@ -8222,6 +8300,7 @@ Use of INITARGS is not supported at this time." #:system-source-file #:system-source-directory #:system-relative-pathname #:system-description #:system-long-description #:system-author #:system-maintainer #:system-licence #:system-license + #:system-version #:definition-dependency-list #:definition-dependency-set #:system-defsystem-depends-on #:system-depends-on #:system-weakly-depends-on #:component-build-pathname #:build-pathname @@ -8243,8 +8322,10 @@ Use of INITARGS is not supported at this time." If no system is found, then signal an error if ERROR-P is true (the default), or else return NIL. A system designator is usually a string (conventionally all lowercase) or a symbol, designating the same system as its downcased name; it can also be a system object (designating itself).")) + (defgeneric system-source-file (system) (:documentation "Return the source file in which system is defined.")) + ;; This is bad design, but was the easiest kluge I found to let the user specify that ;; some special actions create outputs at locations controled by the user that are not affected ;; by the usual output-translations. @@ -8263,6 +8344,7 @@ NB: This interface is subject to change. Please contact ASDF maintainers if you (with no argument) when running an image dumped from the COMPONENT. NB: This interface is subject to change. Please contact ASDF maintainers if you use it.")) + (defmethod component-entry-point ((c component)) nil)) @@ -8287,19 +8369,21 @@ a SYSTEM is redefined and its class is modified.")) (defclass system (module proto-system) ;; Backward-compatibility: inherit from module. ASDF4: only inherit from parent-component. (;; {,long-}description is now inherited from component, but we add the legacy accessors - (description :accessor system-description) - (long-description :accessor system-long-description) - (author :accessor system-author :initarg :author :initform nil) - (maintainer :accessor system-maintainer :initarg :maintainer :initform nil) - (licence :accessor system-licence :initarg :licence - :accessor system-license :initarg :license :initform nil) - (homepage :accessor system-homepage :initarg :homepage :initform nil) - (bug-tracker :accessor system-bug-tracker :initarg :bug-tracker :initform nil) - (mailto :accessor system-mailto :initarg :mailto :initform nil) - (long-name :accessor system-long-name :initarg :long-name :initform nil) + (description :writer (setf system-description)) + (long-description :writer (setf system-long-description)) + (author :writer (setf system-author) :initarg :author :initform nil) + (maintainer :writer (setf system-maintainer) :initarg :maintainer :initform nil) + (licence :writer (setf system-licence) :initarg :licence + :writer (setf system-license) :initarg :license + :initform nil) + (homepage :writer (setf system-homepage) :initarg :homepage :initform nil) + (bug-tracker :writer (setf system-bug-tracker) :initarg :bug-tracker :initform nil) + (mailto :writer (setf system-mailto) :initarg :mailto :initform nil) + (long-name :writer (setf system-long-name) :initarg :long-name :initform nil) ;; Conventions for this slot aren't clear yet as of ASDF 2.27, but whenever they are, they will be enforced. ;; I'm introducing the slot before the conventions are set for maximum compatibility. - (source-control :accessor system-source-control :initarg :source-control :initform nil) + (source-control :writer (setf system-source-control) :initarg :source-control :initform nil) + (builtin-system-p :accessor builtin-system-p :initform nil :initarg :builtin-system-p) (build-pathname :initform nil :initarg :build-pathname :accessor component-build-pathname) @@ -8375,6 +8459,35 @@ NB: The onus is unhappily on the user to avoid clashes." (frob-substrings (coerce-name name) '("/" ":" "\\") "--"))) +;;; System virtual slot readers, recursing to the primary system if needed. +(with-upgradability () + (defvar *system-virtual-slots* '(long-name description long-description + author maintainer mailto + homepage source-control + licence version bug-tracker) + "The list of system virtual slot names.") + (defun system-virtual-slot-value (system slot-name) + "Return SYSTEM's virtual SLOT-NAME value. +If SYSTEM's SLOT-NAME value is NIL and SYSTEM is a secondary system, look in +the primary one." + (or (slot-value system slot-name) + (unless (primary-system-p system) + (slot-value (find-system (primary-system-name system)) + slot-name)))) + (defmacro define-system-virtual-slot-reader (slot-name) + `(defun* ,(intern (concatenate 'string (string :system-) + (string slot-name))) + (system) + (system-virtual-slot-value system ',slot-name))) + (defmacro define-system-virtual-slot-readers () + `(progn ,@(mapcar (lambda (slot-name) + `(define-system-virtual-slot-reader ,slot-name)) + *system-virtual-slots*))) + (define-system-virtual-slot-readers) + (defun system-license (system) + (system-virtual-slot-value system 'licence))) + + ;;;; Pathnames (with-upgradability () @@ -10786,8 +10899,9 @@ Do NOT try to load a .asd file directly with CL:LOAD. Always use ASDF:LOAD-ASD." (defvar *old-asdf-systems* (make-hash-table :test 'equal)) ;; (Private) function to check that a system that was found isn't an asdf downgrade. - ;; Returns T if everything went right, NIL if the system was an ASDF of the same or older version, - ;; that shall not be loaded. Also issue a warning if it was a strictly older version of ASDF. + ;; Returns T if everything went right, NIL if the system was an ASDF at an older version, + ;; or UIOP of the same or older version, that shall not be loaded. + ;; Also issue a warning if it was a strictly older version of ASDF. (defun check-not-old-asdf-system (name pathname) (or (not (member name '("asdf" "uiop") :test 'equal)) (null pathname) @@ -10798,9 +10912,12 @@ Do NOT try to load a .asd file directly with CL:LOAD. Always use ASDF:LOAD-ASD." (read-file-form version-pathname :at (if asdfp '(0) '(2 2 2))))) (old-version (asdf-version))) (cond - ;; Don't load UIOP of the exact same version: we already loaded it as part of ASDF. - ((and (equal old-version version) (equal name "uiop")) nil) - ((version<= old-version version) t) ;; newer or same version: Good! + ;; Same version is OK for ASDF, to allow loading from modified source. + ;; However, do *not* load UIOP of the exact same version: + ;; it was already loaded it as part of ASDF and would only be double-loading. + ;; Be quiet about it, though, since it's a normal situation. + ((equal old-version version) asdfp) + ((version< old-version version) t) ;; newer version: Good! (t ;; old version: bad (ensure-gethash (list (namestring pathname) version) *old-asdf-systems* @@ -10962,6 +11079,8 @@ PREVIOUS-PRIMARY when not null is the primary system for the PREVIOUS system." #:class-for-type #:*default-component-class* #:determine-system-directory #:parse-component-form #:non-toplevel-system #:non-system-system #:bad-system-name + #:*known-systems-with-bad-secondary-system-names* + #:known-system-with-bad-secondary-system-names-p #:sysdef-error-component #:check-component-input #:explain)) (in-package :asdf/parse-defsystem) @@ -11114,7 +11233,7 @@ Please only define ~S and secondary systems with a name starting with ~S (e.g. ~ ;;; "inline methods" (with-upgradability () (defparameter* +asdf-methods+ - '(perform-with-restarts perform explain output-files operation-done-p)) + '(perform-with-restarts perform explain output-files operation-done-p)) (defun %remove-component-inline-methods (component) (dolist (name +asdf-methods+) @@ -11127,19 +11246,55 @@ Please only define ~S and secondary systems with a name starting with ~S (e.g. ~ (component-inline-methods component))) (component-inline-methods component) nil) + (defparameter *standard-method-combination-qualifiers* + '(:around :before :after)) + +;;; Find inline method definitions of the form +;;; +;;; :perform (test-op :before (operation component) ...) +;;; +;;; in REST (which is the plist of all DEFSYSTEM initargs) and define the specified methods. (defun %define-component-inline-methods (ret rest) + ;; find key-value pairs that look like inline method definitions in REST. For each identified + ;; definition, parse it and, if it is well-formed, define the method. (loop* :for (key value) :on rest :by #'cddr :for name = (and (keywordp key) (find key +asdf-methods+ :test 'string=)) :when name :do - (destructuring-bind (op &rest body) value - (loop :for arg = (pop body) - :while (atom arg) - :collect arg :into qualifiers - :finally - (destructuring-bind (o c) arg - (pushnew - (eval `(defmethod ,name ,@qualifiers ((,o ,op) (,c (eql ,ret))) ,@body)) - (component-inline-methods ret))))))) + ;; parse VALUE as an inline method definition of the form + ;; + ;; (OPERATION-NAME [QUALIFIER] (OPERATION-PARAMETER COMPONENT-PARAMETER) &rest BODY) + (destructuring-bind (operation-name &rest rest) value + (let ((qualifiers '())) + ;; ensure that OPERATION-NAME is a symbol. + (unless (and (symbolp operation-name) (not (null operation-name))) + (sysdef-error "Ill-formed inline method: ~S. The first element is not a symbol ~ + designating an operation but ~S." + value operation-name)) + ;; ensure that REST starts with either a cons (potential lambda list, further checked + ;; below) or a qualifier accepted by the standard method combination. Everything else + ;; is ill-formed. In case of a valid qualifier, pop it from REST so REST now definitely + ;; has to start with the lambda list. + (cond + ((consp (car rest))) + ((not (member (car rest) + *standard-method-combination-qualifiers*)) + (sysdef-error "Ill-formed inline method: ~S. Only a single of the standard ~ + qualifiers ~{~S~^ ~} is allowed, not ~S." + value *standard-method-combination-qualifiers* (car rest))) + (t + (setf qualifiers (list (pop rest))))) + ;; REST must start with a two-element lambda list. + (unless (and (listp (car rest)) + (length=n-p (car rest) 2) + (null (cddar rest))) + (sysdef-error "Ill-formed inline method: ~S. The operation name ~S is not followed by ~ + a lambda-list of the form (OPERATION COMPONENT) and a method body." + value operation-name)) + ;; define the method. + (destructuring-bind ((o c) &rest body) rest + (pushnew + (eval `(defmethod ,name ,@qualifiers ((,o ,operation-name) (,c (eql ,ret))) ,@body)) + (component-inline-methods ret))))))) (defun %refresh-component-inline-methods (component rest) ;; clear methods, then add the new ones @@ -11253,6 +11408,13 @@ system names contained using COERCE-NAME. Return the result." (coerce-name (component-system component)))) component))) + (defparameter* *known-systems-with-bad-secondary-system-names* + (list-to-hash-set '("cl-ppcre"))) + (defun known-system-with-bad-secondary-system-names-p (asd-name) + ;; Does .asd file with name ASD-NAME contain known exceptions + ;; that should be screened out of checking for BAD-SYSTEM-NAME? + (gethash asd-name *known-systems-with-bad-secondary-system-names*)) + (defun register-system-definition (name &rest options &key pathname (class 'system) (source-file () sfp) defsystem-depends-on &allow-other-keys) @@ -11270,8 +11432,11 @@ system names contained using COERCE-NAME. Return the result." (let* ((asd-name (and source-file (equal "asd" (fix-case (pathname-type source-file))) (fix-case (pathname-name source-file)))) + ;; note that PRIMARY-NAME is a *syntactically* primary name (primary-name (primary-system-name name))) - (when (and asd-name (not (equal asd-name primary-name))) + (when (and asd-name + (not (equal asd-name primary-name)) + (not (known-system-with-bad-secondary-system-names-p asd-name))) (warn (make-condition 'bad-system-name :source-file source-file :name name)))) (let* (;; NB: handle defsystem-depends-on BEFORE to create the system object, ;; so that in case it fails, there is no incomplete object polluting the build. @@ -11833,8 +11998,17 @@ which is probably not what you want; you probably need to tweak your output tran :static-library (resolve-symlinks* pathname)))) (defun linkable-system (x) - (or (if-let (s (find-system x)) + (or ;; If the system is available as source, use it. + (if-let (s (find-system x)) + (and (output-files 'lib-op s) s)) + ;; If an ASDF upgrade is available from source, but not a UIOP upgrade to that, + ;; then use the asdf/driver system instead of + ;; the UIOP that was disabled by check-not-old-asdf-system. + (if-let (s (and (equal (coerce-name x) "uiop") + (output-files 'lib-op "asdf") + (find-system "asdf/driver"))) (and (output-files 'lib-op s) s)) + ;; If there was no source upgrade, look for modules provided by the implementation. (if-let (p (system-module-pathname (coerce-name x))) (make-prebuilt-system x p)))) @@ -12567,7 +12741,7 @@ after having found a .asd file? True by default.") (recurse-beyond-asds *recurse-beyond-asds*) ignore-cache) (let ((visited (make-hash-table :test 'equalp))) (flet ((collectp (dir) - (unless (and (not ignore-cache) (process-source-registry-cache directory collect)) + (unless (and (not ignore-cache) (process-source-registry-cache dir collect)) (let ((asds (collect-asds-in-directory dir collect))) (or recurse-beyond-asds (not asds))))) (recursep (x) ; x will be a directory pathname @@ -13225,6 +13399,7 @@ system or its dependencies if it has already been loaded." #:system-maintainer #:system-license #:system-licence + #:system-version #:system-source-file #:system-source-directory #:system-relative-pathname