Compilation of Lisp to Native Code

In addition to the byte-compilation, described in the previous chapter, Emacs can also optionally compile Lisp function definitions into a true compiled code, known as native code. This feature uses the libgccjit library, which is part of the GCC distribution, and requires that Emacs be built with support for using that library. It also requires to have GCC and Binutils (the assembler and linker) available on your system for you to be able to native-compile Lisp code. To determine whether the current Emacs process can produce and load natively-compiled Lisp code, call native-comp-available-p (Native-Compilation Functions). Unlike byte-compiled code, natively-compiled Lisp code is executed directly by the machine's hardware, and therefore runs at full speed that the host CPU can provide. The resulting speedup generally depends on what the Lisp code does, but is usually 2.5 to 5 times faster than the corresponding byte-compiled code. Since native code is generally incompatible between different systems, the natively-compiled code is not transportable from one machine to another, it can only be used on the same machine where it was produced or on very similar ones (having the same CPU and run-time libraries). The transportability of natively-compiled code is the same as that of shared libraries (.so or .dll files). Libraries of natively-compiled code include crucial dependencies on Emacs Lisp primitives (What Is a Function) and their calling conventions, and thus Emacs usually won't load natively-compiled code produced by earlier or later Emacs versions; native compilation of the same Lisp code by a different Emacs version will usually produce a natively-compiled library under a unique file name that only that version of Emacs will be able to load. However, the use of unique file names allows to have in the same directory several versions of the same Lisp library natively-compiled by several different versions of Emacs. A non-nil file-local variable binding of no-byte-compile (Byte Compilation) also disables the native compilation of that file. In addition, a similar variable no-native-compile disables just the native compilation of the file. If both no-byte-compile and no-native-compile are specified, the former takes precedence.

Native-Compilation Functions

Native-Compilation is implemented as a side effect of byte-compilation (Byte Compilation). Thus, compiling Lisp code natively always produces its byte code as well, and therefore all the rules and caveats of preparing Lisp code for byte compilation (Compilation Functions) are valid for native-compilation as well. You can natively-compile either a single function or macro definition, or a whole file of Lisp code, with the native-compile function. Natively-compiling a file will produce both the corresponding .elc file with byte code and the .eln file with native code. Native compilation might produce warning or error messages; these are normally recorded in the buffer called *Native-compile-Log*. In interactive sessions, it uses the special LIMPLE mode (native-comp-limple-mode), which sets up font-lock as appropriate for this log, and is otherwise the same as Fundamental mode. Logging of messages resulting from native-compilation can be controlled by the native-comp-verbose variable (Native-Compilation Variables). When Emacs is run non-interactively, messages produced by native-compilation are reported by calling message (Displaying Messages), and are usually displayed on the standard error stream of the terminal from which Emacs was invoked.

native-compile: This function compiles function-or-file into native code. The argument function-or-file can be a function symbol, a Lisp form, or a name (a string) of the file which contains the Emacs Lisp source code to compile. If the optional argument output is provided, it must be a string specifying the name of the file to write the compiled code into. Otherwise, if function-or-file is a function or a Lisp form, this function returns the compiled object, and if function-or-file is a file name, the function returns the full absolute name of the file it created for the compiled code. The output file is by default given the .eln extension. This function runs the final phase of the native compilation, which invokes GCC via libgccjit, in a separate subprocess, which invokes the same Emacs executable as the process that called this function.
batch-native-compile: This function runs native-compilation on files specified on the Emacs command line in batch mode. It must be used only in a batch execution of Emacs, as it kills Emacs upon completion of the compilation. If one or more of the files fail to compile, the Emacs process will attempt to compile all the other files, and will terminate with a non-zero status code. The optional argument for-tarball, if non-nil, tells the function to place the resulting .eln files in the last directory mentioned in native-comp-eln-load-path (Library Search); this is meant to be used as part of building an Emacs source tarball for the first time, when the natively-compiled files, which are absent from the source tarball, should be generated in the build tree instead of the user's cache directory.

Native compilation can be run entirely asynchronously, in a subprocess of the main Emacs process. This leaves the main Emacs process free to use while the compilation runs in the background. This is the method used by Emacs to natively-compile any Lisp file or byte-compiled Lisp file that is loaded into Emacs, when no natively-compiled file for it is available. Note that because of this use of a subprocess, native compilation may produce warning and errors which byte-compilation does not, and lisp code may thus need to be modified to work correctly. See native-comp-async-report-warnings-errors in Native-Compilation Variables for more details.

native-compile-async: This function compiles the named files asynchronously. The argument files should be a single file name (a string) or a list of one or more file and/or directory names. If directories are present in the list, the optional argument recursively should be non-nil to cause the compilation to recurse into those directories. If load is non-nil, Emacs will load each file that it succeeded to compile. The optional argument selector allows control of which of files will be compiled; it can have one of the following values:
nil or omitted: Select all the files and directories in files.
a regular expression string: Select the files and directories whose names match the regexp.
a function: A predicate function, which will be called with each file and directory in files, and should return non-nil if the file or the directory should be selected for compilation.

On systems with multiple CPU execution units, when files names more than one file, this function will normally start several compilation subprocesses in parallel, under the control of native-comp-async-jobs-number (Native-Compilation Variables). The following function allows Lisp programs to test whether native-compilation is available at runtime.

native-comp-available-p: This function returns non-nil if the running Emacs process has the native-compilation support compiled into it. On systems that load libgccjit dynamically, it also makes sure that library is available and can be loaded. Lisp programs that need to know up front whether native-compilation is available should use this predicate.

Native-Compilation Variables

This section documents the variables that control native-compilation.

native-comp-speed: This variable specifies the optimization level for native compilation. Its value should be a number between −1 and 3. Values between 0 and 3 specify the optimization levels equivalent to the corresponding compiler -O0, -O1, etc. command-line options of the compiler. The value −1 means disable native-compilation: functions and files will be only byte-compiled; however, the *.eln files will still be produced, they will just contain the compiled code in bytecode form. (This can be achieved at function granularity by using the (declare (speed -1)) form, Declare Form.) The default value is 2.
native-comp-debug: This variable specifies the level of debugging information produced by native-compilation. Its value should be a number between zero and 3, with the following meaning:
0: No debugging output. This is the default.
1: Emit debugging symbols with the native code. This allows easier debugging of the native code with debuggers such as gdb.
2: Like 1, and in addition dump pseudo-C code.
3: Like 2, and in addition dump the GCC intermediate passes and libgccjit log file.
native-comp-verbose: This variable controls the verbosity of native-compilation by suppressing some or all of the log messages emitted by it. If its value is zero, the default, all of the log messages are suppressed. Setting it to a value between 1 and 3 will allow logging of the messages whose level is above the value. The values have the following interpretations:
0: No logging. This is the default.
1: Log the final LIMPLE representation of the code.
2: Log the LAP, the final LIMPLE, and some additional pass info.
3: Maximum verbosity: log everything.
native-comp-async-jobs-number: This variable determines the maximum number of native-compilation subprocesses that will be started simultaneously. It should be a non-negative number. The default value is zero, which means use half the number of the CPU execution units, or 1 if the CPU has only one execution unit.
native-comp-async-report-warnings-errors: If this variable's value is non-nil, warnings and errors from asynchronous native-compilation subprocesses are reported in the main Emacs session in a buffer named *Warnings*. The default value t means display the resulting buffer. To log warnings without popping up the *Warnings* buffer, set this variable to silent. A common cause for asynchronous native-compilation to produce warnings is compiling a file that is missing some require of a necessary feature. The feature may be loaded into the main emacs, but because native compilation always starts from a subprocess with a pristine environment, that may not be true for the subprocess.
native-comp-async-query-on-exit: If this variable's value is non-nil, Emacs will query upon exiting whether to exit and kill any asynchronous native-compilation subprocesses that are still running, thus preventing the corresponding .eln files from being written. If the value is nil, the default, Emacs will kill these subprocesses without querying.