Emacs Package Management Guide

For Emacs users, optimizing their configuration is fun and a sure way to become an Emacs expert. Generally speaking, a novice’s configuration is pieced together, which is the fastest and most efficient way to learn. As you get deeper into Emacs, the configuration becomes more complex, and it is hard to imagine continuing to have fun with Emacs without refactoring your previously disorganized configuration.

This article will introduce my personal experience in optimizing configuration, the main content: package loading principles and management practices, I hope it will be of some help to readers in optimizing their own configurations.

Package.el issues

It is no exaggeration to say that the high degree of extensibility is the main reason why Emacs continues to live and die for decades. (length package-alist) counts the number of packages installed via package.el, which is 137 for me.

Although package.el provides a convenient way to install packages, it does not provide versioning, which is the most basic feature of any package manager, and I’ve had many times where the feature failed due to package upgrades, which is very frustrating, see here.

There are some solutions in the community, such as straight, borg , but to avoid introducing new problems and to reduce the learning burden. I currently do not use these solutions, but use git’s own submodule to manage some heavily used packages (e.g. lsp-mode/magit). You can then work on upgrades at your leisure, and if you have a problem with an upgrade, you can just fall back to the previous commit. No more worries about being interrupted by tools.

Package loading principles

For packages managed by package.el, users do not need to know how Emacs loads packages in order to use them, but they need to know the details of how to manage them completely on their own. First, the definition of a package is clear.

A package is a collection of one or more ELisp files that Emacs searches in the folder specified by load-path.

Emacs provides two types of high-level interfaces for package autoloading: Autoload and Feature.

Autoload

Autoload functions can declare functions or macros and then load their corresponding files when they are actually used.

1

(autoload filename docstring interactive type)

Generally do not use autoload function directly, but autoload magic comment, then use some function to parse the magic comment to automatically generate autoload function, for example, in my-mode folder there is a file hello-world.el, the content is.

1
2
3
4

;;;###autoload
(defun my-hello ()
(interactive)
(message "hello world"))

Use the following command to generate the autoloads file.

1

(package-generate-autoloads "hello-world" "~/my-mode")

Generate the hello-world-autoloads.el file in the same directory, with the following contents

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

;;; hello-world-autoloads.el --- automatically extracted autoloads
;;
;;; Code:

(add-to-list 'load-path (directory-file-name
                      (or (file-name-directory #$) (car load-path))))


;;;### (autoloads nil "hello-world" "hello-world.el" (0 0 0 0))
;;; Generated autoloads from hello-world.el

(autoload 'my-hello "hello-world" nil t nil)

;;;***

;; Local Variables:
;; version-control: never
;; no-byte-compile: t
;; no-update-autoloads: t
;; coding: utf-8
;; End:
;;; hello-world-autoloads.el ends here

This means that only the first time M-x my-hello goes back to load the hello-world.el file.

Note here that in order for Emacs to recognize the declaration of the my-hello function, it needs to go back and load the hello-world-autoloads.el file. For packages managed through package.el, package.el does the following when downloading the package.

1. parse the dependencies and download them recursively
2. append the package directory to the load-path
3. automatically generate the autoloads file and load it

This allows the user to use the functions provided by the package directly. If you use submodule management, you will need to implement the above operations yourself, which will be described later.

Feature

Feature is another mechanism provided by Emacs to automatically load ELisp files, example of use.

1
2
3
4
5
6

(defun my-hello ()
  (interactive)
  (message "hello world"))

;; feature 名与文件名相同
(provide 'hello-world)

The above code generates a feature called hello-world, and since it has the same name as the file, just use my-hello before (require 'hello-world) and it will go ahead and load hello-world.el automatically.

Load

Load.

1

(load filename &optional missing-ok nomessage nosuffix must-suffix)

The above autoload and feature will call the load function to load the file. load is a relatively low-level API and is not recommended to be called directly by the upper layer.

Submodule Management Package

In the introduction to autoload above, the general steps for downloading a package were described in package.el. Here is a refresher:

1. parse the dependencies and download them recursively
2. append the package directory to the load-path
3. automatically generate the autoloads file and load it

If you use a submodule, you can only download the package itself, so you need to do all three steps above. I currently use use-package to download and configure packages, and the following example shows its usage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

(use-package lsp-mode
  ;; 配置 load-path，lsp-mode 通过 submodule 下载到 ~/.emacs.d/vendor/lsp-mode 目录内
  :load-path ("~/.emacs.d/vendor/lsp-mode" "~/.emacs.d/vendor/lsp-mode/clients")
  :init (setq lsp-keymap-prefix "C-c l")
  ;; 配置 mode 的 hook
  :hook ((go-mode . lsp-deferred))
  ;; 生成 autoloads
  :commands (lsp lsp-deferred)
  ;; 配置 custom 变量
  :custom ((lsp-log-io nil))
  :config
  (require 'lsp-modeline)
  (push "[/\\\\]vendor$" lsp-file-watch-ignored-directories)
  ;; 配置 mode-map 快捷键
  :bind (:map lsp-mode-map
              ("M-." . lsp-find-definition)
              ("M-n" . lsp-find-references)))

As you can see, the use-package macro is very concise and unifies the various configurations of the package, and is highly recommended. Expanding use-package using macroexpand-1 shows that it is not much different from the code we configured manually.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

(progn
  (eval-and-compile
    (add-to-list 'load-path "~/.emacs.d/vendor/lsp-mode"))
  (eval-and-compile
    (add-to-list 'load-path "~/.emacs.d/vendor/lsp-mode/clients"))

  (let
      ((custom--inhibit-theme-enable nil))
    (unless
        (memq 'use-package custom-known-themes)
      (deftheme use-package)
      (enable-theme 'use-package)
      (setq custom-enabled-themes
            (remq 'use-package custom-enabled-themes)))
    (custom-theme-set-variables 'use-package
                                '(lsp-log-io nil nil nil "Customized with use-package lsp-mode")))
  (unless
      (fboundp 'lsp-deferred)
    (autoload #'lsp-deferred "lsp-mode" nil t))
  (unless
      (fboundp 'lsp-find-definition)
    (autoload #'lsp-find-definition "lsp-mode" nil t))
  (unless
      (fboundp 'lsp-find-references)
    (autoload #'lsp-find-references "lsp-mode" nil t))
  (unless
      (fboundp 'lsp)
    (autoload #'lsp "lsp-mode" nil t))
  (condition-case-unless-debug err
      (setq lsp-keymap-prefix "C-c l")
    (error
     (funcall use-package--warning139 :init err)))
  (eval-after-load 'lsp-mode
    '(progn
       (require 'lsp-modeline)
       (push "[/\\\\]vendor$" lsp-file-watch-ignored-directories)
       t)
    (add-hook 'go-mode-hook #'lsp-deferred)
    (bind-keys :package lsp-mode :map lsp-mode-map
      ("M-." . lsp-find-definition)
      ("M-n" . lsp-find-references))
    ))

use-package solves the cumbersome configuration problem, but does not solve the problem of package dependencies, which can only be downloaded one by one (see the Package-Requires declaration of the package for specific dependencies).

1
2
3
4
5
6

;; lsp-mode deps
(use-package spinner
  :defer t)
(use-package lv
  :defer t)
;; ...

use-package will automatically use package.el to download these dependencies if they are not found in the load-path. My approach here is a compromise, for some lighter packages there is no need to manage them with submodule. The reader may find this manual management of dependencies tedious, but in reality the dependencies are likely to be the same for different packages, e.g. dash.el, s.el, f.el, and other such base packages, so there are not many dependencies that actually need to be managed manually.

use-package bootstrap

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

(package-initialize)
(when (not package-archive-contents)
  (package-refresh-contents))

(dolist (p '(use-package))
  (when (not (package-installed-p p))
    (package-install p)))

(setq use-package-always-ensure t
      use-package-verbose t)

;; 后面就可以直接使用 use-package 来安装、配置包了

Common Git Commands

1
2
3
4
5
6
7
8
9

# 修改 .gitmodules 后
git submodule sync

# 更新到最新 commit
git submodule update --init --recursive --remote

# https://stackoverflow.com/a/18854453/2163429
# 更新到 .gitmodules 中指定的 commit
git submodule update --init

For adding and removing submodules, you can do it directly in magit by pressing o under magit-status-mode.