Among the methods of setting up dynamic links, rpath has its own problems, and ld.so.conf is an OS global configuration that may affect other services because of the dynamic library version of a single service, which is not an elegant method.

So how can we make the target service find the corresponding dynamic library without affecting other services, without using rpath and ld.so.conf?

LD_LIBRARY_PATH

For dynamic link library path configuration, in addition to rpath and ld.so.conf, there is also LD_LIBRARY_PATH. As long as the LD_LIBRARY_PATH variable is set in the binary startup environment, glibc will use the path configuration in this variable as one of the paths to find the DLLs, allowing the binary executable to link properly to the DLLs it depends on.

LD_LIBRARY_PATH will take effect “globally” in the current shell session, however, since the author’s service uses systemd to manage its life cycle, according to the above idea, we should only need to inject LD_LIBRARY_PATH into the startup environment before the service starts. This also enables a certain degree of environmental “isolation” with systemd.

The Environment field in the systemd service configuration is used to describe the environment variables of the startup environment, so let’s add LD_LIBRARY_PATH here (below), reload the systemd configuration and restart the service, and observe the effect.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

 [Unit]
 Description=Nginx
 After=syslog.target network.target remote-fs.target nss-lookup.target

 [Service]
 User=triplez
 Group=triplez
 Type=forking
+Environment="LD_LIBRARY_PATH=/your/path/to/openssl/lib:/your/path/to/jemalloc/lib:/your/path/to/luajit/lib"
 PIDFile=/run/nginx/nginx.pid
 RuntimeDirectory=nginx
 RuntimeDirectoryMode=0755
 ExecStartPre=/your/path/to/nginx -t
 ExecStart=/your/path/to/nginx
 ExecReload=/bin/kill -s HUP $MAINPID
 ExecStop=/bin/kill -s QUIT $MAINPID

 PrivateTmp=true
 Restart=always

 [Install]
 WantedBy=multi-user.target

1
2

$ sudo systemctl daemon-reload
$ sudo systemctl restart nginx.service

In fact, for most services, it is sufficient to add environment variables to solve the problem. But unfortunately, since our service needs to listen on ports 1024 and below (e.g. 80, 443), I have performed a setcap operation on the binary executable to give it the ability to listen on lower ports.

1
2

$ sudo getcap /your/path/to/nginx
/your/path/to/nginx = cap_net_bind_service+ep

Capability

Because the file contains capability, glibc ignores LD_LIBRARY_PATH, and the service is still not linked to the dynamic library correctly.

Once you understand why, the next step is clear: first, remove the capability from the binary executable (chown can remove it, as can setcap -ep), and second, use systemd’s service configuration to configure the capability (below).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

 [Unit]
 Description=Nginx
 After=syslog.target network.target remote-fs.target nss-lookup.target

 [Service]
 User=triplez
 Group=triplez
 Type=forking
 Environment="LD_LIBRARY_PATH=/your/path/to/openssl/lib:/your/path/to/jemalloc/lib:/your/path/to/luajit/lib"
 PIDFile=/run/nginx/nginx.pid
 RuntimeDirectory=nginx
 RuntimeDirectoryMode=0755
 ExecStartPre=/your/path/to/nginx -t
 ExecStart=/your/path/to/nginx
 ExecReload=/bin/kill -s HUP $MAINPID
 ExecStop=/bin/kill -s QUIT $MAINPID

+CapabilityBoundingSet=CAP_NET_BIND_SERVICE
+AmbientCapabilities=CAP_NET_BIND_SERVICE
 PrivateTmp=true
 Restart=always

 [Install]
 WantedBy=multi-user.target

Reload the systemd configuration again and restart the service, the problem is solved perfectly.