Django's signaling mechanism

Django describes signal as a “signal dispatcher” that triggers multiple applications, mainly in the form of signals. This article will explain how signals work in Django and how to use them from the perspective of source code analysis.

Signal class

The most common scenario for signal is notifications. For example, if your blog has comments, the system will have a notification mechanism to push the comments to you. If you use signal, you only need to trigger the signal notification when the comment is posted, instead of putting the notification logic after the comment is posted, which greatly reduces the program coupling and facilitates the maintenance of the system later.

Django implements a Signal class, which is used to implement the function of “signal dispatcher”. The working mechanism is shown in the figure below, which is divided into two parts: first, each callback function that needs to be dispatched is registered to signal, and second, the event triggers sender to send the signal.

receiver

The signal maintains a list receiver of callback functions and their ids that connect to the signal. Each receiver must be a callback function and accepts the keyword argument **kwarg, and the signal’s connect method is used to connect the callback function to the signal.

Let’s take a look at the source code of connect, as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

class Signal:

    ...
    
    def connect(self, receiver, sender=None, weak=True, dispatch_uid=None):
            from django.conf import settings

        # If DEBUG is on, check that we got a good receiver
        if settings.configured and settings.DEBUG:
            assert callable(receiver), "Signal receivers must be callable."

            # Check for **kwargs
            if not func_accepts_kwargs(receiver):
                raise ValueError("Signal receivers must accept keyword arguments (**kwargs).")

        if dispatch_uid:
            lookup_key = (dispatch_uid, _make_id(sender))
        else:
            lookup_key = (_make_id(receiver), _make_id(sender))

        if weak:
            ref = weakref.ref
            receiver_object = receiver
            # Check for bound methods
            if hasattr(receiver, '__self__') and hasattr(receiver, '__func__'):
                ref = weakref.WeakMethod
                receiver_object = receiver.__self__
            receiver = ref(receiver)
            weakref.finalize(receiver_object, self._remove_receiver)

        with self.lock:
            self._clear_dead_receivers()
            for r_key, _ in self.receivers:
                if r_key == lookup_key:
                    break
            else:
                self.receivers.append((lookup_key, receiver))
            self.sender_receivers_cache.clear()

The code is very clean and is divided into four parts: checking parameters, getting the ID of receiver, receiver weak references, and locking. Here we mainly look at the last two parts.

receiver weak reference

Preparatory Knowledge.

Weak references: Python’s approach to garbage collection is to mark references, and the role of weak references is to avoid memory leaks caused by circular references. The main principle is that when a weak reference is made to an object, the number of references is not added to the reference mark, so when the strong reference to the object is 0, the system still reclaims it, and the weak reference fails.

method and function: Python’s functions are the same as those of other languages, including function names and function bodies, and supporting formal parameters; compared to functions, methods have an additional layer of class relationships, that is, they are functions defined in classes. This function is used to configure method step by step, see an excerpt of the source code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

PyObject *
PyMethod_New(PyObject *func, PyObject *self, PyObject *klass)
{
    ...
        
    im->im_weakreflist = NULL;
    Py_INCREF(func);
        
    im->im_func = func;
    Py_XINCREF(self);
    im->im_self = self;
    Py_XINCREF(klass);
    im->im_class = klass;
    _PyObject_GC_TRACK(im);
    return (PyObject *)im;
    }

In addition to the function attribute im_func, there is an im_self attribute table self and an im_class attribute table class in method.

Bound Method and Unbound Method: Methods can be divided into bound methods and unbound methods. The difference is that bound methods have an additional layer of instance binding, i.e., bound method invokes a method through an instance, while unbound method invokes a method directly through a class.

Weak references in signal:

Anyone familiar with Python garbage collection should know that Python only garbage collects an object when its reference count is 0. Therefore, all references to callback functions in signal are weakly referenced by default to avoid memory leaks.

First, the weak argument to connect indicates whether to use a weak reference, and the default is True; receiver can be either a function or a method, and the reference to bound method is short-lived, in line with the lifetime of the instance, so the standard weak reference is not enough to keep it, and weakref. WeakMethod to emulate the weak reference of bound method; finally the weakref.finalize method returns a callable terminator object that is called when receiver is garbage collected, unlike normal weak references, the terminator will always be alive before it is called and die after it is called, thus greatly simplifying life cycle management.

Locking

Locks exist to achieve thread safety, which means that when multiple threads are present at the same time, the result still meets expectations. Obviously, the receiver registration process in signal is not inherently thread-safe; the thread-safe method in signal is locking to achieve atomic operation of the connect method.

Locks are defined in the signal’s __init__ method, using Lock from the standard library.

1

self.lock = threading.Lock()

signal encapsulates the three operations of cleaning up weakly referenced objects in the receiver list, adding elements to the receiver list, and cleaning up the global cache dictionary into atomic operations using thread locks, as follows.

1
2
3
4
5
6
7
8

with self.lock:
    self._clear_dead_receivers()
    for r_key, _ in self.receivers:
        if r_key == lookup_key:
            break
    else:
        self.receivers.append((lookup_key, receiver))
    self.sender_receivers_cache.clear()

sender

To be precise, the sender in signal is an identifier to record “who” triggered the signal, what really works is the send method, which is used in the event to trigger the signal to all receivers “in event. Here is the source code for send.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

class Signal:

    ...
    
    def send(self, sender, **named):
        if not self.receivers or self.sender_receivers_cache.get(sender) is NO_RECEIVERS:
            return []

        return [
            (receiver, receiver(signal=self, sender=sender, **named))
            for receiver in self._live_receivers(sender)
        ]

It is easy to see that the process of triggering all the recorded callback functions is synchronous, so signal is not suitable for handling large batches of tasks, but we can rewrite it as asynchronous tasks.

How to use signal

The use of signal requires only two configurations, one is the registration of the callback function, and the other is the event triggering.

There are two ways to register callback functions, one is the regular signal.connect(); the other is that Django signal provides a decorator receiver, which can be decorated by passing in which signal it is, or you can specify sender, and if you don’t specify it, you receive all the information sent by sender. Event triggering is done by calling <your_signal>.send() where it can be triggered. A demo is given below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

from django.dispatch import Signal, receiver

signal = Signal()


@receiver(signal, sender="main")
def my_receiver(sender, **kwargs):
    print("here is my receiver.")
    print("hello sender: {}".format(sender))


if __name__ == "__main__":
    print("begin...")
    signal.send(sender="main")

Output.

1
2
3

begin...
here is my receiver.
hello sender: main

Django’s built-in signals

Django has a number of built-in signals that we can use directly.

Model-related

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

pre_init = ModelSignal(providing_args=["instance", "args", "kwargs"], use_caching=True)  # 对象初始化前
post_init = ModelSignal(providing_args=["instance"], use_caching=True)  #对象初始化后

pre_save = ModelSignal(providing_args=["instance", "raw", "using", "update_fields"],
                       use_caching=True)  # 对象保存修改前
post_save = ModelSignal(providing_args=["instance", "raw", "created", "using", "update_fields"], use_caching=True)  #对象保存修改后

pre_delete = ModelSignal(providing_args=["instance", "using"], use_caching=True)  #对象删除前
post_delete = ModelSignal(providing_args=["instance", "using"], use_caching=True)  #对象删除后

m2m_changed = ModelSignal(
    providing_args=["action", "instance", "reverse", "model", "pk_set", "using"],
    use_caching=True,
)  #ManyToManyField 字段更新后触发

pre_migrate = Signal(providing_args=["app_config", "verbosity", "interactive", "using", "apps", "plan"])  #数据迁移前
post_migrate = Signal(providing_args=["app_config", "verbosity", "interactive", "using", "apps", "plan"])  #数据迁移后

Request related

1
2
3
4

request_started = Signal(providing_args=["environ"])  #request 请求前
request_finished = Signal()  #request 请求后
got_request_exception = Signal(providing_args=["request"])  #request 请求出错后
setting_changed = Signal(providing_args=["setting", "value", "enter"])  #request 请求某些设置被修改后