After I have complained countless times,
NATS JetStream has finally ended its beta phase and entered the RC phase. Finally, I’ve just gotten an official reply that the official version will be released after a few issues are addressed. So on the occasion of this important
NATS-Server feature release, let’s talk about the differences between the NATS product itself and the use of the new features, as well as more potential differences.
Conceptual distinction: NATS-Server / NATS Streaming Server / NATS JetStream
nats) is an open source, cloud-native, high-performance messaging system that is the most basic product of NATS. At its core, it is a publish/subscribe (Pub/Sub) system that allows clients to communicate across services in different clusters
nats without having to pay attention to which service a specific message is on. In other words, a client can publish a message on the server side of any cluster, while trying to read it on any cluster client. In the official feature comparison with other similar message queue products, we can also pipe in a list of features of the product.
nats supports multi-stream multi-service for
pub/sub, load balancing, guaranteed message delivery at most/least once, multi-tenancy and user authentication, and other features. Although it seems to have a lot of advantages, the important reason why
nats is not a widely used message queue is that it lacks some of the most important product features for message queues, such as persistence support and guaranteed one-time delivery of messages. This means that after your message is sent, your message is potentially lost during processing and may even be undelivered.
NATS Streaming Server
NATS Streaming Server (or
stan) is used to try to solve the existing problems with
nats mentioned above.
stan adds persistence features and message delivery policy support. The
nats server comes with
stan cannot be mixed during use. The relationship between
nats is described in the official documentation as follows
NATS clients and NATS Streaming Server clients cannot exchange data with each other. That is, if a NATS Streaming Server client publishes a message on foo, a NATS client subscribed on the same topic will not receive the message. NATS Streaming Server messages are NATS messages consisting of protobuf. NATS Streaming Server has to send ACKs to the producer and receive ACKs from the consumer. If messages are freely exchanged with NATS clients, this can cause problems.
The specific architecture of
stan is shown below.
stan provides persistence and messaging policy support, there were architectural design problems that led to a lot of problems left over from the initial design, such as when you determined that
stan clusters are fixed and cannot be expanded horizontally without limits (#999), like not supporting multi-tenancy (#1122), like clients not being able to actively pull messages and only being pushed etc.
NATS JetStream (or
JetStream) is NATS’ latest architectural design based on the
Raft algorithm to try to solve the above problems. It is different from the original
stan functionality, and provides new persistence features and message delivery policies, as well as support for horizontal scaling. At the same time, the new
JetStream is also optimized for large messages, not as a client of
nats but embedded in
NATS Server as one of the features. In other words, when choosing between these technologies,
JetStream should be the most preferred option. More details can be found in the official guidance document.
NATS JetStream usage
Now that we’ve covered the theory, let’s talk about the actual usage. For now
JetStream is still in RC stage.
Compile and start the client
nats-server source code, unzip it and execute.
This will start a server that supports
Writing JetStream DEMO
Next we look at how to use
JetStream for message publishing/subscription functionality.
In this example, there is a noteworthy feature that needs to be highlighted in addition to the
subscribe message, where we specifically declare the
nats.DeliverNew() option. If not declared, the default is:
nats.DeliverAll(). In addition to these two parameters, there is a
nats.DeliverLast() parameter, which corresponds to each of the 3 ways to start a subscription: the default way
nats.DeliverAll() is to read all messages within the valid lifetime, even those that have already been processed.
nats.DeliverLast() will include the last message in the message queue, even if it has been processed;
nats.DeliverNew() will only process new messages after the subscription.