Comparison of NATS-Server (JetStream) and NATS Streaming Server

sobyte

After I have complained countless times, NATS JetStream has finally ended its beta phase and entered the RC phase. Finally, I’ve just gotten an official reply that the official version will be released after a few issues are addressed. So on the occasion of this important NATS-Server feature release, let’s talk about the differences between the NATS product itself and the use of the new features, as well as more potential differences.

Conceptual distinction: NATS-Server / NATS Streaming Server / NATS JetStream

NATS-Server

NATS-Server (or nats) is an open source, cloud-native, high-performance messaging system that is the most basic product of NATS. At its core, it is a publish/subscribe (Pub/Sub) system that allows clients to communicate across services in different clusters nats without having to pay attention to which service a specific message is on. In other words, a client can publish a message on the server side of any cluster, while trying to read it on any cluster client. In the official feature comparison with other similar message queue products, we can also pipe in a list of features of the product. nats supports multi-stream multi-service for pub/sub, load balancing, guaranteed message delivery at most/least once, multi-tenancy and user authentication, and other features. Although it seems to have a lot of advantages, the important reason why nats is not a widely used message queue is that it lacks some of the most important product features for message queues, such as persistence support and guaranteed one-time delivery of messages. This means that after your message is sent, your message is potentially lost during processing and may even be undelivered.

NATS Streaming Server

The NATS Streaming Server (or stan) is used to try to solve the existing problems with nats mentioned above. stan adds persistence features and message delivery policy support. The nats server comes with stan, but nats and stan cannot be mixed during use. The relationship between stan and nats is described in the official documentation as follows

NATS clients and NATS Streaming Server clients cannot exchange data with each other. That is, if a NATS Streaming Server client publishes a message on foo, a NATS client subscribed on the same topic will not receive the message. NATS Streaming Server messages are NATS messages consisting of protobuf. NATS Streaming Server has to send ACKs to the producer and receive ACKs from the consumer. If messages are freely exchanged with NATS clients, this can cause problems.

The specific architecture of stan is shown below.

sobyte

However, although stan provides persistence and messaging policy support, there were architectural design problems that led to a lot of problems left over from the initial design, such as when you determined that stan clusters are fixed and cannot be expanded horizontally without limits (#999), like not supporting multi-tenancy (#1122), like clients not being able to actively pull messages and only being pushed etc.

NATS JetStream

NATS JetStream (or JetStream) is NATS’ latest architectural design based on the Raft algorithm to try to solve the above problems. It is different from the original stan functionality, and provides new persistence features and message delivery policies, as well as support for horizontal scaling. At the same time, the new JetStream is also optimized for large messages, not as a client of nats but embedded in NATS Server as one of the features. In other words, when choosing between these technologies, JetStream should be the most preferred option. More details can be found in the official guidance document.

NATS JetStream usage

Now that we’ve covered the theory, let’s talk about the actual usage. For now JetStream is still in RC stage.

Compile and start the client

Download the nats-server source code, unzip it and execute.

1
2
3

cd nats-server-master
go build -o nats-server -ldflags="-s -w -buildid=" .
./nats-server -js

This will start a server that supports JetStream functionality.

[54738] 2021/03/02 18:27:02.605197 [INF] Starting nats-server
[54738] 2021/03/02 18:27:02.605236 [INF]   Version:  2.2.0-RC.2
[54738] 2021/03/02 18:27:02.605238 [INF]   Git:      [not set]
[54738] 2021/03/02 18:27:02.605239 [INF]   Name:     NAFWRGQTR2CHMBIKNPE6R3ZTW2BWV2FWPAZREMHI24IYVM6FVHMVIYLQ
[54738] 2021/03/02 18:27:02.605240 [INF]   ID:       NAFWRGQTR2CHMBIKNPE6R3ZTW2BWV2FWPAZREMHI24IYVM6FVHMVIYLQ
[54738] 2021/03/02 18:27:02.605658 [INF] Starting JetStream
[54738] 2021/03/02 18:27:02.606062 [WRN]     _ ___ _____ ___ _____ ___ ___   _   __  __
[54738] 2021/03/02 18:27:02.606076 [WRN]  _ | | __|_   _/ __|_   _| _ \ __| /_\ |  \/  |
[54738] 2021/03/02 18:27:02.606077 [WRN] | || | _|  | | \__ \ | | |   / _| / _ \| |\/| |
[54738] 2021/03/02 18:27:02.606078 [WRN]  \__/|___| |_| |___/ |_| |_|_\___/_/ \_\_|  |_|
[54738] 2021/03/02 18:27:02.606079 [WRN]
[54738] 2021/03/02 18:27:02.606080 [WRN]                _         _
[54738] 2021/03/02 18:27:02.606081 [WRN]               | |__  ___| |_ __ _
[54738] 2021/03/02 18:27:02.606082 [WRN]               | '_ \/ -_)  _/ _` |
[54738] 2021/03/02 18:27:02.606083 [WRN]               |_.__/\___|\__\__,_|
[54738] 2021/03/02 18:27:02.606084 [WRN]
[54738] 2021/03/02 18:27:02.606084 [WRN]          JetStream is a Beta feature
[54738] 2021/03/02 18:27:02.606085 [WRN]     https://github.com/nats-io/jetstream
[54738] 2021/03/02 18:27:02.606092 [INF]
[54738] 2021/03/02 18:27:02.606093 [INF] ----------- JETSTREAM -----------
[54738] 2021/03/02 18:27:02.606095 [INF]   Max Memory:      12.00 GB
[54738] 2021/03/02 18:27:02.606096 [INF]   Max Storage:     35.79 GB
[54738] 2021/03/02 18:27:02.606098 [INF]   Store Directory: "/var/folders/5s/8rczg1gs4wb59y9s22nc3f_r0000gn/T/nats/jetstream"
[54738] 2021/03/02 18:27:02.606099 [INF] ---------------------------------
[54738] 2021/03/02 18:27:02.606399 [INF] Listening for client connections on 0.0.0.0:4222
[54738] 2021/03/02 18:27:02.606512 [INF] Server is ready

Writing JetStream DEMO

Next we look at how to use JetStream for message publishing/subscription functionality.

    // 连接到nats的服务器
	conn, err := nats.Connect("nats://127.0.0.1:4222")
	if err != nil {
		log.Panic(err)
	}
	defer conn.Close()

	// 初始化JetStream功能
	js, err := conn.JetStream()
	if err != nil {
		log.Panic(err)
	}

	// 判断Stream是否存在，如果不存在，那么需要创建这个Stream，否则会导致pub/sub失败
	stream, err := js.StreamInfo(streamName)
	if err != nil {
		log.Println(err) // 如果不存在，这里会有报错
	}
	if stream == nil {
		log.Printf("creating stream %q and subject %q", streamName, subject)
		_, err = js.AddStream(&nats.StreamConfig{
			Name:     streamName,
			Subjects: []string{subject},
			MaxAge:   3 * 24 * time.Hour,
		})
		if err != nil {
			log.Panicln(err)
		}
	}

	// 订阅消息
	sub, err := js.Subscribe(subject, cbHandle, nats.AckAll(), nats.DeliverNew())
	if err != nil {
		log.Panic(err)
		return
	}
	defer sub.Unsubscribe()

	// 发送消息
	js.Publish(subject, []byte("Hello World! "+time.Now().Format(time.RFC3339)))

	time.Sleep(5 * time.Second)
	log.Println("Exiting...")

In this example, there is a noteworthy feature that needs to be highlighted in addition to the subscribe message, where we specifically declare the nats.DeliverNew() option. If not declared, the default is: nats.DeliverAll(). In addition to these two parameters, there is a nats.DeliverLast() parameter, which corresponds to each of the 3 ways to start a subscription: the default way nats.DeliverAll() is to read all messages within the valid lifetime, even those that have already been processed. nats.DeliverLast() will include the last message in the message queue, even if it has been processed; nats.DeliverNew() will only process new messages after the subscription.

Table of Contents