STAN and NATS
NATS -> a pub/sub messaging system (ephemeral by default)
- decouples the API from the backend work
- by default (without persistence) it just fires and forgets -- puts messages in the queue and moves on
STAN -> adds persistence, "exactly-once delivery", replay -- runs on top of NATS
- ensures messages aren't lost if workers crash
- maintains message order, using the names of files on disk to determine sequence
JetStream -> NATS 2.0 built-in streaming (replaces STAN)
The term **streaming does not refer to fast-moving data.**
Streaming refers to an ordered, persistent log of all messages
In the context of Astronomer, this is helpful because the server (Houston) can respond to the client (UI) quickly, while the workers do long_-ish_ running Helm commands behind the scenes.
Here is the stan service in action:
kubectl port-forward -n astronomer astronomer-stan-0 8222:8222
List channels/topics:
curl localhost:8222/streaming/channelsz
{
"cluster_id": "astronomer-stan",
"server_id": "MnDH0pJH2DbzjcQRTr1yxi",
"now": "2025-09-08T18:07:30.058578615Z",
"offset": 0,
"limit": 1024,
"count": 10,
"total": 10,
"names": [
"cluster.config.updated",
"houston-cleanedup-airflow-deployment-db",
"houston.deployment.created",
"houston.deployment.deleted",
"houston.deployment.image.updated",
"houston.deployment.updated",
"houston.deployment.upserted.create",
"houston.deployment.upserted.update",
"houston.deployment.variables.updated",
"houston.refreshed.deployment.task.data"
]
}
specific channel:
❯ curl http://localhost:8222/streaming/channelsz\?channel\=houston.deployment.updated
{
"name": "houston.deployment.updated",
"msgs": 0,
"bytes": 0,
"first_seq": 0,
"last_seq": 0,
"subs_count": 2
}