ΛↃ LAMBDACOMBINE

Symbolic Systems Infrastructure

10 · Event Sourcing

APPEND-ONLY LOG · PROJECTION · COMMANDS · TIME TRAVEL

Your bank balance is not stored, what is stored are the transactions. The balance is a projection: a fold over the list of events. An UPDATE statement throws away the event that caused the change, and event sourcing keeps it.

This is a very simple example of event sourcing, and the reason we are addressing it here is not because it's a part of Datastar itself, but because it's a pattern that is often coupled with CQRS - itself something that is also not strictly a part of Datastar.

As mentioned in the introduction, the exact border between what Datastar includes and what are more generic patterns that can be implemented with it can be often hard to determine when those patterns are new: CQRS allows us to have multiple instructions using a single stream, and event sourcing is often used by the write-side of CQRS to append events to an event store, while the read side listens to those events. Is it useful? It can be, but the purpose of the example here is not to convince that it's needed, merely demonstrate what it looks like.

Events, commands, projections

These three terms are important to understand event sourcing:

The fold in Common Lisp

Using our account example, Balance is a reduce over the event list. There is no balance variable anywhere in this program.

(defun demo-es-balance (events)
  (reduce (lambda (bal evt)
            (ecase (getf evt :type)
              (:deposited (+ bal (getf evt :amount)))
              (:withdrawn (- bal (getf evt :amount)))))
          events :initial-value 0))

ecase makes the projection exhaustive: add a new event type and the compiler flags every call site that does not handle it.

Commands: validate, then append

A command checks a business rule against the current projected state and, if valid, appends an event to the log. It never mutates application state directly; the new state emerges from replaying the event stream. The lock makes the balance check and the append atomic:

(defun cmd-es-withdraw (amount)
  (bt:with-lock-held (*demo-es-lock*)
    (when (and (numberp amount) (plusp amount)
               (>= (demo-es-balance *demo-es-events*) (round amount)))
      (push (list :type :withdrawn :amount (round amount)
                  :time (get-universal-time))
            *demo-es-events*)
      t)))

Two concurrent withdrawals cannot both pass the balance check because they both compete for the same lock.

Time travel

Because every historical event is in the log, you can compute the account state after any event - no separate audit table or history column needed. The Balance after column in the demo is the same fold stopped at each step:

(let ((running 0) (idx 0))
  (dolist (evt chronological-events)
    (incf idx)
    (incf running (* (getf evt :amount)
                     (if (eq (getf evt :type) :deposited) 1 -1)))
    ;; running = balance as of event idx
    ))

When to use it

If you are not doing event sourcing you are losing data - but a lot of times that is not only acceptable, it's the more reasonable approach, since it might be that there's no value in storing state transitions.

Good fits: financial records, compliance audits, analytics that ask what-if questions retroactively, any domain where the history itself is a business artefact. Poor fits: session state, ephemeral caches, anything where the event sequence has no value beyond the current state.

Live Demo

A shared bank account. All connected tabs see the same event log. Withdrawing more than the balance is silently rejected. Reset clears the log (balance returns to zero).

Connecting...

Pure-push SSE and the keyed registry

with-sse with an empty body and :keep-alive t sends heartbeats and keeps the stream open until the client disconnects. All meaningful work happens in :on-connect and :on-disconnect - no explicit loop needed.

The standalone version adds per-client unicast: a browser-generated UUID (crypto.randomUUID) is carried as a signal so a POST handler can reply to exactly one client. reg:make-keyed-sse-registry handles both broadcast and unicast under one lock:

(defvar *clients* (reg:make-keyed-sse-registry "event-sourcing"))

;;; Broadcast to all connected clients.
(defun broadcast () (reg:notify-subscribers *clients* #'push-account-state))

;;; Unicast to the client whose session matches SID.
(defun reply-to (sid &optional (error ""))
  (reg:notify-subscriber *clients* sid
                         (lambda (g) (d*:patch-signals g (list :error error)))))

;;; SSE endpoint: pure-push, sid is the registry key.
(d*:with-sse (gen hunchentoot:*request*
              :keep-alive    t
              :on-connect    (lambda (g) (push-account-state g)
                                         (reg:register *clients* g :key sid))
              :on-disconnect (lambda (g) (reg:unregister *clients* g))))

Full Source (standalone Hunchentoot)

Load with sbcl --load event-sourcing.lisp and visit http://localhost:8989. The standalone version adds per-client error feedback via session IDs.

;;;; -*- Mode: LISP; fill-column: 80; coding: utf-8 -*-

;;;; EVENT-SOURCING.LISP --- Event-sourced bank account
;;;;
;;;; Copyright (C) 2025, 2026 Frederico Muñoz / ΛↃ lambda combine
;;;;
;;;; This file is part of datastar-cl, the Common Lisp SDK for Datastar
;;;;
;;;; License: MIT

;;; Demonstrates event sourcing combined with CQRS:
;;;
;;; 1) Append-only event log as the source of truth - no mutable balance
;;;    variable exists.
;;; 2) State derived by folding (REDUCE) the event log on every read.
;;;    The REDUCE is not a helper: it IS the state (so, "events are the database").
;;; 3) Commands validate business rules, then produce events, and they never
;;;    mutate state directly.
;;; 4) The "Balance after" column in the event log is like time travel: it shows
;;;    the account state at every point in history, computed by the same
;;;    fold as the current balance but stopped at an earlier event.
;;; 5) CQRS-ES: POST commands append events and broadcast to SSE subscribers, with
;;;    the SSE endpoint being pure-push (empty body + :keep-alive).
;;;
;;; Not everything about "event sourcing" is shown, and what is shown is not
;;; necessarily the best or only way to do it. Part of it is by design (for
;;; simplicity we opt for simple solutions without any other dependency), the
;;; other a result of my partial understanding.

;;; Run with: sbcl --load event-sourcing.lisp

;;; SYSTEM DEFINITION ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(ql:quickload '(:hunchentoot :spinneret :bordeaux-threads
                :datastar-cl/hunchentoot :datastar-cl/registry))

(defpackage #:event-sourcing
  (:use #:cl #:hunchentoot)
  (:local-nicknames (:sp :spinneret) (:d* :datastar-cl)
                    (:reg :datastar-cl.registry)
                    (:bt :bordeaux-threads) (:ht :hunchentoot)))

(in-package #:event-sourcing)

;;; EVENT STORE: append-only ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defstruct event id type amount timestamp)

(defvar *events* nil) ; newest first (push prepends)
(defvar *events-lock* (bt:make-lock "events"))

(defun append-event (type amount)
  (bt:with-lock-held (*events-lock*)
    (let ((e (make-event :id (1+ (length *events*))
                         :type type :amount amount
                         :timestamp (get-universal-time))))
      (push e *events*) e)))

(defun all-events ()
  (bt:with-lock-held (*events-lock*)
    (reverse *events*)))                       ; chronological order for fold

;;; PROJECTION: state is a reduce, not a field ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;
;;; There is no variable holding the current balance.  Every call to
;;; CURRENT-BALANCE replays the full log.  BALANCE-FROM is also used by the
;;; event-log renderer to compute the running balance after each event
;;; (the "time travel" column).
;;;
;;; "In event sourcing, state is a left fold". I checked, and a "left fold" is
;;; what normal people call reduce.

(defun balance-from (events)
  (reduce (lambda (bal e)
            (ecase (event-type e)
              (:deposited (+ bal (event-amount e)))
              (:withdrawn (- bal (event-amount e)))))
          events :initial-value 0))

(defun current-balance () (balance-from (all-events)))

;;; COMMANDS: validate, then produce events ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;
;;; Partially the reason why this comes up when CQRS is mentioned - independent
;;; things that happen to play well together.

(defun cmd-deposit (amount)
  (when (plusp amount)
    (append-event :deposited amount) t))

(defun cmd-withdraw (amount)
  (when (and (plusp amount) (>= (current-balance) amount))
    (append-event :withdrawn amount) t))

;;; RENDERING ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defun fmt-time (ut)
  (multiple-value-bind (s m h) (decode-universal-time ut)
    (format nil "~2,'0d:~2,'0d:~2,'0d" h m s)))

(defun render-balance-panel ()
  (sp:with-html-string
    (:div :id "balance-panel"
          (:h2 "Balance: " (:code (format nil "$~D" (current-balance))))
          (:p (:input :type "number" :min "1" :data-bind "amount"
                      :style "width:5em")
              " "
              (:button :|data-on:click| (d*:sse-post "/deposit") "Deposit")
              " "
              (:button :|data-on:click| (d*:sse-post "/withdraw") "Withdraw"))
          (:p :data-show "$error != ''" :style "display:none;color:red"
              :data-text "$error"))))

(defun render-event-log ()
  (let ((events (all-events))
        (running 0))
    (sp:with-html-string
      (:table :id "event-log"
              :style "border-collapse:collapse;width:100%;margin-top:1em"
              (:caption (:em "Event log: the source of truth"))
              (:thead
               (:tr (:th "#") (:th "Event") (:th "Amount")
                    (:th "Balance after") (:th "Time")))
              (:tbody
               (dolist (e events)
                 (incf running (* (event-amount e)
                                  (if (eq (event-type e) :deposited) 1 -1)))
                 (:tr :style "border-top:1px solid #ccc"
                      (:td (event-id e))
                      (:td (string-downcase (symbol-name (event-type e))))
                      (:td (format nil "~D" (event-amount e)))
                      (:td (:strong (format nil "$~D" running)))
                      (:td (fmt-time (event-timestamp e))))))))))

(defun push-account-state (gen)
  (d*:patch-elements gen (render-balance-panel) :selector "#balance-panel")
  (d*:patch-elements gen (render-event-log) :selector "#event-log"))

;;; SUBSCRIPTION (CQRS push layer) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;
;;; *CLIENTS* is a keyed registry: NOTIFY-SUBSCRIBERS broadcasts to all
;;; connected generators; NOTIFY-SUBSCRIBER + sid key targets one.
;;; The sid is a UUID assigned by the browser at page load (crypto.randomUUID)
;;; and carried on every request as a Datastar signal, so POST handlers can
;;; target exactly the requester's stream.

(defvar *clients* (reg:make-keyed-sse-registry "event-sourcing"))

(defun broadcast ()
  (reg:notify-subscribers *clients* #'push-account-state))

;;; REPLY-TO delivers the command outcome asynchronously to one client over its
;;; already-open push stream.  NOTIFY-SUBSCRIBER routes via CALL-WITH-GENERATOR
;;; for correct cross-thread dispatch on all backends.
;;; Empty ERROR string clears a previous rejection (signal persists in the store).

(defun reply-to (sid &optional (error ""))
  (reg:notify-subscriber *clients* sid
                         (lambda (g) (d*:patch-signals g (list :error error)))))

;;; HANDLERS ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(ht:define-easy-handler (index :uri "/") ()
  (setf (ht:content-type*) "text/html")
  (sp:with-html-string
    (:doctype)
    (:html
     (:head (:script :type "module" :src (d*:datastar-url)))
     (:body :data-signals (d*:init-signals :amount 10 :error "" :sid "")
            :data-init (format nil "$sid = crypto.randomUUID(); ~A" (d*:sse-get "/sse"))
            (:h1 "Event-Sourced Bank Account")
            (:p "The balance is never stored: it is derived by replaying the "
                "event log on every read (a " (:em "projection") "). "
                "The " (:strong "Balance after") " column shows the account "
                "state at each point in history.")
            (:p "The event IS the data: the succession of events not only produce the final result, they are in themselves important data.")
            (:raw (render-balance-panel))
            (:raw (render-event-log))))))

;;; Pure-push SSE endpoint (empty body, :keep-alive drives heartbeat).
;;; :on-connect pushes current state immediately and registers with the sid so
;;; POST handlers can unicast back to exactly this client.
(ht:define-easy-handler (sse-handler :uri "/sse") ()
  (d*:with-signals ((sid "sid" "")) ht:*request*
    (d*:with-sse (gen ht:*request*
                 :on-connect    (lambda (g) (push-account-state g) (reg:register *clients* g :key sid))
                 :on-disconnect (lambda (g) (reg:unregister *clients* g))
                 :keep-alive    t))))

;;; Command endpoints: fire-and-forget (204 No Content).
;;;
;;; The POST says "do this if possible" and returns immediately with no body.
;;; The *outcome* arrives asynchronously over the push channel:
;;;   - success:   broadcast new balance+log to all subscribers (incl. requester);
;;;                unicast empty $error to requester to clear any previous rejection.
;;;   - rejection: unicast rejection message to requester's stream only.
;;;
;;; This decouples "command accepted" from "command outcome": between the click
;;; and the result there may be many intervening transactions.

(ht:define-easy-handler (deposit-handler :uri "/deposit") ()
  (d*:with-signals ((amount "amount" 0) (sid "sid" "")) ht:*request*
    (if (cmd-deposit (round amount))
        (progn (broadcast) (reply-to sid))
        (reply-to sid "Invalid amount")))
  (setf (ht:return-code*) ht:+http-no-content+)
  "")

(ht:define-easy-handler (withdraw-handler :uri "/withdraw") ()
  (d*:with-signals ((amount "amount" 0) (sid "sid" "")) ht:*request*
    (if (cmd-withdraw (round amount))
        (progn (broadcast) (reply-to sid))
        (reply-to sid "Insufficient funds")))
  (setf (ht:return-code*) ht:+http-no-content+)
  "")

;;; SEED: three initial events so the log is non-empty at startup ;;;;;;;;;;;;;;
(cmd-deposit 1000)
(cmd-deposit 500)
(cmd-withdraw 200)

;;; STARTUP ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(ht:start (make-instance 'ht:easy-acceptor :port 8989))
(format t "~&Server started on http://localhost:8989~%")

(defroute guide-event-sourcing (:get :text/html))