LLM primitives

The LLM is the reasoning engine, swappable like a database driver. Four operations, gated by the llm capability (auto-granted in run, required under serve):

llm.syn

-- Doc example: LLM operations. The four ops return TEXT and need a real provider,
-- so the doctest verifies the offline/online check; the ops are shown in `summarize`.
intent: "doc example: LLM operations"
require llm

task summarize(report)
    when llm_available()
        give reason about report with context = "be concise"   -- needs a provider
    otherwise
        give "(LLM offline)"

print("summarize(\"a report\") → " + summarize("a report"))    -- offline → (LLM offline)

test "llm_available() is a bool — branch on it instead of guessing"
    assert_eq(type_of(llm_available()), "bool")

-- llm_stream(prompt, context, on_chunk): on_chunk receives each fragment as it is
-- produced; the call returns the full text. Offline it returns the placeholder
-- WITHOUT ever invoking on_chunk (placeholders are not answers).
task on_tok(t)
    print("chunk: " + t)

test "llm_stream: offline → placeholder, on_chunk untouched; online → text"
    when llm_available()
        assert_eq(type_of(llm_stream("Reply with one word: pong", "", on_tok)), "text")
    otherwise
        assert_eq(llm_stream("q", "", on_tok), "[no llm provider]")

The four operations

require llm

let analysis be analyze sales_data for "trends and anomalies"
let action   be decide between ["refund", "replace", "escalate"] given ticket
let email    be generate "response email" given complaint with tone = "empathetic"
let insight  be reason about problem with context = background_data

They take a subject: reason about X, decide between [...] given X, analyze X for "...", generate "..." given X. (A bare reason "literal" also works.)

Validated decisions

A decide between [...] result must be exactly one of the options. Synsema enforces this in

layers: on network providers the choice is forced at the API level (an internal tool whose

schema restricts the answer to your options — zero retries in the common case); the answer is

then normalized (case, punctuation, whole-word containment: "The answer is BLUE" → BLUE); if

it still doesn't match, one retry with feedback; and only then it warns (one-time stderr

notice) and returns the raw response — the chain never breaks, but never silently.

Offline mode

Without a provider configured, the ops return descriptive placeholders (e.g. decide → "[decision pending]"), so programs stay runnable — the chain never breaks. It's never silent, though: the first LLM op that falls back prints a one-time stderr notice pointing you to synsema llm status for the exact diagnosis (which variable is missing, and where). Branch on llm_available() instead of guessing:

when llm_available()
    let s be reason "Summarize: " + text
otherwise
    let s be "(LLM offline)"

Metering & budget — `llm_usage()`, `SYNSEMA_LLM_BUDGET`

Every real provider call is metered. llm_usage() → number of LLM tokens (input + output) consumed by this process so far — introspection, no capability (like llm_available()); 0 offline or before any call. One counter per process, shared under serve too. It is monotonic: time windows are your program's policy, not runtime state.

The host can set a hard token budget with SYNSEMA_LLM_BUDGET (a positive integer, resolved process environ > .env; invalid or 0 → no budget, with a warning). At the ceiling, every LLM op degrades to the marker text [llm budget exceeded: used N of M tokens] — no error, no network call, one stderr notice per process. An LLM op never breaks the agent chain (same philosophy as offline placeholders), so code that must stop on exhaustion branches on the marker or on llm_usage():

require llm
let before be llm_usage()
let answer be reason about question
when contains(answer, "llm budget exceeded")
    give "(budget exhausted — not retrying)"
print("this call cost " + text(llm_usage() - before) + " tokens")

Streaming (`llm_stream`)

llm_stream(prompt, context, on_chunk) generates with the configured provider, invoking the task on_chunk with each text fragment as it is produced, and returns the full text. Same llm gate. Network providers stream the API's SSE text-deltas as real chunks; the embedded local provider streams token by token. (With SYNSEMA_LLM_HTTP_STREAM=0 a network provider falls back to one chunk — the whole response.)

require llm

task on_tok(t)
    print(t)

let full be llm_stream("Explain CSP in one line", "", on_tok)

Offline it returns "[no llm provider]" without invoking on_chunk. Under a serve SSE route it composes with send — define the emitting task inside the stream block (send is a statement, not a function, and only parses there):

serve on 8080
    route "GET /chat"
        stream
            task emit(tok)
                send tok
            let full be llm_stream("Count to ten in words", "", emit)
            send full as "full"

If on_chunk fails (e.g. the send of a disconnected client), generation stops early and the error propagates — recoverable with try/recover.

Configure a provider in Provider config; let the model pick tools in Tool calling.