0035. Sending domains are written through operator endpoints and replicated to project stores in version order
- Status: Accepted (decided by the maintainer)
- Date: 2026-11-13
- Amends: ADR 0034 (settles the replication path its 2026-11-13 amendment left open), ADR 0028 (adds
/operator/*to the API routes), ADR 0005 (operator routes are HTTP-only and have no MCP tool, like theendpointroutes/in/*,/u/*,/provider-events/*, and the approval decision endpoint of ADR 0033), ADR 0009 (adds theSenderSetupStoreport and its contract suite), ADR 0026 (adds thecl_op_credential prefix)
Context
Section titled “Context”ADR 0034 keeps each workspace’s provider tenant and sending domains in the control plane, provisioned by the operator until the domain management API exists, and replicates them to each project store, where claimSend checks the from domain. A store without a replica sends nothing, and skipped:sender_not_verified is terminal. Its 2026-11-13 amendment leaves three things to the replication path: an order between writes, so a delayed older replica cannot bring back a removed domain; a replication on every control-plane write; and a resync that writes the replica of every existing project store, run before or with the deploy of the check and by the development setup.
Facts that shape the path:
- No code writes the control plane today. Workspaces and projects are inserted by SQL: the development seed, the test fixtures, and the operator.
- On Cloudflare a project store is a Durable Object, reachable only through a Worker.
wrangler d1 executecan write D1 but cannot reach a project store. - There are no users or roles yet (ADR 0025 is Proposed), so there is no operator identity to authorize an operator action.
Alternatives considered
Section titled “Alternatives considered”| Choice | Alternatives | Why this one |
|---|---|---|
| A per-workspace counter, incremented in the same transaction as each write, orders replicas | The write’s timestamp; one tombstone per removed domain | A counter has no clock skew between isolates and no ties; the replica stays one whole value, so nothing in the project store has to merge |
| Replication runs synchronously after each write; a failed store is reported and repaired by a resync | Fan-out through the delivery queue; a periodic resync on a cron trigger | The validation phase has few projects per workspace. A cron would apply a new domain only at its next tick, and every message claimed before that is skipped for good |
Operator endpoints on the api Worker with a bearer token |
Only wrangler d1 execute; a separate operator Worker; api actions with an admin key |
SQL alone cannot reach a project store. A separate Worker would duplicate the composition root to host four routes. Provisioning sending domains is not a project’s action: an api action would put it in MCP, where a project’s own AI could verify a domain |
Decision
Section titled “Decision”Control plane
Section titled “Control plane”workspaces.provider_tenant(nullable) andworkspaces.sender_version(starting at 0), andsending_domains(workspace_id, domain, status)withstatuspendingorverifiedand the key(workspace_id, domain).- Domains are stored in normal form: lower case, no trailing dot, and IDN labels as A-labels.
normalizeSendingDomainin core gets it from the WHATWG URL host parser, which applies the UTS 46 mapping and is in the WinterTC minimum common API on both targets, and declaresURLthe waylog.tsdeclaresconsole. It refuses anything the parser refuses, IP addresses, input with userinfo, port, path, or escape characters, labels other than letters, digits, and inner hyphens of at most 63 characters, names of one label or longer than 253 characters, andxn--labels that do not decode or that the parser would not write (it passes them through unchecked; a small RFC 3492 decoder checks them). - Each write of a workspace’s domains or tenant increments
sender_versionin the same transaction (one D1 batch). A write made by SQL instead of the operator endpoints must increment it too; otherwise project stores that already hold that version ignore the change, and a resync does not repair them. - The port is
SenderSetupStore, apart fromControlStore: only the operator endpoints use it, never a request, job, or the send path. Its contract is that a write and its version increment commit together;describeSenderSetupStoreintesting/contractschecks it, and runs on D1 in workerd.
Replication
Section titled “Replication”- The replica is
{ version, providerTenant, verifiedDomains }, read from the control plane in one consistent read. Domains with any other status are left out. - Core’s
replaceSenderReplicaowns the order, as core owns the message state machine: it reads the stored replica’s version and replaces the replica only if the incoming version is greater. A write of the same or an older version changes nothing, so writes can arrive in any order and more than once.ProjectTx.senderReplica.replaceis a plain write. A store with no replica, or one written before versions existed, reads as version -1 and gives way to any control-plane write. - Every operator write is followed by a replication of the workspace’s whole setup to each project the control plane lists for the workspace. Replication mints the scope of each listed project (
scopeForSenderReplication), a verified source in the Tenant model. A project store that fails is logged by ID and reported; it keeps its older replica until a resync. - The resync replicates every workspace, or one on request. It is the backfill for stores that never received a replica: every project store that existed before the deploy reaches project-store schema version 5 (the replica tables, after send approvals at 4) with no replica, and version 6 adds the replica’s version; projects inserted into the control plane by SQL get none either.
Operator endpoints
Section titled “Operator endpoints”PUT /operator/workspaces/:workspaceId/sending-domains/:domainwith{ "status": "pending" | "verified" },DELETEon the same path,PUT /operator/workspaces/:workspaceId/provider-tenantwith{ "providerTenant": string | null }, andPOST /operator/sender-replicas/resyncwith an optional{ "workspaceId" }. Each answers with the version replicated (read after the write, so a concurrent later write’s version can show here) and the project IDs replicated and failed; a resync of every workspace also lists the workspaces whose setup could not be read. The answer is 200 when every store took the replica and 503 when one did not, so the operator repeats the request or resyncs. Repeating any of them is safe.- They are authenticated by
Authorization: Bearer(the scheme in any case) withOPERATOR_TOKEN, a platform secret with the credential prefixcl_op_(32 to 128 characters after it), so log redaction masks it (ADR 0032). Tokens are compared by their SHA-256 hashes. WithoutOPERATOR_TOKENthe routes do not exist (404); with a malformed one, every operator request fails with a configuration error. - They are not actions: they are not in the action registry, MCP, OpenAPI, or the generated docs, and no API key reaches them. ADR 0005’s parity is for a project’s actions; these are operator tooling that the domain management API replaces with
apiactions.
Projects, deploys, and development
Section titled “Projects, deploys, and development”- No code creates projects yet. Until it does, the operator runs the workspace’s resync after inserting a project. When project creation is built, it replicates the workspace’s setup before it returns.
- Deploy order for the sender check and replication: the control-plane migration is applied and the
OPERATOR_TOKENsecret is set before the deploy, and the resync runs right after it. The check and the replication are deployed together: no build that has the check without the replication is ever deployed. Messages claimed between the deploy and the resync are skipped for good; the deploy happens while only the fake provider can be wired (ADR 0031), so no real mail is lost. pnpm dev:setupseedsexample.testas a verified domain of the development workspace and runs the resync through a short-livedwrangler devwith a one-time operator token.pnpm devdoes not enable the operator endpoints unless the developer setsOPERATOR_TOKENin.dev.vars.
Consequences
Section titled “Consequences”- A store’s replica converges on the control plane’s latest setup in any order of delivery, and a removed domain stays removed once its write has reached the store.
- Integration tests provision their sending domains through the operator endpoint, so every send in them goes through the replication path.
- Open:
- The check when a send approval is requested (ADR 0033) is built with the approvals. It can read the project store’s replica, as
claimSenddoes, so the request path does not read the control plane either. - A domain is not unique across workspaces. The operator verifies ownership before provisioning, and SES tenant association is the guard behind it; the domain management API decides uniqueness together with its DNS checks.
- Replication runs in the request, one project store at a time. A workspace with many projects needs fan-out through a queue, within the Worker’s subrequest limit.
- Operator writes are not in the audit log yet.
- If the control plane is restored to a point before later writes, its versions fall below those the project stores hold, and stores ignore every write until the version passes theirs. A restore of the control plane therefore raises each workspace’s
sender_versionabove any version it handed out before the restore, then resyncs. - The SES tenant name format is assumed to be 1 to 64 letters, digits,
_, and-; it is confirmed against the SES documentation before the SES tenant is wired (B4).
- The check when a send approval is requested (ADR 0033) is built with the approvals. It can read the project store’s replica, as