For Engineering

Laurelin internals

How the Laurelin backend is structured. For the data model see Schema reference; for HTTP endpoints see API reference; for sync workers see Sync internals.

The stack

There are no async middlewares. All DB calls are synchronous. The only async work in the request path is readBody(req) (collecting POST/PUT JSON) and the OAuth token refresh paths.

File map

server.js                           ← entrypoint; raw http.createServer
laurelin/
├── db.js                           ← singleton getDb(), initSchema, migrations,
│                                       logChanges(), uuid()
├── routes.js                       ← handleLaurelinRoutes(req, res, db)
├── seed.js                         ← runSeed(db) — INSERT OR IGNORE for demo data
├── meta-db.js                      ← separate metadata DB (rarely touched)
├── meta-seed.js                    ← seed for metadata DB
├── diagnose.js                     ← one-shot health check script
├── merge-duplicate-companies.js    ← admin script for bulk merging
├── notion-sync.js                  ← Notion → Laurelin reconciliation (companies)
├── notion-pipeline-sync.js         ← Notion → Laurelin reconciliation (projects)
├── sync-worker.js                  ← runs adapters on a schedule
└── sync/
    ├── auto-router.js              ← classification cascade
    ├── outlook-adapter.js          ← Microsoft Graph polling
    ├── outlook-oauth.js            ← OAuth flow + token refresh
    ├── slack-adapter.js            ← Events webhook handler
    ├── slack-api.js                ← conversations.history backfill
    ├── slack-oauth.js              ← Slack OAuth flow
    ├── telegram-adapter.js         ← getUpdates poll
    ├── telegram-bot.js             ← read-only bot helper (no send methods)
    ├── signature-parser.js         ← deterministic regex over bodies
    ├── lost-thread-detector.js     ← stale-outbound scanner
    ├── affiliations.js             ← affiliation maintenance
    ├── backlog-cleanup.js          ← batch cleanup of older sync_inbox rows
    └── claude-api.js               ← Anthropic SDK wrapper for draft generation

The request lifecycle

  1. server.js receives a request, hands it to handleLaurelinRoutes(req, res, db, metaDb).
  2. handleLaurelinRoutes defines a local match(method, regex) helper. The body is a long series of if (match(...)) { ... return true } blocks.
  3. The first matching route handles the request and returns true. Subsequent blocks are skipped.
  4. If no route matches, the function returns false and server.js returns a 404.

This is intentionally low-magic. Adding a route is grepping the file for a similar one and copying the pattern.

The match() helper

const match = (method, pattern) => {
  if (req.method !== method) return null;
  const m = pathname.match(pattern);
  return m;
};

Returns the regex match array (truthy with capture groups) on a hit, null otherwise. The m is captured into the outer scope so handlers can read m[1], m[2], etc. for path parameters.

Typical usage:

if ((m = match("GET", /^\/api\/laurelin\/companies\/([^/]+)$/))) {
  const id = m[1];
  // ... return true at the end
}

Helpers used in every route

readBody(req) — async parsing

return readBody(req).then((body) => {
  // body is the parsed JSON
  // ... write response
  return true;
});

For POST/PUT. Always wrap the handler in readBody(req).then(...) and return true immediately so the route is marked handled.

json(res, status, data)

Sets the right headers, writes the JSON-serialized payload, ends the response. The only response helper.

parseUrl(reqUrl)

Returns { pathname, params } where params is the parsed query string. Used at the top of handleLaurelinRoutes to avoid re-parsing per route.

logChanges() — the audit rule

Every UPDATE on companies, people, projects must call:

logChanges(db, "companies", id, oldRecord, newFields, changedBy);

This compares oldRecord (full row before the update) against newFields (the partial update payload) and inserts one row into changelog per changed field. Without this call, the audit trail breaks.

Pattern:

const existing = db.prepare("SELECT * FROM companies WHERE id = ?").get(id);
const updates = { name: body.name, stage: body.stage /* etc */ };
db.prepare(`UPDATE companies SET name = ?, stage = ?, ... WHERE id = ?`)
  .run(updates.name, updates.stage, id);
logChanges(db, "companies", id, existing, updates, body.changed_by);

If you add a new column to companies/people/projects, the next UPDATE that includes it must also pass it through logChanges(). The function ignores unchanged fields, so it's safe to always pass the full update set.

getDb() — the singleton

const { getDb } = require("./db");
const db = getDb();

Returns the singleton better-sqlite3 connection. Calls initSchema(_db) on first call. Subsequent calls return the same instance. WAL mode + foreign keys on.

Don't create your own better-sqlite3 instance — always go through getDb(). Tests stub it.

Schema and migrations

initSchema(db) creates every table (CREATE TABLE IF NOT EXISTS ...). It's idempotent — safe to call on every startup.

Adding a column to an existing table uses the PRAGMA check pattern (see db.js around line 173 for a real example):

const cols = db.prepare("PRAGMA table_info(companies)").all().map((c) => c.name);
if (!cols.includes("review_link")) {
  db.exec("ALTER TABLE companies ADD COLUMN review_link TEXT;");
}

Wrap each migration in its own column-check block. Migrations run after initSchema() on every startup. They must be idempotent.

For column type changes or constraint changes, SQLite forces a table rebuild. The pattern is "create <name>_new, copy data, drop old, rename" — there are several examples in db.js you can crib from.

For schema changes that affect agents or readers, also update the AUTO-SYNC schema table in CLAUDE.md. The generated Schema reference updates automatically on deploy.

Seed data

seed.js exports runSeed(db), called once on server startup after initSchema(). Uses INSERT OR IGNORE everywhere so re-runs are safe. The seed populates a small set of demo companies, the team members, and basic settings.

For a fresh local DB, the seed gets you to a usable state. For a production DB, the seed only adds missing rows — it never overwrites.

Constraints worth knowing

Adding a route — the 30-second version

if (match("GET", /^\/api\/laurelin\/<resource>$/)) {
  const rows = db.prepare("SELECT ...").all();
  json(res, 200, rows);
  return true;
}

if (match("POST", /^\/api\/laurelin\/<resource>$/)) {
  return readBody(req).then((body) => {
    // validate, insert
    const id = uuid();
    db.prepare("INSERT INTO ... VALUES (?, ?, ?)").run(id, body.foo, body.bar);
    json(res, 201, { id });
    return true;
  });
}

if ((m = match("PUT", /^\/api\/laurelin\/<resource>\/([^/]+)$/))) {
  const id = m[1];
  return readBody(req).then((body) => {
    const existing = db.prepare("SELECT * FROM <resource> WHERE id = ?").get(id);
    // ... build update
    logChanges(db, "<resource>", id, existing, updates, body.changed_by);
    json(res, 200, /* updated row */);
    return true;
  });
}

That's the entire pattern. Look at any block in routes.js for a more concrete example.

Server entry & static file serving

server.js does three things, in order, per request:

  1. If the URL is /api/laurelin/*, call handleLaurelinRoutes.
  2. If the URL is /api/recruiting/*, call the recruiting handler.
  3. Otherwise, serve a static file from the appropriate directory.

The static-file serving is intentionally minimal — Nginx handles the bulk of static delivery in production. The Node server fronts the API and the static files only for development.

Logs and error handling

The server logs to stdout. In production, that goes to the systemd journal — journalctl -u valinor-intra -f to tail.

Per-request error handling is per-handler. There's no global error middleware (no Express). If a handler throws, the request will hang and eventually time out — wrap risky work in try/catch and return a JSON error via json(res, 500, { error: "..." }).

Testing

There's no automated test harness for Laurelin routes. Manual testing is the norm:

If you're making a structural change (adding a new resource, refactoring a handler), exercise both the route and the UI in dev before pushing.