theseus/DESIGN-bns-indexer-service.md

# BNS indexing: Ariadne's Thread owns it, everyone reads it

Status: design, revised 2026-10-03.

Already implemented:
- the public raw snapshot, with two copies (see the server tier);
- Theseus's indexer process `bns-indexer.js`, which becomes the last-resort
  fallback here;
- Theseus's quick single-name lookup while no index exists.

## The model

```
 server                         this machine                                     readers
 ──────                         ──────────────────────────────────────────       ───────
 gateway index ─┬─► public       Ariadne's Thread   (scope: All users | Just me)
 (electrum)     │   snapshot ──►   indexer  ──writes──►  local snapshot copies    Theseus
                │   ×2 copies      (sync, may stop,      (read-only for readers)  ──┐
                │   read-only       pause, crash)                 │                │ same
                │                      │ pipe (live, optional)    │                │ source
                └─► public BNS         ▼                          ▼                │ chain
                    indexer       resolver (All browsers mode only) ◄──────────────┘
                    /api/name/<n>  DNS :53 + NRPT + gateway  (All users)
                    (single names) PAC proxy + gateway       (Just me)        other browsers
```

Ariadne's Thread is the parent of everything BNS on a machine. It has two
processes with different jobs:

- **The indexer** builds and maintains the index from the raw beacon
  evidence and writes the local snapshot copies. It is heavy-ish (electrum
  sync), and it is allowed to stop: Economy, "Launch at start" off, a crash,
  an update.
- **The resolver** answers names: for other browsers in All browsers mode,
  through system DNS or the per-user proxy. It is small and does no syncing
  of its own beyond its fallbacks.

**Resolution never depends on the indexer.** The resolver and Theseus read
names through the same fallback chain (below), in which the live indexer is
only the first and freshest source. If it is gone, the answers come from the
local snapshot copies, the public snapshots or the public BNS indexer. The
only thing lost is freshness, and only until a source with newer data
answers.

## Tiers

### Server (done)
- **Live index:** the gateway keeps it over electrum, and serves single names
  at `/api/name/<n>`. This is the **public BNS indexer**. Its answers carry
  records and owner, interpreted by the operator; clients treat them as
  provisional (see below).
- **Raw snapshot:** `cdn-name-snapshot` (VPS, every 10 min) publishes it as
  **two public, read-only copies on two storage backends**:
  - `https://dl.silentmode.st/bns-name-snapshot.json` (the dl vhost's disk)
  - `https://navigate.st/bns/silentmode.bch/bns-name-snapshot.json` (the Sia
    bucket, served by the gateway)

  Both are static, with CORS `*` and no credentials. Each contains the
  history plus the verbose transactions, so clients rebuild the index from
  the evidence themselves. A failed upload of the second copy is retried on
  the next run. Clients try them in that order (`nameListUrl`,
  `nameListMirrorUrl`).
- **Android** keeps using the interpreted `bns-name-index.json`.

### The source chain (the resolver and Theseus alike)

Fastest first. A lookup is answered by the first source that has the name;
the slower sources refresh in the background.

1. **In memory:** whatever the reader has already loaded (normally all of
   it).
2. **The live indexer over the pipe:** `snapshot` on connect, then
   `subscribe` for changes. This is the freshest source when it is running.
3. **The local snapshot copies:** a disk read of milliseconds. There are two,
   so a deleted, moved or half-written file still leaves one:
   - Ariadne's: `C:\ProgramData\Ariadne\` (All users) or
     `%LOCALAPPDATA%\Ariadne\` (Just me), written only by the indexer.
   - The reader's own last-known-good copy. For Theseus that is its profile;
     for the resolver it is beside Ariadne's file.
4. **The public snapshots,** either copy: one download of about 450 KB,
   rebuilt locally from the transactions. It always runs in the background;
   nothing waits for it.
5. **The public BNS indexer** (`/api/name/<n>`): a single-name lookup of
   about 0.2 s. Used when sources 1–3 have no data yet, or the name is
   missing and the data is stale. The answer is **provisional**:
   - It is served at once.
   - It is compared with the verified index when that arrives, and on a
     mismatch the tabs or cached answers are replaced.
   - It is never used for the extension-publisher check, which waits for
     verified data.
6. **Theseus only, last resort:** its own electrum indexer process
   (`bns-indexer.js`), for a portable Theseus with no Ariadne and no CDN.

**Privacy:** sources 4–5 only ever see BNS names.
- The resolver only receives queries for BNS TLDs (NRPT and the PAC file
  route only those).
- Theseus only asks for names under known BCNR TLDs.
- An ordinary web host is never sent to the public indexer.

### Ariadne's Thread: the indexer
- **One implementation** with Theseus's indexer logic (`bns-indexer.js`):
  - warm start from the local snapshot;
  - catch-up from the public snapshots;
  - the electrum delta poll every 30 s;
  - the index rebuilt from the beacon transactions, with `owner`;
  - the TLD set from `tlds.bch`.

  This replaces the daemon's current warm start from the pre-resolved list,
  which has no owners and an operator-interpreted index.
- **Local snapshot:**
  - It uses the raw format, written atomically after every change, and the
    previous file is kept as the second copy.
  - In All users, **users may read it; only SYSTEM and Administrators may
    write it.** The transactions are not checked against block headers, so
    a user-writable copy would let any local process insert names or owners.
  - `policy.json` stays user-writable, as today; it holds settings, not
    data.
- **The pipe:** `\\.\pipe\ariadne-bns`, or `…-<user SID>` for Just me.
  - **Requests:** `hello` returns `{version, network, gen, builtAt,
    snapshotPath}`; `snapshot` and `get <name>` return index data; `poll`
    runs one delta poll on demand.
  - **Push:** `subscribe` sends `{gen, changed}` on every change.
  - **Server check:** a client checks that the server process is Ariadne's
    indexer before trusting it (`GetNamedPipeServerProcessId`, then the
    process image path under Program Files and, for All users, its owner
    SYSTEM). Otherwise a process could take the pipe name while the indexer
    is down and feed false owners to the extension-publisher check. A failed
    check simply skips to source 3.
  - **Not TCP:** a 127.0.0.1 port has no way to tell who is listening.

### Ariadne's Thread: the resolver
- **Runs only in All browsers mode;** Theseus is its own resolver. It is
  small: DNS, proxy and gateway listeners plus the source chain, with no
  electrum.
- **On its own:**
  - With the indexer down it keeps answering from the local copies.
  - It fetches the public snapshot once an hour when the local copies are
    older than that, holding the result in memory only, since the indexer
    owns the files.
  - It uses the public indexer for single misses.
- **If the resolver itself is down,** the task restarts it within a minute.
  With NRPT routing the BNS TLDs to 127.0.0.1 there is no system fallback
  for those names meanwhile (ICANN names are unaffected). Open question: a
  public BNS DNS server as NRPT's second name server, and a public proxy as
  the PAC file's second entry.

### Ariadne's Thread: scope × mode

**Scope** decides who it runs for:
- **All users** (the admin path): system processes for every account on the
  OS.
- **Just me:** runs under one account, with no admin.

**Mode** decides who it resolves for:
- **Theseus only.**
- **All browsers:** every browser on the machine, and for All users every app
  as well.

| | **Theseus only** | **All browsers** (default) |
|---|---|---|
| **All users** (admin, SYSTEM at boot) | indexer only | indexer + resolver: system DNS on :53 with NRPT rules, the local gateway on :80/:443, a machine-wide CA. Every app on the PC resolves BNS names |
| **Just me** (no admin, at logon) | indexer only | indexer + resolver: a PAC file set in the account's own Windows proxy settings (HKCU, no admin) points at a local proxy on a high port, which routes BNS names to the gateway and passes everything else through. The CA goes into the user's certificate store, which Windows confirms once |

Notes on **Just me + All browsers**:
- Chrome, Edge and Brave follow the Windows proxy settings. Firefox follows
  them when set to "use system proxy", its default, but needs the CA in its
  own store, which Ariadne already handles for the machine CA.
- Programs that ignore the system proxy (some command-line tools, apps with
  their own resolver) do not see BNS names. Covering them takes system DNS,
  which needs admin.
- A PAC file is not DNS, so it also works when a browser uses its own
  DNS-over-HTTPS, which bypasses the NRPT route on networks that tamper with
  DNS.

**Where the mode lives:**
- All users: `policy.json` in ProgramData. Just me: `policy.json` in the
  user's profile. The field is `"mode": "theseus" | "all-browsers"`.
- Switching starts or stops the resolver and adds or removes the NRPT rules,
  or the per-user proxy setting.
- **It never touches the indexer.**

### Ariadne's Thread: lifetime and start-up

Both processes are background processes with no window, independent of
Theseus.

| | **All users** | **Just me** |
|---|---|---|
| Runs as | SYSTEM, scheduled tasks | the user, scheduled tasks (hidden) |
| Starts | at OS start, before anyone logs on | at that user's logon, the earliest without admin |
| Keeps running | until shutdown | until that user logs off |

To start before logon, a user switches the scope to All users (one UAC
prompt).

**Task settings, both tasks and both scopes:**
- Restart on failure (5 times, 1 min apart).
- No execution time limit.
- **`-AllowStartIfOnBatteries -DontStopIfGoingOnBatteries`.** Today's task
  uses Windows' defaults, which skip the start on battery and stop the task
  when a laptop unplugs. Power is a setting of Ariadne's (below), not
  something Windows decides by killing the process.
- Below-normal process and I/O priority for the indexer.

**"Launch at start" (on by default):** a switch in Ariadne's Thread settings,
mirrored in Theseus Settings › Plug-ins › Ariadne's Thread.
- **Scope of the switch:** it applies to the **indexer**. The resolver
  follows the mode: in All browsers it always starts, because other browsers
  depend on it and it is small.
- **Where it lives:** `policy.json`, `"startAtBoot": true | false`. The
  running process enables or disables the trigger on its own task, so no
  UAC prompt is needed in either scope.
- **On:** the indexer is up from OS start (All users) or logon (Just me),
  whether or not Theseus is ever opened.
- **Off:** the indexer starts on demand when Theseus launches, by running the
  task. For All users the task's security descriptor lets local users run it,
  but not change it. Until then, names resolve from the source chain as
  usual, in Theseus and in other browsers; only the freshness depends on the
  indexer.
- **Turning it back on** starts the indexer at once as well as at the next
  boot.

**Power: "Performance" or "Economy"**, in Ariadne's Thread settings,
mirrored in Theseus. It lives in `policy.json`: `"power": "performance" |
"economy"`, plus `"economyWhen": "on-battery" | "low-battery"`.

- **Performance (default):** the indexer syncs normally on battery.
- **Economy:** while the condition holds, the indexer **pauses its network
  work**: the electrum poll, the published-snapshot refresh and server
  discovery.
  - Resolution is unaffected. Readers keep answering from memory and the
    local copies, and the public indexer still covers single misses.
  - When the condition clears, the indexer resumes with one immediate
    catch-up poll.
  - **The condition**, `economyWhen`:
    - `on-battery`: running on battery, the old behaviour of Windows' default
      task settings.
    - `low-battery`: Windows' Battery Saver is on, which Windows turns on at
      20% by default.
  - Pausing rather than stopping keeps the warm process and a cheap resume.
    Stopping would be just as safe for resolution, since the readers do not
    depend on the indexer.
- **Detection:**
  - The indexer reads the power state every 60 s and on resume-from-sleep: AC
    or battery and the Battery Saver flag, from `GetSystemPowerStatus` via a
    small helper, or one long-lived PowerShell process, never one process per
    check.
  - Theseus's fallback indexer uses Electron's `powerMonitor`.
- **Desktops** (no battery) never enter Economy, and the switch is hidden.

### Theseus
- **Launch:** the source chain above. In practice: Ariadne's local snapshot
  file (milliseconds, parsed in Theseus's indexer process, never on the main
  thread), then `subscribe` to the pipe for changes.
- **Its own copy:** Theseus keeps its own last-known-good copy in its
  profile, so a missing Ariadne file still leaves one.
- **No index yet:** the quick single-name lookup (source 5) while the public
  snapshot downloads in the background.
- **Never writes Ariadne's files,** and never starts, stops or reconfigures
  the indexer. The one exception is running its task on demand when "Launch
  at start" is off. Its Settings page shows Ariadne's status and edits its
  `policy.json`.

## Installer

- **Ariadne's Thread is always installed** with Theseus. There is no "install
  Ariadne?" question.
- **Setup asks two things on one page:** the scope and the mode.

### Install scope

|  | **All users** (admin, default) | **Just me** (no admin) |
|---|---|---|
| Runs as | SYSTEM scheduled tasks, at boot | per-user scheduled tasks, at logon |
| Indexer + local snapshot | one per machine, `C:\ProgramData\Ariadne\` | one per account, `%LOCALAPPDATA%\Ariadne\` |
| Who may write the snapshot | SYSTEM and Administrators | the account (the same trust level as the Theseus profile) |
| Pipe | `\\.\pipe\ariadne-bns` | `\\.\pipe\ariadne-bns-<user SID>` |
| All browsers via | system DNS + NRPT + gateway | per-user PAC proxy + gateway |

Inno Setup provides the scope choice natively (`PrivilegesRequired=lowest`
with `PrivilegesRequiredOverridesAllowed=dialog`).

**Pre-selection (decided 2026-10-03):** All users when the installing
account is an administrator; **Just me when it is not**, so a standard user
is never sent to a UAC prompt they cannot approve. Both can still pick the
other.

### Mode

- **All browsers (recommended, default):** every browser on this PC resolves
  BNS names. With All users, so does every other app.
- **Theseus only:** BNS names resolve in Theseus; nothing else on the system
  changes.

### Changing either later

From Theseus Settings › Plug-ins › Ariadne's Thread, or Ariadne's own
settings:

- **Mode, Launch at start and Power:** no prompt, in either scope
  (`policy.json`).
- **Scope,** in either direction: one UAC prompt, since it installs or
  removes the SYSTEM tasks. The local snapshot moves to the new location.
- **Both present:** if a machine install and a per-user one exist at the same
  time, readers use the machine one, and the per-user one stops itself.

### Other installer behaviour

- **Updates:** Ariadne keeps its own update lifecycle, chained by Theseus's
  updater the way it chains the add-on updates. An indexer update never
  interrupts resolution.
- **Uninstalling Theseus** asks whether to keep Ariadne's Thread. Keeping it
  is the default in All-browsers mode.

## Migration from today

1. **Lift `bns-indexer.js` into a shared module,** and the source chain into
   a shared reader module. Ariadne's two processes and Theseus use the same
   code and the same tests.
2. **Split Ariadne's daemon** (today one process, `bnsd.js`) into:
   - the indexer: raw snapshot, `owner`, `tlds.bch`, the electrum delta, the
     two local copies with their ACL, the pipe, Power;
   - the resolver: today's DNS, gateway and CA code on the shared reader,
     plus the per-user PAC proxy for Just me.

   Its read-only `/api/*` HTTP endpoints move to the resolver, for the
   gateway and the Theseus status panel.
3. **Theseus:**
   - the source chain;
   - its own snapshot copy;
   - the pipe client with the server-process check;
   - its electrum indexer demoted to the last resort.
4. **Installer:** Ariadne bundled again, with the scope and mode questions.
5. **Pages:** the tools page and theseus.x describe Ariadne's Thread as part
   of Theseus, with All browsers as the default mode.

## Open questions

- **System-level fallback while the resolver restarts:**
  - a public BNS DNS server as NRPT's second name server (All users);
  - a public proxy as the PAC file's second entry (Just me).
- **Portable Theseus:** it never installs Ariadne, so it lives on its own
  copy, the public sources and its own indexer. Should it offer to install
  Ariadne?
- **macOS/Linux:** launchd or systemd units for the same two processes, once
  Theseus ships there.