92260d177e
In Theseus, Pithos now links the person's Theseus ID to their hosted account by itself, once, right after signing in to Silent Mode (or on the first look at an account already signed in): one consent prompt, then Pithos is listed under Settings › Theseus ID. On a Theseus without the add-on API it does nothing, so the add-on can ship before the Theseus release that carries it. The gateway serves /console on its own origin: sign in with the Theseus ID and see the account's state, sign-up, usage and Theseus ID, with a console session that cannot reach files. Three fixed files under a CSP that allows only this origin. The bundled copy in TheseusNavigator/bundled-addons/pithos is rebuilt from Pithos/ with no version bump; the release that ships it bumps the version.
196 lines
10 KiB
Markdown
196 lines
10 KiB
Markdown
# Pithos "On Silent Mode": hosted s3d with your own Sia account
|
|
|
|
**Status:** built 2026-10-04 (Pithos 0.3.7, `hosted/`), not deployed.
|
|
Exercised end to end against a mock indexer; not yet against sia.storage.
|
|
**Scope:** bring-your-own-account only. Paid storage is Hestia, a separate
|
|
product (see "Later").
|
|
|
|
## What the user gets
|
|
|
|
Pithos gains a choice under Settings: **On this computer** (today, unchanged,
|
|
still the default) or **On Silent Mode**. The interface is the same in both
|
|
modes: drives, browser, viewer, Users & keys, share links. Hosted mode adds:
|
|
|
|
- nothing to keep running: apps, S3 clients and share links work with the PC off
|
|
- the same drives from any device, at `https://s3.silentmode.st`
|
|
- per-folder public access and per-person rights (read / add / edit), which a
|
|
bare s3d cannot do
|
|
|
|
The storage stays the user's own sia.storage account, so it costs Silent Mode no
|
|
siacoins. What we pay for is VPS memory and disk and all hosted traffic.
|
|
|
|
## Decisions taken
|
|
|
|
| | |
|
|
|---|---|
|
|
| Model | Bring your own sia.storage account. Silent Mode plans are out of scope here. |
|
|
| Indexer | sia.storage (the user's own account). No Silent Mode indexd. |
|
|
| Encryption | Private files are encrypted on the user's machine before upload, file names included. Anything public or shared by link is stored readable. |
|
|
| Anti-Sybil | A hosted slot needs a BNS name or a small refundable BCH deposit, plus a per-IP sign-up limit. With BYO accounts this protects RAM and bandwidth, not storage. |
|
|
|
|
## Why one s3d per user
|
|
|
|
An s3d process is bound to one Sia account: its connection key, users, keys,
|
|
buckets and upload buffer all live in its data folder (`s3d.db`). That is also
|
|
why Pithos's account switch works by moving the folder. So the server runs
|
|
**one s3d per hosted user**. Each one gets its own data folder, loopback-only
|
|
ports, and admin password.
|
|
|
|
- **Your recovery phrase never leaves your PC.** You connect locally as today.
|
|
Pithos then uploads the connected data folder, which carries only the connection
|
|
key for this one app. The connection uses one of sia.storage's 3 app slots per
|
|
account.
|
|
- **The server holds that connection key** and so could read anything s3d
|
|
stores. This is why private files get the extra client-side layer below.
|
|
- **Idle s3d processes are stopped.** A user's s3d starts on the first request
|
|
(a few seconds) and stops after 30 min with no requests and nothing pending.
|
|
Before it stops, leftovers are flushed to Sia (the auto-flush setting, forced on
|
|
in hosted mode).
|
|
|
|
## Server: `pithos-hosted` on the VPS
|
|
|
|
A Node service in `Pithos/server-hosted/` that reuses `core/`. It does not
|
|
duplicate it.
|
|
|
|
1. **Supervisor.** `/var/lib/pithos-hosted/<uid>/` holds `s3d.yml` and the data.
|
|
Ports come from a pool and are bound to 127.0.0.1 only. A per-user
|
|
`diskUsageLimit` keeps upload buffers bounded, and there's a cap on how many
|
|
s3d run at once (LRU stop). systemd unit, nginx TLS, deployed from the repo.
|
|
2. **Per-user Pithos.** Each user gets one `createPithos()` instance with
|
|
`hostName: 'hosted'`, so the exact same `/api/*` surface is served. Routes that
|
|
make no sense on a server are refused in hosted mode: binary install, address
|
|
settings, login pty, open-external, account switch/restore.
|
|
3. **Sign-in.** The identity is a key from the Theseus vault (new derive path
|
|
`pithos/hosted-identity/v1`; the existing `pithos/sia-recovery/v1` and
|
|
`pithos/credentials/v1` stay untouched). Challenge → signature → session token.
|
|
The desktop and web hosts use a local identity file that can be exported and
|
|
imported. Sign-up runs the anti-Sybil check: a BNS name owned by the key's
|
|
address, or a deposit tx.
|
|
4. **Gateway S3 keys.** Hosted s3d is never exposed. The gateway mints its own
|
|
SigV4 keys, verifies them itself (`core/s3.js` already signs, so verification
|
|
mirrors it) and re-signs to the user's s3d. That's what makes the rules below
|
|
enforceable, and it lets rclone, aws-cli and the like keep working.
|
|
5. **Meters.** Monthly download bytes per user, with a cap (proposed 50 GB per
|
|
month, then throttled) and per-IP sign-up limits.
|
|
|
|
## Per-folder access and rights
|
|
|
|
Rules are stored per user: `{ drive, prefix, who, rights, expires }`.
|
|
|
|
- `who`: `public`, a link token, or a person (BNS name or wallet address, who signs
|
|
in like any user)
|
|
- `rights`: `read` (GET, HEAD, list), `add` (create new keys only: PUT with
|
|
If-None-Match / existence check, no DELETE, no overwrite), `edit` (PUT and DELETE)
|
|
|
|
Every request through the gateway, whether S3 API, `/api`, public URL
|
|
`https://s3.silentmode.st/p/<user>/<drive>/<path>` or share link, is matched
|
|
against the longest prefix rule. List results are filtered to what the caller may
|
|
see. The owner always has full rights. This is what s3d alone can't do: its
|
|
bucket policy is whole-bucket public read only.
|
|
|
|
Encrypted folders can be shared with a person only once key sharing exists (the
|
|
folder key wrapped to the grantee's public key; v2). Until then, the UI offers
|
|
to share those folders as readable copies, and says so.
|
|
|
|
## Client-side encryption
|
|
|
|
The encryption happens in the local Pithos core. In hosted mode it acts as an
|
|
encrypting forwarder between the unchanged UI and the gateway, so plaintext and
|
|
keys never leave the PC.
|
|
|
|
- Key: HKDF from the vault, path `pithos/hosted-encryption/v1`, one key per drive
|
|
(`HKDF(master, drive id)`).
|
|
- Bodies: AES-256-GCM in 1 MiB chunks, each with its own nonce and the chunk
|
|
index authenticated. Range reads and streaming keep working.
|
|
- Names: encrypted per path segment with AES-SIV (deterministic), so folders
|
|
still list and the server sees no file names (decided: in v1). Sizes, folder
|
|
depth and timestamps remain visible.
|
|
- Making an encrypted folder public or shared by link re-uploads it decrypted,
|
|
after a confirmation that says what becomes readable.
|
|
- Losing the vault means losing the files. Same as the rest of Silent Mode.
|
|
|
|
## Moving between local and hosted
|
|
|
|
No file is re-uploaded to Sia. Only s3d's metadata folder moves.
|
|
|
|
**Local → hosted:** flush everything to Sia, stop local s3d, upload the data
|
|
folder (TLS, authenticated), then the gateway starts s3d on it and checks the
|
|
bucket list. The local copy is archived like an account switch, never deleted.
|
|
Pithos refuses to run a local s3d on a data folder that's live on the server:
|
|
two s3d writing one account's metadata would diverge.
|
|
Existing files stay unencrypted. Pithos offers "Encrypt existing private files",
|
|
a background re-upload, and lists what is still readable until that's done.
|
|
|
|
**Hosted → local:** the gateway flushes and stops, Pithos downloads the folder,
|
|
starts local s3d on it, and the gateway deletes its copy after the local start
|
|
succeeds. Encrypted files remain encrypted. Pithos decrypts them on read in both
|
|
modes.
|
|
|
|
s3d must be the same pinned version on both sides (today 0.2.0). The migration
|
|
checks this before it starts.
|
|
|
|
## Build order
|
|
|
|
| Step | What | State |
|
|
|---|---|---|
|
|
| 1 | Supervisor + per-user `createPithos` + sign-in/sessions + anti-Sybil check | done |
|
|
| 2 | Pithos client mode switch, forwarder, hosted status in Overview | done |
|
|
| 3 | Migration both ways, live-elsewhere lock | done |
|
|
| 4 | Encryption (bodies and names) | done |
|
|
| 5 | Gateway SigV4 keys, access rules, public URLs, share links | done |
|
|
| 6 | Meters, limits, docs | done; deploy (DNS for s3.silentmode.st, nginx, systemd) waiting |
|
|
|
|
One change from the plan: "rights for a person" are S3 keys the owner mints
|
|
for one folder (read, add-only or edit), not a second sign-in by the other
|
|
person's wallet. Keys work with every S3 client and need no account on the
|
|
other side; a wallet-bound variant can come with encrypted sharing (v2).
|
|
|
|
Pithos changes ship as extension OTA versions, never a Theseus release.
|
|
|
|
## Open points
|
|
|
|
- Host: the current VPS is at 79% disk. Upload buffers need headroom, so this
|
|
may need a separate box. Measure s3d memory per process first.
|
|
- Download cap per user (proposed 50 GB per month).
|
|
|
|
## Later: Hestia
|
|
|
|
Selling storage is a separate product: Hestia, enterprise software with which
|
|
an operator sets their own storage limits, prices and plans and monetizes
|
|
storage (Silent Mode's own 500 MB free web hosting and paid tiers would be one
|
|
deployment of it). It can reuse pieces built here: the per-account s3d
|
|
supervisor, the gateway's SigV4 and rule enforcement, client-side encryption.
|
|
Pithos stays free of quotas and billing.
|
|
|
|
## Theseus ID
|
|
|
|
A hosted account is its Ed25519 storage key, and stays so: that key owns and
|
|
decrypts the data. A Theseus ID (DESIGN-theseus-id.md §11.3) is added as a way
|
|
to look at the account from a browser that does not hold the key.
|
|
|
|
- **Linking.** In Theseus, right after Pithos signs in to Silent Mode (and
|
|
once per run for an account already signed in), the add-on asks
|
|
`api.theseusId.signIn` for project `pithos`, signed as origin
|
|
`theseus-addon://pithos`, and posts it to `POST /hosted/v1/theseus-id` with
|
|
its owner session. Theseus asks the person once. A Theseus without the
|
|
"theseus-id" add-on API skips this; nothing else changes.
|
|
- **The console.** `GET /console` on the gateway's own origin
|
|
(s3.silentmode.st, on the `pithos` origin list) signs in with
|
|
`window.theseusId` through `POST /hosted/v1/console/session` and gets a
|
|
session with scope `console`. That scope reaches `GET /hosted/v1/me`, its
|
|
own sign-out and the link status, and nothing else: no files, rules, moves
|
|
or forwarded owner API. Console sessions last 7 days.
|
|
- **Routes.** `GET /hosted/v1/theseus-id/challenge` (nonce),
|
|
`POST /hosted/v1/console/session`, and with an owner session
|
|
`GET|POST|DELETE /hosted/v1/theseus-id`. `tid-links.json` at the root maps
|
|
each linked ID to its account; `account.json` carries `theseusId`.
|
|
- **Verification** is `hosted/theseus-id/`: TheseusID/lib's `message.mjs` and
|
|
`verify.mjs` copied verbatim (a test fails if they drift) with libauth doing
|
|
the curve maths.
|
|
- **Deploying.** The server now has two sub-folders, `hosted/theseus-id/` and
|
|
`hosted/console/`. Copying `hosted/*.js` alone misses them; copy both
|
|
folders too.
|
|
- **Not yet.** Signing in to the console from other browsers through
|
|
id.theseus.x (the `pithos` client on the login service), and a browser-app
|
|
sign-in that also unlocks the drives: that needs the encryption keys, which
|
|
a console session must never carry.
|