From 9727ce79d3b7ac6de03303d6beb2692574bdacea Mon Sep 17 00:00:00 2001 From: Local Dev Date: Sun, 4 Oct 2026 01:44:23 +0200 Subject: [PATCH] Pithos: document hosting on Silent Mode and the gateway Records what was built against the plan (names are encrypted from v1, rights for other people are folder-scoped S3 keys), how to run pithos-hosted, and that paid storage belongs to Hestia, a separate product. --- DESIGN-hosted.md | 46 +++++++++++++++++------------- README.md | 38 ++++++++++++++++++++++--- hosted/README.md | 73 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 133 insertions(+), 24 deletions(-) create mode 100644 hosted/README.md diff --git a/DESIGN-hosted.md b/DESIGN-hosted.md index 4aad202..5ae57f7 100644 --- a/DESIGN-hosted.md +++ b/DESIGN-hosted.md @@ -1,9 +1,9 @@ # Pithos "On Silent Mode": hosted s3d with your own Sia account -**Status:** plan, 2026-10-04. Not built. -**Scope:** bring-your-own-account only. Silent Mode-paid storage (500 MB free web -hosting, paid 10/20/50 GB) is a separate Sirius.X project. The gateway below is -built so Sirius.X can reuse it (see "Later"). +**Status:** built 2026-10-04 (Pithos 0.3.7, `hosted/`), not deployed. +Exercised end to end against a mock indexer; not yet against sia.storage. +**Scope:** bring-your-own-account only. Paid storage is Hestia, a separate +product (see "Later"). ## What the user gets @@ -25,7 +25,7 @@ siacoins. What we pay for is VPS memory and disk and all hosted traffic. |---|---| | Model | Bring your own sia.storage account. Silent Mode plans are out of scope here. | | Indexer | sia.storage (the user's own account). No Silent Mode indexd. | -| Encryption | Private files are encrypted on the user's machine before upload. Anything public or shared by link is stored readable. | +| Encryption | Private files are encrypted on the user's machine before upload, file names included. Anything public or shared by link is stored readable. | | Anti-Sybil | A hosted slot needs a BNS name or a small refundable BCH deposit, plus a per-IP sign-up limit. With BYO accounts this protects RAM and bandwidth, not storage. | ## Why one s3d per user @@ -103,8 +103,8 @@ keys never leave the PC. - Bodies: AES-256-GCM in 1 MiB chunks, each with its own nonce and the chunk index authenticated. Range reads and streaming keep working. - Names: encrypted per path segment with AES-SIV (deterministic), so folders - still list and sort and the server sees no file names. *Open point: costs a few - days; bodies-only is the cheaper v1. Recommend doing names too.* + still list and the server sees no file names (decided: in v1). Sizes, folder + depth and timestamps remain visible. - Making an encrypted folder public or shared by link re-uploads it decrypted, after a confirmation that says what becomes readable. - Losing the vault means losing the files. Same as the rest of Silent Mode. @@ -131,14 +131,19 @@ checks this before it starts. ## Build order -| Step | What | Size | +| Step | What | State | |---|---|---| -| 1 | Supervisor + per-user `createPithos` + sign-in/sessions + anti-Sybil check | 3-4 days | -| 2 | Pithos client mode switch, forwarder, hosted status in Overview | 2 days | -| 3 | Migration both ways, live-elsewhere lock | 2 days | -| 4 | Encryption (bodies, then names) | 3-4 days | -| 5 | Gateway SigV4 keys, access rules, public URLs, share links | 3-4 days | -| 6 | Meters, limits, deploy (DNS for s3.silentmode.st, nginx, systemd), docs | 2 days | +| 1 | Supervisor + per-user `createPithos` + sign-in/sessions + anti-Sybil check | done | +| 2 | Pithos client mode switch, forwarder, hosted status in Overview | done | +| 3 | Migration both ways, live-elsewhere lock | done | +| 4 | Encryption (bodies and names) | done | +| 5 | Gateway SigV4 keys, access rules, public URLs, share links | done | +| 6 | Meters, limits, docs | done; deploy (DNS for s3.silentmode.st, nginx, systemd) waiting | + +One change from the plan: "rights for a person" are S3 keys the owner mints +for one folder (read, add-only or edit), not a second sign-in by the other +person's wallet. Keys work with every S3 client and need no account on the +other side; a wallet-bound variant can come with encrypted sharing (v2). Pithos changes ship as extension OTA versions, never a Theseus release. @@ -147,11 +152,12 @@ Pithos changes ship as extension OTA versions, never a Theseus release. - Host: the current VPS is at 79% disk. Upload buffers need headroom, so this may need a separate box. Measure s3d memory per process first. - Download cap per user (proposed 50 GB per month). -- Encrypt file names in v1 or later. -## Later: Sirius.X plans +## Later: Hestia -The Silent Mode-paid tiers (500 MB free web hosting, 10/20/50 GB paid) plug into -the same supervisor. Their "per-user s3d" connects to Silent Mode's sia.storage -account instead of the user's, with quotas enforced at the gateway as Hestia -describes. Sirius.X can take over the gateway as its storage backend. +Selling storage is a separate product: Hestia, enterprise software with which +an operator sets their own storage limits, prices and plans and monetizes +storage (Silent Mode's own 500 MB free web hosting and paid tiers would be one +deployment of it). It can reuse pieces built here: the per-account s3d +supervisor, the gateway's SigV4 and rule enforcement, client-side encryption. +Pithos stays free of quotas and billing. diff --git a/README.md b/README.md index e017cd9..c8facd4 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,37 @@ admin API; Pithos puts a UI over all of it: drag-and-drop, download, delete files and folders, presigned share links, public-read policies. - **Upload pipeline**: pending, uploaded, unpinned and failed objects from - `/stats/uploads`, plus "flush now". + `/stats/uploads`, plus "flush now" and an optional automatic upload of + leftovers after N quiet minutes. +- **On Silent Mode** (optional): run the same s3d on Silent Mode's server + with your own Sia account, with client-side encryption, public folders, + share links and S3 keys for other people. See below. + +## On this computer, or on Silent Mode + +By default s3d runs on your computer. Settings › *Where your drives live* +can move it to `pithos-hosted` on Silent Mode instead, where it keeps working +with your computer off and is reachable from other devices. The interface +stays the same. + +- **Your own Sia account.** Only s3d's data folder moves: its database and any + files still waiting for Sia. The recovery phrase never leaves your + computer, and nothing already on Sia is uploaded again. Moving back works + the same way in reverse. +- **Encrypted before it leaves.** In hosted mode Pithos encrypts file names + (AES-SIV per path segment, so folders still list) and contents (chunked + AES-256-GCM, so range reads stay cheap) on your computer, with keys from + your Theseus vault or, on other hosts, a key file beside `s3d.yml` that + you should back up. See `core/crypt.js`. +- **Sharing a bare s3d cannot do.** Public folders, expiring share links, and + S3 keys for other people limited to one folder with read, add-only or edit + rights. Those files are stored readable, because the people they are for + have no key; stop sharing and Pithos encrypts them again. +- **One slot per person.** Signing up needs a signature from the wallet that + owns a BNS name, a small refundable BCH deposit, or an invite. + +The server is in `hosted/` (see `hosted/README.md`); the design and its +trade-offs are in `DESIGN-hosted.md`. ## One core, three hosts @@ -137,9 +167,9 @@ keys and prefixes and on presigned URLs (`npm test`). - Object versioning, lifecycle rules and multipart-upload cleanup are not in the UI yet. - Desktop installers (electron-builder) are not set up yet. -- Multi-tenant mode, where end users sign in with a wallet and each get - their own s3d user and quota, is the planned follow-up. It is the - Hestia direction in `Hephaestus/DESIGN-hestia.md`. +- Storage paid to Silent Mode (quotas, plans, billing) is not part of + Pithos. That is Hestia (`Hephaestus/DESIGN-hestia.md`), a separate product + for operators who sell storage. ## License diff --git a/hosted/README.md b/hosted/README.md new file mode 100644 index 0000000..39fb197 --- /dev/null +++ b/hosted/README.md @@ -0,0 +1,73 @@ +# pithos-hosted + +The server half of Pithos's *On Silent Mode*: one s3d per person, each on +that person's own Sia account, behind a gateway that adds sign-in, per-folder +access and traffic limits. The client half (encryption, moving drives) is in +`../core/hosted.js` and `../core/hosted-routes.js`. + +## What it serves + +| Path | Who | What | +| --- | --- | --- | +| `/hosted/v1/challenge`, `/hosted/v1/session` | anyone | Ed25519 sign-in; first sign-in needs a sign-up proof | +| `/hosted/v1/me`, `/rules`, `/data/...` | session | account, access rules, moving the s3d data folder in and out | +| `/api/...` | session | the owner's Pithos API, forwarded to their s3d's control server (an allowlist; object routes are refused so nothing bypasses client-side encryption) | +| `/p///` | anyone | files under a public rule | +| `/l/` | anyone | one file under a share link, until it expires | +| everything else | S3 keys | path-style S3. Owner keys go straight to their s3d, which checks the signature; keys minted for other people are verified here, held to their folder and rights, and re-signed with the owner's key | + +s3d processes listen on loopback only (`portBase + 2 * slot`). Each one +starts on the account's first request and is stopped after `idleMinutes`, +leftovers flushed to Sia first; at most `maxRunning` run at once. + +State is files under `root`: `users//` (account, s3d.yml, data, rules, +usage), `sessions.json` (token hashes), `gate.json`, `keys.json`, +`grants.json`, `links.json`. Back up that folder and you have backed up the +service. + +## Sign-up gate + +`gate` in the config enables any of: + +- `bns: { api }`: the user signs `pithos-hosted signup ` with + the wallet that owns a BNS name; `api` is a BNS gateway answering + `/api/name/` with the owner. One slot per name. +- `deposit: { address, minSats, confirmations, electrum: { host, port, tls } }`: + a BCH payment to `address` carrying OP_RETURN `pithos:`. One + slot per transaction. Refunds are an operator decision. +- `invites: ["code", ...]`: one use each. + +`perIpPerDay` limits sign-ups per client address (set `trustProxy` behind +nginx so `X-Forwarded-For` is used). + +## Running it + +```bash +cd /opt/pithos && npm install --omit=dev --omit=optional +cd hosted && npm install --omit=dev +cp deploy/pithos-hosted.example.json /etc/pithos-hosted.json # edit it +node main.js --config /etc/pithos-hosted.json +``` + +`deploy/` has a systemd unit and an nginx site for `s3.silentmode.st` +(streamed bodies, `Host` passed through: s3d checks signatures against the +host the client signed). The s3d binary must be the version Pithos pins +(`core/binary.js`), the same on both sides of a move. + +Stopping the service flushes every running s3d first, so give systemd time +(`TimeoutStopSec=300`). + +## Tests + +`npm test` here covers SigV4 verification, rules and the sign-up gate. +The full path (move up, encrypt, share, grant, move back) needs a real s3d; +it was exercised against a mock indexer during development. + +## Not yet + +- Signed streaming uploads (`STREAMING-AWS4-HMAC-SHA256-PAYLOAD`) from shared + keys are refused; owner keys are unaffected. +- Shared keys and links reach readable copies only. Sharing encrypted files + with a person (wrapping the folder key to theirs) is the next step. +- No quotas here: storage is each person's own Sia account. Paid storage is + Hestia, a separate product.