Pithos: plan for hosting s3d on Silent Mode with the user's own Sia account
One s3d serves one Sia account, so hosting means one s3d per user on the server. Writing down the agreed model (bring your own sia.storage account, client-side encryption except public files, BNS-name-or-deposit gate), how the gateway enforces per-folder rights s3d cannot, and how a data folder moves between PC and server without re-uploading to Sia, before building.
This commit is contained in:
Local Dev
parent
b399d1be46
commit
08da2d897e
1 changed files with 157 additions and 0 deletions
157
DESIGN-hosted.md
Normal file
157
DESIGN-hosted.md
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
# Pithos "On Silent Mode": hosted s3d with your own Sia account
|
||||
|
||||
**Status:** plan, 2026-10-04. Not built.
|
||||
**Scope:** bring-your-own-account only. Silent Mode-paid storage (500 MB free web
|
||||
hosting, paid 10/20/50 GB) is a separate Sirius.X project. The gateway below is
|
||||
built so Sirius.X can reuse it (see "Later").
|
||||
|
||||
## What the user gets
|
||||
|
||||
Pithos gains a choice under Settings: **On this computer** (today, unchanged,
|
||||
still the default) or **On Silent Mode**. The interface is the same in both
|
||||
modes: drives, browser, viewer, Users & keys, share links. Hosted mode adds:
|
||||
|
||||
- nothing to keep running: apps, S3 clients and share links work with the PC off
|
||||
- the same drives from any device, at `https://s3.silentmode.st`
|
||||
- per-folder public access and per-person rights (read / add / edit), which a
|
||||
bare s3d cannot do
|
||||
|
||||
The storage stays the user's own sia.storage account, so it costs Silent Mode no
|
||||
siacoins. What we pay for is VPS memory and disk and all hosted traffic.
|
||||
|
||||
## Decisions taken
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Model | Bring your own sia.storage account. Silent Mode plans are out of scope here. |
|
||||
| Indexer | sia.storage (the user's own account). No Silent Mode indexd. |
|
||||
| Encryption | Private files are encrypted on the user's machine before upload. Anything public or shared by link is stored readable. |
|
||||
| Anti-Sybil | A hosted slot needs a BNS name or a small refundable BCH deposit, plus a per-IP sign-up limit. With BYO accounts this protects RAM and bandwidth, not storage. |
|
||||
|
||||
## Why one s3d per user
|
||||
|
||||
An s3d process is bound to one Sia account: its connection key, users, keys,
|
||||
buckets and upload buffer all live in its data folder (`s3d.db`). That is also
|
||||
why Pithos's account switch works by moving the folder. So the server runs
|
||||
**one s3d per hosted user**. Each one gets its own data folder, loopback-only
|
||||
ports, and admin password.
|
||||
|
||||
- **Your recovery phrase never leaves your PC.** You connect locally as today.
|
||||
Pithos then uploads the connected data folder, which carries only the connection
|
||||
key for this one app. The connection uses one of sia.storage's 3 app slots per
|
||||
account.
|
||||
- **The server holds that connection key** and so could read anything s3d
|
||||
stores. This is why private files get the extra client-side layer below.
|
||||
- **Idle s3d processes are stopped.** A user's s3d starts on the first request
|
||||
(a few seconds) and stops after 30 min with no requests and nothing pending.
|
||||
Before it stops, leftovers are flushed to Sia (the auto-flush setting, forced on
|
||||
in hosted mode).
|
||||
|
||||
## Server: `pithos-hosted` on the VPS
|
||||
|
||||
A Node service in `Pithos/server-hosted/` that reuses `core/`. It does not
|
||||
duplicate it.
|
||||
|
||||
1. **Supervisor.** `/var/lib/pithos-hosted/<uid>/` holds `s3d.yml` and the data.
|
||||
Ports come from a pool and are bound to 127.0.0.1 only. A per-user
|
||||
`diskUsageLimit` keeps upload buffers bounded, and there's a cap on how many
|
||||
s3d run at once (LRU stop). systemd unit, nginx TLS, deployed from the repo.
|
||||
2. **Per-user Pithos.** Each user gets one `createPithos()` instance with
|
||||
`hostName: 'hosted'`, so the exact same `/api/*` surface is served. Routes that
|
||||
make no sense on a server are refused in hosted mode: binary install, address
|
||||
settings, login pty, open-external, account switch/restore.
|
||||
3. **Sign-in.** The identity is a key from the Theseus vault (new derive path
|
||||
`pithos/hosted-identity/v1`; the existing `pithos/sia-recovery/v1` and
|
||||
`pithos/credentials/v1` stay untouched). Challenge → signature → session token.
|
||||
The desktop and web hosts use a local identity file that can be exported and
|
||||
imported. Sign-up runs the anti-Sybil check: a BNS name owned by the key's
|
||||
address, or a deposit tx.
|
||||
4. **Gateway S3 keys.** Hosted s3d is never exposed. The gateway mints its own
|
||||
SigV4 keys, verifies them itself (`core/s3.js` already signs, so verification
|
||||
mirrors it) and re-signs to the user's s3d. That's what makes the rules below
|
||||
enforceable, and it lets rclone, aws-cli and the like keep working.
|
||||
5. **Meters.** Monthly download bytes per user, with a cap (proposed 50 GB per
|
||||
month, then throttled) and per-IP sign-up limits.
|
||||
|
||||
## Per-folder access and rights
|
||||
|
||||
Rules are stored per user: `{ drive, prefix, who, rights, expires }`.
|
||||
|
||||
- `who`: `public`, a link token, or a person (BNS name or wallet address, who signs
|
||||
in like any user)
|
||||
- `rights`: `read` (GET, HEAD, list), `add` (create new keys only: PUT with
|
||||
If-None-Match / existence check, no DELETE, no overwrite), `edit` (PUT and DELETE)
|
||||
|
||||
Every request through the gateway, whether S3 API, `/api`, public URL
|
||||
`https://s3.silentmode.st/p/<user>/<drive>/<path>` or share link, is matched
|
||||
against the longest prefix rule. List results are filtered to what the caller may
|
||||
see. The owner always has full rights. This is what s3d alone can't do: its
|
||||
bucket policy is whole-bucket public read only.
|
||||
|
||||
Encrypted folders can be shared with a person only once key sharing exists (the
|
||||
folder key wrapped to the grantee's public key; v2). Until then, the UI offers
|
||||
to share those folders as readable copies, and says so.
|
||||
|
||||
## Client-side encryption
|
||||
|
||||
The encryption happens in the local Pithos core. In hosted mode it acts as an
|
||||
encrypting forwarder between the unchanged UI and the gateway, so plaintext and
|
||||
keys never leave the PC.
|
||||
|
||||
- Key: HKDF from the vault, path `pithos/hosted-encryption/v1`, one key per drive
|
||||
(`HKDF(master, drive id)`).
|
||||
- Bodies: AES-256-GCM in 1 MiB chunks, each with its own nonce and the chunk
|
||||
index authenticated. Range reads and streaming keep working.
|
||||
- Names: encrypted per path segment with AES-SIV (deterministic), so folders
|
||||
still list and sort and the server sees no file names. *Open point: costs a few
|
||||
days; bodies-only is the cheaper v1. Recommend doing names too.*
|
||||
- Making an encrypted folder public or shared by link re-uploads it decrypted,
|
||||
after a confirmation that says what becomes readable.
|
||||
- Losing the vault means losing the files. Same as the rest of Silent Mode.
|
||||
|
||||
## Moving between local and hosted
|
||||
|
||||
No file is re-uploaded to Sia. Only s3d's metadata folder moves.
|
||||
|
||||
**Local → hosted:** flush everything to Sia, stop local s3d, upload the data
|
||||
folder (TLS, authenticated), then the gateway starts s3d on it and checks the
|
||||
bucket list. The local copy is archived like an account switch, never deleted.
|
||||
Pithos refuses to run a local s3d on a data folder that's live on the server:
|
||||
two s3d writing one account's metadata would diverge.
|
||||
Existing files stay unencrypted. Pithos offers "Encrypt existing private files",
|
||||
a background re-upload, and lists what is still readable until that's done.
|
||||
|
||||
**Hosted → local:** the gateway flushes and stops, Pithos downloads the folder,
|
||||
starts local s3d on it, and the gateway deletes its copy after the local start
|
||||
succeeds. Encrypted files remain encrypted. Pithos decrypts them on read in both
|
||||
modes.
|
||||
|
||||
s3d must be the same pinned version on both sides (today 0.2.0). The migration
|
||||
checks this before it starts.
|
||||
|
||||
## Build order
|
||||
|
||||
| Step | What | Size |
|
||||
|---|---|---|
|
||||
| 1 | Supervisor + per-user `createPithos` + sign-in/sessions + anti-Sybil check | 3-4 days |
|
||||
| 2 | Pithos client mode switch, forwarder, hosted status in Overview | 2 days |
|
||||
| 3 | Migration both ways, live-elsewhere lock | 2 days |
|
||||
| 4 | Encryption (bodies, then names) | 3-4 days |
|
||||
| 5 | Gateway SigV4 keys, access rules, public URLs, share links | 3-4 days |
|
||||
| 6 | Meters, limits, deploy (DNS for s3.silentmode.st, nginx, systemd), docs | 2 days |
|
||||
|
||||
Pithos changes ship as extension OTA versions, never a Theseus release.
|
||||
|
||||
## Open points
|
||||
|
||||
- Host: the current VPS is at 79% disk. Upload buffers need headroom, so this
|
||||
may need a separate box. Measure s3d memory per process first.
|
||||
- Download cap per user (proposed 50 GB per month).
|
||||
- Encrypt file names in v1 or later.
|
||||
|
||||
## Later: Sirius.X plans
|
||||
|
||||
The Silent Mode-paid tiers (500 MB free web hosting, 10/20/50 GB paid) plug into
|
||||
the same supervisor. Their "per-user s3d" connects to Silent Mode's sia.storage
|
||||
account instead of the user's, with quotas enforced at the gateway as Hestia
|
||||
describes. Sirius.X can take over the gateway as its storage backend.
|
||||
Loading…
Add table
Reference in a new issue