9727ce79d3
Records what was built against the plan (names are encrypted from v1, rights for other people are folder-scoped S3 keys), how to run pithos-hosted, and that paid storage belongs to Hestia, a separate product.
176 lines
7.4 KiB
Markdown
176 lines
7.4 KiB
Markdown
# Pithos
|
||
|
||
A control panel for [s3d](https://github.com/SiaFoundation/s3d), the Sia
|
||
Foundation's S3 gateway. s3d itself is managed through a CLI and a two-route
|
||
admin API; Pithos puts a UI over all of it:
|
||
|
||
- **Gateway**: install the pinned s3d release (SHA-256 checked), start, stop,
|
||
restart, live logs, edit `s3d.yml`.
|
||
- **Sia connection**: register s3d with an indexer (sia.storage or your own
|
||
indexd) with an existing recovery phrase or a freshly generated one, and
|
||
approve the connection in the browser.
|
||
- **Users & keys**: create and delete users, issue and revoke access keys,
|
||
copy ready-made `aws`, `rclone` and env-var config.
|
||
- **Buckets**: browse as any user, create and delete buckets, upload by
|
||
drag-and-drop, download, delete files and folders, presigned share links,
|
||
public-read policies.
|
||
- **Upload pipeline**: pending, uploaded, unpinned and failed objects from
|
||
`/stats/uploads`, plus "flush now" and an optional automatic upload of
|
||
leftovers after N quiet minutes.
|
||
- **On Silent Mode** (optional): run the same s3d on Silent Mode's server
|
||
with your own Sia account, with client-side encryption, public folders,
|
||
share links and S3 keys for other people. See below.
|
||
|
||
## On this computer, or on Silent Mode
|
||
|
||
By default s3d runs on your computer. Settings › *Where your drives live*
|
||
can move it to `pithos-hosted` on Silent Mode instead, where it keeps working
|
||
with your computer off and is reachable from other devices. The interface
|
||
stays the same.
|
||
|
||
- **Your own Sia account.** Only s3d's data folder moves: its database and any
|
||
files still waiting for Sia. The recovery phrase never leaves your
|
||
computer, and nothing already on Sia is uploaded again. Moving back works
|
||
the same way in reverse.
|
||
- **Encrypted before it leaves.** In hosted mode Pithos encrypts file names
|
||
(AES-SIV per path segment, so folders still list) and contents (chunked
|
||
AES-256-GCM, so range reads stay cheap) on your computer, with keys from
|
||
your Theseus vault or, on other hosts, a key file beside `s3d.yml` that
|
||
you should back up. See `core/crypt.js`.
|
||
- **Sharing a bare s3d cannot do.** Public folders, expiring share links, and
|
||
S3 keys for other people limited to one folder with read, add-only or edit
|
||
rights. Those files are stored readable, because the people they are for
|
||
have no key; stop sharing and Pithos encrypts them again.
|
||
- **One slot per person.** Signing up needs a signature from the wallet that
|
||
owns a BNS name, a small refundable BCH deposit, or an invite.
|
||
|
||
The server is in `hosted/` (see `hosted/README.md`); the design and its
|
||
trade-offs are in `DESIGN-hosted.md`.
|
||
|
||
## One core, three hosts
|
||
|
||
```
|
||
core/ control server: s3d process, CLI wrapper, SigV4 S3 client, admin API, HTTP API
|
||
ui/ the single-page UI the server serves (plain JS, no build step)
|
||
|
||
bin/pithos-web.js web console -> your browser
|
||
desktop/main.cjs desktop app -> an Electron window
|
||
theseus/ Theseus add-on -> a Theseus tab
|
||
```
|
||
|
||
Every host runs the same `createPithos()` server on a loopback port and
|
||
points a view at it. A feature added to `core/` + `ui/` lands in all three.
|
||
|
||
### Web console
|
||
|
||
```bash
|
||
npm install
|
||
node bin/pithos-web.js
|
||
```
|
||
|
||
It prints a one-time sign-in link for `127.0.0.1:8700`. To reach it from
|
||
another machine, set a password and put a TLS reverse proxy in front:
|
||
|
||
```bash
|
||
PITHOS_PASSWORD='long random secret' node bin/pithos-web.js --listen 0.0.0.0:8700 --allow-host pithos.example.org
|
||
```
|
||
|
||
Other flags: `--config <s3d.yml>`, `--s3d <binary>`, `--bin-dir <dir>`,
|
||
`--password-file <file>`.
|
||
|
||
### Desktop
|
||
|
||
```bash
|
||
npm run desktop
|
||
```
|
||
|
||
The window signs in with the server's one-time secret; nothing else on the
|
||
machine can drive the API. Quitting stops s3d cleanly. Installed s3d builds
|
||
go to the app's user-data folder.
|
||
|
||
### Theseus
|
||
|
||
```bash
|
||
node scripts/build-theseus-addon.mjs
|
||
```
|
||
|
||
builds `dist/theseus-addon/pithos/` (about 0.9 MB). Copy it into
|
||
`TheseusNavigator/bundled-addons/` to bundle it, or publish it as a
|
||
community extension. The dock menu has *Open Pithos* and *Stop s3d*.
|
||
|
||
**Shipping an update.** The add-on updates on its own extension channel, not
|
||
with Theseus releases:
|
||
|
||
1. Bump `version` in `theseus/addon.json` (and `package.json`).
|
||
2. Rebuild the bundled copy:
|
||
`node scripts/build-theseus-addon.mjs ../TheseusNavigator/bundled-addons/pithos`,
|
||
then commit, so new installs carry the same version.
|
||
3. Sign it and publish it on the channel at
|
||
`https://navigate.st/bns/theseus.x/extensions/pithos/` (Sia bucket
|
||
`bns/theseus/extensions/pithos/`), as described in
|
||
`TheseusNavigator/docs/ADDON-UPDATES.md`.
|
||
|
||
Installed copies pick the update up within about 30 seconds of the next
|
||
launch and switch to it on the launch after that. The Theseus version
|
||
stays the same.
|
||
|
||
The add-on leaves out the pty module (12 MB of native code), so on Windows
|
||
the indexer login runs in its own console window and Pithos picks up the
|
||
result when it closes.
|
||
|
||
## Which s3d instance it manages
|
||
|
||
The same one a hand-run `s3d` would use: `S3D_CONFIG_FILE` if set, otherwise
|
||
`%APPDATA%\s3d\s3d.yml` (Windows), `~/Library/Application Support/s3d/s3d.yml`
|
||
(macOS), `/etc/s3d/s3d.yml` (Linux). Every s3d call Pithos makes gets
|
||
`S3D_CONFIG_FILE` pinned, so the daemon and the CLI calls always agree.
|
||
|
||
If another s3d process is already serving that config (a service, or one
|
||
started by hand), the overview shows it as *external* and reads its stats,
|
||
but Pithos won't start a second copy.
|
||
|
||
Pithos needs **s3d v0.2.0 or newer**: `users list` and `keys list` arrived
|
||
in 0.2.0. Older builds get an upgrade banner.
|
||
|
||
## Security model
|
||
|
||
- The server binds loopback by default and refuses Host headers other than
|
||
its own address (DNS-rebinding guard). Binding beyond loopback requires a
|
||
password of at least 12 characters.
|
||
- Sessions are HttpOnly, SameSite=Strict cookies. Every non-GET request also
|
||
needs an `x-pithos: 1` header, so a cross-site form cannot drive the API.
|
||
- The recovery phrase is only written to the s3d login process's terminal.
|
||
It is never stored, logged or sent anywhere else. A generated phrase is
|
||
shown once and must be acknowledged.
|
||
- Downloads are served with a sandboxing CSP and `nosniff`, so a stored HTML
|
||
object cannot run script on the console's origin.
|
||
- s3d keeps access-key secrets in plain text in its own SQLite database.
|
||
Pithos shows them to whoever is signed in to the console, so treat console
|
||
access as root on the gateway.
|
||
|
||
## How it talks to s3d
|
||
|
||
| Need | Mechanism |
|
||
| --- | --- |
|
||
| users, keys, version, registration check | `s3d users …`, `s3d keys …`, `s3d version`, `s3d login` with stdin closed |
|
||
| indexer login | `s3d login` in a pseudo-terminal (it reads the phrase with `term.ReadPassword`) |
|
||
| buckets and objects | the S3 API, signed with SigV4 as the chosen user's first access key |
|
||
| stats, flush | the admin API (`/stats/uploads`, `/objects/flush`) with the configured admin password |
|
||
|
||
`test/sigv4.test.js` cross-checks the signer against the AWS CLI on awkward
|
||
keys and prefixes and on presigned URLs (`npm test`).
|
||
|
||
## Limits and next steps
|
||
|
||
- Uploads are single PUTs, capped at 5 GiB (s3d's part limit). Multipart
|
||
uploads from the browser would lift that.
|
||
- Object versioning, lifecycle rules and multipart-upload cleanup are not in
|
||
the UI yet.
|
||
- Desktop installers (electron-builder) are not set up yet.
|
||
- Storage paid to Silent Mode (quotas, plans, billing) is not part of
|
||
Pithos. That is Hestia (`Hephaestus/DESIGN-hestia.md`), a separate product
|
||
for operators who sell storage.
|
||
|
||
## License
|
||
|
||
MPL-2.0. s3d is MIT, © The Sia Foundation.
|