diff --git a/docs/index.html b/docs/index.html index f52870b..bae460c 100644 --- a/docs/index.html +++ b/docs/index.html @@ -66,7 +66,7 @@ Markdown ↓
- +

Pithos documentation

Using an AI assistant? Give it this page or the Markdown version, then describe what you want to do. Type recovery phrases and secret keys only into s3d or Pithos on your own computer, never into a chat.
@@ -176,6 +176,54 @@ force_path_style = true 
rclone ls s3d:my-drive
 rclone copy ./photos s3d:my-drive/photos --progress

Environment variables for SDKs and tools: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION=us-east-1, AWS_ENDPOINT_URL=http://127.0.0.1:8000.

+

Storage for scripts, backups and AI agents

+

A script, a backup tool or an AI agent uses your storage like any S3 app: an endpoint and an access key. Give each one its own key, so you can see what it does and switch it off alone.

+

1. Give it its own user and key

+
  1. Users & keys › New user, named after the job (backup-agent, claude-notes). Pithos creates its key and a drive of the same name.
  2. Optional: Drives › New drive for that user if you want a differently named drive.
  3. Open the key (Show secret & config) and press Export JSON…. Pick the drive and save the file.
+

The file looks like this:

+ 
{
+  "endpoint": "http://127.0.0.1:8000",
+  "accessKey": "…",
+  "secretKey": "…",
+  "bucket": "backup-agent"
+}
+ +

2. Use it

+

Environment variables (most tools read these):

+ 
export AWS_ACCESS_KEY_ID=$(jq -r .accessKey pithos-backup-agent.json)
+export AWS_SECRET_ACCESS_KEY=$(jq -r .secretKey pithos-backup-agent.json)
+export AWS_REGION=us-east-1
+export AWS_ENDPOINT_URL=$(jq -r .endpoint pithos-backup-agent.json)
+BUCKET=$(jq -r .bucket pithos-backup-agent.json)
+aws s3 cp report.pdf "s3://$BUCKET/reports/report.pdf"
+aws s3 ls "s3://$BUCKET/reports/"
+

Backups with restic (encrypted, deduplicated snapshots; restore any day):

+ 
# the four exports above, then:
+export RESTIC_REPOSITORY="s3:$AWS_ENDPOINT_URL/$BUCKET/restic"
+export RESTIC_PASSWORD_FILE=~/.config/restic/password   # keep this password: without it the backup cannot be read
+restic -o s3.bucket-lookup=path -o s3.region=us-east-1 init
+restic -o s3.bucket-lookup=path -o s3.region=us-east-1 backup ~/Documents
+restic -o s3.bucket-lookup=path -o s3.region=us-east-1 snapshots
+

rclone (sync, mount, serve): add a remote with the four values (provider = Other, force_path_style = true), as in "Using drives from other apps" above, then rclone sync ~/Projects backup:backup-agent/projects.

+

Python (boto3):

+ 
import json, boto3
+from botocore.config import Config
+c = json.load(open("pithos-backup-agent.json"))
+s3 = boto3.client("s3", endpoint_url=c["endpoint"], aws_access_key_id=c["accessKey"],
+                  aws_secret_access_key=c["secretKey"], region_name="us-east-1",
+                  config=Config(s3={"addressing_style": "path"}))
+s3.upload_file("notes.md", c["bucket"], "notes/notes.md")
+print([o["Key"] for o in s3.list_objects_v2(Bucket=c["bucket"]).get("Contents", [])])
+

Node (AWS SDK v3): new S3Client({ endpoint: c.endpoint, region: "us-east-1", forcePathStyle: true, credentials: { accessKeyId: c.accessKey, secretAccessKey: c.secretKey } }).

+

3. Where the agent can reach it

+ +

SSH keys and SFTP

+

s3d speaks only S3: there is no SSH login or SSH key to add. If a tool insists on SFTP, run rclone in front of it:

+ 
rclone serve sftp backup:backup-agent --addr 127.0.0.1:2022 --authorized-keys ~/.ssh/authorized_keys
+

That gives SSH-key logins to one drive, but underneath it is the same S3 key, and rclone has to keep running. For agents and scripts, an S3 key per job (above) is simpler and easier to revoke; use SFTP only for tools that cannot do S3. rclone serve webdav and rclone mount work the same way for apps that want a network drive.

+

Telling an AI agent

+

Give the agent the JSON file path, not the secret in the chat, and a short brief such as:

+

> Use the S3 storage described in pithos-backup-agent.json (keys endpoint, accessKey, secretKey, bucket; region us-east-1; path-style addressing). Only read and write inside that bucket, under backups/. Never print the secret key. Do not delete objects unless I ask.

Common tasks

Troubleshooting

@@ -185,6 +233,7 @@ rclone copy ./photos s3d:my-drive/photos --progress

For AI assistants

You are helping someone run s3d (the Sia S3 gateway), optionally through Pithos. Facts to rely on:

+

For an agent that should store files: ask the user to create a separate S3 user in Pithos and export its key as JSON (Users & keys › Show secret & config › Export JSON…), then read the file instead of having secrets pasted. See "Storage for scripts, backups and AI agents".

Rules: never ask the user to paste their recovery phrase or secret keys into the conversation; have them type secrets only into s3d or Pithos on their own machine. Prefer read-only checks (s3d status, s3d users list, aws s3 ls) before changing anything, and confirm before deleting users, keys, drives or files.

diff --git a/docs/pithos-docs.md b/docs/pithos-docs.md index afd29da..15d9d5c 100644 --- a/docs/pithos-docs.md +++ b/docs/pithos-docs.md @@ -253,6 +253,96 @@ rclone copy ./photos s3d:my-drive/photos --progress **Environment variables** for SDKs and tools: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION=us-east-1`, `AWS_ENDPOINT_URL=http://127.0.0.1:8000`. +## Storage for scripts, backups and AI agents + +A script, a backup tool or an AI agent uses your storage like any S3 app: an endpoint and an access key. Give each one its own key, so you can see what it does and switch it off alone. + +### 1. Give it its own user and key + +1. **Users & keys › New user**, named after the job (`backup-agent`, `claude-notes`). Pithos creates its key and a drive of the same name. +2. Optional: **Drives › New drive** for that user if you want a differently named drive. +3. Open the key (**Show secret & config**) and press **Export JSON…**. Pick the drive and save the file. + +The file looks like this: + +```json +{ + "endpoint": "http://127.0.0.1:8000", + "accessKey": "…", + "secretKey": "…", + "bucket": "backup-agent" +} +``` + +- The key reaches **every drive of that user**; `bucket` only says which one to use. A user's keys cannot see other users' drives, which is why one user per job is the boundary. +- The file is **not encrypted**. Keep it out of git, chats and shared folders. **Save credentials…** makes an encrypted copy instead (password or Theseus vault) for keeping. +- To stop the agent: **Users & keys › Revoke** on its key (or delete the user). Nothing else changes. + +### 2. Use it + +**Environment variables** (most tools read these): + +```bash +export AWS_ACCESS_KEY_ID=$(jq -r .accessKey pithos-backup-agent.json) +export AWS_SECRET_ACCESS_KEY=$(jq -r .secretKey pithos-backup-agent.json) +export AWS_REGION=us-east-1 +export AWS_ENDPOINT_URL=$(jq -r .endpoint pithos-backup-agent.json) +BUCKET=$(jq -r .bucket pithos-backup-agent.json) +aws s3 cp report.pdf "s3://$BUCKET/reports/report.pdf" +aws s3 ls "s3://$BUCKET/reports/" +``` + +**Backups with restic** (encrypted, deduplicated snapshots; restore any day): + +```bash +# the four exports above, then: +export RESTIC_REPOSITORY="s3:$AWS_ENDPOINT_URL/$BUCKET/restic" +export RESTIC_PASSWORD_FILE=~/.config/restic/password # keep this password: without it the backup cannot be read +restic -o s3.bucket-lookup=path -o s3.region=us-east-1 init +restic -o s3.bucket-lookup=path -o s3.region=us-east-1 backup ~/Documents +restic -o s3.bucket-lookup=path -o s3.region=us-east-1 snapshots +``` + +**rclone** (sync, mount, serve): add a remote with the four values (`provider = Other`, `force_path_style = true`), as in "Using drives from other apps" above, then `rclone sync ~/Projects backup:backup-agent/projects`. + +**Python** (boto3): + +```python +import json, boto3 +from botocore.config import Config +c = json.load(open("pithos-backup-agent.json")) +s3 = boto3.client("s3", endpoint_url=c["endpoint"], aws_access_key_id=c["accessKey"], + aws_secret_access_key=c["secretKey"], region_name="us-east-1", + config=Config(s3={"addressing_style": "path"})) +s3.upload_file("notes.md", c["bucket"], "notes/notes.md") +print([o["Key"] for o in s3.list_objects_v2(Bucket=c["bucket"]).get("Contents", [])]) +``` + +**Node** (AWS SDK v3): `new S3Client({ endpoint: c.endpoint, region: "us-east-1", forcePathStyle: true, credentials: { accessKeyId: c.accessKey, secretAccessKey: c.secretKey } })`. + +### 3. Where the agent can reach it + +- **On the same computer** as s3d: the endpoint `http://127.0.0.1:8000` works as is. Local agents (Claude Code, a cron job, a backup app) need nothing else. +- **From another machine or a cloud agent**: `127.0.0.1` points to *that* machine, so it fails. Either move the drives to **Drives on Silent Mode** (the endpoint becomes `https://s3.silentmode.st` and keeps working with your computer off) or put s3d behind your own server with TLS and write that address into the JSON as `endpoint`. +- **Narrower keys**: on Drives on Silent Mode, ⋯ › Share… on a folder can make a key for one folder only: *read*, *add new files* (it can create files but never overwrite or delete one, good for an agent that drops reports or exports) or *add, change and delete*. restic needs the last one, because it removes its own lock files. A plain s3d only has whole-user keys. +- Files a tool uploads wait on this computer until a ~40 MB block fills; automatic upload sends leftovers after 10 quiet minutes, or press **Upload now**. Until then they are not on Sia yet. + +### SSH keys and SFTP + +s3d speaks only S3: there is no SSH login or SSH key to add. If a tool insists on SFTP, run rclone in front of it: + +```bash +rclone serve sftp backup:backup-agent --addr 127.0.0.1:2022 --authorized-keys ~/.ssh/authorized_keys +``` + +That gives SSH-key logins to one drive, but underneath it is the same S3 key, and rclone has to keep running. For agents and scripts, an S3 key per job (above) is simpler and easier to revoke; use SFTP only for tools that cannot do S3. `rclone serve webdav` and `rclone mount` work the same way for apps that want a network drive. + +### Telling an AI agent + +Give the agent the JSON file path, not the secret in the chat, and a short brief such as: + +> Use the S3 storage described in `pithos-backup-agent.json` (keys `endpoint`, `accessKey`, `secretKey`, `bucket`; region `us-east-1`; path-style addressing). Only read and write inside that bucket, under `backups/`. Never print the secret key. Do not delete objects unless I ask. + ## Common tasks - **Give an app its own key.** Users & keys › create a user for the app (or use yours) › Create key. Revoke it later without touching your other keys. @@ -293,4 +383,6 @@ You are helping someone run **s3d** (the Sia S3 gateway), optionally through **P - Buckets ("drives") can be private, public read, or public read + list; nothing finer. - Config lookup: `S3D_CONFIG_FILE`, else the OS default path listed above. +For an agent that should store files: ask the user to create a separate S3 user in Pithos and export its key as JSON (Users & keys › Show secret & config › Export JSON…), then read the file instead of having secrets pasted. See "Storage for scripts, backups and AI agents". + Rules: never ask the user to paste their recovery phrase or secret keys into the conversation; have them type secrets only into s3d or Pithos on their own machine. Prefer read-only checks (`s3d status`, `s3d users list`, `aws s3 ls`) before changing anything, and confirm before deleting users, keys, drives or files.