Commit graph

2 commits

Author SHA1 Message Date
Local Dev
d25ed36e14 prometheus: s3d sidecar + /dl/<sha256> for Sia-backed hosting
Publishers no longer need external hosting at all. The service runs its
own s3d (SiaFoundation gateway) as a Docker sidecar on the internal
compose network; scan-clean uploads land there via PUT; readers fetch
through /dl/<sha256> on prometheus.x which proxies back out from s3d.

- deploy/docker-compose.yml: s3d service added. Image is
  ghcr.io/siafoundation/s3d:latest, data folder bind-mounted at
  ./s3d-data/ so the operator can seed it from a Pithos connection
  export. Only prometheus-internal network — never public.
- deploy/setup-s3d.sh: one-time import of the Pithos connection
  tarball, writes a fresh s3d.yml with a random adminPassword, mints
  an S3 access key for the Prometheus service, appends
  PROMETHEUS_S3_* lines to .env. Idempotent on re-run.
- service/storage.js: endpoint now defaults to the internal s3d
  (http://s3d:8000); publicBase defaults to https://prometheus.x so
  download URLs flow through the service's own /dl handler. Bucket is
  auto-created on first use so a fresh data folder bootstraps.
  Added getObjectStream() for the read side.
- service/downloads.js: GET/HEAD /dl/<sha256> — forwards Range,
  content-type and content-length from s3d, drops s3d-specific
  headers (x-amz-*), sets immutable long-cache. Hash is validated
  against ^[0-9a-f]{64}$ before touching upstream.
- service/server.js: route table wires /dl/* to the handler above.
2026-10-24 01:08:24 +03:00
Local Dev
09bb42f7a6 prometheus: push scan-clean uploads to Sia, auto-fill the download URL
Publishers no longer need to self-host. When the service is configured
with PROMETHEUS_S3_* credentials pointing at the hosted s3d
(s3.silentmode.st), every clean upload is pushed to
s3://${bucket}/${prefix}<sha256>.bin and the public URL comes back on
/api/upload as `hosted_url`. Step 2 of the submit form pre-fills the
Download URL input with that URL — publishers can overwrite if they
prefer their own hosting; by default they ship nothing.

- service/s3.js: SigV4 S3 client vendored from Pithos/core/s3.js so bug
  fixes port by file-copy. Path-style, UNSIGNED-PAYLOAD streaming PUT,
  Content-Length required (s3d chunked encoding does not round-trip
  cleanly for large bodies).
- service/storage.js: thin wrapper exposing isConfigured() and
  uploadScannedFile(). Idempotent on sha256 — same bytes → same key.
  Failure is non-fatal; the publisher can still paste their own URL.
- service/uploads.js: after rename-to-final, push to Sia if configured.
  Added `hosted_url` to the response (null when storage off).
- service/pages.js: step 2 pre-fills download_url when hosted_url is
  present; hint text explains the override path.
- docker-compose.yml: PROMETHEUS_S3_* env plumbed through, with sane
  defaults for bucket/prefix (bns + prometheus-storage/).
2026-10-23 21:33:14 +03:00