Commit graph

7 commits

Author SHA1 Message Date
Local Dev
c9008ef3ba prometheus: VirusTotal enrichment, gated on VIRUSTOTAL_API_KEY
After clamd returns a verdict, scanStream queries VirusTotal by the file's
SHA-256 and attaches the finding under scan.virustotal. The call never
blocks, never changes the overall verdict, and never ships without a key
— enrichment is informational and strictly additive.

Shape in the signed scan report:
  scan.virustotal: {status: 'known', positives, total, scan_date, signatures}
                 | {status: 'unknown'}
                 | {status: 'rate_limited'}
                 | {status: 'error', reason}
                 | (absent)              — enrichment disabled

Rate-limited to VIRUSTOTAL_RATE_PER_MIN (default 4, matching the free
tier) via a sliding window; the service never fails the request when the
quota is spent, it just skips the enrichment that time. Hashes are
cached in-process so a resubmitted file does not re-spend quota.

The upload POST response passes the virustotal field through on both the
success and error branches so the submit UI can display it alongside the
clamd verdict.

uploads.js passes the SHA-256 it already computed into scanStream rather
than making scanStream hash a second time — the current call site is the
only one that needs the hash downstream.
2026-10-08 21:44:34 +02:00
Local Dev
9f16ce3128 prometheus: weekly s3 snapshot of the browse surface to bns/prometheus
scripts/snapshot.mjs renders every public page (home, browse, publish,
boundaries, 8 categories, every active listing) to <out>/<path>/index.html,
plus /sitemap.xml, /robots.txt, and the two static assets. Writing to a
directory next to the service AND uploading via `aws s3 sync` are two
flags on the same command: --out to pick the directory, --upload to push.

Live routes added for /sitemap.xml and /robots.txt so a crawler hitting
the live service sees the same enumeration the snapshot bakes into the
bucket. renderSitemap / renderRobots in service/seo.js are the single
rendering path both call sites use.

Scope deliberately limited to the browse surface — /submit and the API
routes depend on the live backend and were never going to work on a
static mirror. The browse surface IS what the fallback is for: readers
survive an outage, publishers can't.

systemd unit + timer under deploy/ run the snapshot weekly on Sunday
04:00 UTC with 30 min jitter and Persistent=yes so a missed run after a
reboot catches up. Unit uses the sia-storage AWS profile per
[[sia-storage-credentials-location]] and the per-process endpoint URL
baked in via Environment=.

Local dry-run against an empty DB produced 14 HTML files + sitemap +
robots + the two assets.
2026-10-08 21:42:54 +02:00
Local Dev
cb1589cf36 prometheus: admin panel — login, listings, 3-mode toggle, audit log
/admin is wallet-gated. The configured ADMIN_PUBLISHER is a BNS name;
only the on-chain owner of that name can sign in. The sign-in flow is
BIP-137 (same primitive publishers use): the login page shows a fresh
one-shot challenge, the admin signs it in their wallet, pastes the
signature, and the service verifies that it recovers to the current
on-chain owner. Session is a 24-hour ed25519-signed cookie minted by
the service key.

Pages:
  /admin             dashboard — totals, mode state, recent listings + audit
  /admin/login       BIP-137 challenge sign-in
  /admin/listings    all listings, with a Withdraw button per row
  /admin/modes       global + per-category publish-mode toggle
  /admin/audit       the hash-chained audit log

The three publish modes exist as toggles:
  `append`     — any valid listing goes in (launch default)
  `scan-first` — infected uploads are rejected before they land
  `review`     — clean listings wait for human approval
Global mode is the fallback; a per-category override takes precedence
when set. The toggle persists; the review-queue enforcement lands in a
later milestone (the enforcement sketch is in modes.js's header).

Audit log:
  Every admin action (login, set_mode, clear_mode, withdraw_listing) is
  an append-only row with a sha256 chain over the previous row's hash.
  The chain does not prevent edits; it makes retroactive ones loud —
  row N's `prev_hash` disagreeing with row N-1's `this_hash` is the
  tamper signal.

Withdrawal:
  Soft — `withdrawn_at` on the listing row. Browse hides withdrawn
  listings (already the case) and the detail page returns 404 for them.
  Nothing in the chain is undone; this is Prometheus's own view only.

Session design:
  Cookie is `prom_admin=<payload b64url>.<signature b64url>`, signed with
  the ed25519 service key, HttpOnly, SameSite=Strict, Path=/admin,
  24-hour expiry. Reconfiguring ADMIN_PUBLISHER invalidates in-flight
  sessions by name mismatch. Challenges live in a 10-min TTL map and
  are tied to a specific name so one admin's challenge cannot authenticate
  another.

Added:
  service/admin.js         route handlers + dispatch
  service/admin-pages.js   HTML templates for the four pages
  service/admin-session.js challenge + cookie mint/verify + readSession
  service/audit.js         hash-chained append helper
  service/modes.js         publish-mode read/write with override semantics
  schema: settings + audit tables

scripts/test-admin-roundtrip.mjs drives the whole flow (unauth redirect
→ challenge scrape → sign → login → dashboard → set mode → audit
inspection) with the same deterministic test key the listing-roundtrip
script uses. Withdraw-flow spot-check: create a listing, log in as
admin, withdraw it, confirm browse hides it and the detail page returns
404. All seven checks pass.
2026-10-08 21:35:48 +02:00
Local Dev
bb07c73aaf prometheus: browse pages — categories and per-listing detail
/c/<slug>  renders one of the eight categories with every active listing
           in it, newest first, as a grid of cards. Empty category shows
           a "be the first" prompt rather than a dead page.

/app/<pub>/<slug>  renders a listing: icon + name + publisher + version
                   chip, a prominent Download card with size, hash chip,
                   signed scan badge (green/clean | red/infected |
                   neutral/unknown), description, homepage link, who
                   signed, and an endorsements placeholder for when APS1
                   lands.

The home-page and /browse grids now read real counts from the DB — the
hardcoded "0 listings" placeholders are gone. A new helper
`renderCategoriesGrid()` is the one place category cards are rendered,
so the two surfaces always agree.

404s for unknown categories (`/c/nope`) and unknown listings
(`/app/foo/bar`) go through the same notFound page and now actually
return HTTP 404 instead of 200 — pages opt in by returning the
notFound object, server detects it by title and sets the status.

CATEGORIES is the single source of truth for slug/icon/title/blurb;
adding a new category now means one row here plus one entry in the
listings allow-list.

Verified end-to-end with three listings across three categories: counts
update, cards render, detail page shows the signed scan badge, 404s
return 404.
2026-10-08 01:56:52 +02:00
Local Dev
72f7d41902 prometheus: signed listings — BIP-137 publisher sign, SQLite index
POST /api/listing accepts a JSON body with the manifest fields, a
reference to a signed scan report from /api/upload, and a BIP-137
signature. Before the row lands in SQLite the service verifies three
things:

  1. the attached scan_report was signed by this service, is for the
     exact (hash, size) the listing claims, and the verdict is clean —
     a publisher cannot swap in a different file's clean report;

  2. the publisher's signature recovers to a valid cashaddress, and
     that address matches the on-chain owner of the publisher's BNS
     name (gateway /api/name/<name>) — recovery alone is never enough
     per [[recovery-is-not-verification]];

  3. the usual shape checks: category in the allow-list, sha256 is 64
     hex, download URL is http(s), slug matches [a-z0-9-]{1..64}, body
     under 1 MiB.

GET /api/listings and GET /api/listings/<publisher>/<slug> are read-only
endpoints the browse milestone will render from.

MOCK_BNS_OWNER=<cashaddr> short-circuits the gateway call for local
testing without a chipnet gateway reachable — same pattern as MOCK_SCAN.

Dependencies:
- @bitauth/libauth, for the secp256k1 primitives behind BIP-137 recovery
  and the cashaddress encoder. Same primitive Argus's registrar uses, so
  signatures from Argus-compatible wallets verify unchanged.
- node:sqlite (ships with Node 22+) for the listings DB — one file next
  to the service, WAL journaling, no native build step. engines bumped
  to ">=22".

Lifted service/lib/sign-message.js from Argus so Prometheus does not
have to pull the resolver's whole dependency tree in for one verifier.
Removed its NETWORK import; the default address prefix comes from
BNS_NETWORK=chipnet|mainnet instead.

INTERIM: no APS1 record on chain yet. The signed row IS the listing;
when APS1 is defined cross-session the service will add an on-chain
publish step that points at the same signed manifest. The browse and
admin milestones can proceed against this DB in the meantime.

scripts/test-listing-roundtrip.mjs drives the whole flow end-to-end
with a deterministic test key, so a sanity check is a two-command
sequence: start the server with MOCK_SCAN=clean + MOCK_BNS_OWNER=<the
test address>, then run the script. Negative paths (fake scan report,
bad category, duplicate slug) also verified by hand.
2026-10-08 01:48:12 +02:00
Local Dev
c4566b4ea3 prometheus: upload + ClamAV scan + signed scan reports
POST /api/upload accepts a multipart file, spools it to a temp path while
hashing to SHA-256, hands the on-disk copy to clamd via INSTREAM, keeps
the file at uploads/<sha256>.bin on a clean verdict and discards it on
anything else. The response carries the (hash, size, verdict) tuple
signed by the service's ed25519 key — the artifact the manifest step
will attach to the APS1 record later.

For local dev on a box without ClamAV, MOCK_SCAN=clean|infected|error
short-circuits the TCP call while still draining the bytes end-to-end.
CLAMD_HOST / CLAMD_PORT / PROMETHEUS_MAX_UPLOAD_BYTES override the
defaults (127.0.0.1:3310, 2 GiB).

Service key lives at .keys/service.key (gitignored, 0600), generated on
first start if absent. The matching public key is served at
/.well-known/prometheus-service-key.json so clients can cache it per
origin. ed25519 chosen over BIP-137 because the scan report is a service
attestation, not a Bitcoin thing — the key never signs value, never
signs for a user, never touches the chain.

Canonicalisation walks the payload recursively (sorted keys at every
level, no whitespace). The first draft used JSON.stringify with a keys
array, which only serialises top-level keys from the array — nested
objects came out as {}, so the signature verified nothing useful.

/healthz pings clamd (or the mock) so monitors can tell when the scanner
is down.

The /submit page gets a working upload form that drives the endpoint
and renders the full signed response, so the whole chain is testable in
the browser as well as via curl.

One dependency added: busboy. Rolling our own multipart parser was the
obvious alternative; busboy is one small, maintained library with no
runtime deps of its own, and multipart RFC 7578 edge cases are not
places this project earns anything by rediscovering.
2026-10-05 22:41:55 +02:00
Local Dev
2c5df40d77 prometheus: scaffold the standalone service — pages, routing, static assets
New top-level Prometheus/ directory holds the Node service that will own
prometheus.x once the on-chain record flips from the silentmode.st
redirect to ip+tls. Nothing dynamic yet — this milestone delivers the
site skeleton and proves the family's visual conventions port cleanly to
a dynamic server.

Shape:
  Prometheus/service/server.js    node:http listener + map-based router
  Prometheus/service/layout.js    document shell, topnav, footer
  Prometheus/service/pages.js     home / browse / submit / publish / boundaries
  Prometheus/public/style.css     ported silentmode.st tokens + category grid
  Prometheus/public/favicon.svg   flame mark

Routes: /, /browse, /submit, /publish, /boundaries, /static/*.
Dynamic routes (/api/upload, /api/manifest, /api/publish, /api/pay,
/admin/*) land here in later milestones per the architecture doc.

Launch config .claude/launch.json gets a prometheus-service entry on
port 3700 so the preview harness can run it.

The pre-pivot WordPress-based scaffold at site-prometheus-x/ is left
in place for the next session to delete deliberately — the architecture
doc marks it as abandoned for prometheus.x but keeps the Sirius Press
infra fixes it carried in.
2026-10-05 22:22:47 +02:00