b96cb8e151
Splits ProtocolMessages exceeding NIP-44's 65,535-byte plaintext ceiling across multiple gift-wrapped events. Symmetric (both directions), fire-and-forget, backward-compatible via a new transport-level \`extensions\` field on \`dapp_ready\` and \`wallet_ready\`. Resolves the \"Failed to swap: invalid plaintext size\" error on aggregated swap requests and enables signed-tx responses up to the 1 MB BCH consensus limit (~2 MB hex).
357 lines
16 KiB
Markdown
357 lines
16 KiB
Markdown
# Relay transport
|
||
|
||
The relay layer handles everything below the application protocol: WebSocket connectivity,
|
||
message encryption, reconnection, and message queuing. Application code (wallet, dapp) should
|
||
never touch NDK or Nostr directly.
|
||
|
||
## Stack
|
||
|
||
```
|
||
Application messages (hdwalletv1 JSON)
|
||
↓ JSON.stringify
|
||
NDK PrivateDirectMessage (kind 14, rumor)
|
||
↓ NIP-17 gift wrap (kind 1059, encrypted to recipient)
|
||
NDK → Nostr relay (WebSocket)
|
||
↓ stored as kind 1059 event, tagged with recipient pubkey
|
||
NDK ← Nostr relay subscription (filter: kind=1059, #p=<our_pubkey>)
|
||
↓ giftUnwrap → PrivateDirectMessage → JSON.parse
|
||
Application message handler
|
||
```
|
||
|
||
## RelayClient
|
||
|
||
`RelayClient` is the low-level building block. One instance per peer connection.
|
||
|
||
### Responsibilities
|
||
|
||
- Owns an NDK instance configured with explicit relay URLs.
|
||
- Subscribes to `kind:1059` (GiftWrap) events tagged with its own pubkey.
|
||
- Decrypts and unwraps incoming events using NIP-17 `giftUnwrap`.
|
||
- Publishes outbound messages using `giftWrap` addressed to the paired peer.
|
||
- Filters incoming messages by peer pubkey (rejects messages from unknown senders).
|
||
- Discards messages older than `lastProcessedTimestamp` (replay protection).
|
||
- Queues outbound messages until at least one relay is ready.
|
||
|
||
### Construction
|
||
|
||
```typescript
|
||
new RelayClient({
|
||
explicitRelayUrls: string[]; // WebSocket URLs, e.g. ["wss://relay.riften.net:443"]
|
||
signerPrivateKey: Uint8Array; // 32-byte secp256k1 private key (this client's identity)
|
||
pairedPublicKey?: Uint8Array; // 32-byte x-only pubkey of the peer (set after key exchange)
|
||
logNetworkActivity?: boolean; // default true
|
||
})
|
||
```
|
||
|
||
### Key methods
|
||
|
||
```typescript
|
||
connect(): Promise<void>
|
||
// NDK connect, subscribe to GiftWrap events, start waiting for relays.
|
||
|
||
disconnect(): Promise<void>
|
||
// Stop subscription, mark queue not-ready, update lastProcessedTimestamp.
|
||
|
||
relay(message: ProtocolMessage): Promise<void>
|
||
// Send a message. Enqueues if relays not ready. Throws if paired key not set.
|
||
|
||
setPairedPublicKey(key: Uint8Array): void
|
||
// Called after key exchange. Enables outbound messages and incoming peer filtering.
|
||
|
||
isKeyExchangeComplete(): boolean
|
||
// true once pairedPublicKey is set and non-zero.
|
||
|
||
nextSequence(): number
|
||
// Returns a unique sequence number (starts at random offset, increments by 2).
|
||
|
||
getLastProcessedTimestamp() / setLastProcessedTimestamp(ts): void
|
||
// Persist/restore across reconnects to avoid re-delivering buffered messages.
|
||
```
|
||
|
||
### Message queue
|
||
|
||
Before any relay is confirmed as connected, outbound `relay()` calls are enqueued in a
|
||
`MessageQueue`. The client polls every 100 ms for up to 5 seconds for at least one relay with
|
||
status 1 (connected). After that timeout it assumes the relay is ready and flushes the queue
|
||
regardless (avoiding silent message loss on slow connections).
|
||
|
||
On `disconnect()`, the queue is marked not-ready so messages sent during a reconnect gap are
|
||
held rather than dropped.
|
||
|
||
### Replay protection
|
||
|
||
`lastProcessedTimestamp` is set to `now - 2` on the first connection. On reconnect it is updated
|
||
to `now` in `disconnect()`. Any incoming message with `time < lastProcessedTimestamp` is silently
|
||
dropped. This prevents the relay from re-delivering messages that were already handled before a
|
||
disconnect.
|
||
|
||
### Keepalive
|
||
|
||
`RelayClient` creates its `SimplePool` with `enablePing: true`. This enables nostr-tools'
|
||
built-in heartbeat: every 29 seconds the pool pings each connected relay and expects a
|
||
response within 20 seconds. In Node.js this uses native WebSocket ping/pong frames; in
|
||
browsers (where the WebSocket API doesn't expose ping) it falls back to sending a dummy
|
||
subscription request and waiting for EOSE.
|
||
|
||
If a relay fails to respond, nostr-tools closes the WebSocket, which fires the subscription
|
||
`onclose` callback, which emits `"disconnect"` on `RelayClient`, which triggers the
|
||
connection manager's existing reconnect loop. This detects "zombie" TCP connections where the
|
||
socket appears open but the relay is unreachable.
|
||
|
||
Additionally, if a `publishMessage()` call fails (all relays reject the event),
|
||
`RelayClient` emits `"disconnect"` alongside the thrown error. This ensures the reconnect
|
||
loop starts immediately rather than waiting for the next ping cycle.
|
||
|
||
### Sequence numbers
|
||
|
||
`nextSequence()` starts at a random offset in the safe integer range and increments by 2. This
|
||
means two instances are unlikely to collide (they start at different offsets), and the step of 2
|
||
leaves room for error responses at odd offsets if needed in the future.
|
||
|
||
## initiateRelay
|
||
|
||
`initiateRelay()` (in `relay-handler.ts`) wraps `RelayClient` with a reconnect loop:
|
||
|
||
```typescript
|
||
initiateRelay(
|
||
statusCallback: RelayStatusCallback,
|
||
privateKey: Uint8Array,
|
||
initialPairedKey: Uint8Array,
|
||
options: { explicitRelayUrls, reconnectInterval?, maxReconnectAttempts? }
|
||
): () => void // returns cleanup function
|
||
```
|
||
|
||
It calls `statusCallback` with a `RelayUpdatePayload` on every state change:
|
||
|
||
```typescript
|
||
interface RelayUpdatePayload {
|
||
client: RelayClient;
|
||
status: RelayStatus; // { status: "connected" | "reconnecting" | "disconnected" }
|
||
}
|
||
```
|
||
|
||
The reconnect loop is simple: on a disconnect event from the client it waits
|
||
`reconnectInterval` (default 5000 ms) and calls `client.connect()` again. No exponential backoff
|
||
currently — connections are expected to be stable (relay and mobile network) and fast to re-establish.
|
||
|
||
## initiateDappRelay
|
||
|
||
Higher-level helper that bundles credential generation, URI encoding, key exchange handling,
|
||
and reconnection into one call:
|
||
|
||
```typescript
|
||
initiateDappRelay(
|
||
statusCallback: RelayStatusCallback,
|
||
options?: {
|
||
explicitRelayUrls?: string[];
|
||
reconnectInterval?: number;
|
||
maxReconnectAttempts?: number;
|
||
existingCredentials?: { privateKey: string; secret: string };
|
||
}
|
||
): DappRelayResult
|
||
|
||
interface DappRelayResult {
|
||
client: RelayClient;
|
||
uri: string; // wiz:// URI to encode into QR
|
||
credentials: KeyExchangeCredentials;
|
||
events: EventEmitter<{ keyexchangecomplete: [walletPublicKey: Uint8Array] }>;
|
||
cleanup: () => void;
|
||
}
|
||
```
|
||
|
||
Internally it wraps the caller's `statusCallback` to intercept `wallet_ready` messages,
|
||
verify the secret, and call `client.setPairedPublicKey()` before the message is processed by
|
||
`DappConnectionManager`. On reconnect, if `walletPublicKeyNostr` is already known it immediately
|
||
restores the paired key so outbound messages can be sent before the next `wallet_ready` arrives.
|
||
|
||
## initiateWalletRelay
|
||
|
||
```typescript
|
||
initiateWalletRelay(
|
||
statusCallback: RelayStatusCallback,
|
||
options: {
|
||
uri: string; // wiz:// URI from QR scan
|
||
walletPrivateKey: Uint8Array; // wallet's relay identity key (may be ephemeral)
|
||
}
|
||
): { cleanup: () => void }
|
||
```
|
||
|
||
Decodes the URI, sets the dapp's public key as the initial paired key, and starts the relay.
|
||
On each connect/reconnect, the wallet sends `wallet_ready` which carries the key exchange fields
|
||
(`public_key`, `secret`) alongside the session data. This gives the dapp the wallet's pubkey
|
||
atomically with the application handshake data.
|
||
|
||
## Encryption: NIP-17 gift wrap
|
||
|
||
NIP-17 gift wrap works like a sealed envelope:
|
||
|
||
1. **Rumor** (kind 14, `PrivateDirectMessage`): the plaintext message, signed by the sender.
|
||
Content is the JSON payload. Tags include `["p", recipient_pubkey_hex]`.
|
||
2. **Seal** (kind 13): the rumor encrypted with a shared ECDH secret derived from sender's
|
||
private key and recipient's public key. No `p` tag — unlinkable to sender/recipient.
|
||
3. **Wrap** (kind 1059, `GiftWrap`): the seal encrypted to the recipient's public key using a
|
||
freshly generated throwaway key. Has a `["p", recipient_pubkey_hex]` tag so the relay can
|
||
route it to the right subscription.
|
||
|
||
The relay sees only kind 1059 with a recipient tag. It cannot read content or link messages to
|
||
senders. The throwaway outer key means even the relay cannot correlate multiple messages from
|
||
the same sender.
|
||
|
||
NDK handles all three layers in `giftWrap()` / `giftUnwrap()`.
|
||
|
||
## Default relays
|
||
|
||
```
|
||
wss://relay.riften.net:443 (primary)
|
||
wss://relay.cauldron.quest:443 (secondary)
|
||
```
|
||
|
||
Both relays are used by default on both dapp and wallet sides for redundancy. Since Nostr
|
||
relays do not federate (they don't forward events to each other), connecting to multiple relays
|
||
ensures messages are delivered even if one relay is temporarily unavailable.
|
||
|
||
The connection URI encodes only the primary relay; the secondary is added programmatically
|
||
by the library. When a custom relay is specified (via URI or `explicitRelayUrls`), only that
|
||
relay is used — default relays are not auto-added.
|
||
|
||
Nothing in the protocol prevents using any other standard Nostr relay.
|
||
|
||
### Duplicate message handling
|
||
|
||
When subscribed to multiple relays, the same event may arrive from more than one relay.
|
||
nostr-tools `SimplePool.subscribeMany()` deduplicates events by ID — it tracks seen event IDs
|
||
in a per-subscription `_knownIds` set and only fires `onevent` once per unique ID. Since
|
||
`pool.publish(urls, event)` sends the identical event (same ID) to all relays, the receiving
|
||
side's pool delivers it exactly once.
|
||
|
||
## Transport-level extensions
|
||
|
||
Transport-level extensions are capabilities of the relay/gift-wrap transport itself, independent
|
||
of any application protocol. They're distinct from the protocol-level extensions documented in
|
||
[extensions.md](extensions.md), which extend `hdwalletv1` specifically.
|
||
|
||
### Negotiation
|
||
|
||
Both sides advertise transport-level extensions in a base `extensions` field on their handshake
|
||
message (`dapp_ready` / `wallet_ready`). The shape is identical on both sides:
|
||
|
||
```typescript
|
||
interface DappReadyMessage {
|
||
// ...
|
||
extensions?: Record<string, unknown>;
|
||
}
|
||
|
||
interface WalletReadyMessage {
|
||
// ...
|
||
extensions?: Record<string, unknown>;
|
||
}
|
||
```
|
||
|
||
A capability is considered enabled only when **both** sides advertise it. `RelayClient` exposes
|
||
`setPeerCapabilities({...})` so the connection managers can inform it once they've parsed the
|
||
peer's `_ready` message.
|
||
|
||
Extension values are per-extension; today `chunk` uses `{ version: 1 }`. Unknown extension keys
|
||
are ignored by implementations that don't recognise them, so adding new capabilities is always
|
||
backward-compatible — an old peer simply doesn't advertise them and the new side falls back to
|
||
the non-extended behaviour.
|
||
|
||
### Known transport extensions
|
||
|
||
| Extension | Purpose |
|
||
|---|---|
|
||
| `chunk` | Split messages larger than NIP-44's 65,535-byte plaintext ceiling across multiple gift-wrapped events. Symmetric — enabled when both sides advertise. See below. |
|
||
|
||
## Chunking (`chunk` extension)
|
||
|
||
### Why it exists
|
||
|
||
NIP-44 caps plaintext at 65,535 bytes — the length is encoded as a U16BE prefix in the wire
|
||
format, so the limit is structural, not a guardrail that can be raised. NIP-17 gift-wrap
|
||
additionally encrypts twice (rumor inside seal inside wrap), so a single 50+ KB `ProtocolMessage`
|
||
will blow the outer wrap's plaintext budget even when the inner payload itself looks small enough.
|
||
|
||
Real scenarios that exceed the cap:
|
||
|
||
- Aggregated swap `sign_transaction_request` with many pool UTXOs (each with hex `lockingBytecode`
|
||
and `unlockingBytecode` in the per-input `sourceOutputs`).
|
||
- `sign_transaction_response` carrying a signed transaction hex string. Policy-max BCH transactions
|
||
are 100 KB (→ 200 KB hex); consensus-max is 1 MB (→ 2 MB hex).
|
||
|
||
### Wire format
|
||
|
||
```typescript
|
||
interface ChunkMessage extends ProtocolMessage {
|
||
action: "chunk";
|
||
time: number; // shared across all chunks of one logical message
|
||
msgId: string; // random identifier, shared across all chunks
|
||
index: number; // 0-based chunk index within [0, total)
|
||
total: number; // total number of chunks for this msgId
|
||
data: string; // base64 slice of the UTF-8 bytes of JSON.stringify(originalMessage)
|
||
}
|
||
```
|
||
|
||
To reconstruct the original message: concatenate the `data` strings in `index` order, base64-decode
|
||
to bytes, UTF-8-decode to a string, `JSON.parse`.
|
||
|
||
### Sender
|
||
|
||
`RelayClient.publishMessage` measures the UTF-8 byte length of the serialized `ProtocolMessage`.
|
||
If it exceeds the per-message threshold:
|
||
|
||
- If the peer's `chunk` capability is enabled, the message is split into `ChunkMessage`s and each
|
||
is published individually via the same `wrapEvent` path as any other message. No ACKs — see
|
||
"Failure modes" below.
|
||
- If the peer has not advertised `chunk`, `publishMessage` throws a structured error
|
||
(`"Cannot send <action>: message is larger than NIP-44's 65,535-byte ceiling and the peer does
|
||
not advertise the 'chunk' transport extension. Please update the connected wallet/dapp..."`).
|
||
This replaces the cryptic `invalid plaintext size` error from nostr-tools.
|
||
|
||
The per-chunk budget is sized conservatively. Each chunk's raw data is ≤ 30,000 bytes, which after
|
||
base64 expansion (~4/3×), JSON envelope overhead, and the two-layer NIP-17 gift-wrap encryption
|
||
stays well under NIP-44's 65,535-byte ceiling for the outer wrap's plaintext. See
|
||
`packages/core/src/transforms/chunk.ts` for the derivation.
|
||
|
||
### Receiver
|
||
|
||
`RelayClient` owns a `ChunkReassembler` instance. Chunks pass the same peer filter and
|
||
`lastProcessedTimestamp` dedup as any other message, then are routed to the reassembler. When all
|
||
chunks for a `msgId` have arrived (in any order), the bytes are concatenated, decoded, and the
|
||
resulting `ProtocolMessage` is handed to the application-level handler — indistinguishable to
|
||
the application from an unchunked message of equivalent size.
|
||
|
||
The reassembler holds two maps with TTL eviction:
|
||
|
||
- **in-flight buffers** keyed by `msgId` — collects chunks until complete; default TTL 120 s,
|
||
sized to comfortably fit a ~35-chunk 2 MB response under real relay latency.
|
||
- **completed** — tracks `msgId`s we've already delivered, for the same TTL, so late-arriving
|
||
duplicates (e.g. cross-subscription replay after reconnect) don't spawn a second reassembly and
|
||
double-deliver.
|
||
|
||
A background sweeper runs every 10 s while connected and evicts expired entries. Reassembly state
|
||
is not persisted — reconnects rely on the relay replaying events to complete any in-flight
|
||
transfer (see below).
|
||
|
||
### Failure modes
|
||
|
||
| Failure | Behaviour |
|
||
|---|---|
|
||
| Dapp reloads mid-send | Dapp on reload has no in-flight state. User retries → fresh `msgId`, all chunks re-sent. Wallet's partial buffer for the old `msgId` expires via TTL. |
|
||
| Wallet reloads mid-receive | Subscription filter has no `since` clause, so on reconnect the relay re-delivers all events addressed to the wallet. Chunks reassemble fresh. Works as long as the chunks are still within the relay's retention window. |
|
||
| Network blip / all relays reject a chunk | `Promise.allSettled` on publish treats each chunk identically to any other single message. If all relays reject, `publishMessage` throws and `emitDisconnect` fires — same posture as today's sign-request failure. |
|
||
| Relay prunes mid-reassembly | Receiver's partial times out via TTL. Application sees no response; user retries — same failure mode the protocol already has for any lost sign request. |
|
||
|
||
No persistence layer, no per-chunk ACKs, no new round trips. The assembled `ProtocolMessage`'s
|
||
own response (e.g. `sign_transaction_response`) is the effective end-to-end ACK.
|
||
|
||
### Backward compatibility
|
||
|
||
| Dapp | Wallet | Outcome |
|
||
|---|---|---|
|
||
| Old | Old | Unchanged. Oversized message fails at nostr-tools with the raw NIP-44 error. |
|
||
| **New** | Old | Dapp detects absent `chunk` capability and throws a clear upgrade-guidance error instead of the cryptic NIP-44 error. |
|
||
| Old | **New** | Wallet detects absent `chunk` capability and throws a clear error symmetrically. |
|
||
| **New** | **New** | Both advertise, both enable, oversized messages chunk-and-reassemble transparently. Application code is unchanged. |
|
||
|
||
Upgrading the library on both sides is sufficient — no adapter-interface changes, no new
|
||
configuration, no capability opt-in. The extension is always-on when supported.
|