Commit graph

94 commits

Author SHA1 Message Date
Hossein Zoda
e86de7d5f5 ido: hide txchain_entrypoint, expose txchain_head as head txid
The IDO RPC record no longer leaks internal txchain row ids. Drop the
txchain_entrypoint field entirely, and change txchain_head from the
internal ido_txchain.id to the head record's txid (display hex).

list_idos and get_ido_by_offering_token_id resolve this in a single
query via LEFT JOIN ido_txchain ON head_tx.id = ido.txchain_head; a
NULL head yields null. Adds tests for both the resolved and null cases.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-21 01:44:39 +03:00
Hossein Zoda
6728753db0 ido: use preinit_txid hex as public id (rename internal id)
Rename ido.id -> ido.internal_id in the schema and IdoDBRecord, and
expose the preinit_txid hex as the public "id" in all RPC responses.

Public API shape:
  - IdoRpcRecord.id: i64 -> String (hex of preinit_txid). preinit_txid
    field is preserved unchanged.
  - IdoEntryRpcRecord.ido_id: i64 -> String (hex of parent preinit_txid).
  - IdoTxChainRpcRecord.ido_id: i64 -> String (hex of parent preinit_txid).
  - IdoTxChainRpcRecord gains ido_internal_id: i64 (debug endpoint only).
  - Routes /<id>/entries and /<id>/txchain accept the preinit_txid hex
    as the path id; returns 404 IDO_NOT_FOUND on miss.

Internals:
  - New lookup_internal_id_by_preinit_txid helper.
  - list_ido_entries / list_ido_txchain take preinit_txid_hex as input
    so the caller (which already parsed it from the path) avoids the
    extra "preinit_txid by internal_id" lookup.
  - Child table FK columns (ido_entry.ido_id, ido_txchain.ido_id) keep
    their names; only the parent PK and field accesses were renamed.

Notes:
  - Breaking API change for /ido/* endpoints. Clients reading "id" or
    "ido_id" as integers must switch to strings.
  - ido.db has no migration framework: drop the file and re-index on
    deploy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-21 01:44:39 +03:00
Hossein Zoda
210a51f274 ido bcmr ser/deser fixes 2026-06-21 01:44:39 +03:00
Hossein Zoda
c4eaad4cec ido more fixes +two unittests 2026-06-21 01:44:39 +03:00
Hossein Zoda
a944db4702 ido better error reporting 2026-06-21 01:44:39 +03:00
Hossein Zoda
f8ddac6ec3 followup update for ido -> adding ipfs bcmr placeholder 2026-06-21 01:44:39 +03:00
Hossein Zoda
335818460f followup changes to ido contracts 2026-06-21 01:44:39 +03:00
Hossein Zoda
3f8a9cd026 ido more fixes 2026-06-21 01:44:39 +03:00
Hossein Zoda
aa99f775da tiny fix 2026-06-21 01:44:39 +03:00
Hossein Zoda
988426e485 ido, compiles, with claude generated unittests 2026-06-21 01:44:39 +03:00
Hossein Zoda
0629e168c3 ido incomplete c02 2026-06-21 01:44:39 +03:00
Hossein Zoda
ce9dba23cf (WIP) ido 2026-06-21 01:44:39 +03:00
Dagur Valberg Johannsson
9d0eaff22c
tokentoken: add GET /tokentoken/pools (list all active pools)
List every active TokenToken pool (no pair filter) so the frontend can
present the set of pooled token pairs without probing each candidate pair.
Mirrors db_active_pools_for_pair minus the pair WHERE clause; served at a
distinct /pools path (the param'd /pool/active already matches param-less
requests via its Option guards, so reusing it would collide).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 13:48:45 +02:00
Dagur Valberg Johannsson
1614b84674 Upgrade bitcoincash to 0.32 and add TokenToken AMM pool indexing
Migrate off the deprecated bitcoincash 0.29 API (Amount/Version types,
FromHex -> FromStr/parse, non-exhaustive Network) across the indexer and
tests.

Add indexing of native token-A <-> token-B (TokenToken) AMM pools:

- tokentoken_pool / tokentoken_pool_history_entry tables in cauldron.db,
  created via an always-run idempotent migration (no DB_VERSION bump, so
  existing databases upgrade in place)
- block-path indexing sharing the cauldron write transaction and reorg
  undo, with creation/swap/withdrawal state tracking and reserve deltas
- mempool indexing: electrum mempool.get filters on the tokentoken
  contract code (spends) and the CONJURE op_return hint (creations);
  first_seen_timestamp reconciles with mtp on confirmation
- RPC endpoints /tokentoken/pool/active (pair lookup, order-insensitive)
  and /tokentoken/tokens

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 16:18:35 +02:00
jakobsn
dd4ffb5e3b Remove oracle_cash quickfix introduced Apr 24 2026 2026-06-09 13:18:53 +00:00
jakobsn
d8edfe15fb Merge branch 'chartImprovementFirstSeen' into 'master'
Use first time seen for phe same as transactions to make the live price chart sensible.

See merge request riftenlabs/riftenlabs-indexer!85
2026-05-08 14:24:18 +00:00
Jakob Notland
071bd3552b Use first time seen for phe same as transactions to make the live price chart sensible. 2026-05-07 14:25:52 +02:00
Dagur Valberg Johannsson
4d205cada4
Delphi v2 support
Same API from consumer-standpoint
2026-05-04 15:03:40 +02:00
jakobsn
f56cc16d93 using oracles cash feed 2026-04-24 05:19:43 +00:00
jakobsn
460794f067 Count pools that were not withdrawn at the given time. 2026-04-16 12:35:15 +00:00
Dagur Valberg Johannsson
c7eb5bcc36 Added support for indexing Moria contracts 2026-03-30 23:52:29 +02:00
jakobsn
bcab4db2fe Cachecandles 2026-03-25 11:38:10 +00:00
jakobsn
15f7331867 dormant-pools-report-inflated-apy 2026-03-19 08:55:51 +00:00
jakobsn
381cb94295 Commit alternative solution for pool inflation 2026-03-11 09:31:21 +00:00
jakobsn
1877b09b47 Resolve pool id array aapy 2026-03-06 12:08:15 +00:00
Dagur Valberg Johannsson
f421327a5d
Switch to tokio and sqlx
This makes the application fully async; freeing web server threads to
handle new connections when waiting on SQL queries.

Additionally sqlx will allow easier move to a different database if
needed in the future.

Includes some SQL optimizations as well (slow queries more easliy identifiable
with sqlx).
2026-02-18 10:16:51 +01:00
Dagur Valberg Johannsson
ceb0e54dc0
Add 'bcmr' to metrics cache
This removes the need to have a BCMR connction during token search /
listing; also saving 2x lookups per token in results.
2026-02-17 14:06:16 +01:00
Dagur Valberg Johannsson
bac43bbb98
Make DB read pool size configurable 2026-02-10 14:49:28 +01:00
Dagur Valberg Johannsson
16d7ff42b0
bug: Fix AAPY issue; follow up from BLOB change 2026-02-05 15:09:47 +01:00
Dagur Valberg Johannsson
d6fa922154
Reduce file descriptor usage
Misc changes to reduce file descriptor usage for the indexer.
2026-02-04 08:53:27 +01:00
Dagur Valberg Johannsson
774fff6c92 Merge branch 'cache-write-lock' into 'master'
Fix potential API hangs from write lock contention

See merge request riftenlabs/riftenlabs-indexer!60
2026-02-02 20:14:09 +00:00
Dagur Valberg Johannsson
b32a3e54cc Fix cross-database query bug in CRC20 BCMR filtering
The previous commit tried to query bcmr_data table from the crc20
database connection, but bcmr_data exists only in the bcmr database.

Fix by introducing STATE_SKIP_HAS_BCMR to mark tokens that have BCMR
data. The CRC20 fetcher now queries the bcmr database separately to
find candidates with BCMR data, marks them in the crc20 database, and
skips them in future iterations.
2026-02-02 21:03:11 +01:00
Dagur Valberg Johannsson
e1ebc853f5 Fix potential API hangs from write lock contention
- Refactor update_changes_score_volume_and_ranking to use read connections
  for read/compute phases, only acquiring write connection for final writes
- Change first_pool_creation to use try_get() for opportunistic caching,
  preventing thread exhaustion if write pool is busy
- Add linter to flag blocking _w.get() calls in RPC handlers
2026-02-02 21:01:50 +01:00
Dagur Valberg Johannsson
d1cfea4155
Don't fetch CRC20 info for tokens that have BCMR
When BCMR data exist; we prefer to use it. Thus we don't need to fetch
CRC20 data for tokens in this case.
2026-02-02 12:09:45 +01:00
Dagur Valberg Johannsson
beebc2c86f
bugfix: Reverse hash issues with blob 2026-02-02 11:34:18 +01:00
Dagur Valberg Johannsson
cbc2e10acb
bug: Reading TEXT as BLOB
This is a follow-up bugfix from previous "to BLOB" change.
2026-01-23 16:15:42 +01:00
Dagur Valberg Johannsson
2301b45887 Merge branch 'blob' into 'master'
Convert hash columns from TEXT to BLOB storage

Closes #1

See merge request riftenlabs/riftenlabs-indexer!53
2026-01-22 15:20:38 +00:00
Dagur Valberg Johannsson
58fd9595d7
Convert hash columns from TEXT to BLOB storage
Migrate all 32-byte hash fields from TEXT (64-char hex) to BLOB
(32 bytes) for ~50% storage savings on hash columns.

Changes:
- Add src/db/blob.rs with ToBlob/FromBlob traits for hash types
- Update all table schemas to use BLOB for hash columns
- Replace .to_hex() with .to_blob() for DB inserts
- Replace ::from_hex() with ::from_blob() for DB reads
- Use SQL hex() function where API responses need hex strings
- Bump DB_VERSION from 4 to 5

Affected tables: tx, pool, pool_history_entry, utxo_spending,
utxo_funding, user_action, auth_chain_entry, bcmr_data,
bcmr_failure, bcmr_well_known, crc20, crc20_candidates,
delphi_entry, cached_token_metrics

Fixes #1
2026-01-22 08:30:17 +01:00
Dagur Valberg Johannsson
be1499db7c
Fix token search dedup to prefer entries with metadata
When a token has both an auth_chain_entry (without BCMR) and a CRC20
entry (with name/symbol), the dedup logic now keeps the metadata
instead of returning NULL.
2026-01-21 15:23:42 +01:00
Dagur Valberg Johannsson
d85bc2c9db
Update copyright headers 2026-01-21 12:37:13 +01:00
Dagur Valberg Johannsson
1dabcc756c Merge branch 'shutdown' into 'master'
Add graceful shutdown for background threads

See merge request riftenlabs/riftenlabs-indexer!51
2026-01-21 11:36:47 +00:00
Dagur Valberg Johannsson
ec20a2ad75 Add graceful shutdown for background threads
Introduce a shutdown flag and Rocket fairing that signals background
threads to exit cleanly when Ctrl+C is pressed. Affects the indexing
thread, metrics updater, and block receiver loop.
2026-01-21 12:23:08 +01:00
Dagur Valberg Johannsson
46fb62d29b Fix bcmr_failure unique constraint for multi-token transactions
Remove errant UNIQUE INDEX on (utxo, txid) that conflicted with the
PRIMARY KEY (token_id, utxo, txid). A single transaction can update
auth chains for multiple tokens, requiring the same (utxo, txid) pair
to exist with different token_ids. The unique index blocked this.

The index was also unnecessary - EXPLAIN QUERY PLAN shows the PRIMARY
KEY index is used for all bcmr_failure lookups.

Add test covering multi-token failure recording.
2026-01-21 10:17:49 +01:00
Dagur Valberg Johannsson
50cd23dd34
Add RPC call to retrieve pool ID
Add a call that returns a pool ID given an utxo
2026-01-09 15:29:05 +01:00
Dagur Valberg Johannsson
a6c6159eba
Add Chipnet support 2026-01-09 13:46:58 +01:00
jakobsn
8fd14cc65b Manage several entries one txid 2025-11-17 09:21:51 +00:00
jakobsn
e053d52a76 Resolve "Olando auth-header not tracked" 2025-11-07 13:53:47 +00:00
Dagur Valberg Johannsson
861b9d16aa Merge branch 'volume-index' into 'master'
Add a volume index

See merge request riftenlabs/riftenlabs-indexer!41
2025-10-13 06:25:02 +00:00
Dagur Valberg Johannsson
ff3438ca9f
Add index to BCMR lookup 2025-10-08 14:03:20 +02:00
Dagur Valberg Johannsson
8be4cf894e
Add a volume index
Significantly improves cpu load of volume queries
2025-10-08 13:56:36 +02:00