Commit graph

192 commits

Author SHA1 Message Date
Dagur Valberg Johannsson
527b9ba91c
Bump electrum dep to allow larger websocket msg
We need to allow larger messages to index large blocks via websocket
interface.
2026-02-24 10:33:39 +01:00
Dagur Valberg Johannsson
f421327a5d
Switch to tokio and sqlx
This makes the application fully async; freeing web server threads to
handle new connections when waiting on SQL queries.

Additionally sqlx will allow easier move to a different database if
needed in the future.

Includes some SQL optimizations as well (slow queries more easliy identifiable
with sqlx).
2026-02-18 10:16:51 +01:00
Dagur Valberg Johannsson
ceb0e54dc0
Add 'bcmr' to metrics cache
This removes the need to have a BCMR connction during token search /
listing; also saving 2x lookups per token in results.
2026-02-17 14:06:16 +01:00
Dagur Valberg Johannsson
809b13e1f0 Merge branch 'rpc-health' into 'master'
Add /health endpoint

See merge request riftenlabs/riftenlabs-indexer!66
2026-02-10 14:40:01 +00:00
Dagur Valberg Johannsson
3ff73473df
Add /health endpoint 2026-02-10 15:36:21 +01:00
Dagur Valberg Johannsson
52ad406cf1 Merge branch 'db-cfg' into 'master'
Make DB read pool size configurable

See merge request riftenlabs/riftenlabs-indexer!65
2026-02-10 14:31:03 +00:00
Dagur Valberg Johannsson
bac43bbb98
Make DB read pool size configurable 2026-02-10 14:49:28 +01:00
Dagur Valberg Johannsson
36d620eb82 Merge branch 'bug-aapy' into 'master'
bug: Fix AAPY issue; follow up from BLOB change

See merge request riftenlabs/riftenlabs-indexer!64
2026-02-05 14:14:08 +00:00
Dagur Valberg Johannsson
17995ea520
Shorten db lock in BCMR downloader
Don't need to hold a write lock while we wait for BCMR download.
2026-02-05 15:10:07 +01:00
Dagur Valberg Johannsson
16d7ff42b0
bug: Fix AAPY issue; follow up from BLOB change 2026-02-05 15:09:47 +01:00
Dagur Valberg Johannsson
8180efdfe5 Merge branch 'fd-reduce' into 'master'
Reduce file descriptor usage

See merge request riftenlabs/riftenlabs-indexer!63
2026-02-04 08:11:47 +00:00
Dagur Valberg Johannsson
d6fa922154
Reduce file descriptor usage
Misc changes to reduce file descriptor usage for the indexer.
2026-02-04 08:53:27 +01:00
Dagur Valberg Johannsson
02e8ea2d4a Merge branch 'ws' into 'master'
Add WebSocket support

See merge request riftenlabs/riftenlabs-indexer!62
2026-02-03 08:47:12 +00:00
Dagur Valberg Johannsson
7ad0303209
Add WebSocket support 2026-02-03 09:39:12 +01:00
Dagur Valberg Johannsson
774fff6c92 Merge branch 'cache-write-lock' into 'master'
Fix potential API hangs from write lock contention

See merge request riftenlabs/riftenlabs-indexer!60
2026-02-02 20:14:09 +00:00
Dagur Valberg Johannsson
d8929dc913 Merge branch 'crc20-fix' into 'master'
Fix cross-database query bug in CRC20 BCMR filtering

See merge request riftenlabs/riftenlabs-indexer!61
2026-02-02 20:11:19 +00:00
Dagur Valberg Johannsson
b32a3e54cc Fix cross-database query bug in CRC20 BCMR filtering
The previous commit tried to query bcmr_data table from the crc20
database connection, but bcmr_data exists only in the bcmr database.

Fix by introducing STATE_SKIP_HAS_BCMR to mark tokens that have BCMR
data. The CRC20 fetcher now queries the bcmr database separately to
find candidates with BCMR data, marks them in the crc20 database, and
skips them in future iterations.
2026-02-02 21:03:11 +01:00
Dagur Valberg Johannsson
e1ebc853f5 Fix potential API hangs from write lock contention
- Refactor update_changes_score_volume_and_ranking to use read connections
  for read/compute phases, only acquiring write connection for final writes
- Change first_pool_creation to use try_get() for opportunistic caching,
  preventing thread exhaustion if write pool is busy
- Add linter to flag blocking _w.get() calls in RPC handlers
2026-02-02 21:01:50 +01:00
Dagur Valberg Johannsson
055d0bee29 Merge branch 'crc20-optimize' into 'master'
Don't fetch CRC20 info for tokens that have BCMR

See merge request riftenlabs/riftenlabs-indexer!57
2026-02-02 11:38:22 +00:00
Dagur Valberg Johannsson
efdf6c477e Merge branch '500-on-ibd' into 'master'
Do not serve requests during IBD

See merge request riftenlabs/riftenlabs-indexer!58
2026-02-02 11:31:11 +00:00
Dagur Valberg Johannsson
d1cfea4155
Don't fetch CRC20 info for tokens that have BCMR
When BCMR data exist; we prefer to use it. Thus we don't need to fetch
CRC20 data for tokens in this case.
2026-02-02 12:09:45 +01:00
Dagur Valberg Johannsson
2b29590211
Do not serve requests during IBD
Do not serve old data; reject RPC requests during IBD.
2026-02-02 12:08:04 +01:00
Dagur Valberg Johannsson
62f2693bdf Merge branch 'ci-cache' into 'master'
ci: Use lockfile as cache key

See merge request riftenlabs/riftenlabs-indexer!59
2026-02-02 11:07:41 +00:00
Dagur Valberg Johannsson
b956cb743d Merge branch 'blob-fix' into 'master'
bug: Reading TEXT as BLOB

See merge request riftenlabs/riftenlabs-indexer!56
2026-02-02 11:00:10 +00:00
Dagur Valberg Johannsson
98a6d1a0bb
Clippy issue on new rust version 2026-02-02 11:59:08 +01:00
Dagur Valberg Johannsson
0388aa282e
ci: Use lockfile as cache key
This ensures we don't keep stale builds of dependencies around.
2026-02-02 11:52:09 +01:00
Dagur Valberg Johannsson
beebc2c86f
bugfix: Reverse hash issues with blob 2026-02-02 11:34:18 +01:00
Dagur Valberg Johannsson
cbc2e10acb
bug: Reading TEXT as BLOB
This is a follow-up bugfix from previous "to BLOB" change.
2026-01-23 16:15:42 +01:00
Dagur Valberg Johannsson
2301b45887 Merge branch 'blob' into 'master'
Convert hash columns from TEXT to BLOB storage

Closes #1

See merge request riftenlabs/riftenlabs-indexer!53
2026-01-22 15:20:38 +00:00
Dagur Valberg Johannsson
58fd9595d7
Convert hash columns from TEXT to BLOB storage
Migrate all 32-byte hash fields from TEXT (64-char hex) to BLOB
(32 bytes) for ~50% storage savings on hash columns.

Changes:
- Add src/db/blob.rs with ToBlob/FromBlob traits for hash types
- Update all table schemas to use BLOB for hash columns
- Replace .to_hex() with .to_blob() for DB inserts
- Replace ::from_hex() with ::from_blob() for DB reads
- Use SQL hex() function where API responses need hex strings
- Bump DB_VERSION from 4 to 5

Affected tables: tx, pool, pool_history_entry, utxo_spending,
utxo_funding, user_action, auth_chain_entry, bcmr_data,
bcmr_failure, bcmr_well_known, crc20, crc20_candidates,
delphi_entry, cached_token_metrics

Fixes #1
2026-01-22 08:30:17 +01:00
Dagur Valberg Johannsson
d41073d5b7 Merge branch 'aapy-fix' into 'master'
Fix APY overflow when period_start clips pool duration

Closes #18

See merge request riftenlabs/riftenlabs-indexer!55
2026-01-22 06:50:11 +00:00
Dagur Valberg Johannsson
cf702959ca Fix APY overflow when period_start clips pool duration
Interpolate sqrt(K) linearly when the API's period_start is after the
pool's snapshot timestamp. This ensures yield and duration are calculated
over the same effective period, preventing overflow from high yields
being annualized over short durations.

Added unit tests based on failing production data for token d03a0d87...

Fixes #18
2026-01-21 19:16:46 +01:00
Dagur Valberg Johannsson
27c7165633 Merge branch 'bug-bcmr' into 'master'
Fix token search dedup to prefer entries with metadata

Closes #7

See merge request riftenlabs/riftenlabs-indexer!54
2026-01-21 14:33:27 +00:00
Dagur Valberg Johannsson
be1499db7c
Fix token search dedup to prefer entries with metadata
When a token has both an auth_chain_entry (without BCMR) and a CRC20
entry (with name/symbol), the dedup logic now keeps the metadata
instead of returning NULL.
2026-01-21 15:23:42 +01:00
Dagur Valberg Johannsson
66097c1abb Merge branch 'copyright' into 'master'
Update copyright headers

See merge request riftenlabs/riftenlabs-indexer!52
2026-01-21 11:57:48 +00:00
Dagur Valberg Johannsson
d85bc2c9db
Update copyright headers 2026-01-21 12:37:13 +01:00
Dagur Valberg Johannsson
1dabcc756c Merge branch 'shutdown' into 'master'
Add graceful shutdown for background threads

See merge request riftenlabs/riftenlabs-indexer!51
2026-01-21 11:36:47 +00:00
Dagur Valberg Johannsson
ec20a2ad75 Add graceful shutdown for background threads
Introduce a shutdown flag and Rocket fairing that signals background
threads to exit cleanly when Ctrl+C is pressed. Affects the indexing
thread, metrics updater, and block receiver loop.
2026-01-21 12:23:08 +01:00
Dagur Valberg Johannsson
16ed485711 Merge branch 'bug-bcmr' into 'master'
Fix bcmr_failure unique constraint for multi-token transactions

Closes #17

See merge request riftenlabs/riftenlabs-indexer!50
2026-01-21 10:01:23 +00:00
Dagur Valberg Johannsson
6876dac6d2 Merge branch 'cache' into 'master'
Add Cache-Control headers to RPC endpoints for load balancer caching

See merge request riftenlabs/riftenlabs-indexer!49
2026-01-21 10:01:07 +00:00
Dagur Valberg Johannsson
e93d46ec69 Add Cache-Control headers to RPC endpoints for load balancer caching
Introduce CachedJson response wrapper with cache duration based on data mutability:
- CACHE_IMMUTABLE (1 day): historical data with explicit timestamps
- CACHE_BCMR (5 min): token metadata
- CACHE_AGGREGATE (60s): aggregate data that changes with blocks
- CACHE_NONE: real-time data (current price, active pools, latest txs)

Endpoints dynamically select cache duration based on whether time parameters
are explicitly provided (historical/immutable) or defaulted to now (live).
2026-01-21 10:37:55 +01:00
Dagur Valberg Johannsson
46fb62d29b Fix bcmr_failure unique constraint for multi-token transactions
Remove errant UNIQUE INDEX on (utxo, txid) that conflicted with the
PRIMARY KEY (token_id, utxo, txid). A single transaction can update
auth chains for multiple tokens, requiring the same (utxo, txid) pair
to exist with different token_ids. The unique index blocked this.

The index was also unnecessary - EXPLAIN QUERY PLAN shows the PRIMARY
KEY index is used for all bcmr_failure lookups.

Add test covering multi-token failure recording.
2026-01-21 10:17:49 +01:00
Dagur Valberg Johannsson
42aa74fc41 Merge branch '500-fixes' into 'master'
Fix incorrect 5XX responses for user input errors

See merge request riftenlabs/riftenlabs-indexer!48
2026-01-21 08:06:13 +00:00
Dagur Valberg Johannsson
2c76bf09c7 Fix incorrect 5XX responses for user input errors
Return proper 4XX status codes for client errors to prevent load balancers
from misinterpreting input validation failures as server errors. Add
centralized ApiResult type and ApiErrorCode enum for consistent handling.
2026-01-21 08:27:57 +01:00
Dagur Valberg Johannsson
b702abf8a8 Merge branch 'pool-id' into 'master'
Add RPC call to retrieve pool ID

See merge request riftenlabs/riftenlabs-indexer!47
2026-01-13 08:43:57 +00:00
Dagur Valberg Johannsson
50cd23dd34
Add RPC call to retrieve pool ID
Add a call that returns a pool ID given an utxo
2026-01-09 15:29:05 +01:00
Dagur Valberg Johannsson
46c3ff029a Merge branch 'chipnet' into 'master'
Add Chipnet support

See merge request riftenlabs/riftenlabs-indexer!46
2026-01-09 12:56:50 +00:00
Dagur Valberg Johannsson
a6c6159eba
Add Chipnet support 2026-01-09 13:46:58 +01:00
jakobsn
1b7801dff8 Merge branch 'manage-several-entries-one-txid' into 'master'
Manage several entries one txid

See merge request riftenlabs/riftenlabs-indexer!44
2025-11-17 09:21:51 +00:00
jakobsn
8fd14cc65b Manage several entries one txid 2025-11-17 09:21:51 +00:00