Benchmark data · ArrowSpace v0.26.2 · CVE dataset · 18 evaluation prompts

Improved semantic search. Tail quality, improved.

Taumode ranking improves semantic search where cosine flattens: the first relevant CVE lands about 2× higher, tolerant recall stays very high (0.96), and the tail of the ranking keeps its score shape — a tail/head ratio of 0.990 against cosine’s 0.989. Rare, high-severity matches stop fading into the bottom quartile.

95.9%
Tolerant recall@k · taumode τ=0.6
99.0%
Tail/head score ratio · cosine 98.9%
96.4%
NDCG@10 agreement · taumode vs hybrid
2×
First-hit MRR vs cosine · v0.26 runs
Recall@k comparison across cosine, hybrid and taumode rankings: traditional, semantic and tolerant recall (ArrowSpace v0.26.2 CVE benchmark)
Tolerant recall stays high under taumode ranking Full chart ↗
Tail analysis: score distribution across the ranking for cosine vs taumode (ArrowSpace v0.26.2 CVE benchmark)
Tail/head score ratio τ=0.6: 0.990 vs cosine 0.989 Full chart ↗
Version Run τ NDCG@10 vs Cosine Tail/Head Tolerant Recall@k

Latest CVE benchmark runs from pyarrowspace tests/output (v0.22+). Earlier v0.16–v0.17 corpus-scaling runs are omitted; v0.22/v0.24 covered energymaps and arxiv datasets and carry no CVE rows. Tolerant recall was introduced with the semantic-recall suite in v0.26.2.

Tail recall advantage

Spectral search recovers semantically related CVEs that cosine ranks in the bottom quartile — the structurally rare, high-severity edge cases most likely to be missed by traditional embedding search.

View tail analysis chart ↗
View raw benchmark data ↗ ← Back to search