CLARION: Continuous Latent Augmented-Retrieval Inference On N-cores
A parallel CPU reimplementation of the CLaRa continuous RAG architecture, designed to eliminate computational bottlenecks in embedding generation and cosine similarity search via a hybrid NumPy + Cython and OpenMP approach.
Generation Speedup
×4.95
On autoregressive text generation.
Inference Speedup
×2.35
Reduced global forward pass.
Joint Training
×3.16
End-to-end architecture optimization.
Top-K Buffer Memory
\(\mathcal{O}(B \cdot K)\)
Instead of \(\mathcal{O}(B \cdot N)\) thanks to fused kernels.
Multi-Core Speedup Factors Analysis
Fundamental Architecture & Fused Kernel
The optimized backend reorganizes tensors into C-contiguous format to leverage the compiler's SIMD vectorization. Attention is performed in continuous (streaming) mode by blocks to avoid global allocation:
To propagate gradients through discrete Top-K routing, the pipeline implements a Straight-Through Estimator (STE) with a Softmax relaxation during backpropagation:
Research Paper
View the full scientific article in PDF format directly below.
PDF Preview (Expand on wide screens or download the document)