The Linkup Research team released the SPARSEUP model as an open-source learned sparse encoder. The project uses a 149M-parameter ModernBERT backbone and ships under the Apache 2.0 license. Linkup reports an average score of 56.4 nDCG@10 on the BEIR-13 benchmark. In turn, the team describes it as the top public vocabulary-based sparse encoder under 150M parameters.

Now developers can deploy the system immediately in production. The model weights live openly on Hugging Face. Teams can load the code through Transformers or Sentence Transformers by setting the remote code flag. It delivers fast lexical retrieval without the black-box opacity of pure dense embeddings.

Why Build the SPARSEUP model Now?

Most modern neural search setups rely entirely on dense embeddings. In those systems, an encoder compresses a full document into a single continuous vector. Dense vectors excel at broad topic matching. But they struggle with exact keyword matching, brand names, serial numbers, and rare words.

Sparse encoders take a distinct technical path. Instead of fixed latent vectors, they generate weight values over a defined vocabulary dictionary. Each vector dimension maps directly to a human-readable text token. Because of this structure, the outputs integrate cleanly into traditional inverted search indexes. Engineers can inspect the active dimensions to see which terms drove a retrieval match.

Then came the recent release of DenseOn and LateOn from LightOn. That research team published an open dataset, a training recipe, a dense retrieval model, and a late-interaction model. But the ecosystem lacked a matching learned sparse option on the same foundation. Linkup built the SPARSEUP model to fill that precise technical void. By using the same underlying family and data mixture, practitioners can compare dense, sparse, and late-interaction styles directly.

How Did Engineers Train the SPARSEUP model?

Training for the system started from the LateOn-unsupervised base checkpoint. That base lacked a masked language modeling head on top. So the Linkup engineering team grafted the original ModernBERT masked language modeling head back onto the network.

Next, the team ran fine-tuning using the open embeddings-fine-tuning mixture from LightOn. They applied purely contrastive learning objectives across the training corpus. Training did not rely on complex cross-encoder distillation steps. Each query received 7 hard negative samples drawn from a candidate pool of 50, alongside standard in-batch negatives. As well, the entire fine-tuning run ran smoothly on a single NVIDIA H100 GPU.

Overcoming Vocabulary Bloat in Early Tests

Standard baseline implementations ran into immediate trouble during early tests. A standard SPLADE architecture built on this backbone produced huge token bags crammed with useless stopwords. This issue caused severe index bloat and degraded retrieval speed. To solve this dilemma, Linkup applied three core architectural adjustments:

• Logit shifting: The encoder computes the activation formula log(1 + ReLU(x – 15)). Raw masked language modeling logits in ModernBERT sat too high. That baseline elevation saturated the log function and made token bags far too dense at initial steps.
• Per-position top-k selection: Each input token retains only its top 12 vocabulary dimensions before the max pooling step. This change limits expansion per token rather than capping total vector size.
• Case folding: The standard byte-level Byte Pair Encoding tokenizer tracks distinct case tokens separately. The SPARSEUP model collapses those variations onto a single identifier and preserves the highest weight. This adjustment reduced total output dimensions from roughly 50,000 down to approximately 34,000.

Plus, queries and documents use specific prefixes during indexing and inference. The system tags search queries with a [Q] prefix and documents with a [D] prefix. Scoring relies on a clean dot product calculation between query and document vectors. The evaluation pipeline enforces a maximum length of 128 tokens for queries and 512 tokens for indexed text chunks.

How Does the SPARSEUP model Perform on Retrieval Tasks?

Evaluation on the standard BEIR-13 benchmark demonstrates competitive retrieval capability. In tests excluding MS MARCO, the SPARSEUP model posted an average nDCG@10 of 56.4 points. This mark sets a fresh high for open vocabulary-based sparse models under the 150M parameter threshold.

By then, comparisons against alternative sparse models highlighted strong gains across diverse benchmarks:

• SPARSEUP model: 56.4 nDCG@10
• opensearch-neural-sparse-encoding-doc-v3-gte: 54.6 nDCG@10
• opensearch-neural-sparse-encoding-v1: 52.44 nDCG@10
• ModernBERT-VT: 52.4 nDCG@10
• splade-v3: 51.7 nDCG@10
• granite-embedding-30m-sparse: 50.6 nDCG@10
• LACONIC-1B: 58.7 nDCG@10 (achieved at a much larger 1B parameter scale)

Still, controlled comparisons reveal clear trade-offs across model types. When holding the backbone architecture and training data completely fixed, multi-vector and dense models show distinct strengths. LateOn reached an average of 58.9, while DenseOn earned 57.9 against the 56.4 mark of the SPARSEUP model.

Yet context matters when interpreting these baseline results. The SPARSEUP model used approximate search via an inverted index during benchmarking. Meanwhile, the dense baselines reported exact search runs. In specific head-to-head datasets, sparse encoding showed distinct advantages. The sparse system won outright on ArguAna and Touché, and it outperformed DenseOn on HotpotQA. However, it lagged on heavily semantic workloads. Financial queries on FiQA exhibited the widest performance gap, while DBPedia proved challenging as well.

When tested on decontaminated versions of the BEIR benchmark, the gap between architectures shifted noticeably. On decontaminated data, the gap to DenseOn shrank to just 0.17 points. Linkup noted that decontaminated versions of NQ and MS MARCO contain only 21 and 46 queries respectively, meaning those small subsets carry higher statistical noise.

Is the SPARSEUP model Fast in Production Workloads?

Real-world enterprise adoption depends heavily on inference speed and index footprint. The SPARSEUP model maintains a disciplined balance between vector sparsity and keyword recall. On the MS MARCO evaluation set, the encoder averages 47 non-zero terms for queries and 190 non-zero terms for documents. For comparison, splade-v3 averages 25 terms per query and 170 terms per document.

Speed benchmarks demonstrate high throughput for real-time applications. When paired with the Seismic inverted index library, the model achieves over 97% recall relative to exact search. It delivers that accuracy in roughly 380 microseconds per query on a single CPU thread. Linkup researchers highlighted that allowing denser vectors could add 1 to 2 points to the BEIR score. Instead, the team chose to preserve strict sparsity to maintain high operational throughput and lower infrastructure costs.

As the search ecosystem expands, readers can follow developments and more AI coverage on open information retrieval frameworks.