LDARNet-110M

Pretrained LDARNet (~110M params) with learnable DNA tokenization (dynamic chunking + BiMamba-2).

• Paper: arXiv:2606.04552
• Code: ICML-LDARNet

Best among sub-300M models on 15 of 18 Nucleotide Transformer tasks, best overall on 9 of 18.

Using LDARNet?

I am collecting feedback on where this model actually gets used. If you have run it on your own data, I would be glad to know which organism or task, whether you used frozen embeddings or fine-tuned, and what it was compared against. Open a discussion on this repo or email a.ledn2026@gmail.com. Reports of it working badly are as useful as reports of it working.

Files

• model_ckpt_110m.pt — MLM checkpoint with embedded LDarConfig

Download

Clone the code repository and install its dependencies, then download the model files:

hf download darlednik/LDARNet-110M \
  --local-dir models_ckpts

Load

import torch
from ldar.utils.ckpt import load_ldar_from_ckpt

model, cfg = load_ldar_from_ckpt(
    "models_ckpts/model_ckpt_110m.pt",
    device="cuda",
    dtype=torch.bfloat16,
)

Architecture

Component	Layout	d_model
Encoder	m3t1 — 3× BiMamba-2 + 1 local-attention layer	512
Backbone	M10 — 10× BiMamba-2 (+ SwiGLU)	768
Decoder	m4 — 4× BiMamba-2	512

• Compression ratio N = 4
• Byte vocabulary: {A, C, G, T, N, [MASK], <pad>}

Citation

@misc{ledneva2026ldarnetdnaadaptiverepresentation,
      title={LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic Modeling},
      author={Daria Ledneva and Denis Kuznetsov},
      year={2026},
      eprint={2606.04552},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.04552},
}