| pipeline_tag: text-generation | |
| # Parcae-Large-770M | |
| Parcae is a novel stable, looped architecture that addresses instability issues in prior looped language models by recasting looping as a nonlinear time-variant dynamical system over the residual stream. This specific checkpoint is the 770M parameter version, trained on the FineWeb-Edu dataset. | |
| - **Paper:** [Parcae: Scaling Laws For Stable Looped Language Models](https://arxiv.org/abs/2604.12946) | |
| - **Repository:** [https://github.com/sandyresearch/parcae](https://github.com/sandyresearch/parcae) | |
| - **Project Page:** [https://sandyresearch.github.io/parcae/](https://sandyresearch.github.io/parcae/) | |
| ## Installation | |
| To use this model, you can install the official package: | |
| ```bash | |
| pip install parcae-lm | |
| ``` | |
| ## Usage | |
| You can load the pretrained weights and instantiate the model using the following code: | |
| ```python | |
| import parcae_lm | |
| # Load the pretrained model from HuggingFace | |
| model = parcae_lm.from_pretrained("SandyResearch/parcae-large-770m") | |
| ``` | |
| ## Model Details | |
| This model variant has the following configuration: | |
| | Model | Parameters | Prelude Layers | Core Layers | Coda Layers | Model Dim | Default Recurrence | | |
| |-------|-----------|---------|------|------|-----------|------------| | |
| | Parcae-770M | 770M | 6 | 6 | 6 | 1280 | 8 | | |
| ## Citation | |
| ```bibtex | |
| @misc{prairie2026parcaescalinglawsstable, | |
| title={Parcae: Scaling Laws For Stable Looped Language Models}, | |
| author={Hayden Prairie and Zachary Novack and Taylor Berg-Kirkpatrick and Daniel Y. Fu}, | |
| year={2026}, | |
| eprint={2604.12946}, | |
| archivePrefix={arXiv}, | |
| primaryClass={cs.LG}, | |
| url={https://arxiv.org/abs/2604.12946}, | |
| } | |
| ``` |