Constitutional Midtraining - a cho-ai Collection

cho-ai 's Collections

Constitutional Midtraining

updated 5 days ago

Data and models for the paper 'Constitutional Midtraining: Content Presence Drives Alignment Gains' .

Upvote

cho-ai/constitutional-mt-data

Viewer • Updated 22 days ago • 2.35M • 514 • 3

Note Constitutional corpus, generated from values in Anthropic's Constitution. Documents are tagged by cluster and value. Reasoning variants and assembled training mixes available.
geodesic-research/sft-warm-start-200k

Viewer • Updated Apr 13 • 400k • 122

Note Shared instruction-tuning mix (200K examples) used for the value-neutral SFT stage.
geodesic-research/nemotron-super-120b-cc-mt-replay-only-1b

124B • Updated Jul 9 • 7

Note Control, post-MT
geodesic-research/nemotron-super-120b-cc-mt-replay-only-1b-200k-sft

124B • Updated Jul 9 • 9

Note Control, post-SFT
geodesic-research/nemotron-super-120b-cc-mt-replay-only-1b-200k-sft-gsm8k-lora

Updated Jul 14

Note Control, post-BFT
geodesic-research/nemotron-super-120b-cc-mt-curriculum-dr

124B • Updated Jul 9 • 5

Note Curriculum-DR, post-MT
geodesic-research/nemotron-super-120b-cc-mt-curriculum-dr-200k-sft

124B • Updated Jul 9 • 7

Note Curriculum-DR, post-SFT
geodesic-research/nemotron-super-120b-cc-mt-curriculum-dr-200k-sft-gsm8k-lora

Updated Jul 14

Note Curriculum-DR, post-BFT
geodesic-research/nemotron-super-120b-cc-mt-uniform-dr

124B • Updated Jul 9 • 5

Note Uniform-DR, post-MT
geodesic-research/nemotron-super-120b-cc-mt-uniform-dr-200k-sft

124B • Updated Jul 9 • 10

Note Uniform-DR, post-SFT
geodesic-research/nemotron-super-120b-cc-mt-uniform-dr-200k-sft-gsm8k-lora

Updated Jul 14

Note Uniform-DR, post-BFT
geodesic-research/nemotron-super-120b-cc-mt-curriculum-nodr

124B • Updated Jul 9 • 3

Note Curriculum-noDR, post-MT
geodesic-research/nemotron-super-120b-cc-mt-curriculum-nodr-200k-sft

124B • Updated Jul 9 • 5

Note Curriculum-noDR, post-SFT
geodesic-research/nemotron-super-120b-cc-mt-curriculum-nodr-200k-sft-gsm8k-lora

Updated Jul 14

Note Curriculum-noDR, post-BFT
geodesic-research/nemotron-super-120b-cc-mt-uniform-nodr

124B • Updated Jul 9 • 4

Note Uniform-noDR, post-MT
geodesic-research/nemotron-super-120b-cc-mt-uniform-nodr-200k-sft

124B • Updated Jul 9 • 7

Note Uniform-noDR, post-SFT
geodesic-research/nemotron-super-120b-cc-mt-uniform-nodr-200k-sft-gsm8k-lora

Updated Jul 14

Note Uniform-noDR, post-BFT

Upvote