albtad01/STHELAR-Adapt-CellViT-SAM-H-x40

STHELAR-Adapt for CellViT-SAM-H-x40

This repository contains verified, architecture-specific STHELAR-Adapt states for the CellViT-SAM-H-x40 backbone. STHELAR-Adapt specializes CellViT for nuclei instance segmentation and spatial-transcriptomics-informed cell typing in STHELAR 40× H&E images.

The official CellViT-SAM-H-x40 base checkpoint is required separately. It is not redistributed here.

Q/V LoRA and AdaptFormer specialize the pretrained encoder while the decoder body remains frozen and only the final nucleus heads are trained.

Model and target taxonomy

The adapter state uses the selected MSc-thesis PEFT configuration:

• Q/V LoRA with rank 8, alpha 8, and dropout 0;
• AdaptFormer with reduction 16 and GELU;
• trainable final NP, HV, and NT 1×1 heads;
• frozen pretrained encoder weights outside the inserted PEFT modules;
• frozen decoder body;
• required changed mutable normalization buffers included in the state.

The state has 1.1176% trainable parameters relative to the instantiated CellViT-SAM-H-x40 PEFT model. This is a custom composite adapter format, not a drop-in standard PEFT/LoRA package.

STHELAR-derived foreground identities are grouped into five classes: Immune, Stromal, Epithelial, Melanocyte, and Other. Background is retained as the non-foreground class.

Required base checkpoint

Obtain CellViT-SAM-H-x40.pth from the official CellViT project. The expected SHA-256 is:

b324c10fddb0f80f5ab03a0459453a4c4848866934daf63435b46749a6b278cf

The official base weights, FullFT weights, training checkpoints, optimizer state, datasets, and run logs are not part of this release.

Adapter cohorts

The manifest contains 32 unique verified payloads:

Cohort	Count	Scientific role
Complete-slide evaluation	20	KLT plus nine tissues, reciprocal Fold A/B, seed 42
All-slides deployment	10	KLT plus all nine tissues, seed 42
Supplemental verified KLT	2	Fold A/B seed-43 evaluation states

Evaluation adapters were trained without the reciprocal held-out complete slide. All-slides adapters use all designated slides with the established within-slide training/validation construction; they are deployment states and carry no held-out-slide performance claim. Supplemental states are kept separate from the seed-42 core.

The KLT headline aggregate uses all three training seeds. Two seed-44 SAM-H states remain excluded from this publication set because their historical source-config hash provenance is unresolved, even though their result rows remain part of the audited thesis aggregate.

Directory conventions:

adapters/complete_slide_v1/klt/{fold_a,fold_b}/seed42/
adapters/complete_slide_v1/klt/all_slides/seed42/
adapters/complete_slide_v1/tissue_specific/<tissue>/{fold_a,fold_b}/seed42/
adapters/complete_slide_v1/tissue_specific/<tissue>/all_slides/seed42/
adapters/supplemental_klt_replicates/{fold_a,fold_b}/seed43/

Each directory contains the adapter safetensors plus adapter_config.json, checksums.json, and verification.json. The corresponding training config is recorded in adapter_manifest.csv and included under configs/.

One official architecture-matched base can be combined with one verified compact state at a time. SAM-H and CellViT-256 states are not interchangeable.

MSc thesis evaluation

The final predictive evaluation uses reciprocal complete-slide holdout. KLT values are equal-weight reciprocal-fold means after averaging each fold over seeds 42, 43, and 44. Nine-tissue values use seed 42 and equal weighting over the nine tissue-level reciprocal-fold means.

Evaluation	Method	bPQ	mPQ	F1 detection	F1 type
KLT complete-slide	PEFT	0.564	0.228	0.853	0.492
KLT complete-slide	FullFT	0.564	0.214	0.848	0.508
Nine-tissue complete-slide	PEFT	0.442	0.175	0.808	0.381
Nine-tissue complete-slide	FullFT	0.445	0.187	0.806	0.418

Nucleus detection and class-agnostic segmentation are preserved particularly closely; ST-informed typing remains more difficult.

Illustrative SAM-H overlays across the nine tissues, shown as Ground truth, Linear Probing, and the selected PEFT adapter. Colors denote the five grouped cell identities; the selected examples are not a statistically representative sample.

Systems and storage

Method	Trainable	Peak allocated VRAM	Serialized state
PEFT	1.1176%	9.362 GiB	30.369 MiB
FullFT	100%	15.508 GiB	2.607 GiB

For nine architecture-compatible domains, independent FullFT states require 23.463 GiB, compared with 2.874 GiB for one shared base plus nine PEFT states. Parameter efficiency does not imply universal runtime acceleration; timings depend on hardware, data movement, validation, checkpointing, and postprocessing.

Load a verified package

Install a compatible checkout of STHELAR-Adapt and its dependencies, then use the repository's verified release loader:

from utils.sthelar_release_loader import load_verified_adapter

model, evidence = load_verified_adapter(
    base_checkpoint="/path/to/CellViT-SAM-H-x40.pth",
    package_dir=(
        "/path/to/STHELAR-Adapt-CellViT-SAM-H-x40/"
        "adapters/complete_slide_v1/klt/fold_a/seed42"
    ),
    device="cpu",
)

The loader validates the base hash, payload hash, tensor inventory, expected trainable and mutable-buffer keys, and verification metadata before returning an eval-mode model.

To repeat the package-level checks and deterministic forward sanity test:

PYTHONPATH=/path/to/STHELAR-Adapt \
python scripts/verify_public_release_package.py \
  --base-checkpoint /path/to/CellViT-SAM-H-x40.pth \
  --package adapters/complete_slide_v1/klt/fold_a/seed42

The original finalization standard required state_reconstruction: exact_all_tensors and forward_verification: exact_all_output_tensors. See adapter_manifest.csv and verification_summary.json for the release-wide inventory.

Intended use and limitations

This release is intended for research, reproducibility, and evaluation of STHELAR-specific CellViT adaptation. It is not clinically validated and must not be used as a medical device.

• STHELAR contains few complete slides per tissue; reciprocal direction can materially affect results.
• Tissue-specific results use one seed per direction; KLT uses three.
• No patient-, site-, scanner-, or cohort-level generalization claim is made.
• ST-derived identities contain assignment uncertainty and are not interchangeable with morphology-only labels.
• Performance may change with stain, scanner, magnification, preprocessing, taxonomy, or base-checkpoint version.
• Comparisons with CellViT-256 are practical backbone comparisons, not a controlled scaling experiment.

Citation, provenance, and terms

Prepared STHELAR revision: e32a8cdd50eff2d38e237f3729e9ac85bbb5203b. The adapter manifest records payload hashes, roles, configs, and held-out metrics. No unpublished paper citation is asserted.

See CITATION.cff, LICENSE, and NOTICE. Applicable CellViT/HIPT-derived components include the upstream Apache 2.0 and Commons Clause conditions reproduced in LICENSE. Cite the upstream CellViT, STHELAR, Segment Anything, HIPT, and PanNuke resources as applicable. STHELAR data and the official base checkpoint retain their own terms and are not redistributed here.