Configuration Parsing Warning:In config.json: "quantization_config.bits" must be an integer

Note

This was made to test extremely low quants. 70B is usable at 2.15 bpw. A custom strategy for quanting was used, for details check EXL3 repo.

Safetensors

Model size

11B params

Tensor type

F16

I16

Model tree for SicariusSicariiStuff/Negative_LLAMA_70B_EXL3_d_2.15bpw

Base model

Finetuned

Finetuned

Quantized

(23)

this model