Hugging Face's logo

MrMoeeee
/
lamp-models

Text Generation
Safetensors
GGUF
gemma3
fine-tuned
lamp
lighting
smart-home
json
conversational
Model card Files
xet
 Community
MrMoeeee commited on
Commit
4dd92fd
·
verified ·
1 Parent(s): ea2809c

Add lamp-gemma-4b-v2 full fine-tune weights + training logs

Browse files
Files changed (17) hide show
  1. .gitattributes +1 -0
  2. lamp-gemma-4b-v2/README.md +59 -0
  3. lamp-gemma-4b-v2/added_tokens.json +3 -0
  4. lamp-gemma-4b-v2/chat_template.jinja +47 -0
  5. lamp-gemma-4b-v2/config.json +101 -0
  6. lamp-gemma-4b-v2/generation_config.json +13 -0
  7. lamp-gemma-4b-v2/metrics_detailed.json +1014 -0
  8. lamp-gemma-4b-v2/model-00001-of-00002.safetensors +3 -0
  9. lamp-gemma-4b-v2/model-00002-of-00002.safetensors +3 -0
  10. lamp-gemma-4b-v2/model.safetensors.index.json +891 -0
  11. lamp-gemma-4b-v2/special_tokens_map.json +33 -0
  12. lamp-gemma-4b-v2/tokenizer.json +3 -0
  13. lamp-gemma-4b-v2/tokenizer.model +3 -0
  14. lamp-gemma-4b-v2/tokenizer_config.json +0 -0
  15. lamp-gemma-4b-v2/training_config.json +34 -0
  16. lamp-gemma-4b-v2/training_log.json +601 -0
  17. lamp-gemma-4b-v2/training_metrics.csv +85 -0
.gitattributes CHANGED
@@ -37,3 +37,4 @@ lamp-gemma-4b.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
37
  lamp-llama-3b.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
38
  training_results.png filter=lfs diff=lfs merge=lfs -text
39
  lamp-gemma-4b-v2-gguf/lamp-gemma-4b-v2-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 
 
37
  lamp-llama-3b.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
38
  training_results.png filter=lfs diff=lfs merge=lfs -text
39
  lamp-gemma-4b-v2-gguf/lamp-gemma-4b-v2-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
40
+ lamp-gemma-4b-v2/tokenizer.json filter=lfs diff=lfs merge=lfs -text
lamp-gemma-4b-v2/README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: unsloth/gemma-3-4b-it
3
+ library_name: transformers
4
+ model_name: lamp-gemma-4b-v2
5
+ tags:
6
+ - generated_from_trainer
7
+ - sft
8
+ - trl
9
+ - unsloth
10
+ licence: license
11
+ ---
12
+
13
+ # Model Card for lamp-gemma-4b-v2
14
+
15
+ This model is a fine-tuned version of [unsloth/gemma-3-4b-it](https://huggingface.co/unsloth/gemma-3-4b-it).
16
+ It has been trained using [TRL](https://github.com/huggingface/trl).
17
+
18
+ ## Quick start
19
+
20
+ ```python
21
+ from transformers import pipeline
22
+
23
+ question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
24
+ generator = pipeline("text-generation", model="None", device="cuda")
25
+ output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
26
+ print(output["generated_text"])
27
+ ```
28
+
29
+ ## Training procedure
30
+
31
+
32
+
33
+
34
+ This model was trained with SFT.
35
+
36
+ ### Framework versions
37
+
38
+ - TRL: 0.24.0
39
+ - Transformers: 4.57.6
40
+ - Pytorch: 2.10.0
41
+ - Datasets: 4.3.0
42
+ - Tokenizers: 0.22.2
43
+
44
+ ## Citations
45
+
46
+
47
+
48
+ Cite TRL as:
49
+
50
+ ```bibtex
51
+ @misc{vonwerra2022trl,
52
+ title = {{TRL: Transformer Reinforcement Learning}},
53
+ author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
54
+ year = 2020,
55
+ journal = {GitHub repository},
56
+ publisher = {GitHub},
57
+ howpublished = {\url{https://github.com/huggingface/trl}}
58
+ }
59
+ ```
lamp-gemma-4b-v2/added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "<image_soft_token>": 262144
3
+ }
lamp-gemma-4b-v2/chat_template.jinja ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {{ bos_token }}
2
+ {%- if messages[0]['role'] == 'system' -%}
3
+ {%- if messages[0]['content'] is string -%}
4
+ {%- set first_user_prefix = messages[0]['content'] + '
5
+
6
+ ' -%}
7
+ {%- else -%}
8
+ {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
9
+
10
+ ' -%}
11
+ {%- endif -%}
12
+ {%- set loop_messages = messages[1:] -%}
13
+ {%- else -%}
14
+ {%- set first_user_prefix = "" -%}
15
+ {%- set loop_messages = messages -%}
16
+ {%- endif -%}
17
+ {%- for message in loop_messages -%}
18
+ {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
19
+ {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
20
+ {%- endif -%}
21
+ {%- if (message['role'] == 'assistant') -%}
22
+ {%- set role = "model" -%}
23
+ {%- else -%}
24
+ {%- set role = message['role'] -%}
25
+ {%- endif -%}
26
+ {{ '<start_of_turn>' + role + '
27
+ ' + (first_user_prefix if loop.first else "") }}
28
+ {%- if message['content'] is string -%}
29
+ {{ message['content'] | trim }}
30
+ {%- elif message['content'] is iterable -%}
31
+ {%- for item in message['content'] -%}
32
+ {%- if item['type'] == 'image' -%}
33
+ {{ '<start_of_image>' }}
34
+ {%- elif item['type'] == 'text' -%}
35
+ {{ item['text'] | trim }}
36
+ {%- endif -%}
37
+ {%- endfor -%}
38
+ {%- else -%}
39
+ {{ raise_exception("Invalid content type") }}
40
+ {%- endif -%}
41
+ {{ '<end_of_turn>
42
+ ' }}
43
+ {%- endfor -%}
44
+ {%- if add_generation_prompt -%}
45
+ {{'<start_of_turn>model
46
+ '}}
47
+ {%- endif -%}
lamp-gemma-4b-v2/config.json ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Gemma3ForConditionalGeneration"
4
+ ],
5
+ "boi_token_index": 255999,
6
+ "bos_token_id": 2,
7
+ "dtype": "bfloat16",
8
+ "eoi_token_index": 256000,
9
+ "eos_token_id": 106,
10
+ "image_token_index": 262144,
11
+ "initializer_range": 0.02,
12
+ "mm_tokens_per_image": 256,
13
+ "model_type": "gemma3",
14
+ "pad_token_id": 0,
15
+ "text_config": {
16
+ "_sliding_window_pattern": 6,
17
+ "attention_bias": false,
18
+ "attention_dropout": 0.0,
19
+ "attn_logit_softcapping": null,
20
+ "cache_implementation": "hybrid",
21
+ "dtype": "bfloat16",
22
+ "final_logit_softcapping": null,
23
+ "head_dim": 256,
24
+ "hidden_activation": "gelu_pytorch_tanh",
25
+ "hidden_size": 2560,
26
+ "initializer_range": 0.02,
27
+ "intermediate_size": 10240,
28
+ "layer_types": [
29
+ "sliding_attention",
30
+ "sliding_attention",
31
+ "sliding_attention",
32
+ "sliding_attention",
33
+ "sliding_attention",
34
+ "full_attention",
35
+ "sliding_attention",
36
+ "sliding_attention",
37
+ "sliding_attention",
38
+ "sliding_attention",
39
+ "sliding_attention",
40
+ "full_attention",
41
+ "sliding_attention",
42
+ "sliding_attention",
43
+ "sliding_attention",
44
+ "sliding_attention",
45
+ "sliding_attention",
46
+ "full_attention",
47
+ "sliding_attention",
48
+ "sliding_attention",
49
+ "sliding_attention",
50
+ "sliding_attention",
51
+ "sliding_attention",
52
+ "full_attention",
53
+ "sliding_attention",
54
+ "sliding_attention",
55
+ "sliding_attention",
56
+ "sliding_attention",
57
+ "sliding_attention",
58
+ "full_attention",
59
+ "sliding_attention",
60
+ "sliding_attention",
61
+ "sliding_attention",
62
+ "sliding_attention"
63
+ ],
64
+ "max_position_embeddings": 131072,
65
+ "model_type": "gemma3_text",
66
+ "num_attention_heads": 8,
67
+ "num_hidden_layers": 34,
68
+ "num_key_value_heads": 4,
69
+ "query_pre_attn_scalar": 256,
70
+ "rms_norm_eps": 1e-06,
71
+ "rope_local_base_freq": 10000.0,
72
+ "rope_scaling": {
73
+ "factor": 8.0,
74
+ "rope_type": "linear"
75
+ },
76
+ "rope_theta": 1000000.0,
77
+ "sliding_window": 1024,
78
+ "sliding_window_pattern": 6,
79
+ "use_bidirectional_attention": false,
80
+ "use_cache": true,
81
+ "vocab_size": 262208
82
+ },
83
+ "transformers_version": "4.57.6",
84
+ "unsloth_fixed": true,
85
+ "unsloth_version": "2026.2.1",
86
+ "vision_config": {
87
+ "attention_dropout": 0.0,
88
+ "dtype": "bfloat16",
89
+ "hidden_act": "gelu_pytorch_tanh",
90
+ "hidden_size": 1152,
91
+ "image_size": 896,
92
+ "intermediate_size": 4304,
93
+ "layer_norm_eps": 1e-06,
94
+ "model_type": "siglip_vision_model",
95
+ "num_attention_heads": 16,
96
+ "num_channels": 3,
97
+ "num_hidden_layers": 27,
98
+ "patch_size": 14,
99
+ "vision_use_head": false
100
+ }
101
+ }
lamp-gemma-4b-v2/generation_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 2,
3
+ "cache_implementation": "hybrid",
4
+ "do_sample": true,
5
+ "eos_token_id": [
6
+ 1,
7
+ 106
8
+ ],
9
+ "pad_token_id": 0,
10
+ "top_k": 64,
11
+ "top_p": 0.95,
12
+ "transformers_version": "4.57.6"
13
+ }
lamp-gemma-4b-v2/metrics_detailed.json ADDED
@@ -0,0 +1,1014 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "train_losses": [
3
+ {
4
+ "step": 10,
5
+ "loss": 1.5717
6
+ },
7
+ {
8
+ "step": 20,
9
+ "loss": 0.5627
10
+ },
11
+ {
12
+ "step": 30,
13
+ "loss": 0.0742
14
+ },
15
+ {
16
+ "step": 40,
17
+ "loss": 0.0563
18
+ },
19
+ {
20
+ "step": 50,
21
+ "loss": 0.0459
22
+ },
23
+ {
24
+ "step": 60,
25
+ "loss": 0.0409
26
+ },
27
+ {
28
+ "step": 70,
29
+ "loss": 0.0394
30
+ },
31
+ {
32
+ "step": 80,
33
+ "loss": 0.0385
34
+ },
35
+ {
36
+ "step": 90,
37
+ "loss": 0.0398
38
+ },
39
+ {
40
+ "step": 100,
41
+ "loss": 0.0352
42
+ },
43
+ {
44
+ "step": 110,
45
+ "loss": 0.0337
46
+ },
47
+ {
48
+ "step": 120,
49
+ "loss": 0.035
50
+ },
51
+ {
52
+ "step": 130,
53
+ "loss": 0.0348
54
+ },
55
+ {
56
+ "step": 140,
57
+ "loss": 0.0329
58
+ },
59
+ {
60
+ "step": 150,
61
+ "loss": 0.0325
62
+ },
63
+ {
64
+ "step": 160,
65
+ "loss": 0.0333
66
+ },
67
+ {
68
+ "step": 170,
69
+ "loss": 0.0349
70
+ },
71
+ {
72
+ "step": 180,
73
+ "loss": 0.0338
74
+ },
75
+ {
76
+ "step": 190,
77
+ "loss": 0.0351
78
+ },
79
+ {
80
+ "step": 200,
81
+ "loss": 0.032
82
+ },
83
+ {
84
+ "step": 210,
85
+ "loss": 0.0321
86
+ },
87
+ {
88
+ "step": 220,
89
+ "loss": 0.0319
90
+ },
91
+ {
92
+ "step": 230,
93
+ "loss": 0.0305
94
+ },
95
+ {
96
+ "step": 240,
97
+ "loss": 0.03
98
+ },
99
+ {
100
+ "step": 250,
101
+ "loss": 0.0319
102
+ },
103
+ {
104
+ "step": 260,
105
+ "loss": 0.0318
106
+ },
107
+ {
108
+ "step": 270,
109
+ "loss": 0.0323
110
+ },
111
+ {
112
+ "step": 280,
113
+ "loss": 0.0335
114
+ },
115
+ {
116
+ "step": 290,
117
+ "loss": 0.0303
118
+ },
119
+ {
120
+ "step": 300,
121
+ "loss": 0.0309
122
+ },
123
+ {
124
+ "step": 310,
125
+ "loss": 0.0302
126
+ },
127
+ {
128
+ "step": 320,
129
+ "loss": 0.0295
130
+ },
131
+ {
132
+ "step": 330,
133
+ "loss": 0.0287
134
+ },
135
+ {
136
+ "step": 340,
137
+ "loss": 0.029
138
+ },
139
+ {
140
+ "step": 350,
141
+ "loss": 0.0303
142
+ },
143
+ {
144
+ "step": 360,
145
+ "loss": 0.0292
146
+ },
147
+ {
148
+ "step": 370,
149
+ "loss": 0.0275
150
+ },
151
+ {
152
+ "step": 380,
153
+ "loss": 0.0307
154
+ },
155
+ {
156
+ "step": 390,
157
+ "loss": 0.031
158
+ },
159
+ {
160
+ "step": 400,
161
+ "loss": 0.0274
162
+ },
163
+ {
164
+ "step": 410,
165
+ "loss": 0.0287
166
+ },
167
+ {
168
+ "step": 420,
169
+ "loss": 0.0245
170
+ },
171
+ {
172
+ "step": 430,
173
+ "loss": 0.0241
174
+ },
175
+ {
176
+ "step": 440,
177
+ "loss": 0.024
178
+ },
179
+ {
180
+ "step": 450,
181
+ "loss": 0.0229
182
+ },
183
+ {
184
+ "step": 460,
185
+ "loss": 0.0233
186
+ },
187
+ {
188
+ "step": 470,
189
+ "loss": 0.0226
190
+ },
191
+ {
192
+ "step": 480,
193
+ "loss": 0.0227
194
+ },
195
+ {
196
+ "step": 490,
197
+ "loss": 0.0232
198
+ },
199
+ {
200
+ "step": 500,
201
+ "loss": 0.022
202
+ },
203
+ {
204
+ "step": 510,
205
+ "loss": 0.024
206
+ },
207
+ {
208
+ "step": 520,
209
+ "loss": 0.0222
210
+ },
211
+ {
212
+ "step": 530,
213
+ "loss": 0.023
214
+ },
215
+ {
216
+ "step": 540,
217
+ "loss": 0.0219
218
+ },
219
+ {
220
+ "step": 550,
221
+ "loss": 0.0224
222
+ },
223
+ {
224
+ "step": 560,
225
+ "loss": 0.0211
226
+ },
227
+ {
228
+ "step": 570,
229
+ "loss": 0.0235
230
+ },
231
+ {
232
+ "step": 580,
233
+ "loss": 0.0223
234
+ },
235
+ {
236
+ "step": 590,
237
+ "loss": 0.0223
238
+ },
239
+ {
240
+ "step": 600,
241
+ "loss": 0.0206
242
+ },
243
+ {
244
+ "step": 610,
245
+ "loss": 0.0218
246
+ },
247
+ {
248
+ "step": 620,
249
+ "loss": 0.023
250
+ },
251
+ {
252
+ "step": 630,
253
+ "loss": 0.0225
254
+ },
255
+ {
256
+ "step": 640,
257
+ "loss": 0.0232
258
+ },
259
+ {
260
+ "step": 650,
261
+ "loss": 0.0217
262
+ },
263
+ {
264
+ "step": 660,
265
+ "loss": 0.024
266
+ },
267
+ {
268
+ "step": 670,
269
+ "loss": 0.0221
270
+ },
271
+ {
272
+ "step": 680,
273
+ "loss": 0.0223
274
+ },
275
+ {
276
+ "step": 690,
277
+ "loss": 0.0234
278
+ },
279
+ {
280
+ "step": 700,
281
+ "loss": 0.0209
282
+ },
283
+ {
284
+ "step": 710,
285
+ "loss": 0.0231
286
+ },
287
+ {
288
+ "step": 720,
289
+ "loss": 0.0219
290
+ },
291
+ {
292
+ "step": 730,
293
+ "loss": 0.0234
294
+ },
295
+ {
296
+ "step": 740,
297
+ "loss": 0.0223
298
+ },
299
+ {
300
+ "step": 750,
301
+ "loss": 0.0246
302
+ },
303
+ {
304
+ "step": 760,
305
+ "loss": 0.0225
306
+ },
307
+ {
308
+ "step": 770,
309
+ "loss": 0.022
310
+ },
311
+ {
312
+ "step": 780,
313
+ "loss": 0.023
314
+ },
315
+ {
316
+ "step": 790,
317
+ "loss": 0.0221
318
+ },
319
+ {
320
+ "step": 800,
321
+ "loss": 0.0237
322
+ },
323
+ {
324
+ "step": 810,
325
+ "loss": 0.0238
326
+ },
327
+ {
328
+ "step": 820,
329
+ "loss": 0.0216
330
+ }
331
+ ],
332
+ "eval_losses": [
333
+ {
334
+ "step": 411,
335
+ "eval_loss": 0.029586488381028175
336
+ },
337
+ {
338
+ "step": 822,
339
+ "eval_loss": 0.02881857380270958
340
+ }
341
+ ],
342
+ "learning_rates": [
343
+ {
344
+ "step": 10,
345
+ "lr": 4.2857142857142855e-06
346
+ },
347
+ {
348
+ "step": 20,
349
+ "lr": 9.047619047619049e-06
350
+ },
351
+ {
352
+ "step": 30,
353
+ "lr": 1.3809523809523811e-05
354
+ },
355
+ {
356
+ "step": 40,
357
+ "lr": 1.8571428571428575e-05
358
+ },
359
+ {
360
+ "step": 50,
361
+ "lr": 1.999602581704666e-05
362
+ },
363
+ {
364
+ "step": 60,
365
+ "lr": 1.997656803510789e-05
366
+ },
367
+ {
368
+ "step": 70,
369
+ "lr": 1.9940928223333254e-05
370
+ },
371
+ {
372
+ "step": 80,
373
+ "lr": 1.988916418962778e-05
374
+ },
375
+ {
376
+ "step": 90,
377
+ "lr": 1.9821359895443436e-05
378
+ },
379
+ {
380
+ "step": 100,
381
+ "lr": 1.9737625319593338e-05
382
+ },
383
+ {
384
+ "step": 110,
385
+ "lr": 1.9638096279865717e-05
386
+ },
387
+ {
388
+ "step": 120,
389
+ "lr": 1.9522934212726977e-05
390
+ },
391
+ {
392
+ "step": 130,
393
+ "lr": 1.9392325911471154e-05
394
+ },
395
+ {
396
+ "step": 140,
397
+ "lr": 1.9246483223240496e-05
398
+ },
399
+ {
400
+ "step": 150,
401
+ "lr": 1.908564270540864e-05
402
+ },
403
+ {
404
+ "step": 160,
405
+ "lr": 1.891006524188368e-05
406
+ },
407
+ {
408
+ "step": 170,
409
+ "lr": 1.8720035619953517e-05
410
+ },
411
+ {
412
+ "step": 180,
413
+ "lr": 1.8515862068359837e-05
414
+ },
415
+ {
416
+ "step": 190,
417
+ "lr": 1.829787575734995e-05
418
+ },
419
+ {
420
+ "step": 200,
421
+ "lr": 1.806643026151747e-05
422
+ },
423
+ {
424
+ "step": 210,
425
+ "lr": 1.7821900986302995e-05
426
+ },
427
+ {
428
+ "step": 220,
429
+ "lr": 1.7564684559085138e-05
430
+ },
431
+ {
432
+ "step": 230,
433
+ "lr": 1.729519818584944e-05
434
+ },
435
+ {
436
+ "step": 240,
437
+ "lr": 1.7013878974478778e-05
438
+ },
439
+ {
440
+ "step": 250,
441
+ "lr": 1.6721183225762726e-05
442
+ },
443
+ {
444
+ "step": 260,
445
+ "lr": 1.641758569327606e-05
446
+ },
447
+ {
448
+ "step": 270,
449
+ "lr": 1.6103578813326683e-05
450
+ },
451
+ {
452
+ "step": 280,
453
+ "lr": 1.577967190622215e-05
454
+ },
455
+ {
456
+ "step": 290,
457
+ "lr": 1.5446390350150272e-05
458
+ },
459
+ {
460
+ "step": 300,
461
+ "lr": 1.5104274729013759e-05
462
+ },
463
+ {
464
+ "step": 310,
465
+ "lr": 1.4753879955601162e-05
466
+ },
467
+ {
468
+ "step": 320,
469
+ "lr": 1.4395774371516305e-05
470
+ },
471
+ {
472
+ "step": 330,
473
+ "lr": 1.4030538825326115e-05
474
+ },
475
+ {
476
+ "step": 340,
477
+ "lr": 1.3658765730422126e-05
478
+ },
479
+ {
480
+ "step": 350,
481
+ "lr": 1.3281058104123794e-05
482
+ },
483
+ {
484
+ "step": 360,
485
+ "lr": 1.2898028589582202e-05
486
+ },
487
+ {
488
+ "step": 370,
489
+ "lr": 1.2510298462070619e-05
490
+ },
491
+ {
492
+ "step": 380,
493
+ "lr": 1.2118496621273745e-05
494
+ },
495
+ {
496
+ "step": 390,
497
+ "lr": 1.172325857121012e-05
498
+ },
499
+ {
500
+ "step": 400,
501
+ "lr": 1.1325225389442278e-05
502
+ },
503
+ {
504
+ "step": 410,
505
+ "lr": 1.0925042687246592e-05
506
+ },
507
+ {
508
+ "step": 420,
509
+ "lr": 1.0523359562429441e-05
510
+ },
511
+ {
512
+ "step": 430,
513
+ "lr": 1.0120827546488175e-05
514
+ },
515
+ {
516
+ "step": 440,
517
+ "lr": 9.718099547824657e-06
518
+ },
519
+ {
520
+ "step": 450,
521
+ "lr": 9.315828792725438e-06
522
+ },
523
+ {
524
+ "step": 460,
525
+ "lr": 8.91466776582634e-06
526
+ },
527
+ {
528
+ "step": 470,
529
+ "lr": 8.515267151779974e-06
530
+ },
531
+ {
532
+ "step": 480,
533
+ "lr": 8.118274779842888e-06
534
+ },
535
+ {
536
+ "step": 490,
537
+ "lr": 7.724334573094101e-06
538
+ },
539
+ {
540
+ "step": 500,
541
+ "lr": 7.334085503989547e-06
542
+ },
543
+ {
544
+ "step": 510,
545
+ "lr": 6.948160557946404e-06
546
+ },
547
+ {
548
+ "step": 520,
549
+ "lr": 6.567185706638417e-06
550
+ },
551
+ {
552
+ "step": 530,
553
+ "lr": 6.191778892667591e-06
554
+ },
555
+ {
556
+ "step": 540,
557
+ "lr": 5.822549027258994e-06
558
+ },
559
+ {
560
+ "step": 550,
561
+ "lr": 5.460095002604533e-06
562
+ },
563
+ {
564
+ "step": 560,
565
+ "lr": 5.105004720457654e-06
566
+ },
567
+ {
568
+ "step": 570,
569
+ "lr": 4.757854138554502e-06
570
+ },
571
+ {
572
+ "step": 580,
573
+ "lr": 4.419206336408418e-06
574
+ },
575
+ {
576
+ "step": 590,
577
+ "lr": 4.089610601992864e-06
578
+ },
579
+ {
580
+ "step": 600,
581
+ "lr": 3.769601540794344e-06
582
+ },
583
+ {
584
+ "step": 610,
585
+ "lr": 3.459698208680359e-06
586
+ },
587
+ {
588
+ "step": 620,
589
+ "lr": 3.160403269988864e-06
590
+ },
591
+ {
592
+ "step": 630,
593
+ "lr": 2.872202182204903e-06
594
+ },
595
+ {
596
+ "step": 640,
597
+ "lr": 2.59556240854677e-06
598
+ },
599
+ {
600
+ "step": 650,
601
+ "lr": 2.3309326597389668e-06
602
+ },
603
+ {
604
+ "step": 660,
605
+ "lr": 2.0787421662017827e-06
606
+ },
607
+ {
608
+ "step": 670,
609
+ "lr": 1.8393999818379527e-06
610
+ },
611
+ {
612
+ "step": 680,
613
+ "lr": 1.6132943205457607e-06
614
+ },
615
+ {
616
+ "step": 690,
617
+ "lr": 1.400791926534657e-06
618
+ },
619
+ {
620
+ "step": 700,
621
+ "lr": 1.2022374794648229e-06
622
+ },
623
+ {
624
+ "step": 710,
625
+ "lr": 1.0179530353754874e-06
626
+ },
627
+ {
628
+ "step": 720,
629
+ "lr": 8.482375043088664e-07
630
+ },
631
+ {
632
+ "step": 730,
633
+ "lr": 6.933661654769797e-07
634
+ },
635
+ {
636
+ "step": 740,
637
+ "lr": 5.535902207577515e-07
638
+ },
639
+ {
640
+ "step": 750,
641
+ "lr": 4.2913638724465966e-07
642
+ },
643
+ {
644
+ "step": 760,
645
+ "lr": 3.2020652951077256e-07
646
+ },
647
+ {
648
+ "step": 770,
649
+ "lr": 2.2697733218367435e-07
650
+ },
651
+ {
652
+ "step": 780,
653
+ "lr": 1.4960001336235875e-07
654
+ },
655
+ {
656
+ "step": 790,
657
+ "lr": 8.82000793409088e-08
658
+ },
659
+ {
660
+ "step": 800,
661
+ "lr": 4.287712103684061e-08
662
+ },
663
+ {
664
+ "step": 810,
665
+ "lr": 1.370465245426167e-08
666
+ },
667
+ {
668
+ "step": 820,
669
+ "lr": 7.299914438929634e-10
670
+ }
671
+ ],
672
+ "vram_usage": [
673
+ {
674
+ "step": 10,
675
+ "vram_gb": 24.31
676
+ },
677
+ {
678
+ "step": 20,
679
+ "vram_gb": 24.31
680
+ },
681
+ {
682
+ "step": 30,
683
+ "vram_gb": 24.31
684
+ },
685
+ {
686
+ "step": 40,
687
+ "vram_gb": 24.31
688
+ },
689
+ {
690
+ "step": 50,
691
+ "vram_gb": 24.31
692
+ },
693
+ {
694
+ "step": 60,
695
+ "vram_gb": 24.31
696
+ },
697
+ {
698
+ "step": 70,
699
+ "vram_gb": 24.31
700
+ },
701
+ {
702
+ "step": 80,
703
+ "vram_gb": 24.31
704
+ },
705
+ {
706
+ "step": 90,
707
+ "vram_gb": 24.31
708
+ },
709
+ {
710
+ "step": 100,
711
+ "vram_gb": 24.31
712
+ },
713
+ {
714
+ "step": 110,
715
+ "vram_gb": 24.31
716
+ },
717
+ {
718
+ "step": 120,
719
+ "vram_gb": 24.31
720
+ },
721
+ {
722
+ "step": 130,
723
+ "vram_gb": 24.31
724
+ },
725
+ {
726
+ "step": 140,
727
+ "vram_gb": 24.31
728
+ },
729
+ {
730
+ "step": 150,
731
+ "vram_gb": 24.31
732
+ },
733
+ {
734
+ "step": 160,
735
+ "vram_gb": 24.31
736
+ },
737
+ {
738
+ "step": 170,
739
+ "vram_gb": 24.31
740
+ },
741
+ {
742
+ "step": 180,
743
+ "vram_gb": 24.31
744
+ },
745
+ {
746
+ "step": 190,
747
+ "vram_gb": 24.31
748
+ },
749
+ {
750
+ "step": 200,
751
+ "vram_gb": 24.31
752
+ },
753
+ {
754
+ "step": 210,
755
+ "vram_gb": 24.31
756
+ },
757
+ {
758
+ "step": 220,
759
+ "vram_gb": 24.31
760
+ },
761
+ {
762
+ "step": 230,
763
+ "vram_gb": 24.31
764
+ },
765
+ {
766
+ "step": 240,
767
+ "vram_gb": 24.31
768
+ },
769
+ {
770
+ "step": 250,
771
+ "vram_gb": 24.31
772
+ },
773
+ {
774
+ "step": 260,
775
+ "vram_gb": 24.31
776
+ },
777
+ {
778
+ "step": 270,
779
+ "vram_gb": 24.31
780
+ },
781
+ {
782
+ "step": 280,
783
+ "vram_gb": 24.31
784
+ },
785
+ {
786
+ "step": 290,
787
+ "vram_gb": 24.31
788
+ },
789
+ {
790
+ "step": 300,
791
+ "vram_gb": 24.31
792
+ },
793
+ {
794
+ "step": 310,
795
+ "vram_gb": 24.31
796
+ },
797
+ {
798
+ "step": 320,
799
+ "vram_gb": 24.31
800
+ },
801
+ {
802
+ "step": 330,
803
+ "vram_gb": 24.31
804
+ },
805
+ {
806
+ "step": 340,
807
+ "vram_gb": 24.31
808
+ },
809
+ {
810
+ "step": 350,
811
+ "vram_gb": 24.31
812
+ },
813
+ {
814
+ "step": 360,
815
+ "vram_gb": 24.31
816
+ },
817
+ {
818
+ "step": 370,
819
+ "vram_gb": 24.31
820
+ },
821
+ {
822
+ "step": 380,
823
+ "vram_gb": 24.31
824
+ },
825
+ {
826
+ "step": 390,
827
+ "vram_gb": 24.31
828
+ },
829
+ {
830
+ "step": 400,
831
+ "vram_gb": 24.31
832
+ },
833
+ {
834
+ "step": 410,
835
+ "vram_gb": 24.31
836
+ },
837
+ {
838
+ "step": 411,
839
+ "vram_gb": 24.31
840
+ },
841
+ {
842
+ "step": 420,
843
+ "vram_gb": 24.31
844
+ },
845
+ {
846
+ "step": 430,
847
+ "vram_gb": 24.31
848
+ },
849
+ {
850
+ "step": 440,
851
+ "vram_gb": 24.31
852
+ },
853
+ {
854
+ "step": 450,
855
+ "vram_gb": 24.31
856
+ },
857
+ {
858
+ "step": 460,
859
+ "vram_gb": 24.31
860
+ },
861
+ {
862
+ "step": 470,
863
+ "vram_gb": 24.31
864
+ },
865
+ {
866
+ "step": 480,
867
+ "vram_gb": 24.31
868
+ },
869
+ {
870
+ "step": 490,
871
+ "vram_gb": 24.31
872
+ },
873
+ {
874
+ "step": 500,
875
+ "vram_gb": 24.31
876
+ },
877
+ {
878
+ "step": 510,
879
+ "vram_gb": 24.31
880
+ },
881
+ {
882
+ "step": 520,
883
+ "vram_gb": 24.31
884
+ },
885
+ {
886
+ "step": 530,
887
+ "vram_gb": 24.31
888
+ },
889
+ {
890
+ "step": 540,
891
+ "vram_gb": 24.31
892
+ },
893
+ {
894
+ "step": 550,
895
+ "vram_gb": 24.31
896
+ },
897
+ {
898
+ "step": 560,
899
+ "vram_gb": 24.31
900
+ },
901
+ {
902
+ "step": 570,
903
+ "vram_gb": 24.31
904
+ },
905
+ {
906
+ "step": 580,
907
+ "vram_gb": 24.31
908
+ },
909
+ {
910
+ "step": 590,
911
+ "vram_gb": 24.31
912
+ },
913
+ {
914
+ "step": 600,
915
+ "vram_gb": 24.31
916
+ },
917
+ {
918
+ "step": 610,
919
+ "vram_gb": 24.31
920
+ },
921
+ {
922
+ "step": 620,
923
+ "vram_gb": 24.31
924
+ },
925
+ {
926
+ "step": 630,
927
+ "vram_gb": 24.31
928
+ },
929
+ {
930
+ "step": 640,
931
+ "vram_gb": 24.31
932
+ },
933
+ {
934
+ "step": 650,
935
+ "vram_gb": 24.31
936
+ },
937
+ {
938
+ "step": 660,
939
+ "vram_gb": 24.31
940
+ },
941
+ {
942
+ "step": 670,
943
+ "vram_gb": 24.31
944
+ },
945
+ {
946
+ "step": 680,
947
+ "vram_gb": 24.31
948
+ },
949
+ {
950
+ "step": 690,
951
+ "vram_gb": 24.31
952
+ },
953
+ {
954
+ "step": 700,
955
+ "vram_gb": 24.31
956
+ },
957
+ {
958
+ "step": 710,
959
+ "vram_gb": 24.31
960
+ },
961
+ {
962
+ "step": 720,
963
+ "vram_gb": 24.31
964
+ },
965
+ {
966
+ "step": 730,
967
+ "vram_gb": 24.31
968
+ },
969
+ {
970
+ "step": 740,
971
+ "vram_gb": 24.31
972
+ },
973
+ {
974
+ "step": 750,
975
+ "vram_gb": 24.31
976
+ },
977
+ {
978
+ "step": 760,
979
+ "vram_gb": 24.31
980
+ },
981
+ {
982
+ "step": 770,
983
+ "vram_gb": 24.31
984
+ },
985
+ {
986
+ "step": 780,
987
+ "vram_gb": 24.31
988
+ },
989
+ {
990
+ "step": 790,
991
+ "vram_gb": 24.31
992
+ },
993
+ {
994
+ "step": 800,
995
+ "vram_gb": 24.31
996
+ },
997
+ {
998
+ "step": 810,
999
+ "vram_gb": 24.31
1000
+ },
1001
+ {
1002
+ "step": 820,
1003
+ "vram_gb": 24.31
1004
+ },
1005
+ {
1006
+ "step": 822,
1007
+ "vram_gb": 24.31
1008
+ },
1009
+ {
1010
+ "step": 822,
1011
+ "vram_gb": 24.31
1012
+ }
1013
+ ]
1014
+ }
lamp-gemma-4b-v2/model-00001-of-00002.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c8245f0e3f6dac35bc7477c1f19d5baf4c777cbd30ad4dc1b7cd61061215b9c5
3
+ size 4961807832
lamp-gemma-4b-v2/model-00002-of-00002.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:839abe54a6ca4f025ad3ba4ae9af892d18327335f459ce578c1b8cc8b8f2ace2
3
+ size 3639459296
lamp-gemma-4b-v2/model.safetensors.index.json ADDED
@@ -0,0 +1,891 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_parameters": 4300079472,
4
+ "total_size": 8601150944
5
+ },
6
+ "weight_map": {
7
+ "language_model.model.embed_tokens.weight": "model-00001-of-00002.safetensors",
8
+ "language_model.model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
9
+ "language_model.model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
10
+ "language_model.model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
11
+ "language_model.model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
12
+ "language_model.model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
13
+ "language_model.model.layers.0.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
14
+ "language_model.model.layers.0.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
15
+ "language_model.model.layers.0.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
16
+ "language_model.model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
17
+ "language_model.model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
18
+ "language_model.model.layers.0.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
19
+ "language_model.model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
20
+ "language_model.model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
21
+ "language_model.model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
22
+ "language_model.model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
23
+ "language_model.model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
24
+ "language_model.model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
25
+ "language_model.model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
26
+ "language_model.model.layers.1.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
27
+ "language_model.model.layers.1.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
28
+ "language_model.model.layers.1.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
29
+ "language_model.model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
30
+ "language_model.model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
31
+ "language_model.model.layers.1.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
32
+ "language_model.model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
33
+ "language_model.model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
34
+ "language_model.model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
35
+ "language_model.model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
36
+ "language_model.model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
37
+ "language_model.model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
38
+ "language_model.model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
39
+ "language_model.model.layers.10.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
40
+ "language_model.model.layers.10.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
41
+ "language_model.model.layers.10.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
42
+ "language_model.model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
43
+ "language_model.model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
44
+ "language_model.model.layers.10.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
45
+ "language_model.model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
46
+ "language_model.model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
47
+ "language_model.model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
48
+ "language_model.model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
49
+ "language_model.model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
50
+ "language_model.model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
51
+ "language_model.model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
52
+ "language_model.model.layers.11.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
53
+ "language_model.model.layers.11.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
54
+ "language_model.model.layers.11.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
55
+ "language_model.model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
56
+ "language_model.model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
57
+ "language_model.model.layers.11.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
58
+ "language_model.model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
59
+ "language_model.model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
60
+ "language_model.model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
61
+ "language_model.model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
62
+ "language_model.model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
63
+ "language_model.model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
64
+ "language_model.model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
65
+ "language_model.model.layers.12.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
66
+ "language_model.model.layers.12.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
67
+ "language_model.model.layers.12.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
68
+ "language_model.model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
69
+ "language_model.model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
70
+ "language_model.model.layers.12.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
71
+ "language_model.model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
72
+ "language_model.model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
73
+ "language_model.model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
74
+ "language_model.model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
75
+ "language_model.model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
76
+ "language_model.model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
77
+ "language_model.model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
78
+ "language_model.model.layers.13.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
79
+ "language_model.model.layers.13.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
80
+ "language_model.model.layers.13.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
81
+ "language_model.model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
82
+ "language_model.model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
83
+ "language_model.model.layers.13.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
84
+ "language_model.model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
85
+ "language_model.model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
86
+ "language_model.model.layers.14.input_layernorm.weight": "model-00002-of-00002.safetensors",
87
+ "language_model.model.layers.14.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
88
+ "language_model.model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
89
+ "language_model.model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
90
+ "language_model.model.layers.14.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
91
+ "language_model.model.layers.14.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
92
+ "language_model.model.layers.14.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
93
+ "language_model.model.layers.14.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
94
+ "language_model.model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
95
+ "language_model.model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
96
+ "language_model.model.layers.14.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
97
+ "language_model.model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
98
+ "language_model.model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
99
+ "language_model.model.layers.15.input_layernorm.weight": "model-00002-of-00002.safetensors",
100
+ "language_model.model.layers.15.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
101
+ "language_model.model.layers.15.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
102
+ "language_model.model.layers.15.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
103
+ "language_model.model.layers.15.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
104
+ "language_model.model.layers.15.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
105
+ "language_model.model.layers.15.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
106
+ "language_model.model.layers.15.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
107
+ "language_model.model.layers.15.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
108
+ "language_model.model.layers.15.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
109
+ "language_model.model.layers.15.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
110
+ "language_model.model.layers.15.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
111
+ "language_model.model.layers.15.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
112
+ "language_model.model.layers.16.input_layernorm.weight": "model-00002-of-00002.safetensors",
113
+ "language_model.model.layers.16.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
114
+ "language_model.model.layers.16.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
115
+ "language_model.model.layers.16.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
116
+ "language_model.model.layers.16.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
117
+ "language_model.model.layers.16.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
118
+ "language_model.model.layers.16.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
119
+ "language_model.model.layers.16.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
120
+ "language_model.model.layers.16.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
121
+ "language_model.model.layers.16.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
122
+ "language_model.model.layers.16.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
123
+ "language_model.model.layers.16.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
124
+ "language_model.model.layers.16.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
125
+ "language_model.model.layers.17.input_layernorm.weight": "model-00002-of-00002.safetensors",
126
+ "language_model.model.layers.17.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
127
+ "language_model.model.layers.17.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
128
+ "language_model.model.layers.17.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
129
+ "language_model.model.layers.17.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
130
+ "language_model.model.layers.17.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
131
+ "language_model.model.layers.17.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
132
+ "language_model.model.layers.17.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
133
+ "language_model.model.layers.17.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
134
+ "language_model.model.layers.17.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
135
+ "language_model.model.layers.17.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
136
+ "language_model.model.layers.17.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
137
+ "language_model.model.layers.17.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
138
+ "language_model.model.layers.18.input_layernorm.weight": "model-00002-of-00002.safetensors",
139
+ "language_model.model.layers.18.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
140
+ "language_model.model.layers.18.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
141
+ "language_model.model.layers.18.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
142
+ "language_model.model.layers.18.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
143
+ "language_model.model.layers.18.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
144
+ "language_model.model.layers.18.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
145
+ "language_model.model.layers.18.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
146
+ "language_model.model.layers.18.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
147
+ "language_model.model.layers.18.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
148
+ "language_model.model.layers.18.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
149
+ "language_model.model.layers.18.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
150
+ "language_model.model.layers.18.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
151
+ "language_model.model.layers.19.input_layernorm.weight": "model-00002-of-00002.safetensors",
152
+ "language_model.model.layers.19.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
153
+ "language_model.model.layers.19.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
154
+ "language_model.model.layers.19.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
155
+ "language_model.model.layers.19.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
156
+ "language_model.model.layers.19.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
157
+ "language_model.model.layers.19.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
158
+ "language_model.model.layers.19.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
159
+ "language_model.model.layers.19.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
160
+ "language_model.model.layers.19.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
161
+ "language_model.model.layers.19.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
162
+ "language_model.model.layers.19.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
163
+ "language_model.model.layers.19.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
164
+ "language_model.model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
165
+ "language_model.model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
166
+ "language_model.model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
167
+ "language_model.model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
168
+ "language_model.model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
169
+ "language_model.model.layers.2.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
170
+ "language_model.model.layers.2.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
171
+ "language_model.model.layers.2.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
172
+ "language_model.model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
173
+ "language_model.model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
174
+ "language_model.model.layers.2.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
175
+ "language_model.model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
176
+ "language_model.model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
177
+ "language_model.model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
178
+ "language_model.model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
179
+ "language_model.model.layers.20.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
180
+ "language_model.model.layers.20.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
181
+ "language_model.model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
182
+ "language_model.model.layers.20.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
183
+ "language_model.model.layers.20.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
184
+ "language_model.model.layers.20.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
185
+ "language_model.model.layers.20.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
186
+ "language_model.model.layers.20.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
187
+ "language_model.model.layers.20.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
188
+ "language_model.model.layers.20.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
189
+ "language_model.model.layers.20.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
190
+ "language_model.model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
191
+ "language_model.model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
192
+ "language_model.model.layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
193
+ "language_model.model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
194
+ "language_model.model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
195
+ "language_model.model.layers.21.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
196
+ "language_model.model.layers.21.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
197
+ "language_model.model.layers.21.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
198
+ "language_model.model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
199
+ "language_model.model.layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
200
+ "language_model.model.layers.21.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
201
+ "language_model.model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
202
+ "language_model.model.layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
203
+ "language_model.model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
204
+ "language_model.model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
205
+ "language_model.model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
206
+ "language_model.model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
207
+ "language_model.model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
208
+ "language_model.model.layers.22.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
209
+ "language_model.model.layers.22.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
210
+ "language_model.model.layers.22.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
211
+ "language_model.model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
212
+ "language_model.model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
213
+ "language_model.model.layers.22.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
214
+ "language_model.model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
215
+ "language_model.model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
216
+ "language_model.model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
217
+ "language_model.model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
218
+ "language_model.model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
219
+ "language_model.model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
220
+ "language_model.model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
221
+ "language_model.model.layers.23.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
222
+ "language_model.model.layers.23.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
223
+ "language_model.model.layers.23.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
224
+ "language_model.model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
225
+ "language_model.model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
226
+ "language_model.model.layers.23.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
227
+ "language_model.model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
228
+ "language_model.model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
229
+ "language_model.model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
230
+ "language_model.model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
231
+ "language_model.model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
232
+ "language_model.model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
233
+ "language_model.model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
234
+ "language_model.model.layers.24.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
235
+ "language_model.model.layers.24.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
236
+ "language_model.model.layers.24.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
237
+ "language_model.model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
238
+ "language_model.model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
239
+ "language_model.model.layers.24.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
240
+ "language_model.model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
241
+ "language_model.model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
242
+ "language_model.model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
243
+ "language_model.model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
244
+ "language_model.model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
245
+ "language_model.model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
246
+ "language_model.model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
247
+ "language_model.model.layers.25.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
248
+ "language_model.model.layers.25.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
249
+ "language_model.model.layers.25.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
250
+ "language_model.model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
251
+ "language_model.model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
252
+ "language_model.model.layers.25.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
253
+ "language_model.model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
254
+ "language_model.model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
255
+ "language_model.model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
256
+ "language_model.model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
257
+ "language_model.model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
258
+ "language_model.model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
259
+ "language_model.model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
260
+ "language_model.model.layers.26.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
261
+ "language_model.model.layers.26.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
262
+ "language_model.model.layers.26.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
263
+ "language_model.model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
264
+ "language_model.model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
265
+ "language_model.model.layers.26.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
266
+ "language_model.model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
267
+ "language_model.model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
268
+ "language_model.model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
269
+ "language_model.model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
270
+ "language_model.model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
271
+ "language_model.model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
272
+ "language_model.model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
273
+ "language_model.model.layers.27.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
274
+ "language_model.model.layers.27.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
275
+ "language_model.model.layers.27.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
276
+ "language_model.model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
277
+ "language_model.model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
278
+ "language_model.model.layers.27.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
279
+ "language_model.model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
280
+ "language_model.model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
281
+ "language_model.model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
282
+ "language_model.model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
283
+ "language_model.model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
284
+ "language_model.model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
285
+ "language_model.model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
286
+ "language_model.model.layers.28.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
287
+ "language_model.model.layers.28.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
288
+ "language_model.model.layers.28.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
289
+ "language_model.model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
290
+ "language_model.model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
291
+ "language_model.model.layers.28.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
292
+ "language_model.model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
293
+ "language_model.model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
294
+ "language_model.model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
295
+ "language_model.model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
296
+ "language_model.model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
297
+ "language_model.model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
298
+ "language_model.model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
299
+ "language_model.model.layers.29.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
300
+ "language_model.model.layers.29.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
301
+ "language_model.model.layers.29.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
302
+ "language_model.model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
303
+ "language_model.model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
304
+ "language_model.model.layers.29.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
305
+ "language_model.model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
306
+ "language_model.model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
307
+ "language_model.model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
308
+ "language_model.model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
309
+ "language_model.model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
310
+ "language_model.model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
311
+ "language_model.model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
312
+ "language_model.model.layers.3.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
313
+ "language_model.model.layers.3.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
314
+ "language_model.model.layers.3.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
315
+ "language_model.model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
316
+ "language_model.model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
317
+ "language_model.model.layers.3.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
318
+ "language_model.model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
319
+ "language_model.model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
320
+ "language_model.model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
321
+ "language_model.model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
322
+ "language_model.model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
323
+ "language_model.model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
324
+ "language_model.model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
325
+ "language_model.model.layers.30.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
326
+ "language_model.model.layers.30.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
327
+ "language_model.model.layers.30.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
328
+ "language_model.model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
329
+ "language_model.model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
330
+ "language_model.model.layers.30.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
331
+ "language_model.model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
332
+ "language_model.model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
333
+ "language_model.model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
334
+ "language_model.model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
335
+ "language_model.model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
336
+ "language_model.model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
337
+ "language_model.model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
338
+ "language_model.model.layers.31.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
339
+ "language_model.model.layers.31.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
340
+ "language_model.model.layers.31.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
341
+ "language_model.model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
342
+ "language_model.model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
343
+ "language_model.model.layers.31.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
344
+ "language_model.model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
345
+ "language_model.model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
346
+ "language_model.model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
347
+ "language_model.model.layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
348
+ "language_model.model.layers.32.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
349
+ "language_model.model.layers.32.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
350
+ "language_model.model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
351
+ "language_model.model.layers.32.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
352
+ "language_model.model.layers.32.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
353
+ "language_model.model.layers.32.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
354
+ "language_model.model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
355
+ "language_model.model.layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
356
+ "language_model.model.layers.32.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
357
+ "language_model.model.layers.32.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
358
+ "language_model.model.layers.32.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
359
+ "language_model.model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
360
+ "language_model.model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
361
+ "language_model.model.layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
362
+ "language_model.model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
363
+ "language_model.model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
364
+ "language_model.model.layers.33.post_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
365
+ "language_model.model.layers.33.pre_feedforward_layernorm.weight": "model-00002-of-00002.safetensors",
366
+ "language_model.model.layers.33.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
367
+ "language_model.model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
368
+ "language_model.model.layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
369
+ "language_model.model.layers.33.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
370
+ "language_model.model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
371
+ "language_model.model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
372
+ "language_model.model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
373
+ "language_model.model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
374
+ "language_model.model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
375
+ "language_model.model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
376
+ "language_model.model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
377
+ "language_model.model.layers.4.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
378
+ "language_model.model.layers.4.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
379
+ "language_model.model.layers.4.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
380
+ "language_model.model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
381
+ "language_model.model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
382
+ "language_model.model.layers.4.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
383
+ "language_model.model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
384
+ "language_model.model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
385
+ "language_model.model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
386
+ "language_model.model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
387
+ "language_model.model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
388
+ "language_model.model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
389
+ "language_model.model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
390
+ "language_model.model.layers.5.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
391
+ "language_model.model.layers.5.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
392
+ "language_model.model.layers.5.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
393
+ "language_model.model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
394
+ "language_model.model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
395
+ "language_model.model.layers.5.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
396
+ "language_model.model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
397
+ "language_model.model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
398
+ "language_model.model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
399
+ "language_model.model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
400
+ "language_model.model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
401
+ "language_model.model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
402
+ "language_model.model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
403
+ "language_model.model.layers.6.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
404
+ "language_model.model.layers.6.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
405
+ "language_model.model.layers.6.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
406
+ "language_model.model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
407
+ "language_model.model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
408
+ "language_model.model.layers.6.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
409
+ "language_model.model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
410
+ "language_model.model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
411
+ "language_model.model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
412
+ "language_model.model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
413
+ "language_model.model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
414
+ "language_model.model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
415
+ "language_model.model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
416
+ "language_model.model.layers.7.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
417
+ "language_model.model.layers.7.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
418
+ "language_model.model.layers.7.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
419
+ "language_model.model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
420
+ "language_model.model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
421
+ "language_model.model.layers.7.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
422
+ "language_model.model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
423
+ "language_model.model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
424
+ "language_model.model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
425
+ "language_model.model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
426
+ "language_model.model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
427
+ "language_model.model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
428
+ "language_model.model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
429
+ "language_model.model.layers.8.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
430
+ "language_model.model.layers.8.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
431
+ "language_model.model.layers.8.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
432
+ "language_model.model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
433
+ "language_model.model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
434
+ "language_model.model.layers.8.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
435
+ "language_model.model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
436
+ "language_model.model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
437
+ "language_model.model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
438
+ "language_model.model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
439
+ "language_model.model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
440
+ "language_model.model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
441
+ "language_model.model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
442
+ "language_model.model.layers.9.post_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
443
+ "language_model.model.layers.9.pre_feedforward_layernorm.weight": "model-00001-of-00002.safetensors",
444
+ "language_model.model.layers.9.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
445
+ "language_model.model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
446
+ "language_model.model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
447
+ "language_model.model.layers.9.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
448
+ "language_model.model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
449
+ "language_model.model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
450
+ "language_model.model.norm.weight": "model-00002-of-00002.safetensors",
451
+ "multi_modal_projector.mm_input_projection_weight": "model-00001-of-00002.safetensors",
452
+ "multi_modal_projector.mm_soft_emb_norm.weight": "model-00001-of-00002.safetensors",
453
+ "vision_tower.vision_model.embeddings.patch_embedding.bias": "model-00001-of-00002.safetensors",
454
+ "vision_tower.vision_model.embeddings.patch_embedding.weight": "model-00001-of-00002.safetensors",
455
+ "vision_tower.vision_model.embeddings.position_embedding.weight": "model-00001-of-00002.safetensors",
456
+ "vision_tower.vision_model.encoder.layers.0.layer_norm1.bias": "model-00001-of-00002.safetensors",
457
+ "vision_tower.vision_model.encoder.layers.0.layer_norm1.weight": "model-00001-of-00002.safetensors",
458
+ "vision_tower.vision_model.encoder.layers.0.layer_norm2.bias": "model-00001-of-00002.safetensors",
459
+ "vision_tower.vision_model.encoder.layers.0.layer_norm2.weight": "model-00001-of-00002.safetensors",
460
+ "vision_tower.vision_model.encoder.layers.0.mlp.fc1.bias": "model-00001-of-00002.safetensors",
461
+ "vision_tower.vision_model.encoder.layers.0.mlp.fc1.weight": "model-00001-of-00002.safetensors",
462
+ "vision_tower.vision_model.encoder.layers.0.mlp.fc2.bias": "model-00001-of-00002.safetensors",
463
+ "vision_tower.vision_model.encoder.layers.0.mlp.fc2.weight": "model-00001-of-00002.safetensors",
464
+ "vision_tower.vision_model.encoder.layers.0.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
465
+ "vision_tower.vision_model.encoder.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
466
+ "vision_tower.vision_model.encoder.layers.0.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
467
+ "vision_tower.vision_model.encoder.layers.0.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
468
+ "vision_tower.vision_model.encoder.layers.0.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
469
+ "vision_tower.vision_model.encoder.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
470
+ "vision_tower.vision_model.encoder.layers.0.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
471
+ "vision_tower.vision_model.encoder.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
472
+ "vision_tower.vision_model.encoder.layers.1.layer_norm1.bias": "model-00001-of-00002.safetensors",
473
+ "vision_tower.vision_model.encoder.layers.1.layer_norm1.weight": "model-00001-of-00002.safetensors",
474
+ "vision_tower.vision_model.encoder.layers.1.layer_norm2.bias": "model-00001-of-00002.safetensors",
475
+ "vision_tower.vision_model.encoder.layers.1.layer_norm2.weight": "model-00001-of-00002.safetensors",
476
+ "vision_tower.vision_model.encoder.layers.1.mlp.fc1.bias": "model-00001-of-00002.safetensors",
477
+ "vision_tower.vision_model.encoder.layers.1.mlp.fc1.weight": "model-00001-of-00002.safetensors",
478
+ "vision_tower.vision_model.encoder.layers.1.mlp.fc2.bias": "model-00001-of-00002.safetensors",
479
+ "vision_tower.vision_model.encoder.layers.1.mlp.fc2.weight": "model-00001-of-00002.safetensors",
480
+ "vision_tower.vision_model.encoder.layers.1.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
481
+ "vision_tower.vision_model.encoder.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
482
+ "vision_tower.vision_model.encoder.layers.1.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
483
+ "vision_tower.vision_model.encoder.layers.1.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
484
+ "vision_tower.vision_model.encoder.layers.1.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
485
+ "vision_tower.vision_model.encoder.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
486
+ "vision_tower.vision_model.encoder.layers.1.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
487
+ "vision_tower.vision_model.encoder.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
488
+ "vision_tower.vision_model.encoder.layers.10.layer_norm1.bias": "model-00001-of-00002.safetensors",
489
+ "vision_tower.vision_model.encoder.layers.10.layer_norm1.weight": "model-00001-of-00002.safetensors",
490
+ "vision_tower.vision_model.encoder.layers.10.layer_norm2.bias": "model-00001-of-00002.safetensors",
491
+ "vision_tower.vision_model.encoder.layers.10.layer_norm2.weight": "model-00001-of-00002.safetensors",
492
+ "vision_tower.vision_model.encoder.layers.10.mlp.fc1.bias": "model-00001-of-00002.safetensors",
493
+ "vision_tower.vision_model.encoder.layers.10.mlp.fc1.weight": "model-00001-of-00002.safetensors",
494
+ "vision_tower.vision_model.encoder.layers.10.mlp.fc2.bias": "model-00001-of-00002.safetensors",
495
+ "vision_tower.vision_model.encoder.layers.10.mlp.fc2.weight": "model-00001-of-00002.safetensors",
496
+ "vision_tower.vision_model.encoder.layers.10.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
497
+ "vision_tower.vision_model.encoder.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
498
+ "vision_tower.vision_model.encoder.layers.10.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
499
+ "vision_tower.vision_model.encoder.layers.10.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
500
+ "vision_tower.vision_model.encoder.layers.10.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
501
+ "vision_tower.vision_model.encoder.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
502
+ "vision_tower.vision_model.encoder.layers.10.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
503
+ "vision_tower.vision_model.encoder.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
504
+ "vision_tower.vision_model.encoder.layers.11.layer_norm1.bias": "model-00001-of-00002.safetensors",
505
+ "vision_tower.vision_model.encoder.layers.11.layer_norm1.weight": "model-00001-of-00002.safetensors",
506
+ "vision_tower.vision_model.encoder.layers.11.layer_norm2.bias": "model-00001-of-00002.safetensors",
507
+ "vision_tower.vision_model.encoder.layers.11.layer_norm2.weight": "model-00001-of-00002.safetensors",
508
+ "vision_tower.vision_model.encoder.layers.11.mlp.fc1.bias": "model-00001-of-00002.safetensors",
509
+ "vision_tower.vision_model.encoder.layers.11.mlp.fc1.weight": "model-00001-of-00002.safetensors",
510
+ "vision_tower.vision_model.encoder.layers.11.mlp.fc2.bias": "model-00001-of-00002.safetensors",
511
+ "vision_tower.vision_model.encoder.layers.11.mlp.fc2.weight": "model-00001-of-00002.safetensors",
512
+ "vision_tower.vision_model.encoder.layers.11.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
513
+ "vision_tower.vision_model.encoder.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
514
+ "vision_tower.vision_model.encoder.layers.11.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
515
+ "vision_tower.vision_model.encoder.layers.11.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
516
+ "vision_tower.vision_model.encoder.layers.11.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
517
+ "vision_tower.vision_model.encoder.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
518
+ "vision_tower.vision_model.encoder.layers.11.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
519
+ "vision_tower.vision_model.encoder.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
520
+ "vision_tower.vision_model.encoder.layers.12.layer_norm1.bias": "model-00001-of-00002.safetensors",
521
+ "vision_tower.vision_model.encoder.layers.12.layer_norm1.weight": "model-00001-of-00002.safetensors",
522
+ "vision_tower.vision_model.encoder.layers.12.layer_norm2.bias": "model-00001-of-00002.safetensors",
523
+ "vision_tower.vision_model.encoder.layers.12.layer_norm2.weight": "model-00001-of-00002.safetensors",
524
+ "vision_tower.vision_model.encoder.layers.12.mlp.fc1.bias": "model-00001-of-00002.safetensors",
525
+ "vision_tower.vision_model.encoder.layers.12.mlp.fc1.weight": "model-00001-of-00002.safetensors",
526
+ "vision_tower.vision_model.encoder.layers.12.mlp.fc2.bias": "model-00001-of-00002.safetensors",
527
+ "vision_tower.vision_model.encoder.layers.12.mlp.fc2.weight": "model-00001-of-00002.safetensors",
528
+ "vision_tower.vision_model.encoder.layers.12.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
529
+ "vision_tower.vision_model.encoder.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
530
+ "vision_tower.vision_model.encoder.layers.12.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
531
+ "vision_tower.vision_model.encoder.layers.12.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
532
+ "vision_tower.vision_model.encoder.layers.12.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
533
+ "vision_tower.vision_model.encoder.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
534
+ "vision_tower.vision_model.encoder.layers.12.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
535
+ "vision_tower.vision_model.encoder.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
536
+ "vision_tower.vision_model.encoder.layers.13.layer_norm1.bias": "model-00001-of-00002.safetensors",
537
+ "vision_tower.vision_model.encoder.layers.13.layer_norm1.weight": "model-00001-of-00002.safetensors",
538
+ "vision_tower.vision_model.encoder.layers.13.layer_norm2.bias": "model-00001-of-00002.safetensors",
539
+ "vision_tower.vision_model.encoder.layers.13.layer_norm2.weight": "model-00001-of-00002.safetensors",
540
+ "vision_tower.vision_model.encoder.layers.13.mlp.fc1.bias": "model-00001-of-00002.safetensors",
541
+ "vision_tower.vision_model.encoder.layers.13.mlp.fc1.weight": "model-00001-of-00002.safetensors",
542
+ "vision_tower.vision_model.encoder.layers.13.mlp.fc2.bias": "model-00001-of-00002.safetensors",
543
+ "vision_tower.vision_model.encoder.layers.13.mlp.fc2.weight": "model-00001-of-00002.safetensors",
544
+ "vision_tower.vision_model.encoder.layers.13.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
545
+ "vision_tower.vision_model.encoder.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
546
+ "vision_tower.vision_model.encoder.layers.13.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
547
+ "vision_tower.vision_model.encoder.layers.13.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
548
+ "vision_tower.vision_model.encoder.layers.13.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
549
+ "vision_tower.vision_model.encoder.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
550
+ "vision_tower.vision_model.encoder.layers.13.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
551
+ "vision_tower.vision_model.encoder.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
552
+ "vision_tower.vision_model.encoder.layers.14.layer_norm1.bias": "model-00001-of-00002.safetensors",
553
+ "vision_tower.vision_model.encoder.layers.14.layer_norm1.weight": "model-00001-of-00002.safetensors",
554
+ "vision_tower.vision_model.encoder.layers.14.layer_norm2.bias": "model-00001-of-00002.safetensors",
555
+ "vision_tower.vision_model.encoder.layers.14.layer_norm2.weight": "model-00001-of-00002.safetensors",
556
+ "vision_tower.vision_model.encoder.layers.14.mlp.fc1.bias": "model-00001-of-00002.safetensors",
557
+ "vision_tower.vision_model.encoder.layers.14.mlp.fc1.weight": "model-00001-of-00002.safetensors",
558
+ "vision_tower.vision_model.encoder.layers.14.mlp.fc2.bias": "model-00001-of-00002.safetensors",
559
+ "vision_tower.vision_model.encoder.layers.14.mlp.fc2.weight": "model-00001-of-00002.safetensors",
560
+ "vision_tower.vision_model.encoder.layers.14.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
561
+ "vision_tower.vision_model.encoder.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
562
+ "vision_tower.vision_model.encoder.layers.14.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
563
+ "vision_tower.vision_model.encoder.layers.14.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
564
+ "vision_tower.vision_model.encoder.layers.14.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
565
+ "vision_tower.vision_model.encoder.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
566
+ "vision_tower.vision_model.encoder.layers.14.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
567
+ "vision_tower.vision_model.encoder.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
568
+ "vision_tower.vision_model.encoder.layers.15.layer_norm1.bias": "model-00001-of-00002.safetensors",
569
+ "vision_tower.vision_model.encoder.layers.15.layer_norm1.weight": "model-00001-of-00002.safetensors",
570
+ "vision_tower.vision_model.encoder.layers.15.layer_norm2.bias": "model-00001-of-00002.safetensors",
571
+ "vision_tower.vision_model.encoder.layers.15.layer_norm2.weight": "model-00001-of-00002.safetensors",
572
+ "vision_tower.vision_model.encoder.layers.15.mlp.fc1.bias": "model-00001-of-00002.safetensors",
573
+ "vision_tower.vision_model.encoder.layers.15.mlp.fc1.weight": "model-00001-of-00002.safetensors",
574
+ "vision_tower.vision_model.encoder.layers.15.mlp.fc2.bias": "model-00001-of-00002.safetensors",
575
+ "vision_tower.vision_model.encoder.layers.15.mlp.fc2.weight": "model-00001-of-00002.safetensors",
576
+ "vision_tower.vision_model.encoder.layers.15.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
577
+ "vision_tower.vision_model.encoder.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
578
+ "vision_tower.vision_model.encoder.layers.15.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
579
+ "vision_tower.vision_model.encoder.layers.15.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
580
+ "vision_tower.vision_model.encoder.layers.15.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
581
+ "vision_tower.vision_model.encoder.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
582
+ "vision_tower.vision_model.encoder.layers.15.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
583
+ "vision_tower.vision_model.encoder.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
584
+ "vision_tower.vision_model.encoder.layers.16.layer_norm1.bias": "model-00001-of-00002.safetensors",
585
+ "vision_tower.vision_model.encoder.layers.16.layer_norm1.weight": "model-00001-of-00002.safetensors",
586
+ "vision_tower.vision_model.encoder.layers.16.layer_norm2.bias": "model-00001-of-00002.safetensors",
587
+ "vision_tower.vision_model.encoder.layers.16.layer_norm2.weight": "model-00001-of-00002.safetensors",
588
+ "vision_tower.vision_model.encoder.layers.16.mlp.fc1.bias": "model-00001-of-00002.safetensors",
589
+ "vision_tower.vision_model.encoder.layers.16.mlp.fc1.weight": "model-00001-of-00002.safetensors",
590
+ "vision_tower.vision_model.encoder.layers.16.mlp.fc2.bias": "model-00001-of-00002.safetensors",
591
+ "vision_tower.vision_model.encoder.layers.16.mlp.fc2.weight": "model-00001-of-00002.safetensors",
592
+ "vision_tower.vision_model.encoder.layers.16.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
593
+ "vision_tower.vision_model.encoder.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
594
+ "vision_tower.vision_model.encoder.layers.16.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
595
+ "vision_tower.vision_model.encoder.layers.16.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
596
+ "vision_tower.vision_model.encoder.layers.16.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
597
+ "vision_tower.vision_model.encoder.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
598
+ "vision_tower.vision_model.encoder.layers.16.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
599
+ "vision_tower.vision_model.encoder.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
600
+ "vision_tower.vision_model.encoder.layers.17.layer_norm1.bias": "model-00001-of-00002.safetensors",
601
+ "vision_tower.vision_model.encoder.layers.17.layer_norm1.weight": "model-00001-of-00002.safetensors",
602
+ "vision_tower.vision_model.encoder.layers.17.layer_norm2.bias": "model-00001-of-00002.safetensors",
603
+ "vision_tower.vision_model.encoder.layers.17.layer_norm2.weight": "model-00001-of-00002.safetensors",
604
+ "vision_tower.vision_model.encoder.layers.17.mlp.fc1.bias": "model-00001-of-00002.safetensors",
605
+ "vision_tower.vision_model.encoder.layers.17.mlp.fc1.weight": "model-00001-of-00002.safetensors",
606
+ "vision_tower.vision_model.encoder.layers.17.mlp.fc2.bias": "model-00001-of-00002.safetensors",
607
+ "vision_tower.vision_model.encoder.layers.17.mlp.fc2.weight": "model-00001-of-00002.safetensors",
608
+ "vision_tower.vision_model.encoder.layers.17.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
609
+ "vision_tower.vision_model.encoder.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
610
+ "vision_tower.vision_model.encoder.layers.17.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
611
+ "vision_tower.vision_model.encoder.layers.17.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
612
+ "vision_tower.vision_model.encoder.layers.17.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
613
+ "vision_tower.vision_model.encoder.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
614
+ "vision_tower.vision_model.encoder.layers.17.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
615
+ "vision_tower.vision_model.encoder.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
616
+ "vision_tower.vision_model.encoder.layers.18.layer_norm1.bias": "model-00001-of-00002.safetensors",
617
+ "vision_tower.vision_model.encoder.layers.18.layer_norm1.weight": "model-00001-of-00002.safetensors",
618
+ "vision_tower.vision_model.encoder.layers.18.layer_norm2.bias": "model-00001-of-00002.safetensors",
619
+ "vision_tower.vision_model.encoder.layers.18.layer_norm2.weight": "model-00001-of-00002.safetensors",
620
+ "vision_tower.vision_model.encoder.layers.18.mlp.fc1.bias": "model-00001-of-00002.safetensors",
621
+ "vision_tower.vision_model.encoder.layers.18.mlp.fc1.weight": "model-00001-of-00002.safetensors",
622
+ "vision_tower.vision_model.encoder.layers.18.mlp.fc2.bias": "model-00001-of-00002.safetensors",
623
+ "vision_tower.vision_model.encoder.layers.18.mlp.fc2.weight": "model-00001-of-00002.safetensors",
624
+ "vision_tower.vision_model.encoder.layers.18.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
625
+ "vision_tower.vision_model.encoder.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
626
+ "vision_tower.vision_model.encoder.layers.18.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
627
+ "vision_tower.vision_model.encoder.layers.18.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
628
+ "vision_tower.vision_model.encoder.layers.18.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
629
+ "vision_tower.vision_model.encoder.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
630
+ "vision_tower.vision_model.encoder.layers.18.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
631
+ "vision_tower.vision_model.encoder.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
632
+ "vision_tower.vision_model.encoder.layers.19.layer_norm1.bias": "model-00001-of-00002.safetensors",
633
+ "vision_tower.vision_model.encoder.layers.19.layer_norm1.weight": "model-00001-of-00002.safetensors",
634
+ "vision_tower.vision_model.encoder.layers.19.layer_norm2.bias": "model-00001-of-00002.safetensors",
635
+ "vision_tower.vision_model.encoder.layers.19.layer_norm2.weight": "model-00001-of-00002.safetensors",
636
+ "vision_tower.vision_model.encoder.layers.19.mlp.fc1.bias": "model-00001-of-00002.safetensors",
637
+ "vision_tower.vision_model.encoder.layers.19.mlp.fc1.weight": "model-00001-of-00002.safetensors",
638
+ "vision_tower.vision_model.encoder.layers.19.mlp.fc2.bias": "model-00001-of-00002.safetensors",
639
+ "vision_tower.vision_model.encoder.layers.19.mlp.fc2.weight": "model-00001-of-00002.safetensors",
640
+ "vision_tower.vision_model.encoder.layers.19.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
641
+ "vision_tower.vision_model.encoder.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
642
+ "vision_tower.vision_model.encoder.layers.19.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
643
+ "vision_tower.vision_model.encoder.layers.19.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
644
+ "vision_tower.vision_model.encoder.layers.19.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
645
+ "vision_tower.vision_model.encoder.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
646
+ "vision_tower.vision_model.encoder.layers.19.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
647
+ "vision_tower.vision_model.encoder.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
648
+ "vision_tower.vision_model.encoder.layers.2.layer_norm1.bias": "model-00001-of-00002.safetensors",
649
+ "vision_tower.vision_model.encoder.layers.2.layer_norm1.weight": "model-00001-of-00002.safetensors",
650
+ "vision_tower.vision_model.encoder.layers.2.layer_norm2.bias": "model-00001-of-00002.safetensors",
651
+ "vision_tower.vision_model.encoder.layers.2.layer_norm2.weight": "model-00001-of-00002.safetensors",
652
+ "vision_tower.vision_model.encoder.layers.2.mlp.fc1.bias": "model-00001-of-00002.safetensors",
653
+ "vision_tower.vision_model.encoder.layers.2.mlp.fc1.weight": "model-00001-of-00002.safetensors",
654
+ "vision_tower.vision_model.encoder.layers.2.mlp.fc2.bias": "model-00001-of-00002.safetensors",
655
+ "vision_tower.vision_model.encoder.layers.2.mlp.fc2.weight": "model-00001-of-00002.safetensors",
656
+ "vision_tower.vision_model.encoder.layers.2.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
657
+ "vision_tower.vision_model.encoder.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
658
+ "vision_tower.vision_model.encoder.layers.2.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
659
+ "vision_tower.vision_model.encoder.layers.2.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
660
+ "vision_tower.vision_model.encoder.layers.2.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
661
+ "vision_tower.vision_model.encoder.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
662
+ "vision_tower.vision_model.encoder.layers.2.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
663
+ "vision_tower.vision_model.encoder.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
664
+ "vision_tower.vision_model.encoder.layers.20.layer_norm1.bias": "model-00001-of-00002.safetensors",
665
+ "vision_tower.vision_model.encoder.layers.20.layer_norm1.weight": "model-00001-of-00002.safetensors",
666
+ "vision_tower.vision_model.encoder.layers.20.layer_norm2.bias": "model-00001-of-00002.safetensors",
667
+ "vision_tower.vision_model.encoder.layers.20.layer_norm2.weight": "model-00001-of-00002.safetensors",
668
+ "vision_tower.vision_model.encoder.layers.20.mlp.fc1.bias": "model-00001-of-00002.safetensors",
669
+ "vision_tower.vision_model.encoder.layers.20.mlp.fc1.weight": "model-00001-of-00002.safetensors",
670
+ "vision_tower.vision_model.encoder.layers.20.mlp.fc2.bias": "model-00001-of-00002.safetensors",
671
+ "vision_tower.vision_model.encoder.layers.20.mlp.fc2.weight": "model-00001-of-00002.safetensors",
672
+ "vision_tower.vision_model.encoder.layers.20.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
673
+ "vision_tower.vision_model.encoder.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
674
+ "vision_tower.vision_model.encoder.layers.20.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
675
+ "vision_tower.vision_model.encoder.layers.20.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
676
+ "vision_tower.vision_model.encoder.layers.20.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
677
+ "vision_tower.vision_model.encoder.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
678
+ "vision_tower.vision_model.encoder.layers.20.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
679
+ "vision_tower.vision_model.encoder.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
680
+ "vision_tower.vision_model.encoder.layers.21.layer_norm1.bias": "model-00001-of-00002.safetensors",
681
+ "vision_tower.vision_model.encoder.layers.21.layer_norm1.weight": "model-00001-of-00002.safetensors",
682
+ "vision_tower.vision_model.encoder.layers.21.layer_norm2.bias": "model-00001-of-00002.safetensors",
683
+ "vision_tower.vision_model.encoder.layers.21.layer_norm2.weight": "model-00001-of-00002.safetensors",
684
+ "vision_tower.vision_model.encoder.layers.21.mlp.fc1.bias": "model-00001-of-00002.safetensors",
685
+ "vision_tower.vision_model.encoder.layers.21.mlp.fc1.weight": "model-00001-of-00002.safetensors",
686
+ "vision_tower.vision_model.encoder.layers.21.mlp.fc2.bias": "model-00001-of-00002.safetensors",
687
+ "vision_tower.vision_model.encoder.layers.21.mlp.fc2.weight": "model-00001-of-00002.safetensors",
688
+ "vision_tower.vision_model.encoder.layers.21.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
689
+ "vision_tower.vision_model.encoder.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
690
+ "vision_tower.vision_model.encoder.layers.21.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
691
+ "vision_tower.vision_model.encoder.layers.21.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
692
+ "vision_tower.vision_model.encoder.layers.21.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
693
+ "vision_tower.vision_model.encoder.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
694
+ "vision_tower.vision_model.encoder.layers.21.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
695
+ "vision_tower.vision_model.encoder.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
696
+ "vision_tower.vision_model.encoder.layers.22.layer_norm1.bias": "model-00001-of-00002.safetensors",
697
+ "vision_tower.vision_model.encoder.layers.22.layer_norm1.weight": "model-00001-of-00002.safetensors",
698
+ "vision_tower.vision_model.encoder.layers.22.layer_norm2.bias": "model-00001-of-00002.safetensors",
699
+ "vision_tower.vision_model.encoder.layers.22.layer_norm2.weight": "model-00001-of-00002.safetensors",
700
+ "vision_tower.vision_model.encoder.layers.22.mlp.fc1.bias": "model-00001-of-00002.safetensors",
701
+ "vision_tower.vision_model.encoder.layers.22.mlp.fc1.weight": "model-00001-of-00002.safetensors",
702
+ "vision_tower.vision_model.encoder.layers.22.mlp.fc2.bias": "model-00001-of-00002.safetensors",
703
+ "vision_tower.vision_model.encoder.layers.22.mlp.fc2.weight": "model-00001-of-00002.safetensors",
704
+ "vision_tower.vision_model.encoder.layers.22.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
705
+ "vision_tower.vision_model.encoder.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
706
+ "vision_tower.vision_model.encoder.layers.22.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
707
+ "vision_tower.vision_model.encoder.layers.22.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
708
+ "vision_tower.vision_model.encoder.layers.22.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
709
+ "vision_tower.vision_model.encoder.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
710
+ "vision_tower.vision_model.encoder.layers.22.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
711
+ "vision_tower.vision_model.encoder.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
712
+ "vision_tower.vision_model.encoder.layers.23.layer_norm1.bias": "model-00001-of-00002.safetensors",
713
+ "vision_tower.vision_model.encoder.layers.23.layer_norm1.weight": "model-00001-of-00002.safetensors",
714
+ "vision_tower.vision_model.encoder.layers.23.layer_norm2.bias": "model-00001-of-00002.safetensors",
715
+ "vision_tower.vision_model.encoder.layers.23.layer_norm2.weight": "model-00001-of-00002.safetensors",
716
+ "vision_tower.vision_model.encoder.layers.23.mlp.fc1.bias": "model-00001-of-00002.safetensors",
717
+ "vision_tower.vision_model.encoder.layers.23.mlp.fc1.weight": "model-00001-of-00002.safetensors",
718
+ "vision_tower.vision_model.encoder.layers.23.mlp.fc2.bias": "model-00001-of-00002.safetensors",
719
+ "vision_tower.vision_model.encoder.layers.23.mlp.fc2.weight": "model-00001-of-00002.safetensors",
720
+ "vision_tower.vision_model.encoder.layers.23.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
721
+ "vision_tower.vision_model.encoder.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
722
+ "vision_tower.vision_model.encoder.layers.23.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
723
+ "vision_tower.vision_model.encoder.layers.23.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
724
+ "vision_tower.vision_model.encoder.layers.23.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
725
+ "vision_tower.vision_model.encoder.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
726
+ "vision_tower.vision_model.encoder.layers.23.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
727
+ "vision_tower.vision_model.encoder.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
728
+ "vision_tower.vision_model.encoder.layers.24.layer_norm1.bias": "model-00001-of-00002.safetensors",
729
+ "vision_tower.vision_model.encoder.layers.24.layer_norm1.weight": "model-00001-of-00002.safetensors",
730
+ "vision_tower.vision_model.encoder.layers.24.layer_norm2.bias": "model-00001-of-00002.safetensors",
731
+ "vision_tower.vision_model.encoder.layers.24.layer_norm2.weight": "model-00001-of-00002.safetensors",
732
+ "vision_tower.vision_model.encoder.layers.24.mlp.fc1.bias": "model-00001-of-00002.safetensors",
733
+ "vision_tower.vision_model.encoder.layers.24.mlp.fc1.weight": "model-00001-of-00002.safetensors",
734
+ "vision_tower.vision_model.encoder.layers.24.mlp.fc2.bias": "model-00001-of-00002.safetensors",
735
+ "vision_tower.vision_model.encoder.layers.24.mlp.fc2.weight": "model-00001-of-00002.safetensors",
736
+ "vision_tower.vision_model.encoder.layers.24.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
737
+ "vision_tower.vision_model.encoder.layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
738
+ "vision_tower.vision_model.encoder.layers.24.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
739
+ "vision_tower.vision_model.encoder.layers.24.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
740
+ "vision_tower.vision_model.encoder.layers.24.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
741
+ "vision_tower.vision_model.encoder.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
742
+ "vision_tower.vision_model.encoder.layers.24.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
743
+ "vision_tower.vision_model.encoder.layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
744
+ "vision_tower.vision_model.encoder.layers.25.layer_norm1.bias": "model-00001-of-00002.safetensors",
745
+ "vision_tower.vision_model.encoder.layers.25.layer_norm1.weight": "model-00001-of-00002.safetensors",
746
+ "vision_tower.vision_model.encoder.layers.25.layer_norm2.bias": "model-00001-of-00002.safetensors",
747
+ "vision_tower.vision_model.encoder.layers.25.layer_norm2.weight": "model-00001-of-00002.safetensors",
748
+ "vision_tower.vision_model.encoder.layers.25.mlp.fc1.bias": "model-00001-of-00002.safetensors",
749
+ "vision_tower.vision_model.encoder.layers.25.mlp.fc1.weight": "model-00001-of-00002.safetensors",
750
+ "vision_tower.vision_model.encoder.layers.25.mlp.fc2.bias": "model-00001-of-00002.safetensors",
751
+ "vision_tower.vision_model.encoder.layers.25.mlp.fc2.weight": "model-00001-of-00002.safetensors",
752
+ "vision_tower.vision_model.encoder.layers.25.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
753
+ "vision_tower.vision_model.encoder.layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
754
+ "vision_tower.vision_model.encoder.layers.25.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
755
+ "vision_tower.vision_model.encoder.layers.25.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
756
+ "vision_tower.vision_model.encoder.layers.25.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
757
+ "vision_tower.vision_model.encoder.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
758
+ "vision_tower.vision_model.encoder.layers.25.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
759
+ "vision_tower.vision_model.encoder.layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
760
+ "vision_tower.vision_model.encoder.layers.26.layer_norm1.bias": "model-00001-of-00002.safetensors",
761
+ "vision_tower.vision_model.encoder.layers.26.layer_norm1.weight": "model-00001-of-00002.safetensors",
762
+ "vision_tower.vision_model.encoder.layers.26.layer_norm2.bias": "model-00001-of-00002.safetensors",
763
+ "vision_tower.vision_model.encoder.layers.26.layer_norm2.weight": "model-00001-of-00002.safetensors",
764
+ "vision_tower.vision_model.encoder.layers.26.mlp.fc1.bias": "model-00001-of-00002.safetensors",
765
+ "vision_tower.vision_model.encoder.layers.26.mlp.fc1.weight": "model-00001-of-00002.safetensors",
766
+ "vision_tower.vision_model.encoder.layers.26.mlp.fc2.bias": "model-00001-of-00002.safetensors",
767
+ "vision_tower.vision_model.encoder.layers.26.mlp.fc2.weight": "model-00001-of-00002.safetensors",
768
+ "vision_tower.vision_model.encoder.layers.26.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
769
+ "vision_tower.vision_model.encoder.layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
770
+ "vision_tower.vision_model.encoder.layers.26.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
771
+ "vision_tower.vision_model.encoder.layers.26.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
772
+ "vision_tower.vision_model.encoder.layers.26.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
773
+ "vision_tower.vision_model.encoder.layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
774
+ "vision_tower.vision_model.encoder.layers.26.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
775
+ "vision_tower.vision_model.encoder.layers.26.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
776
+ "vision_tower.vision_model.encoder.layers.3.layer_norm1.bias": "model-00001-of-00002.safetensors",
777
+ "vision_tower.vision_model.encoder.layers.3.layer_norm1.weight": "model-00001-of-00002.safetensors",
778
+ "vision_tower.vision_model.encoder.layers.3.layer_norm2.bias": "model-00001-of-00002.safetensors",
779
+ "vision_tower.vision_model.encoder.layers.3.layer_norm2.weight": "model-00001-of-00002.safetensors",
780
+ "vision_tower.vision_model.encoder.layers.3.mlp.fc1.bias": "model-00001-of-00002.safetensors",
781
+ "vision_tower.vision_model.encoder.layers.3.mlp.fc1.weight": "model-00001-of-00002.safetensors",
782
+ "vision_tower.vision_model.encoder.layers.3.mlp.fc2.bias": "model-00001-of-00002.safetensors",
783
+ "vision_tower.vision_model.encoder.layers.3.mlp.fc2.weight": "model-00001-of-00002.safetensors",
784
+ "vision_tower.vision_model.encoder.layers.3.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
785
+ "vision_tower.vision_model.encoder.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
786
+ "vision_tower.vision_model.encoder.layers.3.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
787
+ "vision_tower.vision_model.encoder.layers.3.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
788
+ "vision_tower.vision_model.encoder.layers.3.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
789
+ "vision_tower.vision_model.encoder.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
790
+ "vision_tower.vision_model.encoder.layers.3.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
791
+ "vision_tower.vision_model.encoder.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
792
+ "vision_tower.vision_model.encoder.layers.4.layer_norm1.bias": "model-00001-of-00002.safetensors",
793
+ "vision_tower.vision_model.encoder.layers.4.layer_norm1.weight": "model-00001-of-00002.safetensors",
794
+ "vision_tower.vision_model.encoder.layers.4.layer_norm2.bias": "model-00001-of-00002.safetensors",
795
+ "vision_tower.vision_model.encoder.layers.4.layer_norm2.weight": "model-00001-of-00002.safetensors",
796
+ "vision_tower.vision_model.encoder.layers.4.mlp.fc1.bias": "model-00001-of-00002.safetensors",
797
+ "vision_tower.vision_model.encoder.layers.4.mlp.fc1.weight": "model-00001-of-00002.safetensors",
798
+ "vision_tower.vision_model.encoder.layers.4.mlp.fc2.bias": "model-00001-of-00002.safetensors",
799
+ "vision_tower.vision_model.encoder.layers.4.mlp.fc2.weight": "model-00001-of-00002.safetensors",
800
+ "vision_tower.vision_model.encoder.layers.4.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
801
+ "vision_tower.vision_model.encoder.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
802
+ "vision_tower.vision_model.encoder.layers.4.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
803
+ "vision_tower.vision_model.encoder.layers.4.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
804
+ "vision_tower.vision_model.encoder.layers.4.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
805
+ "vision_tower.vision_model.encoder.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
806
+ "vision_tower.vision_model.encoder.layers.4.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
807
+ "vision_tower.vision_model.encoder.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
808
+ "vision_tower.vision_model.encoder.layers.5.layer_norm1.bias": "model-00001-of-00002.safetensors",
809
+ "vision_tower.vision_model.encoder.layers.5.layer_norm1.weight": "model-00001-of-00002.safetensors",
810
+ "vision_tower.vision_model.encoder.layers.5.layer_norm2.bias": "model-00001-of-00002.safetensors",
811
+ "vision_tower.vision_model.encoder.layers.5.layer_norm2.weight": "model-00001-of-00002.safetensors",
812
+ "vision_tower.vision_model.encoder.layers.5.mlp.fc1.bias": "model-00001-of-00002.safetensors",
813
+ "vision_tower.vision_model.encoder.layers.5.mlp.fc1.weight": "model-00001-of-00002.safetensors",
814
+ "vision_tower.vision_model.encoder.layers.5.mlp.fc2.bias": "model-00001-of-00002.safetensors",
815
+ "vision_tower.vision_model.encoder.layers.5.mlp.fc2.weight": "model-00001-of-00002.safetensors",
816
+ "vision_tower.vision_model.encoder.layers.5.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
817
+ "vision_tower.vision_model.encoder.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
818
+ "vision_tower.vision_model.encoder.layers.5.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
819
+ "vision_tower.vision_model.encoder.layers.5.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
820
+ "vision_tower.vision_model.encoder.layers.5.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
821
+ "vision_tower.vision_model.encoder.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
822
+ "vision_tower.vision_model.encoder.layers.5.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
823
+ "vision_tower.vision_model.encoder.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
824
+ "vision_tower.vision_model.encoder.layers.6.layer_norm1.bias": "model-00001-of-00002.safetensors",
825
+ "vision_tower.vision_model.encoder.layers.6.layer_norm1.weight": "model-00001-of-00002.safetensors",
826
+ "vision_tower.vision_model.encoder.layers.6.layer_norm2.bias": "model-00001-of-00002.safetensors",
827
+ "vision_tower.vision_model.encoder.layers.6.layer_norm2.weight": "model-00001-of-00002.safetensors",
828
+ "vision_tower.vision_model.encoder.layers.6.mlp.fc1.bias": "model-00001-of-00002.safetensors",
829
+ "vision_tower.vision_model.encoder.layers.6.mlp.fc1.weight": "model-00001-of-00002.safetensors",
830
+ "vision_tower.vision_model.encoder.layers.6.mlp.fc2.bias": "model-00001-of-00002.safetensors",
831
+ "vision_tower.vision_model.encoder.layers.6.mlp.fc2.weight": "model-00001-of-00002.safetensors",
832
+ "vision_tower.vision_model.encoder.layers.6.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
833
+ "vision_tower.vision_model.encoder.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
834
+ "vision_tower.vision_model.encoder.layers.6.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
835
+ "vision_tower.vision_model.encoder.layers.6.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
836
+ "vision_tower.vision_model.encoder.layers.6.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
837
+ "vision_tower.vision_model.encoder.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
838
+ "vision_tower.vision_model.encoder.layers.6.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
839
+ "vision_tower.vision_model.encoder.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
840
+ "vision_tower.vision_model.encoder.layers.7.layer_norm1.bias": "model-00001-of-00002.safetensors",
841
+ "vision_tower.vision_model.encoder.layers.7.layer_norm1.weight": "model-00001-of-00002.safetensors",
842
+ "vision_tower.vision_model.encoder.layers.7.layer_norm2.bias": "model-00001-of-00002.safetensors",
843
+ "vision_tower.vision_model.encoder.layers.7.layer_norm2.weight": "model-00001-of-00002.safetensors",
844
+ "vision_tower.vision_model.encoder.layers.7.mlp.fc1.bias": "model-00001-of-00002.safetensors",
845
+ "vision_tower.vision_model.encoder.layers.7.mlp.fc1.weight": "model-00001-of-00002.safetensors",
846
+ "vision_tower.vision_model.encoder.layers.7.mlp.fc2.bias": "model-00001-of-00002.safetensors",
847
+ "vision_tower.vision_model.encoder.layers.7.mlp.fc2.weight": "model-00001-of-00002.safetensors",
848
+ "vision_tower.vision_model.encoder.layers.7.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
849
+ "vision_tower.vision_model.encoder.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
850
+ "vision_tower.vision_model.encoder.layers.7.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
851
+ "vision_tower.vision_model.encoder.layers.7.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
852
+ "vision_tower.vision_model.encoder.layers.7.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
853
+ "vision_tower.vision_model.encoder.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
854
+ "vision_tower.vision_model.encoder.layers.7.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
855
+ "vision_tower.vision_model.encoder.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
856
+ "vision_tower.vision_model.encoder.layers.8.layer_norm1.bias": "model-00001-of-00002.safetensors",
857
+ "vision_tower.vision_model.encoder.layers.8.layer_norm1.weight": "model-00001-of-00002.safetensors",
858
+ "vision_tower.vision_model.encoder.layers.8.layer_norm2.bias": "model-00001-of-00002.safetensors",
859
+ "vision_tower.vision_model.encoder.layers.8.layer_norm2.weight": "model-00001-of-00002.safetensors",
860
+ "vision_tower.vision_model.encoder.layers.8.mlp.fc1.bias": "model-00001-of-00002.safetensors",
861
+ "vision_tower.vision_model.encoder.layers.8.mlp.fc1.weight": "model-00001-of-00002.safetensors",
862
+ "vision_tower.vision_model.encoder.layers.8.mlp.fc2.bias": "model-00001-of-00002.safetensors",
863
+ "vision_tower.vision_model.encoder.layers.8.mlp.fc2.weight": "model-00001-of-00002.safetensors",
864
+ "vision_tower.vision_model.encoder.layers.8.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
865
+ "vision_tower.vision_model.encoder.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
866
+ "vision_tower.vision_model.encoder.layers.8.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
867
+ "vision_tower.vision_model.encoder.layers.8.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
868
+ "vision_tower.vision_model.encoder.layers.8.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
869
+ "vision_tower.vision_model.encoder.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
870
+ "vision_tower.vision_model.encoder.layers.8.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
871
+ "vision_tower.vision_model.encoder.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
872
+ "vision_tower.vision_model.encoder.layers.9.layer_norm1.bias": "model-00001-of-00002.safetensors",
873
+ "vision_tower.vision_model.encoder.layers.9.layer_norm1.weight": "model-00001-of-00002.safetensors",
874
+ "vision_tower.vision_model.encoder.layers.9.layer_norm2.bias": "model-00001-of-00002.safetensors",
875
+ "vision_tower.vision_model.encoder.layers.9.layer_norm2.weight": "model-00001-of-00002.safetensors",
876
+ "vision_tower.vision_model.encoder.layers.9.mlp.fc1.bias": "model-00001-of-00002.safetensors",
877
+ "vision_tower.vision_model.encoder.layers.9.mlp.fc1.weight": "model-00001-of-00002.safetensors",
878
+ "vision_tower.vision_model.encoder.layers.9.mlp.fc2.bias": "model-00001-of-00002.safetensors",
879
+ "vision_tower.vision_model.encoder.layers.9.mlp.fc2.weight": "model-00001-of-00002.safetensors",
880
+ "vision_tower.vision_model.encoder.layers.9.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
881
+ "vision_tower.vision_model.encoder.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
882
+ "vision_tower.vision_model.encoder.layers.9.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
883
+ "vision_tower.vision_model.encoder.layers.9.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
884
+ "vision_tower.vision_model.encoder.layers.9.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
885
+ "vision_tower.vision_model.encoder.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
886
+ "vision_tower.vision_model.encoder.layers.9.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
887
+ "vision_tower.vision_model.encoder.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
888
+ "vision_tower.vision_model.post_layernorm.bias": "model-00001-of-00002.safetensors",
889
+ "vision_tower.vision_model.post_layernorm.weight": "model-00001-of-00002.safetensors"
890
+ }
891
+ }
lamp-gemma-4b-v2/special_tokens_map.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "boi_token": "<start_of_image>",
3
+ "bos_token": {
4
+ "content": "<bos>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ "eoi_token": "<end_of_image>",
11
+ "eos_token": {
12
+ "content": "<end_of_turn>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false
17
+ },
18
+ "image_token": "<image_soft_token>",
19
+ "pad_token": {
20
+ "content": "<pad>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false
25
+ },
26
+ "unk_token": {
27
+ "content": "<unk>",
28
+ "lstrip": false,
29
+ "normalized": false,
30
+ "rstrip": false,
31
+ "single_word": false
32
+ }
33
+ }
lamp-gemma-4b-v2/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
3
+ size 33384568
lamp-gemma-4b-v2/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
3
+ size 4689074
lamp-gemma-4b-v2/tokenizer_config.json ADDED
The diff for this file is too large to render. See raw diff
 
lamp-gemma-4b-v2/training_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "unsloth/gemma-3-4b-it",
3
+ "output_name": "lamp-gemma-4b-v2",
4
+ "fine_tune_type": "full (all parameters)",
5
+ "precision": "bf16",
6
+ "max_seq_length": 4096,
7
+ "train_config": {
8
+ "per_device_train_batch_size": 8,
9
+ "gradient_accumulation_steps": 2,
10
+ "warmup_ratio": 0.05,
11
+ "num_train_epochs": 2,
12
+ "learning_rate": 2e-05,
13
+ "logging_steps": 10,
14
+ "optim": "adamw_torch",
15
+ "weight_decay": 0.01,
16
+ "lr_scheduler_type": "cosine",
17
+ "seed": 42,
18
+ "save_strategy": "epoch",
19
+ "eval_strategy": "epoch",
20
+ "report_to": "none"
21
+ },
22
+ "trainable_params": 4300079472,
23
+ "total_params": 4300079472,
24
+ "trainable_pct": 100.0,
25
+ "training_time_min": 38.1,
26
+ "dataset": {
27
+ "train": 6567,
28
+ "val": 730
29
+ },
30
+ "final_train_loss": 0.0216,
31
+ "final_eval_loss": 0.02881857380270958,
32
+ "gpu": "NVIDIA H200",
33
+ "peak_vram_gb": 24.31
34
+ }
lamp-gemma-4b-v2/training_log.json ADDED
@@ -0,0 +1,601 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "loss": 1.5717,
4
+ "grad_norm": 12.93989086151123,
5
+ "learning_rate": 4.2857142857142855e-06,
6
+ "epoch": 0.024360535931790498,
7
+ "step": 10
8
+ },
9
+ {
10
+ "loss": 0.5627,
11
+ "grad_norm": 3.815218687057495,
12
+ "learning_rate": 9.047619047619049e-06,
13
+ "epoch": 0.048721071863580996,
14
+ "step": 20
15
+ },
16
+ {
17
+ "loss": 0.0742,
18
+ "grad_norm": 2.975243330001831,
19
+ "learning_rate": 1.3809523809523811e-05,
20
+ "epoch": 0.0730816077953715,
21
+ "step": 30
22
+ },
23
+ {
24
+ "loss": 0.0563,
25
+ "grad_norm": 6.168555736541748,
26
+ "learning_rate": 1.8571428571428575e-05,
27
+ "epoch": 0.09744214372716199,
28
+ "step": 40
29
+ },
30
+ {
31
+ "loss": 0.0459,
32
+ "grad_norm": 0.4206625819206238,
33
+ "learning_rate": 1.999602581704666e-05,
34
+ "epoch": 0.1218026796589525,
35
+ "step": 50
36
+ },
37
+ {
38
+ "loss": 0.0409,
39
+ "grad_norm": 0.39415431022644043,
40
+ "learning_rate": 1.997656803510789e-05,
41
+ "epoch": 0.146163215590743,
42
+ "step": 60
43
+ },
44
+ {
45
+ "loss": 0.0394,
46
+ "grad_norm": 0.37986722588539124,
47
+ "learning_rate": 1.9940928223333254e-05,
48
+ "epoch": 0.1705237515225335,
49
+ "step": 70
50
+ },
51
+ {
52
+ "loss": 0.0385,
53
+ "grad_norm": 0.36188265681266785,
54
+ "learning_rate": 1.988916418962778e-05,
55
+ "epoch": 0.19488428745432398,
56
+ "step": 80
57
+ },
58
+ {
59
+ "loss": 0.0398,
60
+ "grad_norm": 0.31783080101013184,
61
+ "learning_rate": 1.9821359895443436e-05,
62
+ "epoch": 0.2192448233861145,
63
+ "step": 90
64
+ },
65
+ {
66
+ "loss": 0.0352,
67
+ "grad_norm": 0.2857855260372162,
68
+ "learning_rate": 1.9737625319593338e-05,
69
+ "epoch": 0.243605359317905,
70
+ "step": 100
71
+ },
72
+ {
73
+ "loss": 0.0337,
74
+ "grad_norm": 0.2981444299221039,
75
+ "learning_rate": 1.9638096279865717e-05,
76
+ "epoch": 0.2679658952496955,
77
+ "step": 110
78
+ },
79
+ {
80
+ "loss": 0.035,
81
+ "grad_norm": 0.25182104110717773,
82
+ "learning_rate": 1.9522934212726977e-05,
83
+ "epoch": 0.292326431181486,
84
+ "step": 120
85
+ },
86
+ {
87
+ "loss": 0.0348,
88
+ "grad_norm": 0.3449748754501343,
89
+ "learning_rate": 1.9392325911471154e-05,
90
+ "epoch": 0.3166869671132765,
91
+ "step": 130
92
+ },
93
+ {
94
+ "loss": 0.0329,
95
+ "grad_norm": 0.29515525698661804,
96
+ "learning_rate": 1.9246483223240496e-05,
97
+ "epoch": 0.341047503045067,
98
+ "step": 140
99
+ },
100
+ {
101
+ "loss": 0.0325,
102
+ "grad_norm": 0.27917301654815674,
103
+ "learning_rate": 1.908564270540864e-05,
104
+ "epoch": 0.3654080389768575,
105
+ "step": 150
106
+ },
107
+ {
108
+ "loss": 0.0333,
109
+ "grad_norm": 0.283848375082016,
110
+ "learning_rate": 1.891006524188368e-05,
111
+ "epoch": 0.38976857490864797,
112
+ "step": 160
113
+ },
114
+ {
115
+ "loss": 0.0349,
116
+ "grad_norm": 0.3458445370197296,
117
+ "learning_rate": 1.8720035619953517e-05,
118
+ "epoch": 0.41412911084043846,
119
+ "step": 170
120
+ },
121
+ {
122
+ "loss": 0.0338,
123
+ "grad_norm": 0.3031023442745209,
124
+ "learning_rate": 1.8515862068359837e-05,
125
+ "epoch": 0.438489646772229,
126
+ "step": 180
127
+ },
128
+ {
129
+ "loss": 0.0351,
130
+ "grad_norm": 0.32958167791366577,
131
+ "learning_rate": 1.829787575734995e-05,
132
+ "epoch": 0.4628501827040195,
133
+ "step": 190
134
+ },
135
+ {
136
+ "loss": 0.032,
137
+ "grad_norm": 0.24969083070755005,
138
+ "learning_rate": 1.806643026151747e-05,
139
+ "epoch": 0.48721071863581,
140
+ "step": 200
141
+ },
142
+ {
143
+ "loss": 0.0321,
144
+ "grad_norm": 0.3140121400356293,
145
+ "learning_rate": 1.7821900986302995e-05,
146
+ "epoch": 0.5115712545676004,
147
+ "step": 210
148
+ },
149
+ {
150
+ "loss": 0.0319,
151
+ "grad_norm": 0.29097917675971985,
152
+ "learning_rate": 1.7564684559085138e-05,
153
+ "epoch": 0.535931790499391,
154
+ "step": 220
155
+ },
156
+ {
157
+ "loss": 0.0305,
158
+ "grad_norm": 0.2553909718990326,
159
+ "learning_rate": 1.729519818584944e-05,
160
+ "epoch": 0.5602923264311814,
161
+ "step": 230
162
+ },
163
+ {
164
+ "loss": 0.03,
165
+ "grad_norm": 0.3180781304836273,
166
+ "learning_rate": 1.7013878974478778e-05,
167
+ "epoch": 0.584652862362972,
168
+ "step": 240
169
+ },
170
+ {
171
+ "loss": 0.0319,
172
+ "grad_norm": 0.2638692557811737,
173
+ "learning_rate": 1.6721183225762726e-05,
174
+ "epoch": 0.6090133982947625,
175
+ "step": 250
176
+ },
177
+ {
178
+ "loss": 0.0318,
179
+ "grad_norm": 0.3281157910823822,
180
+ "learning_rate": 1.641758569327606e-05,
181
+ "epoch": 0.633373934226553,
182
+ "step": 260
183
+ },
184
+ {
185
+ "loss": 0.0323,
186
+ "grad_norm": 0.30428585410118103,
187
+ "learning_rate": 1.6103578813326683e-05,
188
+ "epoch": 0.6577344701583435,
189
+ "step": 270
190
+ },
191
+ {
192
+ "loss": 0.0335,
193
+ "grad_norm": 0.3534463942050934,
194
+ "learning_rate": 1.577967190622215e-05,
195
+ "epoch": 0.682095006090134,
196
+ "step": 280
197
+ },
198
+ {
199
+ "loss": 0.0303,
200
+ "grad_norm": 0.3207934498786926,
201
+ "learning_rate": 1.5446390350150272e-05,
202
+ "epoch": 0.7064555420219245,
203
+ "step": 290
204
+ },
205
+ {
206
+ "loss": 0.0309,
207
+ "grad_norm": 0.26610976457595825,
208
+ "learning_rate": 1.5104274729013759e-05,
209
+ "epoch": 0.730816077953715,
210
+ "step": 300
211
+ },
212
+ {
213
+ "loss": 0.0302,
214
+ "grad_norm": 0.2562502920627594,
215
+ "learning_rate": 1.4753879955601162e-05,
216
+ "epoch": 0.7551766138855055,
217
+ "step": 310
218
+ },
219
+ {
220
+ "loss": 0.0295,
221
+ "grad_norm": 0.2790921926498413,
222
+ "learning_rate": 1.4395774371516305e-05,
223
+ "epoch": 0.7795371498172959,
224
+ "step": 320
225
+ },
226
+ {
227
+ "loss": 0.0287,
228
+ "grad_norm": 0.26000431180000305,
229
+ "learning_rate": 1.4030538825326115e-05,
230
+ "epoch": 0.8038976857490865,
231
+ "step": 330
232
+ },
233
+ {
234
+ "loss": 0.029,
235
+ "grad_norm": 0.26584070920944214,
236
+ "learning_rate": 1.3658765730422126e-05,
237
+ "epoch": 0.8282582216808769,
238
+ "step": 340
239
+ },
240
+ {
241
+ "loss": 0.0303,
242
+ "grad_norm": 0.27419212460517883,
243
+ "learning_rate": 1.3281058104123794e-05,
244
+ "epoch": 0.8526187576126675,
245
+ "step": 350
246
+ },
247
+ {
248
+ "loss": 0.0292,
249
+ "grad_norm": 0.2997708022594452,
250
+ "learning_rate": 1.2898028589582202e-05,
251
+ "epoch": 0.876979293544458,
252
+ "step": 360
253
+ },
254
+ {
255
+ "loss": 0.0275,
256
+ "grad_norm": 0.29384979605674744,
257
+ "learning_rate": 1.2510298462070619e-05,
258
+ "epoch": 0.9013398294762485,
259
+ "step": 370
260
+ },
261
+ {
262
+ "loss": 0.0307,
263
+ "grad_norm": 0.30975568294525146,
264
+ "learning_rate": 1.2118496621273745e-05,
265
+ "epoch": 0.925700365408039,
266
+ "step": 380
267
+ },
268
+ {
269
+ "loss": 0.031,
270
+ "grad_norm": 0.2812424600124359,
271
+ "learning_rate": 1.172325857121012e-05,
272
+ "epoch": 0.9500609013398295,
273
+ "step": 390
274
+ },
275
+ {
276
+ "loss": 0.0274,
277
+ "grad_norm": 0.3010639548301697,
278
+ "learning_rate": 1.1325225389442278e-05,
279
+ "epoch": 0.97442143727162,
280
+ "step": 400
281
+ },
282
+ {
283
+ "loss": 0.0287,
284
+ "grad_norm": 0.2809979319572449,
285
+ "learning_rate": 1.0925042687246592e-05,
286
+ "epoch": 0.9987819732034104,
287
+ "step": 410
288
+ },
289
+ {
290
+ "eval_loss": 0.029586488381028175,
291
+ "eval_runtime": 42.9141,
292
+ "eval_samples_per_second": 17.011,
293
+ "eval_steps_per_second": 4.264,
294
+ "epoch": 1.0,
295
+ "step": 411
296
+ },
297
+ {
298
+ "loss": 0.0245,
299
+ "grad_norm": 0.26576268672943115,
300
+ "learning_rate": 1.0523359562429441e-05,
301
+ "epoch": 1.0219244823386116,
302
+ "step": 420
303
+ },
304
+ {
305
+ "loss": 0.0241,
306
+ "grad_norm": 0.3137070834636688,
307
+ "learning_rate": 1.0120827546488175e-05,
308
+ "epoch": 1.046285018270402,
309
+ "step": 430
310
+ },
311
+ {
312
+ "loss": 0.024,
313
+ "grad_norm": 0.29844993352890015,
314
+ "learning_rate": 9.718099547824657e-06,
315
+ "epoch": 1.0706455542021924,
316
+ "step": 440
317
+ },
318
+ {
319
+ "loss": 0.0229,
320
+ "grad_norm": 0.2821860909461975,
321
+ "learning_rate": 9.315828792725438e-06,
322
+ "epoch": 1.0950060901339829,
323
+ "step": 450
324
+ },
325
+ {
326
+ "loss": 0.0233,
327
+ "grad_norm": 0.339130163192749,
328
+ "learning_rate": 8.91466776582634e-06,
329
+ "epoch": 1.1193666260657735,
330
+ "step": 460
331
+ },
332
+ {
333
+ "loss": 0.0226,
334
+ "grad_norm": 0.32476964592933655,
335
+ "learning_rate": 8.515267151779974e-06,
336
+ "epoch": 1.143727161997564,
337
+ "step": 470
338
+ },
339
+ {
340
+ "loss": 0.0227,
341
+ "grad_norm": 0.30849921703338623,
342
+ "learning_rate": 8.118274779842888e-06,
343
+ "epoch": 1.1680876979293544,
344
+ "step": 480
345
+ },
346
+ {
347
+ "loss": 0.0232,
348
+ "grad_norm": 0.3520543873310089,
349
+ "learning_rate": 7.724334573094101e-06,
350
+ "epoch": 1.1924482338611448,
351
+ "step": 490
352
+ },
353
+ {
354
+ "loss": 0.022,
355
+ "grad_norm": 0.4163361191749573,
356
+ "learning_rate": 7.334085503989547e-06,
357
+ "epoch": 1.2168087697929355,
358
+ "step": 500
359
+ },
360
+ {
361
+ "loss": 0.024,
362
+ "grad_norm": 0.3560805320739746,
363
+ "learning_rate": 6.948160557946404e-06,
364
+ "epoch": 1.241169305724726,
365
+ "step": 510
366
+ },
367
+ {
368
+ "loss": 0.0222,
369
+ "grad_norm": 0.3097788393497467,
370
+ "learning_rate": 6.567185706638417e-06,
371
+ "epoch": 1.2655298416565164,
372
+ "step": 520
373
+ },
374
+ {
375
+ "loss": 0.023,
376
+ "grad_norm": 0.3562930226325989,
377
+ "learning_rate": 6.191778892667591e-06,
378
+ "epoch": 1.289890377588307,
379
+ "step": 530
380
+ },
381
+ {
382
+ "loss": 0.0219,
383
+ "grad_norm": 0.3165253698825836,
384
+ "learning_rate": 5.822549027258994e-06,
385
+ "epoch": 1.3142509135200975,
386
+ "step": 540
387
+ },
388
+ {
389
+ "loss": 0.0224,
390
+ "grad_norm": 0.3553745746612549,
391
+ "learning_rate": 5.460095002604533e-06,
392
+ "epoch": 1.338611449451888,
393
+ "step": 550
394
+ },
395
+ {
396
+ "loss": 0.0211,
397
+ "grad_norm": 0.24926869571208954,
398
+ "learning_rate": 5.105004720457654e-06,
399
+ "epoch": 1.3629719853836784,
400
+ "step": 560
401
+ },
402
+ {
403
+ "loss": 0.0235,
404
+ "grad_norm": 0.3247683346271515,
405
+ "learning_rate": 4.757854138554502e-06,
406
+ "epoch": 1.3873325213154688,
407
+ "step": 570
408
+ },
409
+ {
410
+ "loss": 0.0223,
411
+ "grad_norm": 0.38559386134147644,
412
+ "learning_rate": 4.419206336408418e-06,
413
+ "epoch": 1.4116930572472595,
414
+ "step": 580
415
+ },
416
+ {
417
+ "loss": 0.0223,
418
+ "grad_norm": 0.33405324816703796,
419
+ "learning_rate": 4.089610601992864e-06,
420
+ "epoch": 1.43605359317905,
421
+ "step": 590
422
+ },
423
+ {
424
+ "loss": 0.0206,
425
+ "grad_norm": 0.3149104118347168,
426
+ "learning_rate": 3.769601540794344e-06,
427
+ "epoch": 1.4604141291108403,
428
+ "step": 600
429
+ },
430
+ {
431
+ "loss": 0.0218,
432
+ "grad_norm": 0.33641424775123596,
433
+ "learning_rate": 3.459698208680359e-06,
434
+ "epoch": 1.484774665042631,
435
+ "step": 610
436
+ },
437
+ {
438
+ "loss": 0.023,
439
+ "grad_norm": 0.3544023633003235,
440
+ "learning_rate": 3.160403269988864e-06,
441
+ "epoch": 1.5091352009744214,
442
+ "step": 620
443
+ },
444
+ {
445
+ "loss": 0.0225,
446
+ "grad_norm": 0.38438504934310913,
447
+ "learning_rate": 2.872202182204903e-06,
448
+ "epoch": 1.5334957369062119,
449
+ "step": 630
450
+ },
451
+ {
452
+ "loss": 0.0232,
453
+ "grad_norm": 0.34402596950531006,
454
+ "learning_rate": 2.59556240854677e-06,
455
+ "epoch": 1.5578562728380025,
456
+ "step": 640
457
+ },
458
+ {
459
+ "loss": 0.0217,
460
+ "grad_norm": 0.3061605989933014,
461
+ "learning_rate": 2.3309326597389668e-06,
462
+ "epoch": 1.582216808769793,
463
+ "step": 650
464
+ },
465
+ {
466
+ "loss": 0.024,
467
+ "grad_norm": 0.3494504392147064,
468
+ "learning_rate": 2.0787421662017827e-06,
469
+ "epoch": 1.6065773447015834,
470
+ "step": 660
471
+ },
472
+ {
473
+ "loss": 0.0221,
474
+ "grad_norm": 0.3283305764198303,
475
+ "learning_rate": 1.8393999818379527e-06,
476
+ "epoch": 1.630937880633374,
477
+ "step": 670
478
+ },
479
+ {
480
+ "loss": 0.0223,
481
+ "grad_norm": 0.32319262623786926,
482
+ "learning_rate": 1.6132943205457607e-06,
483
+ "epoch": 1.6552984165651643,
484
+ "step": 680
485
+ },
486
+ {
487
+ "loss": 0.0234,
488
+ "grad_norm": 0.33977144956588745,
489
+ "learning_rate": 1.400791926534657e-06,
490
+ "epoch": 1.679658952496955,
491
+ "step": 690
492
+ },
493
+ {
494
+ "loss": 0.0209,
495
+ "grad_norm": 0.3605083227157593,
496
+ "learning_rate": 1.2022374794648229e-06,
497
+ "epoch": 1.7040194884287454,
498
+ "step": 700
499
+ },
500
+ {
501
+ "loss": 0.0231,
502
+ "grad_norm": 0.32001832127571106,
503
+ "learning_rate": 1.0179530353754874e-06,
504
+ "epoch": 1.7283800243605358,
505
+ "step": 710
506
+ },
507
+ {
508
+ "loss": 0.0219,
509
+ "grad_norm": 0.3382965922355652,
510
+ "learning_rate": 8.482375043088664e-07,
511
+ "epoch": 1.7527405602923265,
512
+ "step": 720
513
+ },
514
+ {
515
+ "loss": 0.0234,
516
+ "grad_norm": 0.3338392674922943,
517
+ "learning_rate": 6.933661654769797e-07,
518
+ "epoch": 1.777101096224117,
519
+ "step": 730
520
+ },
521
+ {
522
+ "loss": 0.0223,
523
+ "grad_norm": 0.33180609345436096,
524
+ "learning_rate": 5.535902207577515e-07,
525
+ "epoch": 1.8014616321559074,
526
+ "step": 740
527
+ },
528
+ {
529
+ "loss": 0.0246,
530
+ "grad_norm": 0.34170418977737427,
531
+ "learning_rate": 4.2913638724465966e-07,
532
+ "epoch": 1.825822168087698,
533
+ "step": 750
534
+ },
535
+ {
536
+ "loss": 0.0225,
537
+ "grad_norm": 0.38255947828292847,
538
+ "learning_rate": 3.2020652951077256e-07,
539
+ "epoch": 1.8501827040194885,
540
+ "step": 760
541
+ },
542
+ {
543
+ "loss": 0.022,
544
+ "grad_norm": 0.384766161441803,
545
+ "learning_rate": 2.2697733218367435e-07,
546
+ "epoch": 1.874543239951279,
547
+ "step": 770
548
+ },
549
+ {
550
+ "loss": 0.023,
551
+ "grad_norm": 0.29543358087539673,
552
+ "learning_rate": 1.4960001336235875e-07,
553
+ "epoch": 1.8989037758830696,
554
+ "step": 780
555
+ },
556
+ {
557
+ "loss": 0.0221,
558
+ "grad_norm": 0.33598384261131287,
559
+ "learning_rate": 8.82000793409088e-08,
560
+ "epoch": 1.9232643118148598,
561
+ "step": 790
562
+ },
563
+ {
564
+ "loss": 0.0237,
565
+ "grad_norm": 0.31653428077697754,
566
+ "learning_rate": 4.287712103684061e-08,
567
+ "epoch": 1.9476248477466505,
568
+ "step": 800
569
+ },
570
+ {
571
+ "loss": 0.0238,
572
+ "grad_norm": 0.3918892443180084,
573
+ "learning_rate": 1.370465245426167e-08,
574
+ "epoch": 1.971985383678441,
575
+ "step": 810
576
+ },
577
+ {
578
+ "loss": 0.0216,
579
+ "grad_norm": 0.3479089140892029,
580
+ "learning_rate": 7.299914438929634e-10,
581
+ "epoch": 1.9963459196102313,
582
+ "step": 820
583
+ },
584
+ {
585
+ "eval_loss": 0.02881857380270958,
586
+ "eval_runtime": 36.179,
587
+ "eval_samples_per_second": 20.177,
588
+ "eval_steps_per_second": 5.058,
589
+ "epoch": 2.0,
590
+ "step": 822
591
+ },
592
+ {
593
+ "train_runtime": 2204.02,
594
+ "train_samples_per_second": 5.959,
595
+ "train_steps_per_second": 0.373,
596
+ "total_flos": 6.007674419658648e+17,
597
+ "train_loss": 0.05371565775098302,
598
+ "epoch": 2.0,
599
+ "step": 822
600
+ }
601
+ ]
lamp-gemma-4b-v2/training_metrics.csv ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ step,train_loss,eval_loss,learning_rate,vram_gb
2
+ 10,1.5717,,4.2857142857142855e-06,24.31
3
+ 20,0.5627,,9.047619047619049e-06,24.31
4
+ 30,0.0742,,1.3809523809523811e-05,24.31
5
+ 40,0.0563,,1.8571428571428575e-05,24.31
6
+ 50,0.0459,,1.999602581704666e-05,24.31
7
+ 60,0.0409,,1.997656803510789e-05,24.31
8
+ 70,0.0394,,1.9940928223333254e-05,24.31
9
+ 80,0.0385,,1.988916418962778e-05,24.31
10
+ 90,0.0398,,1.9821359895443436e-05,24.31
11
+ 100,0.0352,,1.9737625319593338e-05,24.31
12
+ 110,0.0337,,1.9638096279865717e-05,24.31
13
+ 120,0.035,,1.9522934212726977e-05,24.31
14
+ 130,0.0348,,1.9392325911471154e-05,24.31
15
+ 140,0.0329,,1.9246483223240496e-05,24.31
16
+ 150,0.0325,,1.908564270540864e-05,24.31
17
+ 160,0.0333,,1.891006524188368e-05,24.31
18
+ 170,0.0349,,1.8720035619953517e-05,24.31
19
+ 180,0.0338,,1.8515862068359837e-05,24.31
20
+ 190,0.0351,,1.829787575734995e-05,24.31
21
+ 200,0.032,,1.806643026151747e-05,24.31
22
+ 210,0.0321,,1.7821900986302995e-05,24.31
23
+ 220,0.0319,,1.7564684559085138e-05,24.31
24
+ 230,0.0305,,1.729519818584944e-05,24.31
25
+ 240,0.03,,1.7013878974478778e-05,24.31
26
+ 250,0.0319,,1.6721183225762726e-05,24.31
27
+ 260,0.0318,,1.641758569327606e-05,24.31
28
+ 270,0.0323,,1.6103578813326683e-05,24.31
29
+ 280,0.0335,,1.577967190622215e-05,24.31
30
+ 290,0.0303,,1.5446390350150272e-05,24.31
31
+ 300,0.0309,,1.5104274729013759e-05,24.31
32
+ 310,0.0302,,1.4753879955601162e-05,24.31
33
+ 320,0.0295,,1.4395774371516305e-05,24.31
34
+ 330,0.0287,,1.4030538825326115e-05,24.31
35
+ 340,0.029,,1.3658765730422126e-05,24.31
36
+ 350,0.0303,,1.3281058104123794e-05,24.31
37
+ 360,0.0292,,1.2898028589582202e-05,24.31
38
+ 370,0.0275,,1.2510298462070619e-05,24.31
39
+ 380,0.0307,,1.2118496621273745e-05,24.31
40
+ 390,0.031,,1.172325857121012e-05,24.31
41
+ 400,0.0274,,1.1325225389442278e-05,24.31
42
+ 410,0.0287,,1.0925042687246592e-05,24.31
43
+ 411,,0.029586488381028175,,24.31
44
+ 420,0.0245,,1.0523359562429441e-05,24.31
45
+ 430,0.0241,,1.0120827546488175e-05,24.31
46
+ 440,0.024,,9.718099547824657e-06,24.31
47
+ 450,0.0229,,9.315828792725438e-06,24.31
48
+ 460,0.0233,,8.91466776582634e-06,24.31
49
+ 470,0.0226,,8.515267151779974e-06,24.31
50
+ 480,0.0227,,8.118274779842888e-06,24.31
51
+ 490,0.0232,,7.724334573094101e-06,24.31
52
+ 500,0.022,,7.334085503989547e-06,24.31
53
+ 510,0.024,,6.948160557946404e-06,24.31
54
+ 520,0.0222,,6.567185706638417e-06,24.31
55
+ 530,0.023,,6.191778892667591e-06,24.31
56
+ 540,0.0219,,5.822549027258994e-06,24.31
57
+ 550,0.0224,,5.460095002604533e-06,24.31
58
+ 560,0.0211,,5.105004720457654e-06,24.31
59
+ 570,0.0235,,4.757854138554502e-06,24.31
60
+ 580,0.0223,,4.419206336408418e-06,24.31
61
+ 590,0.0223,,4.089610601992864e-06,24.31
62
+ 600,0.0206,,3.769601540794344e-06,24.31
63
+ 610,0.0218,,3.459698208680359e-06,24.31
64
+ 620,0.023,,3.160403269988864e-06,24.31
65
+ 630,0.0225,,2.872202182204903e-06,24.31
66
+ 640,0.0232,,2.59556240854677e-06,24.31
67
+ 650,0.0217,,2.3309326597389668e-06,24.31
68
+ 660,0.024,,2.0787421662017827e-06,24.31
69
+ 670,0.0221,,1.8393999818379527e-06,24.31
70
+ 680,0.0223,,1.6132943205457607e-06,24.31
71
+ 690,0.0234,,1.400791926534657e-06,24.31
72
+ 700,0.0209,,1.2022374794648229e-06,24.31
73
+ 710,0.0231,,1.0179530353754874e-06,24.31
74
+ 720,0.0219,,8.482375043088664e-07,24.31
75
+ 730,0.0234,,6.933661654769797e-07,24.31
76
+ 740,0.0223,,5.535902207577515e-07,24.31
77
+ 750,0.0246,,4.2913638724465966e-07,24.31
78
+ 760,0.0225,,3.2020652951077256e-07,24.31
79
+ 770,0.022,,2.2697733218367435e-07,24.31
80
+ 780,0.023,,1.4960001336235875e-07,24.31
81
+ 790,0.0221,,8.82000793409088e-08,24.31
82
+ 800,0.0237,,4.287712103684061e-08,24.31
83
+ 810,0.0238,,1.370465245426167e-08,24.31
84
+ 820,0.0216,,7.299914438929634e-10,24.31
85
+ 822,,0.02881857380270958,,24.31