ziyjiang wl1982 commited on
Commit
9ddaa07
·
1 Parent(s): a1728b7

Upload Ovis-VL-Embedding-2B.json (#187)

Browse files

- Upload Ovis-VL-Embedding-2B.json (df824933c69f35c8f023a440dd935bf8f8db1682)


Co-authored-by: asha <wl1982@users.noreply.huggingface.co>

Files changed (1) hide show
  1. scores/Ovis-VL-Embedding-2B.json +2047 -0
scores/Ovis-VL-Embedding-2B.json ADDED
@@ -0,0 +1,2047 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "model_name": "Ovis-VL-Embedding-2B",
4
+ "model_size": 2,
5
+ "model_release_date": "2026-09-11",
6
+ "data_source": "",
7
+ "url": "https://github.com/ATH-MaaS/Ovis-VL-Embedding",
8
+ "contact": "",
9
+ "report_generated_date": "2026-09-11T14:50:16.153786"
10
+ },
11
+ "metrics": {
12
+ "image": {
13
+ "ImageNet-1K": {
14
+ "hit@1": 0.844,
15
+ "hit@5": 0.974,
16
+ "hit@10": 0.991,
17
+ "ndcg_linear@1": 0.844,
18
+ "ndcg_linear@5": 0.9182711058852837,
19
+ "ndcg_linear@10": 0.9237460396846193,
20
+ "ndcg_exponential@1": 0.844,
21
+ "ndcg_exponential@5": 0.9182711058852837,
22
+ "ndcg_exponential@10": 0.9237460396846193,
23
+ "precision@1": 0.844,
24
+ "precision@5": 0.1948,
25
+ "precision@10": 0.09910000000000002,
26
+ "recall@1": 0.844,
27
+ "recall@5": 0.974,
28
+ "recall@10": 0.991,
29
+ "f1@1": 0.844,
30
+ "f1@5": 0.3246666666666666,
31
+ "f1@10": 0.1801818181818182,
32
+ "map@1": 0.844,
33
+ "map@5": 0.8993,
34
+ "map@10": 0.9015436507936508,
35
+ "mrr": 0.9019756314004272,
36
+ "num_pred": 1000,
37
+ "num_data": 1000
38
+ },
39
+ "N24News": {
40
+ "hit@1": 0.806,
41
+ "hit@5": 0.972,
42
+ "hit@10": 0.989,
43
+ "ndcg_linear@1": 0.806,
44
+ "ndcg_linear@5": 0.8996360994036589,
45
+ "ndcg_linear@10": 0.9054157622748665,
46
+ "ndcg_exponential@1": 0.806,
47
+ "ndcg_exponential@5": 0.8996360994036589,
48
+ "ndcg_exponential@10": 0.9054157622748665,
49
+ "precision@1": 0.806,
50
+ "precision@5": 0.19440000000000004,
51
+ "precision@10": 0.0989,
52
+ "recall@1": 0.806,
53
+ "recall@5": 0.972,
54
+ "recall@10": 0.989,
55
+ "f1@1": 0.806,
56
+ "f1@5": 0.324,
57
+ "f1@10": 0.17981818181818182,
58
+ "map@1": 0.806,
59
+ "map@5": 0.8751,
60
+ "map@10": 0.877652380952381,
61
+ "mrr": 0.8785059024309024,
62
+ "num_pred": 1000,
63
+ "num_data": 1000
64
+ },
65
+ "HatefulMemes": {
66
+ "hit@1": 0.672,
67
+ "hit@5": 1.0,
68
+ "hit@10": 1.0,
69
+ "ndcg_linear@1": 0.672,
70
+ "ndcg_linear@5": 0.8789449591714382,
71
+ "ndcg_linear@10": 0.8789449591714382,
72
+ "ndcg_exponential@1": 0.672,
73
+ "ndcg_exponential@5": 0.8789449591714382,
74
+ "ndcg_exponential@10": 0.8789449591714382,
75
+ "precision@1": 0.672,
76
+ "precision@5": 0.20000000000000004,
77
+ "precision@10": 0.10000000000000002,
78
+ "recall@1": 0.672,
79
+ "recall@5": 1.0,
80
+ "recall@10": 1.0,
81
+ "f1@1": 0.672,
82
+ "f1@5": 0.3333333333333333,
83
+ "f1@10": 0.18181818181818182,
84
+ "map@1": 0.672,
85
+ "map@5": 0.836,
86
+ "map@10": 0.836,
87
+ "mrr": 0.836,
88
+ "num_pred": 1000,
89
+ "num_data": 1000
90
+ },
91
+ "VOC2007": {
92
+ "hit@1": 0.949,
93
+ "hit@5": 0.994,
94
+ "hit@10": 0.997,
95
+ "ndcg_linear@1": 0.949,
96
+ "ndcg_linear@5": 0.9753070282840965,
97
+ "ndcg_linear@10": 0.9763527759916459,
98
+ "ndcg_exponential@1": 0.949,
99
+ "ndcg_exponential@5": 0.9753070282840965,
100
+ "ndcg_exponential@10": 0.9763527759916459,
101
+ "precision@1": 0.949,
102
+ "precision@5": 0.1988,
103
+ "precision@10": 0.09970000000000001,
104
+ "recall@1": 0.949,
105
+ "recall@5": 0.994,
106
+ "recall@10": 0.997,
107
+ "f1@1": 0.949,
108
+ "f1@5": 0.33133333333333337,
109
+ "f1@10": 0.18127272727272725,
110
+ "map@1": 0.949,
111
+ "map@5": 0.9688666666666667,
112
+ "map@10": 0.9693428571428572,
113
+ "mrr": 0.9695834988540871,
114
+ "num_pred": 1000,
115
+ "num_data": 1000
116
+ },
117
+ "SUN397": {
118
+ "hit@1": 0.844,
119
+ "hit@5": 0.981,
120
+ "hit@10": 0.993,
121
+ "ndcg_linear@1": 0.844,
122
+ "ndcg_linear@5": 0.9213932093861877,
123
+ "ndcg_linear@10": 0.9253102590799435,
124
+ "ndcg_exponential@1": 0.844,
125
+ "ndcg_exponential@5": 0.9213932093861877,
126
+ "ndcg_exponential@10": 0.9253102590799435,
127
+ "precision@1": 0.844,
128
+ "precision@5": 0.1962,
129
+ "precision@10": 0.09930000000000001,
130
+ "recall@1": 0.844,
131
+ "recall@5": 0.981,
132
+ "recall@10": 0.993,
133
+ "f1@1": 0.844,
134
+ "f1@5": 0.327,
135
+ "f1@10": 0.1805454545454545,
136
+ "map@1": 0.844,
137
+ "map@5": 0.90115,
138
+ "map@10": 0.9027884920634921,
139
+ "mrr": 0.9031118589548166,
140
+ "num_pred": 1000,
141
+ "num_data": 1000
142
+ },
143
+ "Place365": {
144
+ "hit@1": 0.536,
145
+ "hit@5": 0.844,
146
+ "hit@10": 0.909,
147
+ "ndcg_linear@1": 0.536,
148
+ "ndcg_linear@5": 0.7016195552374398,
149
+ "ndcg_linear@10": 0.7228315066836783,
150
+ "ndcg_exponential@1": 0.536,
151
+ "ndcg_exponential@5": 0.7016195552374398,
152
+ "ndcg_exponential@10": 0.7228315066836783,
153
+ "precision@1": 0.536,
154
+ "precision@5": 0.1688,
155
+ "precision@10": 0.09090000000000001,
156
+ "recall@1": 0.536,
157
+ "recall@5": 0.844,
158
+ "recall@10": 0.909,
159
+ "f1@1": 0.536,
160
+ "f1@5": 0.2813333333333334,
161
+ "f1@10": 0.16527272727272727,
162
+ "map@1": 0.536,
163
+ "map@5": 0.6541,
164
+ "map@10": 0.6629718253968253,
165
+ "mrr": 0.6676609400012682,
166
+ "num_pred": 1000,
167
+ "num_data": 1000
168
+ },
169
+ "ImageNet-A": {
170
+ "hit@1": 0.825,
171
+ "hit@5": 0.96,
172
+ "hit@10": 0.979,
173
+ "ndcg_linear@1": 0.825,
174
+ "ndcg_linear@5": 0.9024950780950676,
175
+ "ndcg_linear@10": 0.9088189954594407,
176
+ "ndcg_exponential@1": 0.825,
177
+ "ndcg_exponential@5": 0.9024950780950676,
178
+ "ndcg_exponential@10": 0.9088189954594407,
179
+ "precision@1": 0.825,
180
+ "precision@5": 0.19200000000000003,
181
+ "precision@10": 0.0979,
182
+ "recall@1": 0.825,
183
+ "recall@5": 0.96,
184
+ "recall@10": 0.979,
185
+ "f1@1": 0.825,
186
+ "f1@5": 0.32,
187
+ "f1@10": 0.178,
188
+ "map@1": 0.825,
189
+ "map@5": 0.8828833333333335,
190
+ "map@10": 0.8856019841269841,
191
+ "mrr": 0.886322079097992,
192
+ "num_pred": 1000,
193
+ "num_data": 1000
194
+ },
195
+ "ImageNet-R": {
196
+ "hit@1": 0.99,
197
+ "hit@5": 0.998,
198
+ "hit@10": 0.999,
199
+ "ndcg_linear@1": 0.99,
200
+ "ndcg_linear@5": 0.9948033610822347,
201
+ "ndcg_linear@10": 0.9950924259085526,
202
+ "ndcg_exponential@1": 0.99,
203
+ "ndcg_exponential@5": 0.9948033610822347,
204
+ "ndcg_exponential@10": 0.9950924259085526,
205
+ "precision@1": 0.99,
206
+ "precision@5": 0.19960000000000003,
207
+ "precision@10": 0.0999,
208
+ "recall@1": 0.99,
209
+ "recall@5": 0.998,
210
+ "recall@10": 0.999,
211
+ "f1@1": 0.99,
212
+ "f1@5": 0.3326666666666666,
213
+ "f1@10": 0.18163636363636362,
214
+ "map@1": 0.99,
215
+ "map@5": 0.9937,
216
+ "map@10": 0.9937999999999999,
217
+ "mrr": 0.9938238095238096,
218
+ "num_pred": 1000,
219
+ "num_data": 1000
220
+ },
221
+ "ObjectNet": {
222
+ "hit@1": 0.898,
223
+ "hit@5": 0.984,
224
+ "hit@10": 0.992,
225
+ "ndcg_linear@1": 0.898,
226
+ "ndcg_linear@5": 0.9478663914686188,
227
+ "ndcg_linear@10": 0.9505420784184979,
228
+ "ndcg_exponential@1": 0.898,
229
+ "ndcg_exponential@5": 0.9478663914686188,
230
+ "ndcg_exponential@10": 0.9505420784184979,
231
+ "precision@1": 0.898,
232
+ "precision@5": 0.1968,
233
+ "precision@10": 0.09920000000000001,
234
+ "recall@1": 0.898,
235
+ "recall@5": 0.984,
236
+ "recall@10": 0.992,
237
+ "f1@1": 0.898,
238
+ "f1@5": 0.328,
239
+ "f1@10": 0.18036363636363634,
240
+ "map@1": 0.898,
241
+ "map@5": 0.9355,
242
+ "map@10": 0.936656746031746,
243
+ "mrr": 0.9370399992036949,
244
+ "num_pred": 1000,
245
+ "num_data": 1000
246
+ },
247
+ "Country211": {
248
+ "hit@1": 0.323,
249
+ "hit@5": 0.605,
250
+ "hit@10": 0.714,
251
+ "ndcg_linear@1": 0.323,
252
+ "ndcg_linear@5": 0.47224088133566955,
253
+ "ndcg_linear@10": 0.507676921125093,
254
+ "ndcg_exponential@1": 0.323,
255
+ "ndcg_exponential@5": 0.47224088133566955,
256
+ "ndcg_exponential@10": 0.507676921125093,
257
+ "precision@1": 0.323,
258
+ "precision@5": 0.121,
259
+ "precision@10": 0.0714,
260
+ "recall@1": 0.323,
261
+ "recall@5": 0.605,
262
+ "recall@10": 0.714,
263
+ "f1@1": 0.323,
264
+ "f1@5": 0.20166666666666674,
265
+ "f1@10": 0.1298181818181818,
266
+ "map@1": 0.323,
267
+ "map@5": 0.4281333333333333,
268
+ "map@10": 0.44287023809523807,
269
+ "mrr": 0.45413581524330265,
270
+ "num_pred": 1000,
271
+ "num_data": 1000
272
+ },
273
+ "OK-VQA": {
274
+ "hit@1": 0.699,
275
+ "hit@5": 0.906,
276
+ "hit@10": 0.944,
277
+ "ndcg_linear@1": 0.699,
278
+ "ndcg_linear@5": 0.813266898426621,
279
+ "ndcg_linear@10": 0.8256544311854532,
280
+ "ndcg_exponential@1": 0.699,
281
+ "ndcg_exponential@5": 0.813266898426621,
282
+ "ndcg_exponential@10": 0.8256544311854532,
283
+ "precision@1": 0.699,
284
+ "precision@5": 0.1812,
285
+ "precision@10": 0.09440000000000001,
286
+ "recall@1": 0.699,
287
+ "recall@5": 0.906,
288
+ "recall@10": 0.944,
289
+ "f1@1": 0.699,
290
+ "f1@5": 0.302,
291
+ "f1@10": 0.17163636363636361,
292
+ "map@1": 0.699,
293
+ "map@5": 0.7820166666666667,
294
+ "map@10": 0.7871857142857142,
295
+ "mrr": 0.7899532341100536,
296
+ "num_pred": 1000,
297
+ "num_data": 1000
298
+ },
299
+ "A-OKVQA": {
300
+ "hit@1": 0.792,
301
+ "hit@5": 0.969,
302
+ "hit@10": 0.986,
303
+ "ndcg_linear@1": 0.792,
304
+ "ndcg_linear@5": 0.8908451781766863,
305
+ "ndcg_linear@10": 0.8963139574726099,
306
+ "ndcg_exponential@1": 0.792,
307
+ "ndcg_exponential@5": 0.8908451781766863,
308
+ "ndcg_exponential@10": 0.8963139574726099,
309
+ "precision@1": 0.792,
310
+ "precision@5": 0.19380000000000003,
311
+ "precision@10": 0.09860000000000002,
312
+ "recall@1": 0.792,
313
+ "recall@5": 0.969,
314
+ "recall@10": 0.986,
315
+ "f1@1": 0.792,
316
+ "f1@5": 0.323,
317
+ "f1@10": 0.17927272727272725,
318
+ "map@1": 0.792,
319
+ "map@5": 0.8644833333333334,
320
+ "map@10": 0.8667253968253968,
321
+ "mrr": 0.8674301498708,
322
+ "num_pred": 1000,
323
+ "num_data": 1000
324
+ },
325
+ "DocVQA": {
326
+ "hit@1": 0.965,
327
+ "hit@5": 1.0,
328
+ "hit@10": 1.0,
329
+ "ndcg_linear@1": 0.965,
330
+ "ndcg_linear@5": 0.9859835624653087,
331
+ "ndcg_linear@10": 0.9859835624653087,
332
+ "ndcg_exponential@1": 0.965,
333
+ "ndcg_exponential@5": 0.9859835624653087,
334
+ "ndcg_exponential@10": 0.9859835624653087,
335
+ "precision@1": 0.965,
336
+ "precision@5": 0.20000000000000004,
337
+ "precision@10": 0.10000000000000002,
338
+ "recall@1": 0.965,
339
+ "recall@5": 1.0,
340
+ "recall@10": 1.0,
341
+ "f1@1": 0.965,
342
+ "f1@5": 0.3333333333333333,
343
+ "f1@10": 0.18181818181818182,
344
+ "map@1": 0.965,
345
+ "map@5": 0.9811166666666665,
346
+ "map@10": 0.9811166666666665,
347
+ "mrr": 0.9811166666666665,
348
+ "num_pred": 1000,
349
+ "num_data": 1000
350
+ },
351
+ "InfographicsVQA": {
352
+ "hit@1": 0.844,
353
+ "hit@5": 0.947,
354
+ "hit@10": 0.972,
355
+ "ndcg_linear@1": 0.844,
356
+ "ndcg_linear@5": 0.902638316406286,
357
+ "ndcg_linear@10": 0.9107326108578717,
358
+ "ndcg_exponential@1": 0.844,
359
+ "ndcg_exponential@5": 0.902638316406286,
360
+ "ndcg_exponential@10": 0.9107326108578717,
361
+ "precision@1": 0.844,
362
+ "precision@5": 0.18940000000000004,
363
+ "precision@10": 0.09720000000000002,
364
+ "recall@1": 0.844,
365
+ "recall@5": 0.947,
366
+ "recall@10": 0.972,
367
+ "f1@1": 0.844,
368
+ "f1@5": 0.3156666666666667,
369
+ "f1@10": 0.17672727272727273,
370
+ "map@1": 0.844,
371
+ "map@5": 0.8875333333333334,
372
+ "map@10": 0.8908825396825397,
373
+ "mrr": 0.8920316554824581,
374
+ "num_pred": 1000,
375
+ "num_data": 1000
376
+ },
377
+ "ChartQA": {
378
+ "hit@1": 0.775,
379
+ "hit@5": 0.904,
380
+ "hit@10": 0.936,
381
+ "ndcg_linear@1": 0.775,
382
+ "ndcg_linear@5": 0.8468373558732847,
383
+ "ndcg_linear@10": 0.8572437475999061,
384
+ "ndcg_exponential@1": 0.775,
385
+ "ndcg_exponential@5": 0.8468373558732847,
386
+ "ndcg_exponential@10": 0.8572437475999061,
387
+ "precision@1": 0.775,
388
+ "precision@5": 0.18080000000000002,
389
+ "precision@10": 0.09360000000000002,
390
+ "recall@1": 0.775,
391
+ "recall@5": 0.904,
392
+ "recall@10": 0.936,
393
+ "f1@1": 0.775,
394
+ "f1@5": 0.3013333333333334,
395
+ "f1@10": 0.17018181818181818,
396
+ "map@1": 0.775,
397
+ "map@5": 0.8275666666666666,
398
+ "map@10": 0.8318944444444444,
399
+ "mrr": 0.834309637896972,
400
+ "num_pred": 1000,
401
+ "num_data": 1000
402
+ },
403
+ "Visual7W": {
404
+ "hit@1": 0.682,
405
+ "hit@5": 0.937,
406
+ "hit@10": 0.976,
407
+ "ndcg_linear@1": 0.682,
408
+ "ndcg_linear@5": 0.8253092042040207,
409
+ "ndcg_linear@10": 0.8381285521309954,
410
+ "ndcg_exponential@1": 0.682,
411
+ "ndcg_exponential@5": 0.8253092042040207,
412
+ "ndcg_exponential@10": 0.8381285521309954,
413
+ "precision@1": 0.682,
414
+ "precision@5": 0.1874,
415
+ "precision@10": 0.09760000000000002,
416
+ "recall@1": 0.682,
417
+ "recall@5": 0.937,
418
+ "recall@10": 0.976,
419
+ "f1@1": 0.682,
420
+ "f1@5": 0.3123333333333333,
421
+ "f1@10": 0.17745454545454545,
422
+ "map@1": 0.682,
423
+ "map@5": 0.7874000000000001,
424
+ "map@10": 0.7928091269841269,
425
+ "mrr": 0.7944490171219577,
426
+ "num_pred": 1000,
427
+ "num_data": 1000
428
+ },
429
+ "ScienceQA": {
430
+ "hit@1": 0.812,
431
+ "hit@5": 0.964,
432
+ "hit@10": 0.992,
433
+ "ndcg_linear@1": 0.812,
434
+ "ndcg_linear@5": 0.8963072118307027,
435
+ "ndcg_linear@10": 0.90556908485278,
436
+ "ndcg_exponential@1": 0.812,
437
+ "ndcg_exponential@5": 0.8963072118307027,
438
+ "ndcg_exponential@10": 0.90556908485278,
439
+ "precision@1": 0.812,
440
+ "precision@5": 0.1928,
441
+ "precision@10": 0.09920000000000001,
442
+ "recall@1": 0.812,
443
+ "recall@5": 0.964,
444
+ "recall@10": 0.992,
445
+ "f1@1": 0.812,
446
+ "f1@5": 0.3213333333333333,
447
+ "f1@10": 0.1803636363636363,
448
+ "map@1": 0.812,
449
+ "map@5": 0.8735,
450
+ "map@10": 0.8774408730158729,
451
+ "mrr": 0.8778529149448191,
452
+ "num_pred": 1000,
453
+ "num_data": 1000
454
+ },
455
+ "VizWiz": {
456
+ "hit@1": 0.721,
457
+ "hit@5": 0.838,
458
+ "hit@10": 0.88,
459
+ "ndcg_linear@1": 0.721,
460
+ "ndcg_linear@5": 0.7863728948704011,
461
+ "ndcg_linear@10": 0.7998954775237987,
462
+ "ndcg_exponential@1": 0.721,
463
+ "ndcg_exponential@5": 0.7863728948704011,
464
+ "ndcg_exponential@10": 0.7998954775237987,
465
+ "precision@1": 0.721,
466
+ "precision@5": 0.1676,
467
+ "precision@10": 0.088,
468
+ "recall@1": 0.721,
469
+ "recall@5": 0.838,
470
+ "recall@10": 0.88,
471
+ "f1@1": 0.721,
472
+ "f1@5": 0.2793333333333334,
473
+ "f1@10": 0.16,
474
+ "map@1": 0.721,
475
+ "map@5": 0.7689499999999999,
476
+ "map@10": 0.7744944444444444,
477
+ "mrr": 0.7792803368501767,
478
+ "num_pred": 1000,
479
+ "num_data": 1000
480
+ },
481
+ "GQA": {
482
+ "hit@1": 0.944,
483
+ "hit@5": 0.998,
484
+ "hit@10": 1.0,
485
+ "ndcg_linear@1": 0.944,
486
+ "ndcg_linear@5": 0.9758905729332252,
487
+ "ndcg_linear@10": 0.9766029873074413,
488
+ "ndcg_exponential@1": 0.944,
489
+ "ndcg_exponential@5": 0.9758905729332252,
490
+ "ndcg_exponential@10": 0.9766029873074413,
491
+ "precision@1": 0.944,
492
+ "precision@5": 0.19960000000000003,
493
+ "precision@10": 0.10000000000000002,
494
+ "recall@1": 0.944,
495
+ "recall@5": 0.998,
496
+ "recall@10": 1.0,
497
+ "f1@1": 0.944,
498
+ "f1@5": 0.3326666666666666,
499
+ "f1@10": 0.18181818181818182,
500
+ "map@1": 0.944,
501
+ "map@5": 0.96825,
502
+ "map@10": 0.9685833333333334,
503
+ "mrr": 0.9685833333333334,
504
+ "num_pred": 1000,
505
+ "num_data": 1000
506
+ },
507
+ "TextVQA": {
508
+ "hit@1": 0.971,
509
+ "hit@5": 0.991,
510
+ "hit@10": 0.997,
511
+ "ndcg_linear@1": 0.971,
512
+ "ndcg_linear@5": 0.9830871589268615,
513
+ "ndcg_linear@10": 0.984939020634719,
514
+ "ndcg_exponential@1": 0.971,
515
+ "ndcg_exponential@5": 0.9830871589268615,
516
+ "ndcg_exponential@10": 0.984939020634719,
517
+ "precision@1": 0.971,
518
+ "precision@5": 0.19820000000000002,
519
+ "precision@10": 0.09970000000000001,
520
+ "recall@1": 0.971,
521
+ "recall@5": 0.991,
522
+ "recall@10": 0.997,
523
+ "f1@1": 0.971,
524
+ "f1@5": 0.33033333333333337,
525
+ "f1@10": 0.18127272727272725,
526
+ "map@1": 0.971,
527
+ "map@5": 0.9803333333333333,
528
+ "map@10": 0.9810472222222222,
529
+ "mrr": 0.9812983877233878,
530
+ "num_pred": 1000,
531
+ "num_data": 1000
532
+ },
533
+ "VisDial": {
534
+ "hit@1": 0.897,
535
+ "hit@5": 0.979,
536
+ "hit@10": 0.992,
537
+ "ndcg_linear@1": 0.897,
538
+ "ndcg_linear@5": 0.9436086348140248,
539
+ "ndcg_linear@10": 0.9479933025626941,
540
+ "ndcg_exponential@1": 0.897,
541
+ "ndcg_exponential@5": 0.9436086348140248,
542
+ "ndcg_exponential@10": 0.9479933025626941,
543
+ "precision@1": 0.897,
544
+ "precision@5": 0.1958,
545
+ "precision@10": 0.09920000000000001,
546
+ "recall@1": 0.897,
547
+ "recall@5": 0.979,
548
+ "recall@10": 0.992,
549
+ "f1@1": 0.897,
550
+ "f1@5": 0.3263333333333333,
551
+ "f1@10": 0.1803636363636363,
552
+ "map@1": 0.897,
553
+ "map@5": 0.93155,
554
+ "map@10": 0.9334646825396825,
555
+ "mrr": 0.9341015123765124,
556
+ "num_pred": 1000,
557
+ "num_data": 1000
558
+ },
559
+ "CIRR": {
560
+ "hit@1": 0.671,
561
+ "hit@5": 0.925,
562
+ "hit@10": 0.971,
563
+ "ndcg_linear@1": 0.671,
564
+ "ndcg_linear@5": 0.8117962476240327,
565
+ "ndcg_linear@10": 0.8272013465445694,
566
+ "ndcg_exponential@1": 0.671,
567
+ "ndcg_exponential@5": 0.8117962476240327,
568
+ "ndcg_exponential@10": 0.8272013465445694,
569
+ "precision@1": 0.671,
570
+ "precision@5": 0.18500000000000003,
571
+ "precision@10": 0.0971,
572
+ "recall@1": 0.671,
573
+ "recall@5": 0.925,
574
+ "recall@10": 0.971,
575
+ "f1@1": 0.671,
576
+ "f1@5": 0.30833333333333335,
577
+ "f1@10": 0.1765454545454545,
578
+ "map@1": 0.671,
579
+ "map@5": 0.7736500000000001,
580
+ "map@10": 0.7803246031746032,
581
+ "mrr": 0.7815721178146946,
582
+ "num_pred": 1000,
583
+ "num_data": 1000
584
+ },
585
+ "VisualNews_t2i": {
586
+ "hit@1": 0.771,
587
+ "hit@5": 0.911,
588
+ "hit@10": 0.949,
589
+ "ndcg_linear@1": 0.771,
590
+ "ndcg_linear@5": 0.8484677441279109,
591
+ "ndcg_linear@10": 0.8609079183927358,
592
+ "ndcg_exponential@1": 0.771,
593
+ "ndcg_exponential@5": 0.8484677441279109,
594
+ "ndcg_exponential@10": 0.8609079183927358,
595
+ "precision@1": 0.771,
596
+ "precision@5": 0.1822,
597
+ "precision@10": 0.09490000000000001,
598
+ "recall@1": 0.771,
599
+ "recall@5": 0.911,
600
+ "recall@10": 0.949,
601
+ "f1@1": 0.771,
602
+ "f1@5": 0.30366666666666664,
603
+ "f1@10": 0.17254545454545456,
604
+ "map@1": 0.771,
605
+ "map@5": 0.8274166666666667,
606
+ "map@10": 0.8326380952380953,
607
+ "mrr": 0.8345641029497133,
608
+ "num_pred": 1000,
609
+ "num_data": 1000
610
+ },
611
+ "VisualNews_i2t": {
612
+ "hit@1": 0.823,
613
+ "hit@5": 0.937,
614
+ "hit@10": 0.96,
615
+ "ndcg_linear@1": 0.823,
616
+ "ndcg_linear@5": 0.8859213629727339,
617
+ "ndcg_linear@10": 0.8934758003702307,
618
+ "ndcg_exponential@1": 0.823,
619
+ "ndcg_exponential@5": 0.8859213629727339,
620
+ "ndcg_exponential@10": 0.8934758003702307,
621
+ "precision@1": 0.823,
622
+ "precision@5": 0.18740000000000004,
623
+ "precision@10": 0.09600000000000002,
624
+ "recall@1": 0.823,
625
+ "recall@5": 0.937,
626
+ "recall@10": 0.96,
627
+ "f1@1": 0.823,
628
+ "f1@5": 0.3123333333333333,
629
+ "f1@10": 0.17454545454545453,
630
+ "map@1": 0.823,
631
+ "map@5": 0.8687,
632
+ "map@10": 0.8718880952380953,
633
+ "mrr": 0.8733286120590936,
634
+ "num_pred": 1000,
635
+ "num_data": 1000
636
+ },
637
+ "MSCOCO_t2i": {
638
+ "hit@1": 0.832,
639
+ "hit@5": 0.965,
640
+ "hit@10": 0.989,
641
+ "ndcg_linear@1": 0.832,
642
+ "ndcg_linear@5": 0.904883657979837,
643
+ "ndcg_linear@10": 0.912859965724698,
644
+ "ndcg_exponential@1": 0.832,
645
+ "ndcg_exponential@5": 0.904883657979837,
646
+ "ndcg_exponential@10": 0.912859965724698,
647
+ "precision@1": 0.832,
648
+ "precision@5": 0.19300000000000003,
649
+ "precision@10": 0.0989,
650
+ "recall@1": 0.832,
651
+ "recall@5": 0.965,
652
+ "recall@10": 0.989,
653
+ "f1@1": 0.832,
654
+ "f1@5": 0.32166666666666666,
655
+ "f1@10": 0.1798181818181818,
656
+ "map@1": 0.832,
657
+ "map@5": 0.8847166666666667,
658
+ "map@10": 0.8881333333333334,
659
+ "mrr": 0.8887739927556104,
660
+ "num_pred": 1000,
661
+ "num_data": 1000
662
+ },
663
+ "MSCOCO_i2t": {
664
+ "hit@1": 0.782,
665
+ "hit@5": 0.961,
666
+ "hit@10": 0.986,
667
+ "ndcg_linear@1": 0.782,
668
+ "ndcg_linear@5": 0.8818492817043732,
669
+ "ndcg_linear@10": 0.8901450958595967,
670
+ "ndcg_exponential@1": 0.782,
671
+ "ndcg_exponential@5": 0.8818492817043732,
672
+ "ndcg_exponential@10": 0.8901450958595967,
673
+ "precision@1": 0.782,
674
+ "precision@5": 0.19220000000000004,
675
+ "precision@10": 0.09860000000000002,
676
+ "recall@1": 0.782,
677
+ "recall@5": 0.961,
678
+ "recall@10": 0.986,
679
+ "f1@1": 0.782,
680
+ "f1@5": 0.3203333333333333,
681
+ "f1@10": 0.17927272727272725,
682
+ "map@1": 0.782,
683
+ "map@5": 0.85505,
684
+ "map@10": 0.8585964285714285,
685
+ "mrr": 0.8593999682223262,
686
+ "num_pred": 1000,
687
+ "num_data": 1000
688
+ },
689
+ "NIGHTS": {
690
+ "hit@1": 0.705,
691
+ "hit@5": 0.988,
692
+ "hit@10": 0.996,
693
+ "ndcg_linear@1": 0.705,
694
+ "ndcg_linear@5": 0.868057699630398,
695
+ "ndcg_linear@10": 0.8707129824382781,
696
+ "ndcg_exponential@1": 0.705,
697
+ "ndcg_exponential@5": 0.868057699630398,
698
+ "ndcg_exponential@10": 0.8707129824382781,
699
+ "precision@1": 0.705,
700
+ "precision@5": 0.19760000000000003,
701
+ "precision@10": 0.09960000000000002,
702
+ "recall@1": 0.705,
703
+ "recall@5": 0.988,
704
+ "recall@10": 0.996,
705
+ "f1@1": 0.705,
706
+ "f1@5": 0.3293333333333333,
707
+ "f1@10": 0.18109090909090905,
708
+ "map@1": 0.705,
709
+ "map@5": 0.827,
710
+ "map@10": 0.8281357142857143,
711
+ "mrr": 0.8283506224712107,
712
+ "num_pred": 1000,
713
+ "num_data": 1000
714
+ },
715
+ "WebQA": {
716
+ "hit@1": 0.902,
717
+ "hit@5": 0.987,
718
+ "hit@10": 0.994,
719
+ "ndcg_linear@1": 0.902,
720
+ "ndcg_linear@5": 0.9522627688825706,
721
+ "ndcg_linear@10": 0.9545822486453416,
722
+ "ndcg_exponential@1": 0.902,
723
+ "ndcg_exponential@5": 0.9522627688825706,
724
+ "ndcg_exponential@10": 0.9545822486453416,
725
+ "precision@1": 0.902,
726
+ "precision@5": 0.19740000000000005,
727
+ "precision@10": 0.09940000000000002,
728
+ "recall@1": 0.902,
729
+ "recall@5": 0.987,
730
+ "recall@10": 0.994,
731
+ "f1@1": 0.902,
732
+ "f1@5": 0.329,
733
+ "f1@10": 0.18072727272727268,
734
+ "map@1": 0.902,
735
+ "map@5": 0.9402666666666668,
736
+ "map@10": 0.941256746031746,
737
+ "mrr": 0.9414657871821664,
738
+ "num_pred": 1000,
739
+ "num_data": 1000
740
+ },
741
+ "FashionIQ": {
742
+ "hit@1": 0.449,
743
+ "hit@5": 0.703,
744
+ "hit@10": 0.806,
745
+ "ndcg_linear@1": 0.449,
746
+ "ndcg_linear@5": 0.5860140468014696,
747
+ "ndcg_linear@10": 0.6195579052883298,
748
+ "ndcg_exponential@1": 0.449,
749
+ "ndcg_exponential@5": 0.5860140468014696,
750
+ "ndcg_exponential@10": 0.6195579052883298,
751
+ "precision@1": 0.449,
752
+ "precision@5": 0.1406,
753
+ "precision@10": 0.08059999999999999,
754
+ "recall@1": 0.449,
755
+ "recall@5": 0.703,
756
+ "recall@10": 0.806,
757
+ "f1@1": 0.449,
758
+ "f1@5": 0.23433333333333337,
759
+ "f1@10": 0.14654545454545456,
760
+ "map@1": 0.449,
761
+ "map@5": 0.5469166666666666,
762
+ "map@10": 0.5609043650793651,
763
+ "mrr": 0.5686339504443645,
764
+ "num_pred": 1000,
765
+ "num_data": 1000
766
+ },
767
+ "Wiki-SS-NQ": {
768
+ "hit@1": 0.794,
769
+ "hit@5": 0.945,
770
+ "hit@10": 0.966,
771
+ "ndcg_linear@1": 0.794,
772
+ "ndcg_linear@5": 0.8786886964830362,
773
+ "ndcg_linear@10": 0.8854287381224059,
774
+ "ndcg_exponential@1": 0.794,
775
+ "ndcg_exponential@5": 0.8786886964830362,
776
+ "ndcg_exponential@10": 0.8854287381224059,
777
+ "precision@1": 0.794,
778
+ "precision@5": 0.18900000000000003,
779
+ "precision@10": 0.09660000000000002,
780
+ "recall@1": 0.794,
781
+ "recall@5": 0.945,
782
+ "recall@10": 0.966,
783
+ "f1@1": 0.794,
784
+ "f1@5": 0.315,
785
+ "f1@10": 0.17563636363636362,
786
+ "map@1": 0.794,
787
+ "map@5": 0.85625,
788
+ "map@10": 0.8590031746031745,
789
+ "mrr": 0.8601126539188485,
790
+ "num_pred": 1000,
791
+ "num_data": 1000
792
+ },
793
+ "OVEN": {
794
+ "hit@1": 0.765,
795
+ "hit@5": 0.933,
796
+ "hit@10": 0.975,
797
+ "ndcg_linear@1": 0.765,
798
+ "ndcg_linear@5": 0.8580477012989404,
799
+ "ndcg_linear@10": 0.8719618855951725,
800
+ "ndcg_exponential@1": 0.765,
801
+ "ndcg_exponential@5": 0.8580477012989404,
802
+ "ndcg_exponential@10": 0.8719618855951725,
803
+ "precision@1": 0.765,
804
+ "precision@5": 0.18660000000000002,
805
+ "precision@10": 0.09750000000000003,
806
+ "recall@1": 0.765,
807
+ "recall@5": 0.933,
808
+ "recall@10": 0.975,
809
+ "f1@1": 0.765,
810
+ "f1@5": 0.311,
811
+ "f1@10": 0.17727272727272725,
812
+ "map@1": 0.765,
813
+ "map@5": 0.8327166666666667,
814
+ "map@10": 0.8386563492063492,
815
+ "mrr": 0.8399013048255178,
816
+ "num_pred": 1000,
817
+ "num_data": 1000
818
+ },
819
+ "EDIS": {
820
+ "hit@1": 0.932,
821
+ "hit@5": 0.993,
822
+ "hit@10": 0.997,
823
+ "ndcg_linear@1": 0.932,
824
+ "ndcg_linear@5": 0.9670173730346142,
825
+ "ndcg_linear@10": 0.9682264324408215,
826
+ "ndcg_exponential@1": 0.932,
827
+ "ndcg_exponential@5": 0.9670173730346142,
828
+ "ndcg_exponential@10": 0.9682264324408215,
829
+ "precision@1": 0.932,
830
+ "precision@5": 0.19860000000000003,
831
+ "precision@10": 0.09970000000000001,
832
+ "recall@1": 0.932,
833
+ "recall@5": 0.993,
834
+ "recall@10": 0.997,
835
+ "f1@1": 0.932,
836
+ "f1@5": 0.331,
837
+ "f1@10": 0.18127272727272725,
838
+ "map@1": 0.932,
839
+ "map@5": 0.9581666666666667,
840
+ "map@10": 0.9586166666666667,
841
+ "mrr": 0.9587475140056022,
842
+ "num_pred": 1000,
843
+ "num_data": 1000
844
+ },
845
+ "MSCOCO": {
846
+ "hit@1": 0.863,
847
+ "hit@5": 0.955,
848
+ "hit@10": 0.971,
849
+ "ndcg_linear@1": 0.863,
850
+ "ndcg_linear@5": 0.9152218081312857,
851
+ "ndcg_linear@10": 0.9205935201993672,
852
+ "ndcg_exponential@1": 0.863,
853
+ "ndcg_exponential@5": 0.9152218081312857,
854
+ "ndcg_exponential@10": 0.9205935201993672,
855
+ "precision@1": 0.863,
856
+ "precision@5": 0.19100000000000003,
857
+ "precision@10": 0.09710000000000002,
858
+ "recall@1": 0.863,
859
+ "recall@5": 0.955,
860
+ "recall@10": 0.971,
861
+ "f1@1": 0.863,
862
+ "f1@5": 0.31833333333333336,
863
+ "f1@10": 0.1765454545454545,
864
+ "map@1": 0.863,
865
+ "map@5": 0.9017000000000001,
866
+ "map@10": 0.904034126984127,
867
+ "mrr": 0.9055296763404475,
868
+ "num_pred": 1000,
869
+ "num_data": 1000
870
+ },
871
+ "RefCOCO": {
872
+ "hit@1": 0.984,
873
+ "hit@5": 1.0,
874
+ "hit@10": 1.0,
875
+ "ndcg_linear@1": 0.984,
876
+ "ndcg_linear@5": 0.9940948760571433,
877
+ "ndcg_linear@10": 0.9940948760571433,
878
+ "ndcg_exponential@1": 0.984,
879
+ "ndcg_exponential@5": 0.9940948760571433,
880
+ "ndcg_exponential@10": 0.9940948760571433,
881
+ "precision@1": 0.984,
882
+ "precision@5": 0.20000000000000004,
883
+ "precision@10": 0.10000000000000002,
884
+ "recall@1": 0.984,
885
+ "recall@5": 1.0,
886
+ "recall@10": 1.0,
887
+ "f1@1": 0.984,
888
+ "f1@5": 0.3333333333333333,
889
+ "f1@10": 0.18181818181818182,
890
+ "map@1": 0.984,
891
+ "map@5": 0.992,
892
+ "map@10": 0.992,
893
+ "mrr": 0.992,
894
+ "num_pred": 1000,
895
+ "num_data": 1000
896
+ },
897
+ "RefCOCO-Matching": {
898
+ "hit@1": 0.965,
899
+ "hit@5": 1.0,
900
+ "hit@10": 1.0,
901
+ "ndcg_linear@1": 0.965,
902
+ "ndcg_linear@5": 0.9889753306357154,
903
+ "ndcg_linear@10": 0.9889753306357154,
904
+ "ndcg_exponential@1": 0.965,
905
+ "ndcg_exponential@5": 0.9889753306357154,
906
+ "ndcg_exponential@10": 0.9889753306357154,
907
+ "precision@1": 0.965,
908
+ "precision@5": 0.20000000000000004,
909
+ "precision@10": 0.10000000000000002,
910
+ "recall@1": 0.965,
911
+ "recall@5": 1.0,
912
+ "recall@10": 1.0,
913
+ "f1@1": 0.965,
914
+ "f1@5": 0.3333333333333333,
915
+ "f1@10": 0.18181818181818182,
916
+ "map@1": 0.965,
917
+ "map@5": 0.9855,
918
+ "map@10": 0.9855,
919
+ "mrr": 0.9825,
920
+ "num_pred": 1000,
921
+ "num_data": 1000
922
+ },
923
+ "Visual7W-Pointing": {
924
+ "hit@1": 0.995,
925
+ "hit@5": 1.0,
926
+ "hit@10": 1.0,
927
+ "ndcg_linear@1": 0.995,
928
+ "ndcg_linear@5": 0.9977796420679489,
929
+ "ndcg_linear@10": 0.9977796420679489,
930
+ "ndcg_exponential@1": 0.995,
931
+ "ndcg_exponential@5": 0.9977796420679489,
932
+ "ndcg_exponential@10": 0.9977796420679489,
933
+ "precision@1": 0.995,
934
+ "precision@5": 0.20000000000000004,
935
+ "precision@10": 0.10000000000000002,
936
+ "recall@1": 0.995,
937
+ "recall@5": 1.0,
938
+ "recall@10": 1.0,
939
+ "f1@1": 0.995,
940
+ "f1@5": 0.3333333333333333,
941
+ "f1@10": 0.18181818181818182,
942
+ "map@1": 0.995,
943
+ "map@5": 0.9970333333333333,
944
+ "map@10": 0.9970333333333333,
945
+ "mrr": 0.9970333333333333,
946
+ "num_pred": 1000,
947
+ "num_data": 1000
948
+ }
949
+ },
950
+ "video": {
951
+ "K700": {
952
+ "hit@1": 0.742,
953
+ "hit@5": 0.915,
954
+ "hit@10": 0.941,
955
+ "ndcg_linear@1": 0.742,
956
+ "ndcg_linear@5": 0.8362515412973999,
957
+ "ndcg_linear@10": 0.845000380031472,
958
+ "ndcg_exponential@1": 0.742,
959
+ "ndcg_exponential@5": 0.8362515412973999,
960
+ "ndcg_exponential@10": 0.845000380031472,
961
+ "precision@1": 0.742,
962
+ "precision@5": 0.183,
963
+ "precision@10": 0.0941,
964
+ "recall@1": 0.742,
965
+ "recall@5": 0.915,
966
+ "recall@10": 0.941,
967
+ "f1@1": 0.742,
968
+ "f1@5": 0.305,
969
+ "f1@10": 0.17109090909090907,
970
+ "map@1": 0.742,
971
+ "map@5": 0.8098666666666666,
972
+ "map@10": 0.8136777777777778,
973
+ "mrr": 0.8163987574339042,
974
+ "num_pred": 1000,
975
+ "num_data": 1000
976
+ },
977
+ "SmthSmthV2": {
978
+ "hit@1": 0.934,
979
+ "hit@5": 0.99,
980
+ "hit@10": 0.995,
981
+ "ndcg_linear@1": 0.934,
982
+ "ndcg_linear@5": 0.9656779053318224,
983
+ "ndcg_linear@10": 0.9672560349844582,
984
+ "ndcg_exponential@1": 0.934,
985
+ "ndcg_exponential@5": 0.9656779053318224,
986
+ "ndcg_exponential@10": 0.9672560349844582,
987
+ "precision@1": 0.934,
988
+ "precision@5": 0.19800000000000004,
989
+ "precision@10": 0.09950000000000002,
990
+ "recall@1": 0.934,
991
+ "recall@5": 0.99,
992
+ "recall@10": 0.995,
993
+ "f1@1": 0.934,
994
+ "f1@5": 0.33,
995
+ "f1@10": 0.18090909090909088,
996
+ "map@1": 0.934,
997
+ "map@5": 0.9574166666666667,
998
+ "map@10": 0.9580452380952381,
999
+ "mrr": 0.9584103174603174,
1000
+ "num_pred": 1000,
1001
+ "num_data": 1000
1002
+ },
1003
+ "HMDB51": {
1004
+ "hit@1": 0.805,
1005
+ "hit@5": 0.969,
1006
+ "hit@10": 0.991,
1007
+ "ndcg_linear@1": 0.805,
1008
+ "ndcg_linear@5": 0.8970430569474465,
1009
+ "ndcg_linear@10": 0.9041189942531923,
1010
+ "ndcg_exponential@1": 0.805,
1011
+ "ndcg_exponential@5": 0.8970430569474465,
1012
+ "ndcg_exponential@10": 0.9041189942531923,
1013
+ "precision@1": 0.805,
1014
+ "precision@5": 0.1938,
1015
+ "precision@10": 0.09910000000000002,
1016
+ "recall@1": 0.805,
1017
+ "recall@5": 0.969,
1018
+ "recall@10": 0.991,
1019
+ "f1@1": 0.805,
1020
+ "f1@5": 0.323,
1021
+ "f1@10": 0.18018181818181817,
1022
+ "map@1": 0.805,
1023
+ "map@5": 0.8726833333333334,
1024
+ "map@10": 0.8755793650793651,
1025
+ "mrr": 0.8761436609590073,
1026
+ "num_pred": 1000,
1027
+ "num_data": 1000
1028
+ },
1029
+ "UCF101": {
1030
+ "hit@1": 0.965,
1031
+ "hit@5": 0.999,
1032
+ "hit@10": 1.0,
1033
+ "ndcg_linear@1": 0.965,
1034
+ "ndcg_linear@5": 0.9861897521142866,
1035
+ "ndcg_linear@10": 0.9865052169910724,
1036
+ "ndcg_exponential@1": 0.965,
1037
+ "ndcg_exponential@5": 0.9861897521142866,
1038
+ "ndcg_exponential@10": 0.9865052169910724,
1039
+ "precision@1": 0.965,
1040
+ "precision@5": 0.1998,
1041
+ "precision@10": 0.10000000000000002,
1042
+ "recall@1": 0.965,
1043
+ "recall@5": 0.999,
1044
+ "recall@10": 1.0,
1045
+ "f1@1": 0.965,
1046
+ "f1@5": 0.333,
1047
+ "f1@10": 0.18181818181818182,
1048
+ "map@1": 0.965,
1049
+ "map@5": 0.9816666666666667,
1050
+ "map@10": 0.9817916666666667,
1051
+ "mrr": 0.9817916666666667,
1052
+ "num_pred": 1000,
1053
+ "num_data": 1000
1054
+ },
1055
+ "Breakfast": {
1056
+ "hit@1": 0.8198614318706697,
1057
+ "hit@5": 0.9976905311778291,
1058
+ "hit@10": 1.0,
1059
+ "ndcg_linear@1": 0.8198614318706697,
1060
+ "ndcg_linear@5": 0.9231163842968108,
1061
+ "ndcg_linear@10": 0.9239390336896699,
1062
+ "ndcg_exponential@1": 0.8198614318706697,
1063
+ "ndcg_exponential@5": 0.9231163842968108,
1064
+ "ndcg_exponential@10": 0.9239390336896699,
1065
+ "precision@1": 0.8198614318706697,
1066
+ "precision@5": 0.19953810623556584,
1067
+ "precision@10": 0.1,
1068
+ "recall@1": 0.8198614318706697,
1069
+ "recall@5": 0.9976905311778291,
1070
+ "recall@10": 1.0,
1071
+ "f1@1": 0.8198614318706697,
1072
+ "f1@5": 0.3325635103926097,
1073
+ "f1@10": 0.1818181818181818,
1074
+ "map@1": 0.8198614318706697,
1075
+ "map@5": 0.8975365665896844,
1076
+ "map@10": 0.8979214780600461,
1077
+ "mrr": 0.8979214780600461,
1078
+ "num_pred": 433,
1079
+ "num_data": 433
1080
+ },
1081
+ "MVBench": {
1082
+ "hit@1": 0.5595,
1083
+ "hit@5": 1.0,
1084
+ "hit@10": 1.0,
1085
+ "ndcg_linear@1": 0.5595,
1086
+ "ndcg_linear@5": 0.8056059162693463,
1087
+ "ndcg_linear@10": 0.8056059162693463,
1088
+ "ndcg_exponential@1": 0.5595,
1089
+ "ndcg_exponential@5": 0.8056059162693463,
1090
+ "ndcg_exponential@10": 0.8056059162693463,
1091
+ "precision@1": 0.5595,
1092
+ "precision@5": 0.20000000000000004,
1093
+ "precision@10": 0.10000000000000002,
1094
+ "recall@1": 0.5595,
1095
+ "recall@5": 1.0,
1096
+ "recall@10": 1.0,
1097
+ "f1@1": 0.5595,
1098
+ "f1@5": 0.3333333333333333,
1099
+ "f1@10": 0.18181818181818182,
1100
+ "map@1": 0.5595,
1101
+ "map@5": 0.7396375000000001,
1102
+ "map@10": 0.7396375000000001,
1103
+ "mrr": 0.7396375000000001,
1104
+ "num_pred": 4000,
1105
+ "num_data": 4000
1106
+ },
1107
+ "Video-MME": {
1108
+ "hit@1": 0.5685185185185185,
1109
+ "hit@5": 1.0,
1110
+ "hit@10": 1.0,
1111
+ "ndcg_linear@1": 0.5685185185185185,
1112
+ "ndcg_linear@5": 0.8045340665050689,
1113
+ "ndcg_linear@10": 0.8045340665050689,
1114
+ "ndcg_exponential@1": 0.5685185185185185,
1115
+ "ndcg_exponential@5": 0.8045340665050689,
1116
+ "ndcg_exponential@10": 0.8045340665050689,
1117
+ "precision@1": 0.5685185185185185,
1118
+ "precision@5": 0.19999999999999996,
1119
+ "precision@10": 0.09999999999999998,
1120
+ "recall@1": 0.5685185185185185,
1121
+ "recall@5": 1.0,
1122
+ "recall@10": 1.0,
1123
+ "f1@1": 0.5685185185185185,
1124
+ "f1@5": 0.3333333333333334,
1125
+ "f1@10": 0.18181818181818188,
1126
+ "map@1": 0.5685185185185185,
1127
+ "map@5": 0.7386419753086418,
1128
+ "map@10": 0.7386419753086418,
1129
+ "mrr": 0.7386419753086418,
1130
+ "num_pred": 2700,
1131
+ "num_data": 2700
1132
+ },
1133
+ "NExTQA": {
1134
+ "hit@1": 0.739140588510042,
1135
+ "hit@5": 1.0,
1136
+ "hit@10": 1.0,
1137
+ "ndcg_linear@1": 0.739140588510042,
1138
+ "ndcg_linear@5": 0.8850558738514491,
1139
+ "ndcg_linear@10": 0.8850558738514491,
1140
+ "ndcg_exponential@1": 0.739140588510042,
1141
+ "ndcg_exponential@5": 0.8850558738514491,
1142
+ "ndcg_exponential@10": 0.8850558738514491,
1143
+ "precision@1": 0.739140588510042,
1144
+ "precision@5": 0.19999999999999996,
1145
+ "precision@10": 0.09999999999999998,
1146
+ "recall@1": 0.739140588510042,
1147
+ "recall@5": 1.0,
1148
+ "recall@10": 1.0,
1149
+ "f1@1": 0.739140588510042,
1150
+ "f1@5": 0.33333333333333337,
1151
+ "f1@10": 0.18181818181818182,
1152
+ "map@1": 0.739140588510042,
1153
+ "map@5": 0.8461777985365093,
1154
+ "map@10": 0.8461777985365093,
1155
+ "mrr": 0.8459442628055425,
1156
+ "num_pred": 8564,
1157
+ "num_data": 8564
1158
+ },
1159
+ "EgoSchema": {
1160
+ "hit@1": 0.52,
1161
+ "hit@5": 1.0,
1162
+ "hit@10": 1.0,
1163
+ "ndcg_linear@1": 0.52,
1164
+ "ndcg_linear@5": 0.7737136799082726,
1165
+ "ndcg_linear@10": 0.7737136799082726,
1166
+ "ndcg_exponential@1": 0.52,
1167
+ "ndcg_exponential@5": 0.7737136799082726,
1168
+ "ndcg_exponential@10": 0.7737136799082726,
1169
+ "precision@1": 0.52,
1170
+ "precision@5": 0.2,
1171
+ "precision@10": 0.1,
1172
+ "recall@1": 0.52,
1173
+ "recall@5": 1.0,
1174
+ "recall@10": 1.0,
1175
+ "f1@1": 0.52,
1176
+ "f1@5": 0.3333333333333333,
1177
+ "f1@10": 0.18181818181818182,
1178
+ "map@1": 0.52,
1179
+ "map@5": 0.6984666666666667,
1180
+ "map@10": 0.6984666666666667,
1181
+ "mrr": 0.6984666666666667,
1182
+ "num_pred": 500,
1183
+ "num_data": 500
1184
+ },
1185
+ "ActivityNetQA": {
1186
+ "hit@1": 0.856,
1187
+ "hit@5": 1.0,
1188
+ "hit@10": 1.0,
1189
+ "ndcg_linear@1": 0.856,
1190
+ "ndcg_linear@5": 0.9468538845142899,
1191
+ "ndcg_linear@10": 0.9468538845142899,
1192
+ "ndcg_exponential@1": 0.856,
1193
+ "ndcg_exponential@5": 0.9468538845142899,
1194
+ "ndcg_exponential@10": 0.9468538845142899,
1195
+ "precision@1": 0.856,
1196
+ "precision@5": 0.20000000000000004,
1197
+ "precision@10": 0.10000000000000002,
1198
+ "recall@1": 0.856,
1199
+ "recall@5": 1.0,
1200
+ "recall@10": 1.0,
1201
+ "f1@1": 0.856,
1202
+ "f1@5": 0.3333333333333333,
1203
+ "f1@10": 0.18181818181818182,
1204
+ "map@1": 0.856,
1205
+ "map@5": 0.928,
1206
+ "map@10": 0.928,
1207
+ "mrr": 0.928,
1208
+ "num_pred": 1000,
1209
+ "num_data": 1000
1210
+ },
1211
+ "DiDeMo": {
1212
+ "hit@1": 0.6563745019920318,
1213
+ "hit@5": 0.8615537848605578,
1214
+ "hit@10": 0.9183266932270916,
1215
+ "ndcg_linear@1": 0.6563745019920318,
1216
+ "ndcg_linear@5": 0.7681091358820242,
1217
+ "ndcg_linear@10": 0.7865920580741488,
1218
+ "ndcg_exponential@1": 0.6563745019920318,
1219
+ "ndcg_exponential@5": 0.7681091358820242,
1220
+ "ndcg_exponential@10": 0.7865920580741488,
1221
+ "precision@1": 0.6563745019920318,
1222
+ "precision@5": 0.17231075697211157,
1223
+ "precision@10": 0.09183266932270918,
1224
+ "recall@1": 0.6563745019920318,
1225
+ "recall@5": 0.8615537848605578,
1226
+ "recall@10": 0.9183266932270916,
1227
+ "f1@1": 0.6563745019920318,
1228
+ "f1@5": 0.2871845949535193,
1229
+ "f1@10": 0.16696848967765301,
1230
+ "map@1": 0.6563745019920318,
1231
+ "map@5": 0.7368027888446214,
1232
+ "map@10": 0.7445025453740592,
1233
+ "mrr": 0.7477687713528596,
1234
+ "num_pred": 1004,
1235
+ "num_data": 1004
1236
+ },
1237
+ "MSR-VTT": {
1238
+ "hit@1": 0.544,
1239
+ "hit@5": 0.787,
1240
+ "hit@10": 0.847,
1241
+ "ndcg_linear@1": 0.544,
1242
+ "ndcg_linear@5": 0.6744944567642283,
1243
+ "ndcg_linear@10": 0.6942079673985265,
1244
+ "ndcg_exponential@1": 0.544,
1245
+ "ndcg_exponential@5": 0.6744944567642283,
1246
+ "ndcg_exponential@10": 0.6942079673985265,
1247
+ "precision@1": 0.544,
1248
+ "precision@5": 0.1574,
1249
+ "precision@10": 0.0847,
1250
+ "recall@1": 0.544,
1251
+ "recall@5": 0.787,
1252
+ "recall@10": 0.847,
1253
+ "f1@1": 0.544,
1254
+ "f1@5": 0.26233333333333336,
1255
+ "f1@10": 0.154,
1256
+ "map@1": 0.544,
1257
+ "map@5": 0.6369833333333333,
1258
+ "map@10": 0.6453083333333334,
1259
+ "mrr": 0.6511262404575208,
1260
+ "num_pred": 1000,
1261
+ "num_data": 1000
1262
+ },
1263
+ "MSVD": {
1264
+ "hit@1": 0.7507462686567165,
1265
+ "hit@5": 0.9477611940298507,
1266
+ "hit@10": 0.9701492537313433,
1267
+ "ndcg_linear@1": 0.7507462686567165,
1268
+ "ndcg_linear@5": 0.8619178893506118,
1269
+ "ndcg_linear@10": 0.8692299619117174,
1270
+ "ndcg_exponential@1": 0.7507462686567165,
1271
+ "ndcg_exponential@5": 0.8619178893506118,
1272
+ "ndcg_exponential@10": 0.8692299619117174,
1273
+ "precision@1": 0.7507462686567165,
1274
+ "precision@5": 0.18955223880597014,
1275
+ "precision@10": 0.0970149253731343,
1276
+ "recall@1": 0.7507462686567165,
1277
+ "recall@5": 0.9477611940298507,
1278
+ "recall@10": 0.9701492537313433,
1279
+ "f1@1": 0.7507462686567165,
1280
+ "f1@5": 0.31592039800995037,
1281
+ "f1@10": 0.17639077340569884,
1282
+ "map@1": 0.7507462686567165,
1283
+ "map@5": 0.8328109452736316,
1284
+ "map@10": 0.8358736081497274,
1285
+ "mrr": 0.8371798326034822,
1286
+ "num_pred": 670,
1287
+ "num_data": 670
1288
+ },
1289
+ "VATEX": {
1290
+ "hit@1": 0.5379633765073694,
1291
+ "hit@5": 0.8211255024564538,
1292
+ "hit@10": 0.9012952210808397,
1293
+ "ndcg_linear@1": 0.5379633765073694,
1294
+ "ndcg_linear@5": 0.6901082987688059,
1295
+ "ndcg_linear@10": 0.7164456939727849,
1296
+ "ndcg_exponential@1": 0.5379633765073694,
1297
+ "ndcg_exponential@5": 0.6901082987688059,
1298
+ "ndcg_exponential@10": 0.7164456939727849,
1299
+ "precision@1": 0.5379633765073694,
1300
+ "precision@5": 0.16422510049129074,
1301
+ "precision@10": 0.09012952210808398,
1302
+ "recall@1": 0.5379633765073694,
1303
+ "recall@5": 0.8211255024564538,
1304
+ "recall@10": 0.9012952210808397,
1305
+ "f1@1": 0.5379633765073694,
1306
+ "f1@5": 0.273708500818818,
1307
+ "f1@10": 0.1638718583783345,
1308
+ "map@1": 0.5379633765073694,
1309
+ "map@5": 0.6463599821348817,
1310
+ "map@10": 0.6574712704793098,
1311
+ "mrr": 0.6618887868381477,
1312
+ "num_pred": 4478,
1313
+ "num_data": 4478
1314
+ },
1315
+ "YouCook2": {
1316
+ "hit@1": 0.354513998112614,
1317
+ "hit@5": 0.6388801509908776,
1318
+ "hit@10": 0.7448883296634161,
1319
+ "ndcg_linear@1": 0.354513998112614,
1320
+ "ndcg_linear@5": 0.5043217757954078,
1321
+ "ndcg_linear@10": 0.5387300811645056,
1322
+ "ndcg_exponential@1": 0.354513998112614,
1323
+ "ndcg_exponential@5": 0.5043217757954078,
1324
+ "ndcg_exponential@10": 0.5387300811645056,
1325
+ "precision@1": 0.354513998112614,
1326
+ "precision@5": 0.12777603019817552,
1327
+ "precision@10": 0.07448883296634161,
1328
+ "recall@1": 0.354513998112614,
1329
+ "recall@5": 0.6388801509908776,
1330
+ "recall@10": 0.7448883296634161,
1331
+ "f1@1": 0.354513998112614,
1332
+ "f1@5": 0.2129600503302926,
1333
+ "f1@10": 0.1354342417569848,
1334
+ "map@1": 0.354513998112614,
1335
+ "map@5": 0.4596938240536856,
1336
+ "map@10": 0.4739751693903943,
1337
+ "mrr": 0.48322195240420857,
1338
+ "num_pred": 3179,
1339
+ "num_data": 3179
1340
+ },
1341
+ "QVHighlight": {
1342
+ "hit@1": 0.7903970452446907,
1343
+ "hit@5": 0.96398891966759,
1344
+ "hit@10": 1.0,
1345
+ "ndcg_linear@1": 0.7903970452446907,
1346
+ "ndcg_linear@5": 0.887501860589501,
1347
+ "ndcg_linear@10": 0.8989929238766683,
1348
+ "ndcg_exponential@1": 0.7903970452446907,
1349
+ "ndcg_exponential@5": 0.887501860589501,
1350
+ "ndcg_exponential@10": 0.8989929238766683,
1351
+ "precision@1": 0.7903970452446907,
1352
+ "precision@5": 0.19279778393351799,
1353
+ "precision@10": 0.09999999999999998,
1354
+ "recall@1": 0.7903970452446907,
1355
+ "recall@5": 0.96398891966759,
1356
+ "recall@10": 1.0,
1357
+ "f1@1": 0.7903970452446907,
1358
+ "f1@5": 0.32132963988919677,
1359
+ "f1@10": 0.18181818181818185,
1360
+ "map@1": 0.7903970452446907,
1361
+ "map@5": 0.8616497383810404,
1362
+ "map@10": 0.8663002535578713,
1363
+ "mrr": 0.8663002535578713,
1364
+ "num_pred": 1083,
1365
+ "num_data": 1083
1366
+ },
1367
+ "Charades-STA": {
1368
+ "hit@1": 0.42090784044016505,
1369
+ "hit@5": 0.7702888583218707,
1370
+ "hit@10": 1.0,
1371
+ "ndcg_linear@1": 0.42090784044016505,
1372
+ "ndcg_linear@5": 0.6047098951040933,
1373
+ "ndcg_linear@10": 0.678425439459782,
1374
+ "ndcg_exponential@1": 0.42090784044016505,
1375
+ "ndcg_exponential@5": 0.6047098951040933,
1376
+ "ndcg_exponential@10": 0.678425439459782,
1377
+ "precision@1": 0.42090784044016505,
1378
+ "precision@5": 0.15405777166437412,
1379
+ "precision@10": 0.09999999999999998,
1380
+ "recall@1": 0.42090784044016505,
1381
+ "recall@5": 0.7702888583218707,
1382
+ "recall@10": 1.0,
1383
+ "f1@1": 0.42090784044016505,
1384
+ "f1@5": 0.25676295277395694,
1385
+ "f1@10": 0.18181818181818185,
1386
+ "map@1": 0.42090784044016505,
1387
+ "map@5": 0.5497936726272352,
1388
+ "map@10": 0.5799043689002423,
1389
+ "mrr": 0.5799043689002423,
1390
+ "num_pred": 727,
1391
+ "num_data": 727
1392
+ },
1393
+ "MomentSeeker": {
1394
+ "hit@1": 0.517478152309613,
1395
+ "hit@5": 0.9413233458177278,
1396
+ "hit@10": 1.0,
1397
+ "ndcg_linear@1": 0.517478152309613,
1398
+ "ndcg_linear@5": 0.7480508014838803,
1399
+ "ndcg_linear@10": 0.7710570645889006,
1400
+ "ndcg_exponential@1": 0.517478152309613,
1401
+ "ndcg_exponential@5": 0.7480508014838803,
1402
+ "ndcg_exponential@10": 0.7710570645889006,
1403
+ "precision@1": 0.517478152309613,
1404
+ "precision@5": 0.21098626716604243,
1405
+ "precision@10": 0.11448189762796507,
1406
+ "recall@1": 0.48642322097378277,
1407
+ "recall@5": 0.931023720349563,
1408
+ "recall@10": 0.9980233042030796,
1409
+ "f1@1": 0.495664149971266,
1410
+ "f1@5": 0.33784962436647836,
1411
+ "f1@10": 0.20243391474664882,
1412
+ "map@1": 0.517478152309613,
1413
+ "map@5": 0.68182670966847,
1414
+ "map@10": 0.6929560086102688,
1415
+ "mrr": 0.6972569010958524,
1416
+ "num_pred": 1602,
1417
+ "num_data": 1602
1418
+ }
1419
+ },
1420
+ "visdoc": {
1421
+ "ViDoRe_arxivqa": {
1422
+ "hit@1": 0.862,
1423
+ "hit@5": 0.954,
1424
+ "hit@10": 0.97,
1425
+ "ndcg_linear@1": 0.862,
1426
+ "ndcg_linear@5": 0.914760961406135,
1427
+ "ndcg_linear@10": 0.9199750921832451,
1428
+ "ndcg_exponential@1": 0.862,
1429
+ "ndcg_exponential@5": 0.914760961406135,
1430
+ "ndcg_exponential@10": 0.9199750921832451,
1431
+ "precision@1": 0.862,
1432
+ "precision@5": 0.19080000000000003,
1433
+ "precision@10": 0.097,
1434
+ "recall@1": 0.862,
1435
+ "recall@5": 0.954,
1436
+ "recall@10": 0.97,
1437
+ "f1@1": 0.862,
1438
+ "f1@5": 0.318,
1439
+ "f1@10": 0.17636363636363636,
1440
+ "map@1": 0.862,
1441
+ "map@5": 0.9013666666666666,
1442
+ "map@10": 0.9035373015873015,
1443
+ "mrr": 0.9050097363346488,
1444
+ "num_pred": 500,
1445
+ "num_data": 500
1446
+ },
1447
+ "ViDoRe_docvqa": {
1448
+ "hit@1": 0.49889135254988914,
1449
+ "hit@5": 0.6940133037694013,
1450
+ "hit@10": 0.7516629711751663,
1451
+ "ndcg_linear@1": 0.49889135254988914,
1452
+ "ndcg_linear@5": 0.600866891439063,
1453
+ "ndcg_linear@10": 0.6197249682766942,
1454
+ "ndcg_exponential@1": 0.49889135254988914,
1455
+ "ndcg_exponential@5": 0.600866891439063,
1456
+ "ndcg_exponential@10": 0.6197249682766942,
1457
+ "precision@1": 0.49889135254988914,
1458
+ "precision@5": 0.14146341463414633,
1459
+ "precision@10": 0.07716186252771617,
1460
+ "recall@1": 0.4951958610495196,
1461
+ "recall@5": 0.6876940133037693,
1462
+ "recall@10": 0.7458610495195861,
1463
+ "f1@1": 0.4963045084996304,
1464
+ "f1@5": 0.2332774351425715,
1465
+ "f1@10": 0.13910708655719745,
1466
+ "map@1": 0.49889135254988914,
1467
+ "map@5": 0.5703178122690318,
1468
+ "map@10": 0.5783282300355471,
1469
+ "mrr": 0.5890218522555442,
1470
+ "num_pred": 451,
1471
+ "num_data": 451
1472
+ },
1473
+ "ViDoRe_infovqa": {
1474
+ "hit@1": 0.868421052631579,
1475
+ "hit@5": 0.9554655870445344,
1476
+ "hit@10": 0.97165991902834,
1477
+ "ndcg_linear@1": 0.868421052631579,
1478
+ "ndcg_linear@5": 0.9165450266480737,
1479
+ "ndcg_linear@10": 0.9222824240779839,
1480
+ "ndcg_exponential@1": 0.868421052631579,
1481
+ "ndcg_exponential@5": 0.9165450266480737,
1482
+ "ndcg_exponential@10": 0.9222824240779839,
1483
+ "precision@1": 0.868421052631579,
1484
+ "precision@5": 0.1910931174089069,
1485
+ "precision@10": 0.09736842105263156,
1486
+ "recall@1": 0.868421052631579,
1487
+ "recall@5": 0.9544534412955465,
1488
+ "recall@10": 0.97165991902834,
1489
+ "f1@1": 0.868421052631579,
1490
+ "f1@5": 0.31839213418160783,
1491
+ "f1@10": 0.1769721506563612,
1492
+ "map@1": 0.868421052631579,
1493
+ "map@5": 0.9035762483130905,
1494
+ "map@10": 0.9061001863633442,
1495
+ "mrr": 0.9075480386176507,
1496
+ "num_pred": 494,
1497
+ "num_data": 494
1498
+ },
1499
+ "ViDoRe_tabfquad": {
1500
+ "hit@1": 0.9107142857142857,
1501
+ "hit@5": 0.9857142857142858,
1502
+ "hit@10": 0.9964285714285714,
1503
+ "ndcg_linear@1": 0.9107142857142857,
1504
+ "ndcg_linear@5": 0.9539250179792418,
1505
+ "ndcg_linear@10": 0.9574505070530973,
1506
+ "ndcg_exponential@1": 0.9107142857142857,
1507
+ "ndcg_exponential@5": 0.9539250179792418,
1508
+ "ndcg_exponential@10": 0.9574505070530973,
1509
+ "precision@1": 0.9107142857142857,
1510
+ "precision@5": 0.1971428571428571,
1511
+ "precision@10": 0.09964285714285714,
1512
+ "recall@1": 0.9107142857142857,
1513
+ "recall@5": 0.9857142857142858,
1514
+ "recall@10": 0.9964285714285714,
1515
+ "f1@1": 0.9107142857142857,
1516
+ "f1@5": 0.3285714285714287,
1517
+ "f1@10": 0.18116883116883123,
1518
+ "map@1": 0.9107142857142857,
1519
+ "map@5": 0.9430952380952381,
1520
+ "map@10": 0.9445833333333334,
1521
+ "mrr": 0.9446382783882784,
1522
+ "num_pred": 280,
1523
+ "num_data": 280
1524
+ },
1525
+ "ViDoRe_tatdqa": {
1526
+ "hit@1": 0.6105710814094775,
1527
+ "hit@5": 0.8657351154313487,
1528
+ "hit@10": 0.9228432563791008,
1529
+ "ndcg_linear@1": 0.6105710814094775,
1530
+ "ndcg_linear@5": 0.7492547365066389,
1531
+ "ndcg_linear@10": 0.7678678059475559,
1532
+ "ndcg_exponential@1": 0.6105710814094775,
1533
+ "ndcg_exponential@5": 0.7492547365066389,
1534
+ "ndcg_exponential@10": 0.7678678059475559,
1535
+ "precision@1": 0.6105710814094775,
1536
+ "precision@5": 0.17375455650060753,
1537
+ "precision@10": 0.09258809234507896,
1538
+ "recall@1": 0.6088092345078979,
1539
+ "recall@5": 0.8648845686512757,
1540
+ "recall@10": 0.9219927095990279,
1541
+ "f1@1": 0.6092639640634779,
1542
+ "f1@5": 0.28915118902968245,
1543
+ "f1@10": 0.1681910232335506,
1544
+ "map@1": 0.6105710814094775,
1545
+ "map@5": 0.7101660591332523,
1546
+ "map@10": 0.7179439333449055,
1547
+ "mrr": 0.7222121121375645,
1548
+ "num_pred": 1646,
1549
+ "num_data": 1646
1550
+ },
1551
+ "ViDoRe_shiftproject": {
1552
+ "hit@1": 0.78,
1553
+ "hit@5": 0.97,
1554
+ "hit@10": 0.99,
1555
+ "ndcg_linear@1": 0.78,
1556
+ "ndcg_linear@5": 0.886446094618285,
1557
+ "ndcg_linear@10": 0.8922273911446429,
1558
+ "ndcg_exponential@1": 0.78,
1559
+ "ndcg_exponential@5": 0.886446094618285,
1560
+ "ndcg_exponential@10": 0.8922273911446429,
1561
+ "precision@1": 0.78,
1562
+ "precision@5": 0.19399999999999995,
1563
+ "precision@10": 0.09899999999999999,
1564
+ "recall@1": 0.78,
1565
+ "recall@5": 0.97,
1566
+ "recall@10": 0.99,
1567
+ "f1@1": 0.78,
1568
+ "f1@5": 0.3233333333333334,
1569
+ "f1@10": 0.18000000000000008,
1570
+ "map@1": 0.78,
1571
+ "map@5": 0.8581666666666666,
1572
+ "map@10": 0.8601666666666665,
1573
+ "mrr": 0.8605238095238096,
1574
+ "num_pred": 100,
1575
+ "num_data": 100
1576
+ },
1577
+ "ViDoRe_syntheticDocQA_artificial_intelligence": {
1578
+ "hit@1": 0.96,
1579
+ "hit@5": 1.0,
1580
+ "hit@10": 1.0,
1581
+ "ndcg_linear@1": 0.96,
1582
+ "ndcg_linear@5": 0.9852371901428583,
1583
+ "ndcg_linear@10": 0.9852371901428583,
1584
+ "ndcg_exponential@1": 0.96,
1585
+ "ndcg_exponential@5": 0.9852371901428583,
1586
+ "ndcg_exponential@10": 0.9852371901428583,
1587
+ "precision@1": 0.96,
1588
+ "precision@5": 0.19999999999999996,
1589
+ "precision@10": 0.09999999999999998,
1590
+ "recall@1": 0.96,
1591
+ "recall@5": 1.0,
1592
+ "recall@10": 1.0,
1593
+ "f1@1": 0.96,
1594
+ "f1@5": 0.3333333333333335,
1595
+ "f1@10": 0.1818181818181819,
1596
+ "map@1": 0.96,
1597
+ "map@5": 0.98,
1598
+ "map@10": 0.98,
1599
+ "mrr": 0.98,
1600
+ "num_pred": 100,
1601
+ "num_data": 100
1602
+ },
1603
+ "ViDoRe_syntheticDocQA_energy": {
1604
+ "hit@1": 0.92,
1605
+ "hit@5": 0.97,
1606
+ "hit@10": 0.98,
1607
+ "ndcg_linear@1": 0.92,
1608
+ "ndcg_linear@5": 0.946232126232897,
1609
+ "ndcg_linear@10": 0.9495654595662305,
1610
+ "ndcg_exponential@1": 0.92,
1611
+ "ndcg_exponential@5": 0.946232126232897,
1612
+ "ndcg_exponential@10": 0.9495654595662305,
1613
+ "precision@1": 0.92,
1614
+ "precision@5": 0.19399999999999995,
1615
+ "precision@10": 0.09799999999999998,
1616
+ "recall@1": 0.92,
1617
+ "recall@5": 0.97,
1618
+ "recall@10": 0.98,
1619
+ "f1@1": 0.92,
1620
+ "f1@5": 0.3233333333333334,
1621
+ "f1@10": 0.17818181818181827,
1622
+ "map@1": 0.92,
1623
+ "map@5": 0.9383333333333332,
1624
+ "map@10": 0.9397619047619047,
1625
+ "mrr": 0.9407989417989419,
1626
+ "num_pred": 100,
1627
+ "num_data": 100
1628
+ },
1629
+ "ViDoRe_syntheticDocQA_government_reports": {
1630
+ "hit@1": 0.92,
1631
+ "hit@5": 0.99,
1632
+ "hit@10": 0.99,
1633
+ "ndcg_linear@1": 0.92,
1634
+ "ndcg_linear@5": 0.9602371901428584,
1635
+ "ndcg_linear@10": 0.9602371901428584,
1636
+ "ndcg_exponential@1": 0.92,
1637
+ "ndcg_exponential@5": 0.9602371901428584,
1638
+ "ndcg_exponential@10": 0.9602371901428584,
1639
+ "precision@1": 0.92,
1640
+ "precision@5": 0.19799999999999998,
1641
+ "precision@10": 0.09899999999999999,
1642
+ "recall@1": 0.92,
1643
+ "recall@5": 0.99,
1644
+ "recall@10": 0.99,
1645
+ "f1@1": 0.92,
1646
+ "f1@5": 0.3300000000000001,
1647
+ "f1@10": 0.18000000000000008,
1648
+ "map@1": 0.92,
1649
+ "map@5": 0.95,
1650
+ "map@10": 0.95,
1651
+ "mrr": 0.9509090909090909,
1652
+ "num_pred": 100,
1653
+ "num_data": 100
1654
+ },
1655
+ "ViDoRe_syntheticDocQA_healthcare_industry": {
1656
+ "hit@1": 0.94,
1657
+ "hit@5": 1.0,
1658
+ "hit@10": 1.0,
1659
+ "ndcg_linear@1": 0.94,
1660
+ "ndcg_linear@5": 0.9739278926071439,
1661
+ "ndcg_linear@10": 0.9739278926071439,
1662
+ "ndcg_exponential@1": 0.94,
1663
+ "ndcg_exponential@5": 0.9739278926071439,
1664
+ "ndcg_exponential@10": 0.9739278926071439,
1665
+ "precision@1": 0.94,
1666
+ "precision@5": 0.19999999999999996,
1667
+ "precision@10": 0.09999999999999998,
1668
+ "recall@1": 0.94,
1669
+ "recall@5": 1.0,
1670
+ "recall@10": 1.0,
1671
+ "f1@1": 0.94,
1672
+ "f1@5": 0.3333333333333335,
1673
+ "f1@10": 0.1818181818181819,
1674
+ "map@1": 0.94,
1675
+ "map@5": 0.965,
1676
+ "map@10": 0.965,
1677
+ "mrr": 0.965,
1678
+ "num_pred": 100,
1679
+ "num_data": 100
1680
+ },
1681
+ "ViDoRe_esg_reports_human_labeled_v2": {
1682
+ "hit@1": 0.6346153846153846,
1683
+ "hit@5": 0.8076923076923077,
1684
+ "hit@10": 0.8846153846153846,
1685
+ "ndcg_linear@1": 0.6217948717948717,
1686
+ "ndcg_linear@5": 0.6349849833101007,
1687
+ "ndcg_linear@10": 0.6766977182692209,
1688
+ "ndcg_exponential@1": 0.618131868131868,
1689
+ "ndcg_exponential@5": 0.6329197164606668,
1690
+ "ndcg_exponential@10": 0.6764553588010223,
1691
+ "precision@1": 0.6346153846153846,
1692
+ "precision@5": 0.22692307692307695,
1693
+ "precision@10": 0.14615384615384613,
1694
+ "recall@1": 0.4778846153846154,
1695
+ "recall@5": 0.6620421245421244,
1696
+ "recall@10": 0.7614468864468864,
1697
+ "f1@1": 0.5119329388560158,
1698
+ "f1@5": 0.305936220642103,
1699
+ "f1@10": 0.2243630238539741,
1700
+ "map@1": 0.6346153846153846,
1701
+ "map@5": 0.5877297008547009,
1702
+ "map@10": 0.6096105514855515,
1703
+ "mrr": 0.7138235429100814,
1704
+ "num_pred": 52,
1705
+ "num_data": 52
1706
+ },
1707
+ "ViDoRe_biomedical_lectures_v2_multilingual": {
1708
+ "hit@1": 0.54375,
1709
+ "hit@5": 0.8671875,
1710
+ "hit@10": 0.925,
1711
+ "ndcg_linear@1": 0.54375,
1712
+ "ndcg_linear@5": 0.6054111423513364,
1713
+ "ndcg_linear@10": 0.6414367435231766,
1714
+ "ndcg_exponential@1": 0.54375,
1715
+ "ndcg_exponential@5": 0.6054111423513364,
1716
+ "ndcg_exponential@10": 0.6414367435231766,
1717
+ "precision@1": 0.54375,
1718
+ "precision@5": 0.280625,
1719
+ "precision@10": 0.17921875000000004,
1720
+ "recall@1": 0.34893668831168834,
1721
+ "recall@5": 0.6428547953272218,
1722
+ "recall@10": 0.7504492436179018,
1723
+ "f1@1": 0.3920716089466089,
1724
+ "f1@5": 0.34705303637334894,
1725
+ "f1@10": 0.2615462040342555,
1726
+ "map@1": 0.54375,
1727
+ "map@5": 0.5308493923611112,
1728
+ "map@10": 0.5506469900025983,
1729
+ "mrr": 0.6820034012938432,
1730
+ "num_pred": 640,
1731
+ "num_data": 640
1732
+ },
1733
+ "ViDoRe_economics_reports_v2_multilingual": {
1734
+ "hit@1": 0.6637931034482759,
1735
+ "hit@5": 0.9051724137931034,
1736
+ "hit@10": 0.9439655172413793,
1737
+ "ndcg_linear@1": 0.6637931034482759,
1738
+ "ndcg_linear@5": 0.5690289622254149,
1739
+ "ndcg_linear@10": 0.5555497958939227,
1740
+ "ndcg_exponential@1": 0.6637931034482759,
1741
+ "ndcg_exponential@5": 0.5690289622254149,
1742
+ "ndcg_exponential@10": 0.5555497958939227,
1743
+ "precision@1": 0.6637931034482759,
1744
+ "precision@5": 0.4844827586206897,
1745
+ "precision@10": 0.39525862068965517,
1746
+ "recall@1": 0.10850117235362683,
1747
+ "recall@5": 0.29896154639880523,
1748
+ "recall@10": 0.4237747398378403,
1749
+ "f1@1": 0.1579940079456488,
1750
+ "f1@5": 0.30111517740287175,
1751
+ "f1@10": 0.3318009802129794,
1752
+ "map@1": 0.6637931034482759,
1753
+ "map@5": 0.45995689655172417,
1754
+ "map@10": 0.41782926652461755,
1755
+ "mrr": 0.7757483714298299,
1756
+ "num_pred": 232,
1757
+ "num_data": 232
1758
+ },
1759
+ "ViDoRe_esg_reports_v2_multilingual": {
1760
+ "hit@1": 0.4692982456140351,
1761
+ "hit@5": 0.7543859649122807,
1762
+ "hit@10": 0.8947368421052632,
1763
+ "ndcg_linear@1": 0.4692982456140351,
1764
+ "ndcg_linear@5": 0.47363958068528034,
1765
+ "ndcg_linear@10": 0.5278569087905044,
1766
+ "ndcg_exponential@1": 0.4692982456140351,
1767
+ "ndcg_exponential@5": 0.47363958068528034,
1768
+ "ndcg_exponential@10": 0.5278569087905044,
1769
+ "precision@1": 0.4692982456140351,
1770
+ "precision@5": 0.23070175438596496,
1771
+ "precision@10": 0.16622807017543864,
1772
+ "recall@1": 0.22204469507101088,
1773
+ "recall@5": 0.5130325814536341,
1774
+ "recall@10": 0.6723318713450294,
1775
+ "f1@1": 0.27248698736902005,
1776
+ "f1@5": 0.2758553826318319,
1777
+ "f1@10": 0.23304408809134242,
1778
+ "map@1": 0.4692982456140351,
1779
+ "map@5": 0.3855445906432748,
1780
+ "map@10": 0.41057432354033235,
1781
+ "mrr": 0.6025485930412379,
1782
+ "num_pred": 228,
1783
+ "num_data": 228
1784
+ },
1785
+ "VisRAG_ArxivQA": {
1786
+ "hit@1": 0.8370098039215687,
1787
+ "hit@5": 0.9387254901960784,
1788
+ "hit@10": 0.9607843137254902,
1789
+ "ndcg_linear@1": 0.8370098039215687,
1790
+ "ndcg_linear@5": 0.8937608913411579,
1791
+ "ndcg_linear@10": 0.9009141318725857,
1792
+ "ndcg_exponential@1": 0.8370098039215687,
1793
+ "ndcg_exponential@5": 0.8937608913411579,
1794
+ "ndcg_exponential@10": 0.9009141318725857,
1795
+ "precision@1": 0.8370098039215687,
1796
+ "precision@5": 0.18774509803921569,
1797
+ "precision@10": 0.09607843137254901,
1798
+ "recall@1": 0.8370098039215687,
1799
+ "recall@5": 0.9387254901960784,
1800
+ "recall@10": 0.9607843137254902,
1801
+ "f1@1": 0.8370098039215687,
1802
+ "f1@5": 0.3129084967320262,
1803
+ "f1@10": 0.17468805704099824,
1804
+ "map@1": 0.8370098039215687,
1805
+ "map@5": 0.8786151960784314,
1806
+ "map@10": 0.8815772836912543,
1807
+ "mrr": 0.8829700739994205,
1808
+ "num_pred": 816,
1809
+ "num_data": 816
1810
+ },
1811
+ "VisRAG_ChartQA": {
1812
+ "hit@1": 0.8412698412698413,
1813
+ "hit@5": 0.9365079365079365,
1814
+ "hit@10": 0.9523809523809523,
1815
+ "ndcg_linear@1": 0.8412698412698413,
1816
+ "ndcg_linear@5": 0.8943055298348217,
1817
+ "ndcg_linear@10": 0.8988938604112962,
1818
+ "ndcg_exponential@1": 0.8412698412698413,
1819
+ "ndcg_exponential@5": 0.8943055298348217,
1820
+ "ndcg_exponential@10": 0.8988938604112962,
1821
+ "precision@1": 0.8412698412698413,
1822
+ "precision@5": 0.1873015873015872,
1823
+ "precision@10": 0.09523809523809518,
1824
+ "recall@1": 0.8412698412698413,
1825
+ "recall@5": 0.9365079365079365,
1826
+ "recall@10": 0.9523809523809523,
1827
+ "f1@1": 0.8412698412698413,
1828
+ "f1@5": 0.3121693121693121,
1829
+ "f1@10": 0.17316017316017318,
1830
+ "map@1": 0.8412698412698413,
1831
+ "map@5": 0.8801587301587303,
1832
+ "map@10": 0.8817460317460317,
1833
+ "mrr": 0.8835523818669886,
1834
+ "num_pred": 63,
1835
+ "num_data": 63
1836
+ },
1837
+ "VisRAG_MP-DocVQA": {
1838
+ "hit@1": 0.8358714043993232,
1839
+ "hit@5": 0.9560067681895094,
1840
+ "hit@10": 0.9661590524534687,
1841
+ "ndcg_linear@1": 0.8358714043993232,
1842
+ "ndcg_linear@5": 0.9030703546122169,
1843
+ "ndcg_linear@10": 0.9062989424027735,
1844
+ "ndcg_exponential@1": 0.8358714043993232,
1845
+ "ndcg_exponential@5": 0.9030703546122169,
1846
+ "ndcg_exponential@10": 0.9062989424027735,
1847
+ "precision@1": 0.8358714043993232,
1848
+ "precision@5": 0.19120135363790183,
1849
+ "precision@10": 0.09661590524534686,
1850
+ "recall@1": 0.8358714043993232,
1851
+ "recall@5": 0.9560067681895094,
1852
+ "recall@10": 0.9661590524534687,
1853
+ "f1@1": 0.8358714043993232,
1854
+ "f1@5": 0.3186689227298365,
1855
+ "f1@10": 0.1756652822642671,
1856
+ "map@1": 0.8358714043993232,
1857
+ "map@5": 0.8851663846587704,
1858
+ "map@10": 0.8864656353235033,
1859
+ "mrr": 0.8879222956395415,
1860
+ "num_pred": 591,
1861
+ "num_data": 591
1862
+ },
1863
+ "VisRAG_SlideVQA": {
1864
+ "hit@1": 0.9514388489208633,
1865
+ "hit@5": 0.9928057553956835,
1866
+ "hit@10": 0.9964028776978417,
1867
+ "ndcg_linear@1": 0.9514388489208633,
1868
+ "ndcg_linear@5": 0.9623683640825111,
1869
+ "ndcg_linear@10": 0.9655625858960156,
1870
+ "ndcg_exponential@1": 0.9514388489208633,
1871
+ "ndcg_exponential@5": 0.9623683640825111,
1872
+ "ndcg_exponential@10": 0.9655625858960156,
1873
+ "precision@1": 0.9514388489208633,
1874
+ "precision@5": 0.24676258992805752,
1875
+ "precision@10": 0.1248201438848921,
1876
+ "recall@1": 0.8237410071942446,
1877
+ "recall@5": 0.9820143884892086,
1878
+ "recall@10": 0.9910071942446043,
1879
+ "f1@1": 0.8663069544364508,
1880
+ "f1@5": 0.3872901678657075,
1881
+ "f1@10": 0.2191519511663397,
1882
+ "map@1": 0.9514388489208633,
1883
+ "map@5": 0.9482763788968825,
1884
+ "map@10": 0.9500670891857942,
1885
+ "mrr": 0.9705979590193544,
1886
+ "num_pred": 556,
1887
+ "num_data": 556
1888
+ },
1889
+ "VisRAG_InfoVQA": {
1890
+ "hit@1": 0.8802228412256268,
1891
+ "hit@5": 0.9860724233983287,
1892
+ "hit@10": 0.9944289693593314,
1893
+ "ndcg_linear@1": 0.8802228412256268,
1894
+ "ndcg_linear@5": 0.9400365862338241,
1895
+ "ndcg_linear@10": 0.942892788329613,
1896
+ "ndcg_exponential@1": 0.8802228412256268,
1897
+ "ndcg_exponential@5": 0.9400365862338241,
1898
+ "ndcg_exponential@10": 0.942892788329613,
1899
+ "precision@1": 0.8802228412256268,
1900
+ "precision@5": 0.19721448467966574,
1901
+ "precision@10": 0.09944289693593314,
1902
+ "recall@1": 0.8802228412256268,
1903
+ "recall@5": 0.9860724233983287,
1904
+ "recall@10": 0.9944289693593314,
1905
+ "f1@1": 0.8802228412256268,
1906
+ "f1@5": 0.328690807799443,
1907
+ "f1@10": 0.18080526715624215,
1908
+ "map@1": 0.8802228412256268,
1909
+ "map@5": 0.9243964716805942,
1910
+ "map@10": 0.9256648759782464,
1911
+ "mrr": 0.9259750810934655,
1912
+ "num_pred": 718,
1913
+ "num_data": 718
1914
+ },
1915
+ "VisRAG_PlotQA": {
1916
+ "hit@1": 0.6199304750869061,
1917
+ "hit@5": 0.8632676709154113,
1918
+ "hit@10": 0.9420625724217845,
1919
+ "ndcg_linear@1": 0.6199304750869061,
1920
+ "ndcg_linear@5": 0.7491456567501216,
1921
+ "ndcg_linear@10": 0.7753467659947195,
1922
+ "ndcg_exponential@1": 0.6199304750869061,
1923
+ "ndcg_exponential@5": 0.7491456567501216,
1924
+ "ndcg_exponential@10": 0.7753467659947195,
1925
+ "precision@1": 0.6199304750869061,
1926
+ "precision@5": 0.17265353418308227,
1927
+ "precision@10": 0.09420625724217846,
1928
+ "recall@1": 0.6199304750869061,
1929
+ "recall@5": 0.8632676709154113,
1930
+ "recall@10": 0.9420625724217845,
1931
+ "f1@1": 0.6199304750869061,
1932
+ "f1@5": 0.2877558903051372,
1933
+ "f1@10": 0.17128410407668812,
1934
+ "map@1": 0.6199304750869061,
1935
+ "map@5": 0.711201235998455,
1936
+ "map@10": 0.7224305210689914,
1937
+ "mrr": 0.7246660449329103,
1938
+ "num_pred": 863,
1939
+ "num_data": 863
1940
+ },
1941
+ "ViDoSeek-page-fixed": {
1942
+ "hit@1": 0.7653239929947461,
1943
+ "hit@5": 0.9693520140105079,
1944
+ "hit@10": 0.9859894921190894,
1945
+ "ndcg_linear@1": 0.7653239929947461,
1946
+ "ndcg_linear@5": 0.8805740446728709,
1947
+ "ndcg_linear@10": 0.8859719270708654,
1948
+ "ndcg_exponential@1": 0.7653239929947461,
1949
+ "ndcg_exponential@5": 0.8805740446728709,
1950
+ "ndcg_exponential@10": 0.8859719270708654,
1951
+ "precision@1": 0.7653239929947461,
1952
+ "precision@5": 0.19387040280210155,
1953
+ "precision@10": 0.09859894921190891,
1954
+ "recall@1": 0.7653239929947461,
1955
+ "recall@5": 0.9693520140105079,
1956
+ "recall@10": 0.9859894921190894,
1957
+ "f1@1": 0.7653239929947461,
1958
+ "f1@5": 0.32311733800350273,
1959
+ "f1@10": 0.17927081674892534,
1960
+ "map@1": 0.7653239929947461,
1961
+ "map@5": 0.8504086398131931,
1962
+ "map@10": 0.8526467767492285,
1963
+ "mrr": 0.8533107483629478,
1964
+ "num_pred": 1142,
1965
+ "num_data": 1142
1966
+ },
1967
+ "ViDoSeek-doc": {
1968
+ "hit@1": 0.9964973730297724,
1969
+ "hit@5": 0.9982486865148862,
1970
+ "hit@10": 0.9982486865148862,
1971
+ "ndcg_linear@1": 0.6722124927028604,
1972
+ "ndcg_linear@5": 0.8245917360155337,
1973
+ "ndcg_linear@10": 0.7998334240804187,
1974
+ "ndcg_exponential@1": 0.44058043532649477,
1975
+ "ndcg_exponential@5": 0.7078106015035031,
1976
+ "ndcg_exponential@10": 0.7278358127539598,
1977
+ "precision@1": 0.9964973730297724,
1978
+ "precision@5": 0.9057793345008758,
1979
+ "precision@10": 0.8061295971978986,
1980
+ "recall@1": 0.058748286026062344,
1981
+ "recall@5": 0.26588614485387413,
1982
+ "recall@10": 0.46432028220053345,
1983
+ "f1@1": 0.11020431579671368,
1984
+ "f1@5": 0.4027884901547074,
1985
+ "f1@10": 0.5760065974102732,
1986
+ "map@1": 0.9964973730297724,
1987
+ "map@5": 0.8967600700525393,
1988
+ "map@10": 0.790665598405153,
1989
+ "mrr": 0.9974570854301065,
1990
+ "num_pred": 1142,
1991
+ "num_data": 1142
1992
+ },
1993
+ "MMLongBench-page-fixed": {
1994
+ "hit@1": 0.5119331742243437,
1995
+ "hit@5": 0.698090692124105,
1996
+ "hit@10": 0.7541766109785203,
1997
+ "ndcg_linear@1": 0.5119331742243437,
1998
+ "ndcg_linear@5": 0.5651734278679845,
1999
+ "ndcg_linear@10": 0.5892310388140871,
2000
+ "ndcg_exponential@1": 0.5119331742243437,
2001
+ "ndcg_exponential@5": 0.5651734278679845,
2002
+ "ndcg_exponential@10": 0.5892310388140871,
2003
+ "precision@1": 0.5119331742243437,
2004
+ "precision@5": 0.18186157517899762,
2005
+ "precision@10": 0.10560859188544153,
2006
+ "recall@1": 0.40360126150698944,
2007
+ "recall@5": 0.6205307421297875,
2008
+ "recall@10": 0.685782475281282,
2009
+ "f1@1": 0.4355096574070321,
2010
+ "f1@5": 0.2680566555443714,
2011
+ "f1@10": 0.17498498807234028,
2012
+ "map@1": 0.5119331742243437,
2013
+ "map@5": 0.5250089498806683,
2014
+ "map@10": 0.5373743239671896,
2015
+ "mrr": 0.5946821333816329,
2016
+ "num_pred": 838,
2017
+ "num_data": 838
2018
+ },
2019
+ "MMLongBench-doc": {
2020
+ "hit@1": 0.7231503579952268,
2021
+ "hit@5": 0.8126491646778043,
2022
+ "hit@10": 0.8424821002386634,
2023
+ "ndcg_linear@1": 0.5083532219570406,
2024
+ "ndcg_linear@5": 0.5303117612876772,
2025
+ "ndcg_linear@10": 0.5079193965835419,
2026
+ "ndcg_exponential@1": 0.35492669621547906,
2027
+ "ndcg_exponential@5": 0.4490330748532223,
2028
+ "ndcg_exponential@10": 0.45702082629074353,
2029
+ "precision@1": 0.7231503579952268,
2030
+ "precision@5": 0.5801909307875895,
2031
+ "precision@10": 0.5107398568019093,
2032
+ "recall@1": 0.02337436345545821,
2033
+ "recall@5": 0.09151651589906322,
2034
+ "recall@10": 0.1565633063253933,
2035
+ "f1@1": 0.04481901328603392,
2036
+ "f1@5": 0.15171251843886188,
2037
+ "f1@10": 0.22465064990988276,
2038
+ "map@1": 0.7231503579952268,
2039
+ "map@5": 0.5462649164677804,
2040
+ "map@10": 0.46275377462927186,
2041
+ "mrr": 0.7659964982692293,
2042
+ "num_pred": 838,
2043
+ "num_data": 838
2044
+ }
2045
+ }
2046
+ }
2047
+ }