@@ -117,9 +117,9 @@ <h3 class="fw-light text-nowrap">
117
117
alt ="blog "
118
118
class ="img-fluid "
119
119
/> </ a >
120
- < a href ="https://arxiv.org/html/2406.11927v1 "
120
+ < a href ="https://arxiv.org/abs/2410.01999v1#:~:text=View%20a%20PDF%20of%20the%20paper%20titled%20CodeMMLU:%20A%20Multi-Task "
121
121
> < img
122
- src ="https://img.shields.io/badge/2406.11927 -red?style=for-the-badge&label=arXiv "
122
+ src ="https://img.shields.io/badge/2410.01999 -red?style=for-the-badge&label=arXiv "
123
123
alt ="leaderboard "
124
124
class ="img-fluid "
125
125
/> </ a >
@@ -131,13 +131,13 @@ <h3 class="fw-light text-nowrap">
131
131
/> </ a >
132
132
</ div >
133
133
< div class ="d-flex flex-row justify-content-center gap-3 ">
134
- < a href ="https://github.com/FSoft-AI4Code/RepoExec "
134
+ < a href ="https://github.com/FSoft-AI4Code/CodeMMLU "
135
135
> < img
136
136
src ="https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge&logo=github&logoColor=white "
137
137
alt ="github "
138
138
class ="img-fluid "
139
139
/> </ a >
140
- < a href ="https://github.com/FSoft-AI4Code/RepoExec/blob/master/paper/main.pdf "
140
+ < a href ="https://arxiv.org/abs/2410.01999v1#:~:text=View%20a%20PDF%20of%20the%20paper%20titled%20CodeMMLU:%20A%20Multi-Task "
141
141
> < img
142
142
src ="https://img.shields.io/badge/📝 paper-%23121011.svg?style=for-the-badge "
143
143
alt ="paper "
@@ -175,12 +175,12 @@ <h3>📝 Notes</h3>
175
175
< ol >
176
176
< li >
177
177
Evaluated using
178
- < a href ="https://github.com/FSoft-AI4Code/RepoExec "
179
- > RepoExec </ a
178
+ < a href ="https://github.com/FSoft-AI4Code/CodeMMLU "
179
+ > CodeMMLU </ a
180
180
>
181
181
</ li >
182
182
< li >
183
- Models are ranked according to Pass@1 using greedy decoding.
183
+ Models are ranked according to Accuracy using greedy decoding.
184
184
</ li >
185
185
<!-- <li>
186
186
<i>Complete</i> vs <i>Instruct</i>:
@@ -220,6 +220,12 @@ <h3>🤗 More Leaderboards</h3>
220
220
benchmarks and leaderboards, such as:
221
221
< div class ="inline-block mt-3 ">
222
222
< ol >
223
+ < li >
224
+ < a
225
+ href ="https://repoexec.github.io/ "
226
+ > RepoExec Leaderboard</ a
227
+ >
228
+ </ li >
223
229
< li >
224
230
< a
225
231
href ="https://bigcode-bench.github.io/ "
0 commit comments