April 19, 2021
In the spirit of community building, the Amsterdam ELLIS unit is supporting students teams to tackle data challenges, sprints, and hackathons in machine learning topics. We support diverse teams of students looking to participate in such challenges with coaching from ELLIS faculty members and compute power.
The unit’s first team of PhD, MSc, and BSc students are tackling Google’s BIG-Bench Challenge.
The team, led by Mario Giuglianelli, Jaap Jumelet, and Hugh Mee Wong, includes four PhD, seven MSc, and 2 BSc students. They will additionally receive feedback and coaching from five ELLIS faculty members, who will guide the team as they participate in the challenge.
Google’s Beyond the imitation game Benchmark (BIG-Bench) is a collaborative benchmarking challenge for large language models. BIG-Bench aims to examine questions on the future and scalability of large language models, and especially to test, find, and determine their capabilities and limitations.
“By soliciting benchmark tasks from the community we hope to provide a diverse and large scale benchmark […] Our goal is to test model capabilities quantitatively, either establishing or disproving possible limitations of these models.” (Call for benchmark tasks).
In the next few weeks, the ELLIS team will work together to design tasks that challenge or exceed the capabilities of current models in order to establish or disprove their possible limitations. We wish them luck!
Mario Giuglianello
PhD, ILLC
Jaap Jumelet
PhD, ILLC
Hugh Mee Wong
MSc AI & MoL
Jelle Bosscher
MSc Logic
Dilyar Buzan
BSc AI
Omar Elbaghdadi
MSc AI
Rahel Habacker
MSc AI
Maartje ter Hoeve
PhD, NLP & IR
Aman Hussain
MSc AI
Agnieszka Kluska
BSc Linguistics
Mátyás Schubert
MSc AI
Clemencia Siro
PhD, NLP
Jeroen Taal
MSc AI