In the spirit of community building, the Amsterdam ELLIS unit is supporting students teams to tackle data challenges, sprints, and hackathons in machine learning topics. We support diverse teams of students looking to participate in such challenges with coaching from ELLIS faculty members and compute power. 

The unit’s first team of PhD, MSc, and BSc students are tackling Google’s BIG-Bench Challenge.

The team, led by Mario Giuglianelli, Jaap Jumelet, and Hugh Mee Wong, includes four PhD, seven MSc, and 2 BSc students. They will additionally receive feedback and coaching from five ELLIS faculty members, who will guide the team as they participate in the challenge.

Google’s Beyond the imitation game Benchmark (BIG-Bench) is a collaborative benchmarking challenge for large language models. BIG-Bench aims to examine questions on the future and scalability of large language models, and especially to test, find, and determine their capabilities and limitations.

“By soliciting benchmark tasks from the community we hope to provide a diverse and large scale benchmark […] Our goal is to test model capabilities quantitatively, either establishing or disproving possible limitations of these models.” (Call for benchmark tasks).

In the next few weeks, the ELLIS team will work together to design tasks that challenge or exceed the capabilities of current models in order to establish or disprove their possible limitations. We wish them luck!