Eijkelboom conducted multiple research projects for his thesis as part of the ELLIS unit Amsterdam’s MSc Honours Programme under a joint supervision of dr. ir. Erik Bekkers (University of Amsterdam) and prof. dr. Michael Bronstein (Oxford University). The programme gave him the chance to advance and apply his research skills and gain first-hand experience in an international research setting focusing on geometric deep learning.
What started out as his initial research with his honours supervisor, as well as his honours thesis project, have turned into two published papers – one in the main track and one in the Topology, Algebra, and Geometry for ML workshop – in the International Conference on Machine Learning (ICML) in 2023 entitled “E(n) Equivariant Message Passing Simplicial Networks” and ” Can strong structural encoding reduce the importance of Message Passing?”. In 2023, Eijkelboom was also awarded the ELLIS unit Amsterdam Travel Grant sponsored by Qualcomm to present his papers.
MSc Honours Programme dishes up Eijkelboom’s international collaboration experience
With his supervisor at UvA, Erik Bekkers, he approached his second supervisor based in Oxford, Michael Bronstein. Later, he was introduced to Michael Bronstein’s colleague, Dr. Francesco Di Giovanni. For him, both collaborations within the UvA and outside were great and fruitful.
He highly values the collaborative dynamics he gained through the programme.
“It’s so nice to work with people that are extremely smart and motivated and have cool ideas. You can just talk about difficult things and have fun; for both projects, it really felt as if we were doing research together rather than me being supervised.”
Eijkelboom highlighted that his highest point as an honours student is getting the opportunity to present at the ICML 2023. “ICML approximately has a 20% acceptance rate. I got reviews, and they were varying from just declining the paper to accepting the paper. So, I’m in a very fortunate position that I got to publish”.
His thesis is woven from his published papers at ICML 2023, including the second work which is his research as an honours student. His quest in the last leg of his study as a Master’s AI student was connecting his major works together with a cohesive story.
Previously in 2022, Eijkelboom was able to present at the NeurIPS Conference’s Journal Showcase Poster Session for his paper “FACT course ([Re] Reproduction Study of Variational Fair Clustering)” which was published in ReScience C Journal (volume 8). He also received the ELLIS unit Amsterdam Travel Grant to fund his travel to the NeurIPS Conference 2022.
How Honours programme aided him to reach his goal and inspired him to collaborate better
Reflecting on his honours journey, Eijkelboom sees the program as a golden ticket to meet more people who are already working in his research area of interest. Thus far, he has met many interesting people whom he reached out to. These are the ones who subsequently aided him in extending his academic ‘network’ even further. As a result, he has been working on several collaborations for a while, which he never expected before. In most of the projects he is working on, he plays a different role, typically a supervising one.
“That’s what I realised with this paper. At some point people also know you.. . They reach out to you and ask you to do a talk or interviews, or ask for collaborations. Since I published my paper, I have maybe four, five, six extra projects going on, which has been really nice because I get to explore more ideas I would not be able to conduct on my own.”
The pleasant professional collaboration experience Eijkelboom previously had has inspired him to be a good collaborator for his projects. He is also currently working with another master’s student, Tin Hadži Veljković, to develop a website for mathematical prerequisite knowledge which will be a full-on website next year. The aim of the platform is to help solve a common problem faced by people who are eager to understand mathematics better. “The hard part about mathematics is that wherever you start with mathematics, you can always go deeper. Maths is the currency or the core of deep learning; it does not only define things, but it’s also like the way we reason about things and the way in which we formulate our thoughts.”
“You know, not knowing mathematics is quite brutal. The closest comparison I can think of is in studying a foreign language. If you’re not fluent in a language, reading a text is super frustrating. You don’t get all the words, or words or grammatical structures appear where you don’t expect them. It’s very similar to reading a maths-heavy paper, where you just don’t understand the content or don’t understand why certain notions are defined in the way they are. And even if you know all symbols, sometimes the overarching picture is still vague. A bit like how just knowing German well enough to understand all words in a work by Kafka does not mean you can appreciate it as a native speaker would. You will typically still get away with a shallow understanding.
Reflecting on his Master’s and Honours programme preparation
This took Eijkelboom a trip down the memory lane, recalling his honours journey, almost one and a half years ago at this point. His interest in the intersection between mathematics and deep learning led him to carefully planning his master’s study and his application for the ELLIS MSc Honours Programme.
“To make the best of your study, start early. Try to picture what you’d like to do and identify in what background knowledge you are deficient and work on that. At some point I realised that in my bachelor’s I didn’t really learn the proper mathematics that I needed to do the research I wanted to do. I studied this material to catch up, and to gain more experience, I started a reading group surrounding the topic of geometric deep learning.”
Eijkelboom then started a reading group with logic, AI, and a few physics students where they watched lectures and critically discussed different topics together. “I felt I understood a bit better how geometric deep learning works. When I felt comfortable, I approached Erik Bekkers to see if he wanted to do a project together… that project is actually the paper I published.”
Further looking back, he chose AI for his bachelor’s and master’s studies in the first place as it offers him enough room to explore within the field he is invested in. “It’s so broad and ubiquitously applicable. If you want to, you can really steer your career or research in pretty much any direction.”
The important step he took was knowing what he wants to do. After eventually studying in an AI program and seeing firsthand what different people are doing, and being in direct contact with many staff members, has really opened his eyes to new possibilities.
Satisfying his curiosity while navigating his research path
“The program is tough. I feel that because people are so preoccupied with trying to catch up all the time during the master’s, they don’t have much time or room to let things digest a bit. You have to allow yourself to think a bit more long-term, even though you’re constantly on deadlines.”
His advice would be to those who are seeing themselves in this field to plan the study and research well. But at the same time, he sees the importance of giving oneself an opportunity to still discover new things and not be worried about coming in here fully prepared.
Eventually, Eijkelboom decided to focus his research on geometric and topological deep learning.
“Don’t be scared, but also don’t wait too long to forge your path. You can also get to a point in the program where you do not know what you want to do, as if you only know a little about a lot of things but do not understand anything well”.
“I think the biggest challenge is definitely just the speed at which everything goes. People are constantly pushing for a new publication. In other fields, it’s very normal to just spend a lot of time working on a paper and fine-tuning it and thinking it over more and more and more, making sure it’s actually fully done when you send it off”.
Eijkelboom sees that, subsequently, the pressure is put on master’s students interested in research, where they feel as if they are expected to have a publication in their master’s studies. But, he notices the high stress not only with students but also for example with reviewers during conferences.
Finding a good balance between his study, side projects, fermented coffee, and hosting a barbecue
Growing up in Amsterdam, Eijkelboom has had the opportunity to have a lot of friends and meet people from different parts of the world who can teach him something quite different from AI. Besides his studies, he is mostly preoccupied by cooking different cuisines and preparing fermented coffee. Both equally brought him joy in between his study and side projects.
He finds that balancing different aspects in life is key to staying mentally healthy. He acknowledges his good time management skill where he can do a lot of work by being efficient about his time allocation while, at the same time, hosting a large group barbecue with his friends.
“I really don’t feel any form of overwork which is remotely like burning out as I have a lot of things outside of it that I also enjoy. I feel like a lot of people also get just stuck because they are constantly with their heads in their work. You just need to go outside, sit in the sun, have a drink, go have some fun and come back to it”.
Eijkelboom will continue his education, pursuing a PhD degree at University of Amsterdam in September 2023.
The ELLIS unit Amsterdam MSc Honours Programme aims at providing masters’ students in artificial intelligence, computer science, computational linguistics, and related fields a first-hand experience with international research collaboration, and connecting them to Europe’s best researchers through the ELLIS network. Find out more information about the MSc Honours programme.