BLOGS
Advancing AI Frontiers: The ELLIS Amsterdam MSc Honours Programme (1/2)
The ELLIS unit in Amsterdam is dedicated to developing and retaining a new wave of AI researchers through its partnerships, which are crucial for achieving scientific distinction and pioneering research in modern AI across Europe. To achieve this, the ELLIS unit created a MSc Honours programme in 2020.
The goal of the programme is to enable excellent MSc students to have their AI thesis projects supervised by ELLIS researchers in different countries. The programme therefore does not only ensure diverse and constructive feedback from researchers of different institutions, but also gives the students the opportunity to expand their network within their area of interest and experience different research environments. Finally, working with high-level researchers enables the students to research state-of-the-art AI topics and be on the front of pushing the boundaries of AI-research in Europe.
We interviewed 13 MSc Honours students and highlighted how the programme helped them reach their academic goals, their key takeaways from the programme, and what is next for them. This post will cover the first half of the students.
We would like to congratulate all of the ELLIS unit Amsterdam’s MSc Honours students for their exceptional achievements!
Avik Pal
Thesis project title: Hyperbolic embedding of multiple hierarchies
Supervisors:
– Dr. Pascal Mettes (ELLIS Member, University of Amsterdam)
– Prof. Fabio Galasso (ELLIS Member, Sapienza University of Rome)
Research Summary:
Recent trends in AI lean towards large-scale multimodal learning (e.g., CLIP) which requires an immense amount of data. This study aims to explore the potential of creating a semantic multimodal representation space by leveraging interpretable hyperbolic embeddings as an inductive prior. The initial focus is on Vision and Language modes, aiming to further accelerate training while reducing data requirements. The proposed methodology involves exploring hyperbolic representation learning methods and alignment mechanisms, incorporating insights from existing works.
MSc Honours Opportunity:
I am currently collaborating with Ph.D. students from PINLab at Sapienza University on my research topic. My research visit facilitated by the MSc Honours Programme would enable closer collaboration with them. Further, I am looking forward to presenting my work and findings along with all the interesting discussions and insights I’ll obtain from Fabio and his team.
Jona Ruthardt
Thesis project title: Large Language Models as Visual Experts
Supervisors:
– Dr. Gertjan Burghouts (ELLIS Member, Senior Researcher at TNO)
– Dr. Yuki Asano (ELLIS Member, University of Amsterdam)
– Prof. Dr. Serge Belongie (ELLIS Fellow, University of Copenhagen)
Research Summary:
The recent surge in popularity of large language models (LLMs) can be attributed to their reasoning abilities, capacity to generate meaningful text, and the vast repository of embedded encyclopedic knowledge. Especially their comprehensive understanding of real-world concepts can be exploited in the vision domain to generalize to previously unseen objects, even when annotated data is not readily available.
The project aims to leverage LLMs for computer vision tasks that traditionally relied heavily on expert annotations and humans-in-the-loop for competitive performance. By tapping into the wealth of class-specific information within LLMs, Jona and his supervisors explore ways to diminish the reliance on expensive manual annotations without compromising performance on downstream tasks. Owing to this cross-modal knowledge transfer, resulting approaches promise to be scalable to more diverse datasets and also applicable to low-resource computer vision problems.
MSc Honours Opportunity:
Being part of the MSc Honours Programme provides me with the exciting opportunity to collaborate with Prof. Serge Belongie. His profound expertise in the broader field of Computer Vision, particularly within the targeted research direction, will serve as a significant asset to our project. Thanks to facilitating and funding a multi-week research visit, the MSc Honours Programme will provide me with a firsthand understanding of the Pioneer Centre for AI and foster connections with fellow researchers. The inclusion of three distinguished supervisors with complementary backgrounds and mentorship further enhances my learning opportunities and elevates the project’s overall quality. These invaluable experiences will equip me with key insights that will inform and guide my future academic and professional aspirations.
Robin Sasse
Thesis project title: Hierarchical Auto-Vocabulary Segmentation
Supervisors:
– Dr. Martin Oswald (ELLIS Member, University of Amsterdam)
– Dr. Francis Engelmann (ELLIS Member, ETH Zurich)
– MSc Osman Ülger (University of Amsterdam)
Research Summary:
While humans have the ability to recognize a near-infinite amount of objects, automated segmentation systems usually rely on a fixed number of objects, corresponding to the dataset used during training. Open-vocabulary segmentation methods address this issue by leveraging user input in the form of a list of objects presented alongside the image. Notwithstanding, these methods still rely on user interaction in order to provide results. Recently researchers have proposed new methods, which remove the need for human interaction and instead, rely on large visual-language models to generate labels from the image itself (we call these methods auto-vocabulary segmentation methods). In hierarchical segmentation multiple levels of class abstractions are identified at once (e.g. tire is part of a wheel is part of a car). Hierarchical auto-vocabulary segmentation yields a pixel-level classification with automatic labelling and hierarchical structuring all in one model pipeline.
MSc Honours Opportunity:
With ELLIS I have the chance to spent a month at the ETH Zurich, arguably Europe’s leading research institution in AI and especially Computer Vision. While I will lay the ground-work of my research in Amsterdam, I plan to refine my approach and broaden its use cases towards the end in Zurich. With Dr. Francis Engelmann, a leading researcher in 3D scene segmentation joins the project and I hope to receive valuable input from his experience in 3D, which I can either apply to my 2D segmentation method or find a way to extend my method into the 3D world. Spanning my network across the UvA and ETHZ, I hope to establish valuable connections across two of Europe’s most prestigious AI research institutions.
Egoitz Gonzalez
Thesis project title: SLAM Expert System for Learning Optimal Tracking Strategies
Supervisors:
– Dr. Martin R. Oswald (ELLIS Member, University of Amsterdam)
– Prof. Dr. Marc Pollefeys (ELLIS Member, ETH Zürich)
Research Summary & MSc Honours Opportunity:
Simultaneous Localization and Mapping (SLAM) is a well-studied approach in Computer Vision for 3D scene representation from video captured by a moving camera. My research will focus on improving the most recent Dense Neural SLAM approaches by developing new techniques that will improve the robustness, tacking performance and fidelity of the 3D scene model. This project not only allows me employ diverse methodologies and acquire knowledge about 3D scene representation and SLAM but also helps me better understand how actual research projects work and how collaboration can lead to innovative ideas. Working with people that inspire and motivate you plays a key role, as well as learning from them and cultivating new skills that I could apply in my future projects.
Thanks to the ELLIS Honours Programme, I will have the opportunity to spend some time at ETH Zürich, collaborating with both supervisors and connecting with additional specialists in the field. This is an excellent opportunity that I likely wouldn’t have had access to without ELLIS, and it promises to greatly benefit both my personal and professional development. Engaging with other research groups, expanding my professional network, and learning from experts are undoubtedly the most effective ways of enriching my research experience and fostering personal growth.
Walter Simonichi
Thesis project title: Are all you need your nearest neighbors and their gradients?
Supervisors:
– Dr. Yuki Asano (ELLIS Member, University of Amsterdam)
– Dr. Andrei Bursuc (ELLIS Member, Deputy Scientific Director at valeo.ai)
Research Summary:
The k-nearest neighbors algorithm has been shown to scale well to large amounts of data and has demonstrated potential even before neural networks were relevant. This algorithm is currently being used as a downstream task to evaluate the representations learned by vision transformers and as a simple recipe for visual in-context learning. This research project aims to investigate whether this algorithm’s performance can be further improved by incorporating the gradients resulting from one or more self-supervised losses.
MSc Honours Opportunity:
I am excited to participate in the ELLIS MSc Honours Program, which will allow me to visit Paris, one of the leading European hubs for AI research, and join the Valeo.ai research lab as a visiting student researcher. I look forward to meeting, collaborating, and discussing ideas with the research group, which will significantly impact my thesis work. Moreover, the visit will also help me build my research network, laying the groundwork for future research projects.
Madhura Pawar
Thesis project title: Fair Link Recommendations to Improve Centrality of Minority Groups in Social Networks
Supervisors:
– Dr. Fernando P. Santos (ELLIS Member, University of Amsterdam)
– Dr. Fariba Karimi (TU Graz)
Research Summary:
Algorithmic recommendations in online social media platforms can impact how social networks grow and, importantly, individuals’ opinions and visibility. Madhura and her supervisors aim to investigate how link recommendations that result from a combination of node and network features can affect network growth. They are currently working on understanding how fair link recommendation algorithms are towards minority groups. Currently, they have found that some of these algorithms underrepresent minorities. Hence, they also aim to develop an interventional algorithm to improve the visibility of minorities. This project involves a combination of methods at the interface of computational social sciences and AI: opinion dynamics simulation, complex network analysis, and graph learning/recommendations.
MSc Honours Opportunity:
The ELLIS MSc Honours Program provides me with an ideal opportunity to learn and collaborate with researchers from another university. I very much look forward to sharing my current work with Ana’s team. It would be an insightful (and fun) journey where I look forward to receiving their feedback as well as exploring & understanding the problems they are working on.
Valentinos Pariza
Thesis project title: Improving In-Context-Learning Abilities of Vision Models with k-Nearest Neighbors
Supervisors:
– Dr. Yuki Asano (ELLIS Member, University of Amsterdam)
– Dr. Gertjan Burghouts (ELLIS Member, Senior Researcher at TNO)
– MSc Mohammad Reza Salehi (University of Amsterdam)
– Dr. Francesco Locatello (Ellis Member, Institute of Science and Technology Austria)
Research Summary:
Recent years have seen a surge in NLP with In-Context Learning (ICL), but Computer Vision (CV) lags, relying on task-specific models. Efforts to bridge this gap focus on training Generalist Vision Transformers, Prompt Diffusion, and Representation Learning Models with k-NN operators for patch retrieval. Research of the latter one aims to enhance the Dense k-NN Retrieval Evaluation by incorporating the k-NN operator in dense encoder training. However, current methods underutilize the k-NN operator, prompting Valentinos and his supervisors to propose a new pretraining strategy to optimize patch representations for clustering and improve the Dense k-NN Retrieval Evaluation.
MSc Honours Opportunity:
Visiting the Institute of Science and Technology Austria (ISTA) for my Master Thesis as part of this Ellis programme promises valuable insights and cutting-edge research with Dr. Francessco. In addition to that, collaborating with researchers and PhD students there will broaden my perspective, and will provide me with invaluable feedback. Both of these interactions will allow me to solidify and extend further my Master Thesis work. Last, this opportunity will allow me to establish connections within the research community there that could open doors to future collaborations.
See more information regarding the MSc Honours programme.