RESEARCH

Excellent research is our primary focus. The Amsterdam ELLIS unit hosts over 30 researchers from the Uni­ver­sity of Amsterdam, with expertise covering an expansive range of topics. In addition to working with these top researchers and scientists, we have also built collaborations with leading AI enterprises in the region through our sponsorship program. Together with these sponsors, we perform world-class research with high relevance and applications for industry, specifically in Deep Vision and Deep Learning, Automotive and Computer Vision.

MACHINE LEARNING FOUNDATIONS

My research is focused on building autonomous systems using probabilistic and statistical modelling. These systems should not simply demonstrate artificial intelligence but artificial humility as well. They need to be transparent about their beliefs and willing to admit when they might be wrong. My research consists of formulating general algorithms that use probabilistic reasoning to answer these questions.

Topic areas: deep learning, Bayesian statistics, probabilistic modelling, generative models, anomaly detection

My research is focused on geometric machine learning with a focus on the mathematical and algorithmic foundations of deep learning. My research is guided by an ambition to solve core problems in medical image analysis, whilst aiming for generic solutions that have a wide application scope. My current research focus is on (generalizations of) group convolutional neural networks and the improvement of computational and representational efficiency through sparse and adaptive learning mechanisms.

Topic areas: deep learning, Bayesian statistics, probabilistic modelling, generative models, anomaly detection, medical image analysis, group convolutional neural networks, (sub-)Riemannian geometry

My research focuses on machine learning for autonomous robots in perceptually challenging environments. Currently, I’m working on new ways to exploit known robot models and/or simulators to make reinforcement learning more efficient. I am looking to use a generative model of the robot to characterise its belief over unknown parameters, and pre-training a policy that learns to trade-off exploration and exploitation based on this characterisation.

Topic areas: reinforcement learning, robotics, machine learning

My group combines probabilistic programming with deep learning to develop probabilistic models for machine learning, data science, and artificial intelligence. I am one of the creators of Anglican, a probabilistic programming system that is closely integrated with Clojure. Our group currently develops Probabilistic Torch, a library for deep generative models that extends PyTorch. Many of my students collaborate with other faculty to develop applications to neuroscience, health, natural language processing, and robotics.

Topic areas: probabilistic programming, differentiable programming, variational inference, Monte Carlo methods

My research is focused on the design and study of statistical methods for the estimation of causal models from data and their use in predicting the effects of interventions on systems. Most of the applications I work on are in the biological sciences, the medical domain and in designing recommender systems.

Topic areas: causality, causal modeling, causal inference, causal discovery

Prof. Dr. Max Welling is a research chair in Machine Learning at the University of Amsterdam and leads AMLAB. He is also VP at Qualcomm Technologies and fellow of CIFAR and ELLIS. He co-directs two ICAI labs: QUVA and Delta labs. He is furthermore in the founding board of the ELLIS society and directs the Amsterdam ELLIS unit. Prof. Welling main interest is geometric deep learning, graphical and generative models, and quantum ML.

Topic areas: geometric deep learning, graphical and generative models, quantum machine learning

Patrick Forré’s research is centered around the study of mathematical aspects of machine learning, like the analysis of causal and graphical models, information theory and conditional independence structures, geometric deep learning and topology, etc. Furthermore, he is enthusiastic about applications of machine learning techniques to scientific data problems and enjoys collaborating with researchers from other fields of sciences.

Topic areas: causal and graphical models, information theory, conditional independence structures, geometric deep learning, topology

My research group designs mathematically well-founded machine learning methods for automatic hyper-parameter tuning in online convex optimization. This includes identifying statistically ‘easy’ situations (e.g. low noise, small norm optimal parameters, etc.) in which it is possible to learn more efficiently, with less data. We then construct adaptive methods that exploit these easy cases when present, but automatically fall back to slower robust learning strategies when there is no easy structure. In addition, we have recently started on a formal mathematical analysis of explainability methods, which explain the black-box decisions made by machine learning systems to a user. In the past I have also worked on topics in information theory, model selection for Bayesian statistics, and PAC-Bayesian concentration inequalities.

Topic areas: online convex optimization, learning theory, explainable machine learning, model selection

My research focuses on causality and causality-inspired machine learning, i.e. applications of causal inference to machine learning that would improve its robustness, safety and sample-efficiency. My expertise is in learning causal relations jointly from different observational and experimental settings, especially in the case of latent confounders and small samples, as well as methods to design experiments that would allow one to learn causal relations in a sample-efficient and intervention-efficient way. Recently, I have been focusing on causality-inspired domain adaptation, both in the context of policy transfer in RL and for improving the translation of medical insights from mice to humans. 

Topic areas: causality, causal inference, causality-inspired ML, robustness, safety

COMPUTER VISION

Arnold Smeulders is professor of artificial intelligence at the University of Amsterdam and currently envoy for ELLIS to the EU. He is recipient of the Korteweg medaillon and old enough to receive the ACM SIGMM lifetime award. He is member of the Academia Europeana. He has graduated 60 PhD students.

Topic areas: computer vision

Prof. dr. Cees Snoek heads the Video & Image Sense Lab, where we make sense of video and images with artificial and human intelligence. We study computer vision, deep learning and cognitive science. The VIS Lab also embeds four public-private AI labs. QUVA Lab with Qualcomm, Delta Lab with Bosch, Atlas Lab with TomTom and AIM Lab with the Inception Institute of Artificial Intelligence. For our most recent work please check: https://ivi.fnwi.uva.nl/vislab/.

Topic areas: computer vision, deep learning, cognitive science

My research focuses on three axes: Temporal Machine Learning and Dynamics, Efficient Vision, and Machine Learning for Oncology. Here are some research questions I’m interested in. How do we model these complex and perhaps continuous or even online temporal and spatiotemporal data? Is there an ordinary or partial or stochastic differential equation that corresponds to ImageNet? Is there a link between the consistent and mystical learning behind deep neural networks and chaotic behaviors (or lack thereof) in dynamical systems? Is there a causal connection between neural networks and dynamics? Can learn spatiotemporal models that generalize beyond static and stationary data? These and many more are some fundamental questions that I and colleagues are trying to answer.

Topic areas: temporal machine learning and dynamics, efficient vision, machine learning for oncology

We research multimedia analytics by developing AI techniques for getting the richest information possible from the data (image/video/text/graphs) interactions surpassing human and machine intelligence, and visualizations blending it all in effective interfaces for applications in health, forensics and law enforcement, cultural heritage, urban livability, and social media analysis.

Topic areas: multimedia integration, interactive learning, visual analytics

My research is focused on automatically understanding the content of videos, with a focus on bridging the gap between deep video learning and information from prior semantic and symbolic knowledge. The research agenda includes non-Euclidean manifolds for video representations, recognising actions without training examples, learning the common structure between video examples, and reasoning beyond action labels.

Topic areas: computer vision, video understanding, structured video representations

My research is focusing on physics based computer vision: Light traveling in the 3D world interacts with the scene through intricate processes before being captured by a camera. These processes result in the dazzling effects like color and shading, complex surface and material appearance, different weathering, just to name a few. Physics based vision aims to invert the processes to recover the scene properties, such as shape, reflectance, light distribution, medium properties, etc., from the images by modelling and analyzing the imaging process to extract desired features or information.

Topic areas: computer vision, physics based vision, perception based vision, 3D geometry

Our research is focused on computer vision and deep learning, and in particular image processing, 3D object understanding and human behavior analysis with industrial and societal applications.

Topic areas: computer vision

My research is focused on computer vision, machine learning and medical image analysis. My current research topics include meta-learning, variational Bayesian inference and their applications to few-shot learning, domain generalisation, continual learning, multi-task learning, multi-modal learning, semantic segmentation and automatic report generation. I am also intrigued by interdisciplinary topics, e.g., memory and attention mechanisms, between cognitive science and artificial intelligence.

Topic areas: meta-learning, variational Bayesian inference, out-of-distribution generalisation, neural memory

Data is becoming an abundant data source which we can only fully leverage by developing techniques that do not require manual human annotations. This is particularly true for image and video data, and why my research focuses on self-supervised learning: Here, we aim to learn meaningful representations and solve various tasks all without using annotations, making them possible to scale and generalize to new and larger settings without the burden of using labels.

Topic areas: computer vision, self-supervised learning, representation learning, multi-modal learning

NATURAL LANGUAGE PROCESSING

My research focuses on automated information access, in particular access across languages. Topics of interest to me are: Statistical Machine Translation, Cross-Language Information Retrieval, Data Mining for Natural Language Processing.

Topic areas: machine translation, natural language processing

My research is in the area of natural language processing, with a specific focus on machine learning for natural language understanding tasks. My current interests include few-shot learning and meta-learning, cognitively-inspired models of language, joint modelling of language and vision, and multilingual NLP. My work also explores practical applications of NLP that can have direct societal impact, for instance, in the areas of hate speech and misinformation detection. At the UvA, I lead the Amsterdam Natural Language Understanding Lab, actively collaborating with industrial partners, such as Google, Facebook and Deloitte.

Topic areas: natural language processing, machine learning, meta-learning, cognitive science

My research is focused on building the necessary methods and technology to enable human – machine collaboration for retrieving information by obtaining a deeper understanding of user preferences and decisions during information seeking activities, developing algorithms that can interact with users during information seeking activities, and building the appropriate evaluation methodology to measure progress.

Topic areas: information retrieval, recommender systems, conversational AI, language technology

My research is primarily on natural language understanding (e.g., question answering, information extraction, and semantic parsing) and language generation (e.g., summarization and machine translation). In my group, we are specifically interested in developing methods for reducing the need for expensive human annotation (semi-supervised, self-learning, integrating inductive biases), making the models robust under changes in data distribution (including systematic, compositional generalization) and building models interpretable to human users.

Topic areas: natural language processing, generation, interpretability, systematic generalization

Our research concentrates on statistical learning for language understanding and for modeling human language processing phenomena. Earlier work focused on developing statistical learning algorithms for NLP and on devising structured statistical models for machine translation, paraphrasing, semantic and morpho-syntactic parsing. We collaborate with industrial partners for the exchange of knowledge and research outcomes leading to the development and deployment of actual systems in practical settings.

Topic areas: language understanding, machine translation, paraphrasing, parsing, statistical NLP

Our research is focused on developing intelligent technology to connect people to information. Topics include search engines, recommender systems, and conversational assistants. We follow a data-driven, machine learning-based approach to contribute to fundamental knowledge in the area of information retrieval with new algorithms, new models, and new evaluation methodology.

Topic areas: information retrieval, search engine, recommender system, conversational assistant

My group investigates intelligent systems that support people in their work with data and information from diverse sources. This includes addressing problems related to the preparation, management, integration and reuse of both structured and unstructured data. Topics include: data management for machine learning, information integration, causality-inspired machine learning, automated knowledge graph construction, data provenance.

Topic areas: knowledge graphs, data management for ML, data reuse, data provenance

My research is focused on computational linguistics, cognitive modelling and artificial intelligence in order to understand how we use language to communicate with each other in situated environments and how dialogue interaction shapes learning — about the world and about language itself. Topics include: computational pragmatics, visually grounded language and visual reasoning, conversational agents and learning from interaction, language variation and change in communities of speakers.

Topic areas: natural language processing, dialogue, visual grounding, cognition

My group does research in natural language processing, with a focus on interpretability techniques and the cognitive, neural relevance of modern language models, and venturing into the domains of music processing and language evolution. Our contributions include work on iterated learning, techniques for analyzing grammar learning in children and non-human animals, constituency and dependency parsing, tree-shaped LSTMs and other neural networks, interpretability techniques like diagnostic probes and Shapley-based attributions, and correlating brain activity to word and sentence embeddings. We published both at AI venues (NeurIPS, ACL, EMNLP, JAIR) and in cognitive science journals and conferences (PNAS, TopiCS, CogSci).

Topic areas: natural language processing, music processing, language evolution

My work concerns the design of models and algorithms that learn to represent, understand, and generate language data. Examples of specific problems I am interested in include language modelling, machine translation, syntactic parsing, textual entailment, text classification, and question answering. I also develop techniques to approach general machine learning problems such as probabilistic inference, gradient and density estimation.

Topic areas: natural language processing, statistics, machine learning, approximate inference, global optimisation, formal languages, computational linguistics

I work on responsible machine learning and representation learning for natural language processing. I am interested in tasks and applications where commonsense and real-world knowledge are necessary, including vision & language and applications in medicine and psychology.

Topic areas: natural language processing, vision & language, commonsense knowledge, medicine, psychology

MACHINE LEARNING FOR HEALTH

Our research aims at enhancing patient care by designing and enabling leading edge AI technologies in healthcare. Our expertise lies in developing, validating and clinically integrating socially responsible AI solutions to solve data analysis challenges encountered in different steps of the patient pathway: from prevention and triage, through diagnosis and decision making, to care delivery and management. Particularly, our research is focused on solving clinical challenges in medical image analysis, especially in the fields of radiology, cardiology, neonatology, and ophthalmology.

Topic areas: medical image analysis, healthcare, deep learning, radiology, ophthalmology

My research is focused on visual perception in the human brain using various brain imaging techniques, such as M/EEG, fMRI, TMS and ECoG, in combination with computational models, including both neurophysiologically-informed encoding models and deep neural networks developed in AI. The goal of my research is to find out how the human brain perceives and understands real-world scenes and videos.

Topic areas: computational neuroscience, deep learning, human perception, scene understanding, neuro imaging

Our research aims at enhancing patient care by designing and enabling leading edge AI technologies in healthcare. Our expertise lies in developing, validating and clinically integrating socially responsible AI solutions to solve data analysis challenges encountered in different steps of the patient pathway: from prevention and triage, through diagnosis and decision making, to care delivery and management. Particularly, our research is focused on solving clinical challenges in medical image analysis, especially in the fields of radiology, cardiology, neonatology, and ophthalmology.

Topic areas: medical image analysis, healthcare, deep learning, radiology, ophthalmology