Distinguished Lecture Series

At Robert Bosch Campus Renningen, 23 May 2023

Abstract

In this talk, I will argue for the need for sensorimotor capabilities in intelligent systems and virtual avatars in the 3D world. We argue that to achieve natural and effective communication with humans, machines need to understand human interactions and their 3D environment. Similarly, to enable immersive and realistic experiences with virtual avatars in virtual spaces, machines need to synthesize human motion and behaviour in 3D virtual replicas of the world. I will describe some of our works to capture and generate 3D human motion and interaction jointly with virtual replicas of the 3D world, and emphasise the current challenges and opportunities to make avatars more human like.

Homepage

At Robert Bosch Campus Renningen, 21 March 2023

Abstract

Recent advances in AI have shown that near-miraculous examples of generalization can emerge when large models are pre-trained on massive datasets. Beyond the horizon of these highly visible breakthroughs, which have predominantly focused on images and text, there lies a tremendous potential for AI in domains where the data modalities and prediction problems are much more diverse. In this talk, I will discuss where I see opportunities to apply AI to the modeling and design of physical systems in science and engineering. In particular, I will discuss how programming language design can help us accelerate development of models for specific applications. I will discuss our work on probabilistic programming, which enables application of Bayesian inference methods to stochastic simulators, and the development of inference programming, which train neural networks to accelerate inference. I will close with some perspectives on problems that we will be working on in the next few years, including problems at the UvA Bosch Delta Lab.

Homepage

At Robert Bosch Campus Renningen, 7 February 2023

Abstract

Ideally, the distribution of the latent representations within any neural network should depend only on the task, the data, the loss, and other architecture-specific constraints. However, factors such as the random weights initialization, hyperparameters, or other sources of randomness may induce incoherent latent spaces that hinder any form of reuse. In this talk I'll report an important (and somewhat surprising) empirical observation: under the same data and modeling choices, distinct latent spaces typically differ by an unknown quasi-isometric transformation; that is, in each space, the distances between the encodings do not change. I'll then show how simply adopting pairwise similarities as an alternative data representation leads to guaranteed isometry invariance of the latent spaces, effectively enabling latent space communication: from zero-shot model stitching to latent space comparison between diverse settings. Several validation experiments will follow on different datasets, spanning various modalities (images, text, graphs), tasks (e.g., classification, reconstruction) and architectures (e.g., CNNs, GCNs, transformers).

Homepage

At Robert Bosch Campus Renningen, 11 July 2023

Abstract

Recently, deep generative modeling has become the most prominent paradigm for learning powerful representations of our (visual) world and for generating novel samples thereof. Consequently, this is now serving as the main building block of numerous approaches and practical applications. This talk will contrast the most commonly used generative models to date with a particular focus on denoising diffusion probabilistic models, the core of the currently leading approaches to visual synthesis. Despite their enormous potential, these models come with their own specific limitations. We will then discuss a solution, latent diffusion models a.k.a. "Stable Diffusion", that significantly improves the efficiency of diffusion models. Now billions of training samples can be summarized in compact representations of just a few gigabyte so that the approach runs on consumer hardware. Making high-quality visual synthesis accessible to everyone has revolutionized the way we create visual content and spurred research and the development of numerous novel applications. We will then discuss recent extensions that cast an interesting perspective on future generative modelling: Rather than having powerful likelihood models memorize local image details, we focus their representational power on scene composition. Time permitting, the talk will also cover approaches to post-hoc interpretation of the learned neural representations and to learning semantic similarity.

Homepage

At Robert Bosch Campus Renningen, 08 August 2023

Abstract

A long-standing vision has been to create intelligent robots capable of learning from the world around them to assist humans in everyday tasks from domestic chores to transportation. However, most robots deployed today are still tailored for specific tasks and environments, avoiding contact with humans. Although the past decade has witnessed unprecedented advances in machine learning techniques for various autonomy tasks, they have increased the dependency on manually annotated labels or reward engineering, which are both environment- and task-specific. Moreover, as different robots have different hardware configurations (e.g., sensor modalities, viewpoints, locomotion), the transferability of these learned autonomy modules has become even more challenging. To achieve our goal of ubiquitous robots, we need to develop learning methods for robot autonomy that generalize effectively across diverse tasks, robots, and unstructured environments.

In this talk, I will present our efforts in alleviating the aforementioned challenges in service robots ranging from autonomous vehicles to mobile manipulators. Specifically, I will discuss three fundamental aspects of learning autonomy at scale: 1) learning multiple diverse tasks simultaneously by sharing knowledge and exploiting complementary cues, 2) learning to adapt tasks across different robots and environments, and 3) learning efficiently with minimal human supervision. These techniques have not only facilitated setting the new state-of-the-art, they have opened doors to a wide variety of new applications in human-centered environments. Lastly, I will conclude the talk by presenting our ongoing efforts to address fairness in robot learning for ensuring safe, trustworthy, and responsible innovation, which is crucial for both scalability and fostering acceptance in society.

Homepage

At Robert Bosch Campus Renningen, 11 September 2023

Abstract

Much existing work in reinforcement learning involves environments that are either intentionally neutral, lacking a role for cooperation and competition, or intentionally simple, when agents need imagine nothing more than that they are playing versions of themselves. Richer game theoretic notions become important as these constraints are relaxed. For humans, this encompasses issues that concern utility, such as envy and guilt, and that concern inference, such as recursive modeling of other players. I will discuss some our work in this direction using the framework of interactive partially observable Markov decision-processes, illustrating deception, scepticism, threats and irritation.

Homepage

At Robert Bosch Campus Renningen, 05 October 2023

Abstract

Machine learning has the potential to catalyse a complete transformation in healthcare, but researchers in our field are still hamstrung by a lack of access to high-quality data, which is the result of perfectly valid concerns regarding privacy. With the aim of overcoming these issues, our lab has devoted substantial resources to researching machine learning techniques for synthetic data generation and assessment. It is our shared belief that synthetic data could revolutionise how we access and interact with healthcare datasets. This is a very complicated but uniquely important endeavour, which combines conceptual and technical challenges and has required considerable open-mindedness as we approach it: we have even needed to come up with new ways of understanding notions such as data quality and fairness. We will explain the issues stemming from lack of access to high-quality healthcare datasets. We will explain how synthetic data can help overcome these issues, and will introduce a range of our lab’s own approaches for synthetic data generation and assessment, while also pointing towards some future research directions.

Homepage

At Robert Bosch Campus Renningen, 18 October 2022

Abstract

In the recent years, Artificial Intelligence has seen a tremendous boost in interest, and it is widely considered key to future information technology. Much of this interest has been fueled by major advances in machine learning, which allow for solving a range of problems building on a data-driven, sub-symbolic approach. For the solutions available, however, important concerns are robustness, safety, explainability and trustworthiness in general. Apparently, symbolic approaches to AI that are built on formal (logic-based) models have much to offer in this regard. We consider Answer Set Programming (ASP), a prominent rule-based such approach that has been gaining popularity for declarative problem solving in many AI applications and beyond. We then consider developments in ASP to facilitate neuro-symbolic AI, aiming to bridge sub-symbolic and symbol-based AI, in order to enhance the capabilities of modern AI systems. We discuss opportunities for ASP, such as reasoning, model-building, and explainability, as well as challenges, e.g. dealing with uncertainty and seamless integration, which provide directions for future research.

Homepage

At Robert Bosch Campus Renningen, 24 June 2020

Abstract

Visual perception has become one of the most important sensing modalities for modern robot systems. This is mainly due to the large density of acquired information and the comparably low costs of recent RGB-, stereo-, and depth cameras. These benefits, however, stand in contrast to some important challenges that come with robotic applications. These include insufficient annotated training data, real-time constraints, and the demand for reliable confidence estimates that stems from the safety relevance of physical interactions. Therefore, at DLR, we investigate perception algorithms that both exploit the high potentials of recent Deep Learning approaches and additionally involve specific models to tackle these challenges. This leads to predictions with epistemic uncertainty, fast inference routines across different domains, and a reduced need for real annotated data by leveraging synthetically generated training samples. In my talk, I will present our recent results of this research applied to challenging problems such as object classification, 6D object pose estimation, and 3D scene reconstruction from single images.

Homepage

DLR

At Robert Bosch Campus Renningen, 4 February 2020

Abstract

Massive data collection holds the promise of a better understanding of complex phenomena and ultimately, of better decisions. An exciting opportunity in this regard stems from the growing availability of perturbation / intervention data (manufacturing, advertisement, education, genomics, etc.). In order to obtain mechanistic insights from such data, a major challenge is the integration of different data modalities (video, audio, interventional, observational, etc.). Using genomics as an example, I will first discuss our recent work on coupling autoencoders in the latent space to integrate and translate between data of very different modalities such as sequencing and imaging. I will then present a framework for integrating observational and interventional data for causal structure discovery and characterize the causal relationships that are identifiable from such data. We end by a theoretical analysis of autoencoders linking overparameterization to memorization. In particular, I will characterize the implicit bias of overparameterized autoencoders and show that such networks trained using standard optimization methods implement associative memory. Collectively, our results have major implications for planning and learning from interventions in various application domains.

Biography

Google Scholar

At Robert Bosch Campus Renningen, 28 November 2019

Abstract

Conventional machine vision and audition faces a fundamental latency-power tradeoff, where decreasing latency means increasing sensor sample rate and therefore power consumption. The event sensors developed by our Sensors group and other organizations can solve this problem by using brain-inspired data-driven event output. I will show recent results from our lab using dynamic vision sensor event camera datasets we collected for driving and human body pose estimation. I will also discuss convolutional and recurrent deep neural network hardware accelerators developed by the Sensors group that exploit sparsity to achieve state of the art power efficiency and throughput.

Homepage

At Robert Bosch Campus Renningen, 14 October 2019

Abstract

Deep learning approaches, mostly in the form of convolutional neural networks (CNNs), have taken the field of computer vision by storm. While the progress in recent years has been astounding, their limited interpretability and their demands for ever-larger datasets pose important limitations today. After reviewing some of the challenges in creating large-scale datasets, I will turn to how ideas from classical computer vision can be used to design deep neural networks in computer vision. I will discuss two specific directions. First, I will show how we can adopt techniques from traditional image processing in deep networks for image understanding toward improving their interpretability and accuracy. Second, I will discuss how concepts from optimization and image statistics can be used to design parsimonious network architectures toward improved generalization in a variety of application areas.

Homepage

At Robert Bosch Campus Renningen, 5 September 2019

Abstract

Autonomous quadrotors will soon play a major role in search-and-rescue and remote-inspection missions, where a fast response is crucial. Quadrotors have the potential to navigate quickly through unstructured environments, enter and exit buildings through narrow gaps, and fly through collapsed buildings. However, their speed and maneuverability are still far from those of birds. Indeed, agile navigation through unknown, indoor environments poses a number of challenges for robotics research in terms of perception, state estimation, planning, and control. In this talk, I will show that tightly-coupled perception and control is crucial in order to plan trajectories that improve the quality of perception. Also, I will talk about our recent results on event-based vision to enable low latency sensory motor control and navigation in both low light and dynamic environments, where traditional vision sensors fail.

Homepage

At Robert Bosch Campus Renningen, 8 July 2019

Abstract

A common view is that up to 80% of work on a data mining project is involved in data understanding, cleaning and preparation, yet machine learning has not focused very much on these topics. I will describe issues around data parsing, obtaining (or inferring) a data dictionary, data integration, entity resolution, structural variability, identifying and repairing missing data, and anomaly detection and repair. I will discuss work from the AIDA project at the Alan Turing Institute which addresses some of these issues.

(Joint work with Taha Ceritli, James Geddes, Zoubin Ghahramani, Ernesto Jimenez-Ruiz, Ian Horrocks, Alfredo Nazabal, Tomas Petricek, Charles Sutton, and Gerrit Van Den Burg. AIDA research project https://www.turing.ac.uk/research/research-projects/artificial-intelligence-data-analytics)

Homepage

At Robert Bosch Campus Renningen, 28 May 2019

Abstract

Many practical problems arising in engineering can be formulated as so-called mixed-integer nonlinear optimization problems. On the one hand, this includes integer decisions, e.g., values that count or turn components on or off. On the other hand, nonlinear physical behavior can be modeled this way. Solving such optimization problems is very challenging. Nevertheless, in recent years, solution approaches and software has been developed that allows to solve such problems with reasonable size to global optimality.

In this talk, I will explain the basic ideas of such approaches and illustrate them on several examples, including water and gas network operation and design, as well as the design of truss structures. One particular important feature of these approaches is that the solutions should be robust, i.e., they should still be valid after small fluctuations of certain data like demand or load. For example, a truss structure should not break if the load is slightly different from a given nominal value. I will explain how robustness can be covered in mixed-integer nonlinear optimization problems and illustrate the effect on the mentioned examples.

Biography

At Robert Bosch Campus Renningen, 10 April 2019

Abstract

Energy minimization methods have been among the most successful approaches to inverse problems in imaging for decades. In such methods, the data formation process is expressed via an explicit mathematical model. Recently, purely data-driven approaches have been extremely successful, outperforming the classical techniques on various benchmarks. In this talk I will discuss advantages and drawbacks of the respective approaches, and present some of our recent research on combing model and learning based approaches to obtain versatile and provably controllable algorithms that can still benefit from large data sets of exemplary solutions.

Homepage

At Robert Bosch Campus Renningen, 4 February 2019

Abstract

More and more machine learning systems make data-driven decisions in the real work, in increasingly higher-stakes applications. This has caused substantial interest in reinforcement learning - the field of learning to make decisions from data - which has seen stunning recent empirical breakthroughs. At its heart is the challenge of trading exploration - collecting data for learning better models - and exploitation - using the estimate to make decisions. In many applications, however, exploration is a potentially dangerous proposition, as it requires experimenting with actions that have unknown consequences. Hence, most prior work has confined exploration to simulated environments.

In this talk, Prof. Dr. Andreas Krause will present his work towards rigorously reasoning about safety of exploration in reinforcement learning. He will first discuss a model-free approach, where he seeks to optimize an unknown reward function subject to unknown constraints. Both reward and constraints are revealed through noisy experiments, and safety requires that no infeasible action is chosen at any point. He will also discuss model-based approaches, where he learns about system dynamics through exploration, yet need to verify safety of the estimated policy. His approaches use Bayesian inference over the objective, constraints and dynamics, and - under some regularity conditions - are guaranteed to be both safe and complete, i.e., converge to a natural notion of reachable optimum. He will also discuss extensions to the model-based setting, and present experiments on several real-world applications.

Homepage