Schedule

Subscribe to our calendar for the most recent schedule.

September 25th 3:00pm CET - Luigi Acerbi

Title: Accelerating Bayesian Inference and Data Acquisition via Amortization

Abstract:

Many critical applications demand systems that can both strategically acquire the most informative data and instantaneously perform inference based upon it. Bayesian inference and Bayesian experimental design offer principled mathematical means for reasoning under uncertainty and for strategically gathering data, respectively. While foundational, both methods introduce notorious computational challenges. In recent years, amortized solutions have been proposed to address these issues by pre-training neural networks via synthetic data, significantly reducing computational costs at deployment. In this talk, I will first introduce our Amortized Conditioning Engine (ACE), a flexible amortized inference framework that affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for both discrete and continuous data and latents. I will then share our latest work, the Amortized active Learning and INference Engine (ALINE). ALINE combines the advantages of amortized inference and experimental design into a single, unified framework. It is capable of rapidly proposing valuable data points while simultaneously performing fast, flexible inference based on the collected data, thus seamlessly closing the loop between active data acquisition and real-time reasoning.

October 2nd 3:00pm CET - Carolin Benjamins

Title: carps: A Framework for Comparing N Hyperparameter Optimizers on M Benchmarks

Abstract:

Hyperparameter Optimization (HPO) is crucial to develop well-performing machine learning models. In order to ease prototyping and benchmarking of HPO methods, we propose carps, a benchmark framework for Comprehensive Automated Research Performance Studies allowing to evaluate N optimizers on M benchmark tasks. In this first release of carps, we focus on the four most important types of HPO task types: blackbox, multi-fidelity, multi-objective and multi-fidelity-multi-objective. With 3336 tasks from 5 community benchmark collections and 28 variants of 9 optimizer families, we offer the biggest go-to library to date to evaluate and compare HPO methods. The carps framework relies on a purpose-built, lightweight interface, gluing together optimizers and benchmark tasks. It also features an analysis pipeline, facilitating the evaluation of optimizers on benchmarks. However, navigating a huge number of tasks while developing and comparing methods can be computationally infeasible. To address this, we obtain a subset of representative tasks by minimizing the star discrepancy of the subset, in the space spanned by the full set. As a result, we propose an initial subset of 10 to 30 diverse tasks for each task type, and include functionality to re-compute subsets as more benchmarks become available, enabling efficient evaluations. We also establish a first set of baseline results on these tasks as a measure for future comparisons. With carps (https://github.com/automl/CARP-S), we make an important step in the standardization of HPO evaluation.

October 16th 3:00pm CET - Jan van Rijn

Title: Multi-Objective AutoML: Towards Accurate and Robust models

Abstract:

Machine learning models have to adhere to many requirements. Beyond the need to be accurate and small enough to be deployed on satellite hardware, they need to be robust against various types of domain shifts. Automated machine learning has been successful in supporting data scientists in selecting appropriate machine learning architectures, as well as optimising hyperparameters. By doing so, data scientists can focus their attention on more important tasks.

During the Horizon TAILOR project, we have seen a demand for AutoML techniques to not only provide solutions that are accurate but also those that are trustworthy according to several relevant criteria. Many models are known to be vulnerable to various types of input perturbations, whereas robustness against such deviations is an important criterion of trustworthiness. In this talk, I will summarise various projects we have done towards this goal, which envision AutoML solutions that specifically address the robustness of neural networks, but also include what hyperparameters are important to tune.