Program | Speakers | Registration | Organizers | Venue

The second Human-aligned AI Summer School (HAAISS) will be held in Prague from 25th to 28th July. The focus of the second year will be on “optimization and decision making,” including subtopics such as understanding agent incentives, open-source game theory, and boundaries between game theory and machine learning. We will also cover the latest trends in AI alignment research and broader framings of AI alignment research.

Previous year

Format of the school

The school is focused on teaching approaches and frameworks, less on presentation of the latest research results. The content of the school is mostly technical - it is assumed the attendees understand current ML approaches such as deep learning. The intended audience of the school are researchers interested in learning more about the AI alignment topics, PhD students, researchers working in ML/AI outside academia, and talented students.


Thursday, July 25

Venue: Faculty of Mathematics and Physics

9:00-10:00 Registration
10:00-10:30 Opening session - Jan Kulveit
10:30-10:50 Coffee break
10:50-12:20 Agent incentives - Tom Everitt

12:30-14:00 Lunch (catered)

14:00-15:30 Agent incentives II - Tom Everitt
15:30-15:50 Coffee break
16:00-17:00 Game Theory Foundations for AI Researchers - Michael Dennis
17:00-17:10 Short break
17:10-18:30 Panel on AI alignement agendas - Michael Dennis, Tom Everitt, Vanessa Kosoy, Jan Kulveit, Chris van Merwijk, Ludwig Schubert

19:00-21:30 Welcome reception

Friday, July 26

Venue: Faculty of Mathematics and Physics

9:30-10:30 Learning theoretic approach to AI alignment - Vanessa Kosoy
10:30-10:50 Coffee break
10:50-11:50 Interpretability - Ludwig Schubert
11:50-12:20 Coffee break
12:10-13:10 Mesa-optimizers - Vladimir Mikulik and Chris van Merwijk

13:10-14:10 Lunch (catered)

14:10-15:20 Translucent Game Theory - Michael Dennis
15:20-15:30 Short break
15:30-17:30 Breakout sessions / research ideas brainstorming
17:30-18:00 Snacks, walk to the church
18:00-20:00 Organ concerto (st. Nicolaus church)

(Dinner individually)

Saturday, July 27

Venue: Faculty of Mathematics and Physics

10:00-10:30 Lightning talks (early career researchers)
10:30-11:00 Coffee break
11:00-12:00 Mild optimization - Ryan Carey
12:00-12:30 Coffee break
12:30-13:00 Alignment for predictive processing agents - Jan Kulveit

13:00-14:30 Lunch (catered)

14:30-14:50 AI Safety via Debate and its applications - Vojta Kovarik
14:50-15:20 Coffee break
15:20-16:20 Learning theoretic approach to AI alignment - Vanessa Kosoy
16:20-16:40 Coffee break
16:40-17:40 Panel on careers in AI alignment - Michael Dennis, Ludwig Schubert, Ryan Carey, Rose Hadshar, Tomas Gavenciak

19:00-22:00 School dinner, Cerna Labut

Sunday, July 28

Venue: Faculty of Mathematics and Physics

10:00-11:00 Overview of strategical considerations - Shahar Avin
11:00-11:20 Coffee break
11:20-11:30 Flash talks (3m)
11:30-12:30 Panel discussion on strategy - Shahar Avin, Michael Dennis, Ludwig Schubert, Ryan Carrey, Jan Kulveit
12:30-12:50 Closing session

13:00-14:00 Lunch (catered)


Tom Everitt
Research Scientist, DeepMind

Tom Everitt is a research scientist in AI safety at DeepMind focusing on research of incentives of powerful RL agents. His thesis at the Australian National University supervised by Marcus Hutter, Towards Safe Artificial General Intelligence, was the first PhD thesis specifically devoted to AI safety. He also won the AI Alignment Prize for research on reward tampering and the Kurzweil prize for best AGI paper for research on self-modification of utility functions in rational agents.

Ryan Carey
Research Fellow, FHI

Ryan Carey works at Future of Humanity Institute (Oxford University) on AI safety. Previously, he has worked on research engineering for Ought and as a research assistant for the Alignment for Machine Learning Systems agenda at the Machine Intelligence Research Institute. Prior to that, he obtained a masters in bioinformatics and theoretical systems biology from Imperial College London. Before that, he worked as a medical doctor.

Michael Dennis
PhD student, CHAI

Michael Dennis works on his PhD on AI safety at Center for Human-Compatible AI, University of California, Berkeley. He is an expert on open-source game theory (i.e. agents seeing each others' source code). Before moving to work on AI alignment he worked on computational geometry.

Vladimir Mikulik
Computer science student, University of Oxford

Vladimir Mikulik studies philosophy and computer science at the University of Oxford, co-founded MIRIxOxford, and has coauthored MIRI's paper on mesa-optimization and the inner alignment problem.

Shahar Avin
Research Associate, CSER, University of Cambridge

Shahar’s research at the Centre for the Study of Existential Risk examines challenges and opportunities in the implementation of risk mitigation strategies, particularly in areas involving high uncertainty and heterogenous or conflicting interests and incentives. He's mixing anthropological methods and agent-based modelling.

Vanessa Kosoy
Research Associate, MIRI

Vanessa's research aims at mathematical formalization of general intelligence and value alignment, using tools from computational learning theory and algorithmic information theory. Such mathematical models serve to elucidate the potential failure modes of AGI, clarify confusing conceptual questions, and lead to AI algorithms satisfying theoretical guarantees that imply safety and effectiveness under clear and (ultimately) realistic assumptions. Prior to her work on AI alignment, Vanessa was an algorithm engineer specializing in computer vision.

Ludwig Schubert
Research Engineer, OpenAI (Clarity/Safety Team)

Ludwig is a research engineer on Chris Olah’s Clarity team at OpenAI. Clarity focuses on interpretability research: what happens in the so-called “hidden” layers of deep neutral networks. Early work with Alexander Mordvintsev at Google Brain included DeepDream, a technique which today maybe best known for its artistic applications. The team has since developed more targeted methods (Feature Visualization, Building Blocks of Interpretability, Activation Atlas) and continues their work towards building a “microscope for deep learning” as part of OpenAI’s safety efforts. Ludwig also helps run Distill, a web-native machine learning journal that aims at clear explanations of machine learning.

Chris van Merwijk
Research scholar, FHI

Chris van Merwijk works at the Future of Humanity institute on AI safety, and has coauthored MIRI's paper on mesa-optimization and the inner alignment problem.

Registration and fees

Applications for the summer school are open until 15th June.

Apply for participation.

Regular school fee is € 200. Student fee is € 100. Thanks to our sponsors, limited financial assistance, including partial travel costs reimbursement, is available for participants who want to work on AI alignment research but travel or registration costs would prevent them from attending the school.


The conference is organized by

Program: Jan Kulveit (main coordinator), Tomáš Gavenčiak,Jan Romportl
Operations: Hana Kalivodova


Faculty of Mathematics and Physics, Charles University
Malostranske náměstí 25, Praha 1
First floor - the way from the building entrance will be signposted.

Sněmovní 7 event & coworking space
Sněmovní 7, Praha 1


EA Long-term future fund