Program | Speakers | Registration | Organizers | Venue

The second Human-aligned AI Summer School will be held in Prague from 25th to 28th July. The focus of the second year will be on “optimization and decision making,” including subtopics such as understanding agent incentives, open-source game theory, and boundaries between game theory and machine learning. We will also cover the latest trends in AI alignment research and broader framings of AI alignment research.

Previous year

Format of the school

The school is focused on teaching approaches and frameworks, less on presentation of the latest research results. The content of the school is mostly technical - it is assumed the attendees understand current ML approaches such as deep learning. The intended audience of the school are researchers interested in learning more about the AI alignment topics, PhD students, researchers working in ML/AI outside academia, and talented students.

Program (beta)

Thursday, July 25

Venue: Faculty of Mathematics and Physics

9:00-10:00 Registration
10:00-10:30 Opening session
10:30-10:50 Overview
10:50-11:10 Coffee break
11:10-12:20 Morning session I

12:30-14:00 Lunch (catered)

14:00-15:30 Afternoon session I
15:30-15:50 Coffee break
16:00-17:00 Afternoon session II
17:00-18:00 Panel session

19:00-21:30 Welcome reception

Friday, July 26

Venue: Faculty of Mathematics and Physics

9:30-11:00 Morning session I
11:00-11:20 Coffee break
11:20-12:30 Morning session II

12:30-13:30 Lunch (catered)

13:30-14:30 Afternoon session I
14:30-15:00 Coffee break
15:30-17:00 Breakout sessions / research ideas brainstorming
17:00-17:20 Coffee break
17:20-18:00 Fireside chat

(Dinner individually)

20:00-22:00 Evening program TBD

Saturday, July 27

Venue: Faculty of Mathematics and Physics

9:30-10:0 Lightning talks (early career researchers)
11:00-11:20 Coffee break
11:20-12:30 Morning session I

12:30-14:00 Lunch (catered)

14:00-15:40 Afternoon session I
15:40-16:00 Coffee break
16:00-17:00 Discussion in small groups
17:00-18:00 Lightning talks (early career researchers)

19:00-22:00 School dinner

Sunday, July 28

Venue: Faculty of Mathematics and Physics

9:30-11:00 Strategic considerations in AI alignment
11:00-11:20 Coffee break
11:20-12:40 Panel discussion

12:40-13:00 Closing session

13:00-14:00 Lunch

Speakers (preliminary, incomplete)

Tom Everitt
Research Scientist, DeepMind

Tom Everitt is a research scientist in AI safety at DeepMind focusing on research of incentives of powerful RL agents. His thesis at the Australian National University supervised by Marcus Hutter, Towards Safe Artificial General Intelligence, was the first PhD thesis specifically devoted to AI safety. He also won the AI Alignment Prize for research on reward tampering and the Kurzweil prize for best AGI paper for research on self-modification of utility functions in rational agents.

Ryan Carey
Research Fellow, FHI

Ryan Carey works at Future of Humanity Institute (Oxford University) on AI safety. Previously, he has worked on research engineering for Ought and as a research assistant for the Alignment for Machine Learning Systems agenda at the Machine Intelligence Research Institute. Prior to that, he obtained a masters in bioinformatics and theoretical systems biology from Imperial College London. Before that, he worked as a medical doctor.

Michael Dennis
PhD student, CHAI

Michael Dennis works on his PhD on AI safety at Center for Human-Compatible AI, University of California, Berkeley. He is an expert on open-source game theory (i.e. agents seeing each others' source code). Before moving to work on AI alignment he worked on computational geometry.

Vladimir Mikulik
Computer science student, University of Oxford

Vladimir Mikulik studies philosophy and computer science at the University of Oxford, co-founded MIRIxOxford, and worked with MIRI researchers on the inner alignment problem.

Shahar Avin
Research Associate, CSER, University of Cambridge

Shahar’s research at the Centre for the Study of Existential Risk examines challenges and opportunities in the implementation of risk mitigation strategies, particularly in areas involving high uncertainty and heterogenous or conflicting interests and incentives. He's mixing anthropological methods and agent-based modelling.

Vanessa Kosoy
Research Associate, MIRI

Vanessa's research aims at mathematical formalization of general intelligence and value alignment, using tools from computational learning theory and algorithmic information theory. Such mathematical models serve to elucidate the potential failure modes of AGI, clarify confusing conceptual questions, and lead to AI algorithms satisfying theoretical guarantees that imply safety and effectiveness under clear and (ultimately) realistic assumptions. Prior to her work on AI alignment, Vanessa was an algorithm engineer specializing in computer vision.

Ludwig Schubert
Research Engineer, OpenAI (Clarity/Safety Team)

Ludwig is a research engineer on Chris Olah’s Clarity team at OpenAI. Clarity focuses on interpretability research: what happens in the so-called “hidden” layers of deep neutral networks. Early work with Alexander Mordvintsev at Google Brain included DeepDream, a technique which today maybe best known for its artistic applications. The team has since developed more targeted methods (Feature Visualization, Building Blocks of Interpretability, Activation Atlas) and continues their work towards building a “microscope for deep learning” as part of OpenAI’s safety efforts. Ludwig also helps run Distill, a web-native machine learning journal that aims at clear explanations of machine learning.

Chris van Merwijk
Research scholar, FHI

Chris van Merwijk works at the Future of Humanity institute on AI safety, and has coauthored MIRI's paper on mesa-optimization and the inner alignment problem.

Registration and fees

Applications for the summer school are open until 15th June.

Apply for participation.

Regular school fee is € 200. Student fee is € 100. Thanks to our sponsors, limited financial assistance, including partial travel costs reimbursement, is available for participants who want to work on AI alignment research but travel or registration costs would prevent them from attending the school.


The conference is organized by

Program: Jan Kulveit (main coordinator), Tomáš Gavenčiak,Jan Romportl
Operations: Hana Kalivodova


Faculty of Mathematics and Physics, Charles University
Malostranske náměstí 25, Praha 1
First floor - the way from the building entrance will be signposted.

Sněmovní 7 event & coworking space
Sněmovní 7, Praha 1


EA Long-term future fund