FAQs

Goals

Structure & Content

  • We welcome applications from people who fit most or all of the following criteria:

    • Care about AI safety, and making future development of AI go well;

    • Relatively strong math skills (e.g. about one year's worth of university-level applied math);

    • Strong coders (e.g. have a CS degree / work experience in SWE, or have worked on personal projects involving a lot of coding);

    • Have experience coding in Python;

    • Would be able to travel to London for 4-5 weeks for the programme duration;

    • We are open to people of all levels of experience, whether they are still in school or have already graduated.

    Note - these criteria are mainly intended as guidelines. If you're uncertain whether you meet these criteria, or you don't meet some of them but still think you might be a good fit for the program, please do apply! You can also reach out to us directly, at info@arena.education.

  • We hope that working through this course will give participants an opportunity to gain skills not just in ML engineering, but also more general software engineering skills such as how to structure a codebase, and the adoption of good coding practices. This should help prepare them for applying to research engineering roles at prominent AI safety orgs such as Anthropic, Apollo Research, FAR, METR etc.

    Additionally, at the end of each course, participants will have produced their own GitHub repo containing the projects they worked on throughout the course. This will be a great asset when transitioning to technical work in AI safety.

    Lastly, we hope that sharing the office space with other alignment organisations and independent researchers will lead to productive discussions, networking & collaboration.

  • The programme will likely involve the following:

    • Talks and Q&As with AI safety researchers;

    • Social activities in and around London, in the evenings and over the weekends;

    • Hackathons;

    • Group discussions on AI safety-related topics.

  • Yes! We expect to run 2-3 iterations per year.

  • This programme owes a lot to MLAB, and many parts of it were heavily inspired by MLAB. We feel that the main way ARENA sets itself apart is with the longer duration (5 weeks), giving more time for deep dives into topics, and working on open-ended projects under supervision.

Logistics

  • The programme will take place in the LISA workspace (the London Initiative for Safe AI). LISA is also home to organisations (e.g., Apollo Research, BlueDot Impact), several other AI safety researcher development programmes (e.g., LASR Labs, MATS extension, PIBBS, Pivotal), and many individual researchers (independent and externally affiliated).

  • We will cover all reasonable travel expenses (which will vary depending on where the participant is from) as well as VISA assistance where necessary. Accommodation and meals, drinks, and snacks will also all be included.

    We endeavour to ensure that money is not a barrier for promising candidates wishing to attend the programme.

  • Applications for ARENA 5.0 (April 28 – May 30) are currently open! Apply here.

  • There will be three steps:

    1. Fill out an application form (this is designed to take <1 hour).

    2. Perform a coding assessment.

    3. Interview virtually with one of us, so we can find out more about your background and interests in this course.

  • We are currently not running the programme remotely.

  • The ARENA programme is split into four main chapters:

    1. Fundamentals;

    2. Transformers and Interpretability;

    3. Reinforcement Learning;

    4. Model Evaluations.

    Plus a 5th section for paper replications and a Capstone Project. For more details, see our homepage.

  • At the start of the program, most days will involve of pair programming, working through structured exercises designed to cover all the essential material in a particular chapter. The purpose is to get you more familiar with the material in a hands-on way. There will also usually be a short selection of required readings in the morning.

    As we move through the course, some chapters will transition into more open-ended material. Much of this will still be structured (e.g. in the Mechanistic Interpretability section, there is a large set of structured exercises to choose from), but you’ll have more choice over which things you want to study in more depth. You’ll also hopefully be able to do some independent projects, e.g. experiments, large-scale implementations, paper replications, or other bonus content. There will still be TA supervision during these sections, but the goal is for you to develop your own research and implementation skills. You may also want to work on group projects with other participants during this time instead, if you prefer.

    Each day will be roughly the length of a normal working day (9am-5pm), although there will be more flexibility in working hours during the days of more open-ended projects. There is no compulsory attendance on weekends, but we may organise AI safety discussion groups or social events during this time. The office space will be available 24/7 for anyone who wants to use it outside regular hours.

  • The main ML library we use will be PyTorch. During more open-ended projects you’re welcome to use different libraries, but the exercises will all be based around PyTorch, and fixing bugs might be harder if participants are all using different libraries.

    During the chapter on Transformers and Interpretability, we’ll also use TransformerLens, a library developed by Neel Nanda.

  • Pair programming will be structured in a driver/navigator way. This is where the pair alternates between the roles of driver and navigator at regular intervals (e.g., every half hour).

    The driver sits in front of the keyboard. Their job is to actually code up the functions and solutions to the exercises. The low-level implementation details will be their responsibility.

    The navigator will be giving high-level directions to the driver, and will also be responsible for spotting mistakes in the driver’s code.

    Note that this is just a loose suggestion – every pair will find the style that works best for them. But we strongly recommend that you at least give this style a try.

  • Yes, we will be sending you prerequisite reading and exercises covering material such as PyTorch, einops and linear algebra (this will be in the form of a Colab notebook). We expect that these will take approximately 1-2 days to complete.

For any other questions about ARENA, please reach out to us.