FAQ

Purpose

Who is this camp for?

We serve students and early-career researchers who want to work on AI existential safety. We also welcome people switching careers or those coming from unusual backgrounds if they have the appropriate skills.

Our participants are familiar with one or more fields relevant to AI safety research, such as machine learning, computer science, mathematics, economics, or cognitive psychology. The average participant has around the expertise of a master’s student.

Repeat participants: We’re happy to have you back if you can add value by guiding other participants through AISC and through the topics you’ve built expertise in.

Read more about our criteria here.

I want to work on a safety topic that is likely not a priority for reducing existential risk. Should I still apply?

We appreciate all efforts that contribute to safer AI!
→ To collaborate on addressing present-day problems in society, we recommend you check out other research communities such as Fairness, Accountability, and Transparency in ML or Robust and Responsible AI.

→ To try collaborating on research to safeguard society’s long-term trajectory for the common good of all, apply to AISC.

AI developments pose existential risks to future generations. If an existential risk comes to pass, society’s trajectory will end. You may have reason to prioritise other systemic risks though. We welcome AI research proposals for preventing lock-in of societal interactions (in which people are barred from making wiser decisions), as well as future risks of astronomical suffering.

To learn about our long-term perspective on safety research, check out the 80,000 Hours Podcast or AI Alignment Podcast. Each interview comes with transcripts and links to mentioned papers, if you prefer to read!

I want to explore doing research in AI strategy, governance, community-building, or another non-technical area. Is this the camp for me?

That depends on how you approach it. At past editions, participants often collaborated on research in machine learning, the mathematics of agents, or human preferences. We have hosted some non-technical teams, a few of which received grants or jobs in governance for their work, and a few of which lacked analytical precision needed to clarify considerations for the community.

If you have built expertise in a non-STEM field and are carefully thinking through a non-technical topic, do apply with a tentative proposal! We value your perspective.

We are open to specific stakeholder or policy-oriented research projects with a solid grounding in internal needs assessments, policy science, historical research, and/or fundamentals of underlying technology. We will not accept broad, outward-facing community-building and strategy topics as it’s harder for us to appraise and connect you with experienced mentors there, harder for teams to falsify their assumptions, and easier to spread incorrect or unconstructive memes through publication.

Your application will not stand on your initial proposal but on the clarity and coherence of your underlying reasoning. If you’re accepted, you may also hear from our reviewers that the proposal itself does not seem worth pursuing further. Either way, we encourage you to stay open to alternative research directions!

 

Application

What is the application procedure?

Step 1: Application

You make a copy of the application form, fill it in and share it with aisc5-applications@googlegroups.com.

Step 2: Review

Two people review your submitted answers and make an invite/decline recommendation. Each reviewer has been involved in AISC and AI safety research for a while. If we find that you fulfill our criteria, we will send you an invitation to a half-hour interview. We aim to respond within one week.

Step 3: Interview

One of us will invite you to a 30-minute video call. Please be prepared and on time. While your answers on the form have greater weight than the interview, the interviewer still judges your performance based on the criteria here. She will ask you to tell her about yourself and she will ask clarifying questions about the answers you submitted through the application form.

You will have only five minutes to ask questions about the camp. If you have more questions, please sign up for one of the Q&A calls (21 Nov, 2 Dec) or send us an email.

How can I prepare for the interview?

Prepare to tell us about yourself for three to five minutes. Look through the answers you gave on the application form and make sure you can answer questions about what you wrote. Rehearse a couple of times.

For the interview, dress one level up from how you normally dress, point a light source at your face, put the camera at eye level and tidy up your background. Do this early so that you will be on time.

(By the way, if you want to prepare for future job interviews, we recommend the MT Interview Series. We don’t expect this level of preparedness for your AISC interview.)

How long do I have to wait for a decision?

Three weeks. Our goal is to send you an invitation to the camp (or rejection) three weeks after the day you submit your application. If you apply just before the deadline, it might take longer, since many people apply just before the deadline and we have limited capacity.

How much will the camp cost if I get accepted?

Attendance is free. If the camp experience helped you make next career steps, do signal this value to us with a donation. This pays for organisers to run next editions.

 

Program

How is the virtual camp different from previous physical editions?

You will join research weekend sprints online rather than our usual 9-day mountain retreat. We carefully designed the 2021 virtual program so you can have insightful discussions and work flexibly in your timezone. See the schedule here.

How is AISC different from the 2019 AI Safety Research Program?

AISC enables AI Safety career aspirants to test their fit for a research role. AISRP served participants who had advanced further in their research career.

At AISRP, senior researchers joined over several days to give input for selecting research directions. At AISC, you get to ask senior researchers questions on prioritisation on the first day and then discuss ideas further with PhD-level mentors.

AISRP organisers are considering a new format for 2021, so keep an eye out!

How are teams and topics chosen?

We’ll start by spending several weeks trying to generate, refine, and select project proposals. The goal is to have every participant work on a project that is:

  • Achievable within the time frame of the camp
  • Has the potential to be impactful and useful to the field of AI safety
  • Interesting and exciting for them to work on

This will involve getting advice from senior AI safety researchers about how to choose quality research projects and looking over research agendas and ideas submitted by external AI safety researchers. We’ll also be running proposals by external mentors for feedback on which ones are most promising.

Once we have a list of promising topics, everyone will submit a form of which topics they are most interested in. We will then form the group into teams of 3-5, taking into account participant’s interest, as well as taking into account factors such as timezones and creating balanced teams.

What will the research process look like?

Between team formation in mid-February and presentations at the end of May, teams will work together to finalize their research plan, complete any background reading, perform their research, and work to present their results to the rest of the camp and potentially to the public.

We expect most groups to have regular meetings once or twice every week. Additionally we’ll have three “research sprints,” one on the first weekend of every month, to give a chance for each team to get together for dedicated focused work. Throughout the process, organizers will also be checking in with the teams to help ensure that the project is going smoothly and the team is on track to be able to reach a satisfying completion.

By the end of May we expect every team to give a presentation on their research and write up a description of what they have attempted and accomplished. Of course, not every project is successful: many promising ideas don’t pan out, or require significantly more effort than originally hoped. Many teams continue their work after the camp, either to finish up any loose ends or to ready their work for publication in a journal or blog post.

How will external mentors be involved?

We hope to be able to connect teams with interested mentors who can offer some guidance on the topic and AI Safety research in general. For example, these mentors might be talented alumni of AISC, PhD students working on AI Safety relevant topics, or FHI research scholars. While we are placing an emphasis this year on finding mentors who have the time and interest to be helpful to the teams, the role of mentors is to provide advice and feedback, not to direct and oversee the project as a PhD advisor might.

We’ll also be encouraging teams to reach out to more senior mentors who are experts in the area of their project. While the more senior researchers can be very helpful for giving quick feedback on the project plans, they are generally not available to act as a consistent mentor.

What are the research sprints?

These weekends are an opportunity for participants to focus on AISC work and make substantial progress on their projects. We expect participants to minimize other commitments and be able to devote 12-16 hours of work over the course of the weekend. These weekends are also tied to milestones in the research process in order to help ensure that teams are not falling behind the pace they need to reach a satisfying conclusion to their projects.

The exact structure of the weekends is largely up to each individual team. However, you can expect a mixture of:

  • Individual work
  • Coworking sessions with your team
  • Meetings with your team
  • Check-ins with organizers
  • Optional social activities
Can I join more than one team?

No, you won’t be able to participate in two teams at the same time. However, you’re very welcome to give feedback to another team or discuss their topic during breaks.

Can I participate without joining a team?

No. The idea of the camp is to allow collaborators to come together, so joining a team is crucial to the camp experience.

How much time does AISC require?

The time required varies between the different teams. We expect that successful teams will spend 3-6 hours collaborating on research doing AISC activities during a typical work week. During research sprints we expect participants to spend significantly more time – between 12 and 16 hours over the course of the weekend.

Ideally by the end of May, teams bring their projects to a satisfying conclusion. However, your team is likely to still have work you want to finish up, especially to polish your work in order to present it to the AI Safety community. Expect to put in 5-15 hours to summarise your work –and significantly more if you want to turn your research into a paper or a high-quality blog post.

How will AISC help me develop my career?

At AI Safety Camp, you will be able to

  • connect with new collaborators and mentors.
  • learn common research concepts, practices, and mindsets in the community
  • learn beyond the basics by actually doing research in an area that interests you.
  • find out how much you enjoy and are able to contribute to research on the frontiers.
  • publish work that helps established research organisations better discern where you may be able to contribute.
  • consider your next career steps in AI safety (including for roles in operations, entrepreneurship, or earning-to-give/save).

 

Still have questions? Contact us at contact@aisafety.camp.