About the job

Job summary

Summary

The AI Safety Institute (AISI) is looking for a visionary Head Cyber Misuse Evaluations. You will spearhead the development of a strategy for building and running cutting-edge evaluations of cyber-attack uplift from next-generation frontier AI models. You’ll also build a skilled team and forge relationships with key partners in AI labs and national security agencies. This position offers a unique opportunity to push forward an emerging field while part of an organisation that is a unique and fast-growing experiment in AI research and governance.

The AI Safety Institute

The AI Safety Institute is the first state-backed organisation focused on frontier AI safety for the public interest. We launched at the AI Safety Summit because we believe taking responsible action on this extraordinary technology requires a capable and empowered group of technical experts within government. Our staff includes senior alumni from OpenAI, Google DeepMind, startups and the UK government, and ML professors from Oxford and Cambridge. We are now calling on the world’s top technical talent to build the institute from the ground up. This is a truly unique opportunity to help shape AI safety at an international level.

We have ambitious goals and need to move fast. Our top priorities are to:

• Develop and conduct evaluations on advanced AI systems. We will characterise safety-relevant capabilities, understand the safety and security of systems, and assess their societal impacts.
• Drive foundational AI safety research. We will launch moonshot research projects and convene world-class external researchers
• Facilitate information exchange. We will establish clear information-sharing channels between the Institute and other national and international actors. These include policymakers, international partners, private companies, academia, civil society, and the broader public.

Job description

The Cyber Misuse Workstream

As AI capabilities evolve rapidly, we expect a significant shift in the global cybersecurity paradigm. Of particular concern is the potential for AI systems to grant novice actors dangerous capabilities, expanding the number of actors who could carry out serious cyber attacks. To improve our ability to understand this threat, AISI plans to conduct research on a broad spectrum of cyber activities, including sophisticated threat modelling and the development of cyber ranges with a view towards developing and deploying evaluations for cyber attack uplift capabilities (e.g vulnerability discovery) by next-generation frontier AI systems.

This work is pivotal to an informed governmental and societal response to a potentially significant emerging threat. Even more importantly, current evaluation methodologies in this field are predominantly ad-hoc and qualitative; our ambition is to reshape the cyber evaluations ecosystem in a more rigorous and holistic direction.

To do this, the Cyber Misuse Evaluations Team will collaborate with an array of critical partners include AI labs, academia and the national security community.

The Role

We are hiring for a Head of Safeguard Analysis to lead and build a team to develop and conduct analyses of how well the safety and security components (“safeguards”) of frontier AI systems stand up to a variety of threats. This role presents an exceptional opportunity for a candidate to innovate and expand the understanding of state-of-the-art safeguards.

This role will involve:

• Leading the strategy for the Safeguards team, to design high quality analyses of safeguards. You will be responsible for delivering AISI’s safeguards portfolio, through in-house research and external commissions.
• Developing evaluations aimed at assessing how safeguards stand up against a range of attack vectors, drawing from techniques in ML evaluations and a range of security expertise.
• Ensure evaluations are scientifically robust, actionable, communicable and consider the current state of research.
• Building and leading a team by identifying talent gaps, leading recruitment to address those gaps and being responsible for the final hiring decision for members of technical staff on the workstream.
• Collaborating with AISI teams on specific risks areas to combine safeguard evaluations with evaluations of specific dangerous capabilities, and the AISI platform team to build tooling to perform such evaluations.
• Working closely with partners in the national security community, as well as other partners in industry, academia, and civil society.
• Communicating research outputs in core AISI products through government communications, research publications, and high-profile events like AI summits and conferences.
• Leading longer-term research efforts aimed at better understanding the efficacy of system safeguards.

Person specification

Requirements and Experience

You would be a great fit if you:

• Are motivated by a strong desire to ensure positive outcomes for all of humanity from the creation of AI systems.
• Have substantia experience of cybersecurity or similarly relevant experience with a security focus. You might have experience participating in red teams, penetration testing, CTFs, vulnerability patching, malware detection and reverse engineering.
• Have experience conducting research around benchmarks and measurements of either AI models or security, specifically in the realm of large language models and cybersecurity.
• Can set and steer research agendas towards the most impactful avenues of research and development.
• Have an interest in the intersection of cybersecurity and AI. You might be an academic with a focus on Reinforcement Learning for cybersecurity, have a background in offensive cyber security or have experience in security research with an interest in LLMs.
• Have leadership experience; you can manage a team of dedicated engineers and researchers, able to inspire, guide and maximize the potential of a diverse team.
• Can thrive in complexity and are comfortable navigating a space with high uncertainty, where the landscape is changing rapidly.
• Are comfortable adapting to input from others, collaborating with non-technical stakeholders and can articulate complex concepts to a wide variety of audiences.
• Are passionate about AISI’s mission, to develop the sociotechnical infrastructure needed to understand the risks of advanced AI and support its governance.

We do not expect candidates to fulfil all these qualifications and we urge you to apply if you are inspired by this work. The role holder will be required to obtain DV clearance.

Core requirements

• You should be able to spend at least 4 days per week on working with us.
• You should be able to join us for at least 12 months.
• You should be able work from our office in London (Whitehall) for parts of the week, but we provide flexibility for remote work.

Benefits

Alongside your salary of £105,000, Department for Science, Innovation & Technology contributes £14,850 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

Things you need to know

Selection process details

Please apply with a copy of your CV and a personal statement (500 word limit) setting out your answers to the questions below:

• Why are you interested in joining the AISI right now?
• What are the most important technical projects UK government needs to undertake within the next few months to advance AI Safety?
• What do you want to be doing to drive those projects forward?
• Is there anything else about your skills, experience or background you’d like us to be aware of?

Sift and interview process

In the event of a large number of applicants, applications will be sifted on the personal statement.

Candidates should expect to go through some or all of the following stages once an application has been submitted:-

Stage 1:- Coding Test - To be completed within 48 hours

Stage 2:- Sifting if successful at coding test

Stage 3:- Initial Interview

Stage 4:- Pre Interview take home test

Stage 5:- Formal interview and review of take home test

Stage 6:- Third stage interview

Stage 7:- Senior Civil Servant interview

Sift and interview dates

Expected Timeline subject to change

Sift dates: w/c 29/01/2024

Interview dates: w/c 05/02/2024

Interview Location: MS Teams.

Candidates are asked to note the above timetable, exercising flexibility through the recruitment and selection process.

If you have any questions about the role or your application, then please send an email to AISI-TalentRecruitment@dsit.gov.uk

Further Information

Reasonable Adjustment

We are proud to be a disability confident leader and we welcome applications from disabled candidates and candidates with long-term conditions.

We fully support adjustments throughout our recruitment process and we encourage candidates to discuss their adjustment needs by emailing the job contact which can be found under the contact point for applicants section.

We do not have an exhaustive list of adjustments that we support but just some examples include additional time to complete your application form, behaviour questions up to 72 hours prior to interview and having extra time at interview.

If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'Contact point for applicants' section.

If successful and transferring from another Government Department a criminal record check may be carried out.

New entrants are expected to join on the minimum of the pay band.

A location based reserve list of successful candidates will be kept for 12 months. Should another role become available within that period you may be offered this position.

Please note terms and conditions are attached. Please take time to read the document to determine how these may affect you.

Any move to the Department for Science, Innovation and Technology from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility https://www.childcarechoices.gov.uk

DSIT does not normally offer full home working (i.e. working at home); but we do offer a variety of flexible working options (including occasionally working from home). 

In order to process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service on your behalf.

However, we recognise in exceptional circumstances some candidates will want to send their completed forms direct. If you will be doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading. 

Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant’s details held on the IFD will be refused employment. 

A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5 year period following a dismissal for carrying out internal fraud against government.

Feedback

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a criminal record check. People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window). The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria. The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.