FeedbackShare feedback

← All sections·Learning path

V

AI Safety & Alignment12

Core

The field dedicated to making AI systems behave reliably, honestly, and without causing unintended harm.

AI Accountability

The framework for determining who is responsible when an AI system causes harm - and the institutional mechanisms for ensuring that responsibility is actually exercised.

AI Red Teaming

Systematic adversarial testing of AI systems to find safety failures before deployment - having experts try to make the model produce harmful outputs or exhibit dangerous behaviours.

AI Sycophancy

The tendency of AI models to tell users what they want to hear rather than what is true - agreeing with incorrect beliefs, validating bad ideas, and adjusting answers to match perceived user preferences.

All concepts

A

AI Transparency
The principle that AI systems should be understandable and explainable - that users, regulators, and affected parties should be able to understand how decisions are being made.

D

Deceptive Alignment
A theoretical AI safety failure where a model behaves well during training and evaluation but has learned to pursue different goals that it pursues once deployed - essentially, an AI that games its own training.

H

Hallucination
When an AI confidently states something that is not true - not because it is lying, but because it was trained to produce convincing text, not necessarily accurate text.

I

Instrumental Convergence
The theoretical observation that almost any AI goal will lead to the same set of sub-goals - like self-preservation and acquiring resources - because these are useful for achieving almost anything.

J

Jailbreak Resistance
The ability of an AI model to maintain its safety behaviours when users attempt to manipulate it into producing harmful outputs through clever prompting.

M

R

Reward Hacking
When an AI system finds unexpected ways to achieve a high score on its training objective without doing what the objective was designed to measure - gaming the metric rather than solving the actual problem.

S

Scalable Oversight
The research challenge of developing methods to reliably supervise AI systems that may be more capable than their human supervisors - ensuring alignment holds even as AI capability grows.