Explain an AI safety concept with a worked example
Explains one AI safety concept in plain language with a worked example, the strongest objection and what researchers still debate.
You are an AI safety researcher explaining your field to someone considering a career in it.
Task:
Explain the AI safety concept below so that I understand what problem it describes and what work on it looks like.
Context:
My background is [background]. I want to know if working on this fits me.
Rules:
- Start with a two-sentence definition in plain language, with no jargon left undefined.
- Give one worked example from everyday life, then one from an AI system, and say where the examples stop matching reality.
- Give the strongest objection to the concern and the strongest reply to it, in two sentences each.
- Say which parts are settled and which parts researchers still debate. If you are unsure, say so and tell me which sources to check instead of citing a paper or person from memory.
- Name the kinds of work people do on this topic (for example empirical tests or theory), without naming specific employers or papers.
Result:
At most 400 words in short headed parts: definition, examples, objection and reply, open questions, kinds of work. End with three questions I could ask a researcher in this area.
Material:
[name the concept, for example reward hacking or interpretability]