AI Safety & Alignment Researcher

Job Overview

Help us ensure that Superintelligence remains a force for good. We are looking for a philosophical yet technical AI Safety Researcher to join our Ethics & Alignment team. You will work on the "Alignment Problem"—ensuring that our powerful AI systems are helpful, honest, and harmless, and that they adhere effectively to human values.

Key Responsibilities

• RLHF Optimization: Develop and refine Reinforcement Learning from Human Feedback (RLHF) pipelines to steer model behavior.
• Benchmarking: Create novel datasets and benchmarks to measure subtle model biases, toxicity, and hallucination rates.
• Interpretability: Research techniques to open the "black box" of neural networks and understand why a model made a specific decision (Mechanistic Interpretability).
• Policy & Governance: Collaborate with legal and policy teams to draft "Safe AI" usage guidelines and deployment protocols.
• Red Teaming: Try to "break" the model ethically to find failure modes before deployment.

Mandatory Requirements

• Academic Background: PhD or MS in Computer Science, Cognitive Science, Statistics, or Mathematics.
• Technical Depth: Deep understanding of Transformer architectures, Reinforcement Learning (PPO/DPO), and Probability Theory.
• Safety Frameworks: Familiarity with concepts like Constitution AI, Safety Gym, and Inverse Reinforcement Learning.
• Programming: Strong Python skills (PyTorch/JAX) to implement research papers.

Nice to Have (Bonus)

• Published papers in top conferences (NeurIPS, ICML, ICLR).
• Experience with "Scalable Oversight" techniques.
• A strong public writing portfolio on AI alignment or ethics.

What We Offer

• Mission: Work on the most important technical problem of the 21st century.
• Collaboration: Research partnerships with top labs and academia.
• Flexibility: Deep focus time is respected; minimal meetings.