Software Engineer
US$30 – US$100 / hr base salary
Posted 16 Oct 2026
Advertised as “Software Engineer – AI Evaluation Expert (Contractual)”
BranchFactor Summary
The original posting with the fluff stripped out
- Expected experience
- 2+ yrs
- Management level
- Individual contributor
- Employment
- Remote
- Contract
- Contract
| Base salary | US$30 – US$100 / hr |
|---|
Field of studyComputer Science, Engineering (or related)
Study levelBachelor's
About Patronus AI
Patronus AI is a frontier lab developing simulation research and infrastructure to accelerate progress toward human-aligned AGI. We are on a mission to simulate all of the world’s intelligence.
We are the team behind some of the earliest and most influential research in AI evaluation like FinanceBench, Lynx, SimpleSafetyTests, CopyrightCatcher, Humanity’s Last Exam, and more. We are formerly AI researchers and engineers from companies like Meta AI, Amazon AGI, and Google. Our customers include foundation model labs and Fortune 500 enterprises like Adobe. We are backed by top-tier investors like Lightspeed Venture Partners, Notable Capital, Stanford University, Noam Brown, Gokul Rajaram, and more.
About the roleWe are looking for experienced software engineers (task-based contract) to design realistic, challenging tasks for frontier AI models and to judge how well those models handle them. You'll apply your day-to-day engineering judgment to show where AI gets code, reasoning and technical instructions right, and where it falls short.
What you’ll do
Depending on the project, you may:
- Design realistic, multi-step coding and technical tasks that test an AI model's ability to reason, write code and follow precise instructions.
- Build tasks around realistic workspace materials such as codebases, technical specs, architecture docs and datasets.
- Evaluate AI-generated code and technical content for correctness, efficiency, clarity and alignment with real-world engineering practice.
- Grade model outputs against rubrics and ground-truth answers, and write clear, structured feedback.
- Help improve how AI understands programming, system design, data workflows and engineering best practices.
- Work independently and asynchronously, meet deadlines, and collaborate with the Patronus team to keep quality high.
- Test and refine content to reflect real-world complexity.
- Review other contributors’ work for accuracy, clarity, and practical relevance.
- Incorporate feedback promptly and flag gaps or unclear guidance.