Responsibilities
- Developing and maintaining MCP-compatible evaluation servers
- Implementing logic to check agent actions against scenario definitions
- Creating or extending tools that writers and QAs use to test agents
- Working closely with infrastructure engineers to ensure compatibility
- Occasionally helping with test writing or debug sessions when needed
Requirements
- 4+ years of Python development experience, ideally in backend or tools
- Solid experience building APIs, testing frameworks, or protocol-based interfaces
- Understanding of Docker, Linux CLI, and HTTP-based communication
- Ability to integrate new tools into existing infrastructures
- Familiarity with how LLM agents are prompted, executed, and evaluated
- Clear documentation and communication skills - you’ll work with QA and writers
Nice to Have
- Experience with Model Context Protocol (MCP) or similar structured agent-server interfaces
- Knowledge of FastAPI or similar async web frameworks
- Experience working with LLM logs, scoring functions, or sandbox environments
- Ability to support dev environments (devcontainers, CI configs, linters)
- JS experience
Benefits
- Get paid for your expertise, with rates that can go up to $50/hour depending on your skills, experience, and project needs.
- Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments.
- Participate in an advanced AI project and gain valuable experience to enhance your portfolio.
- Influence how future AI models understand and communicate in your field of expertise.
Work Arrangement
Remote (Country)
Additional Information
- Candidates must submit their resume in English and indicate their level of English proficiency.
- Flexible, remote, freelance project that fits around your primary professional or academic commitments.
