MCP & Tools Python Developer - Agent Evaluation Infrastructure
Mindrift • United States
Education
Not stated
Type
Hourly
Pay
$80/hr equivalent
Mindrift pays per task and shows the pay as an hourly rate.
Last checked
14d ago
Checked
About this role
From the Mindrift listing
This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency. At Mindrift , innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. What we do The Mindrift platform, launched and powered by Toloka , connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe. Who we're looking for Calling all security researchers, engineers, and penetration testers with a strong foundation in problem-solving, offensive security, and AI-related risk assessment. If you thrive on digging into complex systems, uncovering hidden vulnerabilities, and thinking creatively under constraints, join us! We’re looking for someone who can bring a hands-on approach to technical challenges, whether breaking into systems to expose weaknesses or building secure tools and processes. We value contributors with a passion for continuous learning, experimentation, and adaptability. About the project We’re on the hunt for hands-on Python engineers for a new project focused on developing Model Context Protocol (MCP) servers and internal tools for running and evaluating agent behavior. You’ll implement base methods for agent action verification, integrate with internal and client infrastructures, and help fill tooling gaps across the team. What you’ll be doing: Developing and maintaining MCP-compatible evaluation servers Implementing logic to check agent actions against scenario definitions Creating or extending tools that writers and QAs use to test agents Working closely with infrastructure engineers to ensure compatibility Occasionally helping with test writing or debug sessions when needed Although we’re only looking for experts for this current project, contributors with consistent high-quality submissions may receive an invitation for ongoing collaboration across future projects. How to get started: Apply to this post, qualify, and get the chance to contribute to a project aligned with your skills, on your own schedule. Shape the future of AI while building tools that benefit everyone.
Requirements
- Must be eligible to work in United States
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in Software Engineering
Interview prep
This listing mentions this tool. Brush up before a technical screen:
Why this role
This MCP & Tools Python Developer - Agent Evaluation Infrastructure position pays $80/hr equivalent. You need solid Software Engineering knowledge, and you learn the AI training steps on the job.
Skills and categories
More jobs in these fields:
Related jobs
Senior Data Scraping Engineer (Python)
Mindrift • Software Engineering
$45 /hr equivalent
Senior Python Data Scraping Engineer (Freelance)
Mindrift • Software Engineering
$40 /hr equivalent
Freelance Data Scraping Engineer (Python)
Mindrift • Software Engineering
$37 /hr equivalent
Freelance Data Scraping Engineer (Python)
Mindrift • Software Engineering
$37 /hr equivalent
Verified reviews
Community reviews
Leave your review
Common questions
What does a typical Mindrift task involve?
Read a scenario, write a short prompt for the AI (around 100 words), evaluate two AI responses to it, fact-check the outputs, then write a brief explanation (around 50 words) on which response is better against the project rubric. The focus is clarity, safety, and rule-following, not creative writing.
Who is Mindrift good for?
Freelance writers, editors and generalist AI tutors with strong English and good research skills. You don't need a tech degree. Mindrift is built by the data company Toloka, and it also opens better-paid projects in fields such as cybersecurity, medicine and law to verified experts.
What does task-based AI training work look like?
Short, hands-on tasks such as recording videos, sorting images, rating AI answers or checking data. Most take 5 to 60 minutes each.
Can I choose my own hours?
Usually, yes. You log in when you want, pick up a task, finish it and submit it, with no meetings. Other people work from the same pool of tasks, so some days there may be no work until more is added.
What does Software Engineering work look like for a MCP & Tools Python Developer - Agent Evaluation Infrastructure?
You check AI answers to Software Engineering questions, correct their errors and write expert reasoning. As a MCP & Tools Python Developer - Agent Evaluation Infrastructure, you're hired for that judgment, so hands-on Software Engineering experience matters.
How much does this specific role pay?
This listing is posted at $80/hr equivalent, an hourly rate. Pay can change after we check the listing, so confirm the rate on the platform's application page.
Can I apply from outside the United States?
This role is open only to people based in the United States. The limit is usually about where the platform can legally hire, so applying from elsewhere rarely leads to an offer. Check the description for tax or right-to-work rules before you apply.