Data Analyst (AI Evaluation)
micro1 • Remote, United States
Education
Master's
Type
Hourly
Pay
$30-60/hr
Your rate depends on your experience and the task.
Actively hiring
30 openings
Checked
Only apply if this role fits your skills and interests.
About this role
From the micro1 listing
Role Title: Data Analyst (AI Evaluation) Role Type: Contractor Location: Remote, US-based. micro1 is engaging Data Analysts to contribute their knowledge and expertise to a customer’s data quality project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters. Scope of Work
- Review datasets and detailed task outputs for accuracy, completeness, and adherence to guidelines.
- Identify and flag inconsistencies, errors, and outputs that deviate from provided instructions.
- Apply structured rubrics and evaluation criteria consistently across large data sets.
- Exercise sound judgment and document rationale in ambiguous scenarios where guidelines are unclear.
- Escalate patterns or recurring data issues for further analysis rather than addressing them individually.
- Provide clear, actionable written and verbal feedback pinpointing issues and suggesting resolution paths.
- Maintain meticulous attention to detail while working through high volumes of repetitive data review assignments. Preferred Qualifications
- 3–5+ years of experience in data review, data quality, quality assurance, or analytical roles.
- Proven expertise in reviewing, validating, and critiquing data or deliverables based on explicit criteria.
- Exceptional ability to follow detailed written instructions and apply complex evaluation rubrics precisely.
- Demonstrated comfort with ambiguity, including making justifiable decisions when guidelines are incomplete.
- Advanced attention to detail and error-spotting capabilities across repetitive or high-volume work.
- Strong written and verbal communication skills for documenting findings and providing feedback.
- Baseline fluency with spreadsheets (sorting, filtering, simple formulas); SQL or hands-on data querying is a plus.
- Experience with data annotation, labeling, or rubric-based evaluation is advantageous.
Requirements
- Data Analysis
- Data Review
- Attention to detail
- Handling ambiguity
- Error spotting
- Must be eligible to work in United States
How long hiring takes
On the platforms we refer people to, half of hires came within about 25 days of applying. Time from applying to hire:
Data Analyst interview prep
Questions Data Analyst interviewers often ask. They aren't taken from this listing.
What's your process for cleaning a dataset before starting analysis?
I start by profiling the data for missing values, duplicates, and outliers, then decide case by case whether to impute, drop, or flag issues based on how they'd affect the specific analysis. I document every transformation so the cleaning process is reproducible and reviewable, not just a black box.
How do you decide which statistical test is appropriate for comparing two groups?
I check the distribution of the data and whether the groups are independent or paired first, since that determines whether a t-test, a non-parametric alternative like the Mann-Whitney U test, or a paired test is appropriate. Choosing a test based on convention rather than the data's actual properties leads to unreliable conclusions.
What makes a data visualization misleading, even if the underlying numbers are correct?
Common issues include truncated axes that exaggerate differences, using pie charts for data with too many categories to compare visually, or choosing a chart type that implies a trend the data doesn't actually support. I try to match the chart type to the specific comparison I want the viewer to make.
How do you handle outliers in a dataset when you're not sure if they represent errors or real extreme values?
I investigate the source of the outlier before deciding how to handle it, since a data entry error should be corrected or removed, while a genuine extreme value, like a real high-value transaction, should usually stay in the dataset. Removing outliers automatically without checking their origin risks losing real signal.
Key responsibilities
- Review datasets and detailed task outputs for accuracy, completeness, and adherence to guidelines.
- Identify and flag inconsistencies, errors, and outputs that deviate from provided instructions.
- Apply structured rubrics and evaluation criteria consistently across large data sets.
- Exercise sound judgment and document rationale in ambiguous scenarios where guidelines are unclear.
Why this role
Micro1's Data Analyst role pays $31-$60 per hour for people who can review large business data sets while protecting personally identifiable information. The work centers on document review, data analysis and quality assurance on a project built around business data rather than general datasets. Candidates should be comfortable applying data privacy standards to every record they touch, since PII handling is as central to the role as the analysis itself.
Skills and categories
More jobs in these fields:
Related jobs
Verified reviews
Community reviews
Leave your review
Common questions
Does micro1 monitor your computer while you work?
On many projects, yes. Time-tracking tools take screenshots now and then to confirm your hours. Check the project's requirements before you accept it.
What does the micro1 application process look like?
You start with a video interview with Zara, micro1's AI interviewer. Treat it like a video call: good light, clear sound and spoken answers to technical questions. A person reviews the recording, and some roles add a short skills test.
What does task-based AI training work look like?
Short, hands-on tasks such as recording videos, sorting images, rating AI answers or checking data. Most take 5 to 60 minutes each.
What does Data Science work look like for a Data Analyst (AI Evaluation)?
You check AI answers to Data Science questions, correct their errors and write expert reasoning. As a Data Analyst (AI Evaluation), you're hired for that judgment, so hands-on Data Science experience matters.
What specific skills does this listing call for?
Data Analysis, Data Review, Attention to detail, and Handling ambiguity are named directly in the listing. If you don't have hands-on experience with these, expect the screening to test for them.
What does the 'openings' count on this listing mean?
It's how many people micro1 still wants to hire for this role: 30 when we last checked. A high number means more places to fill. The screening stays the same.
How much does this specific role pay?
This listing is posted at $30-60/hr, an hourly rate. Where you land in the range depends on your background and the assessment. Pay can change after we check the listing, so confirm the rate on the platform's application page.
What happens when I click Apply on this listing?
Apply opens micro1's site, where you create an account and follow its sign-up steps.
Do I need a Master's to qualify?
This role lists a master's degree. The domain assessment usually matters more, and if you pass it the degree is often secondary. Some platforms check credentials, so list your qualifications accurately.
Can I apply from outside the United States?
This role is open only to people based in the United States. The limit is usually about where the platform can legally hire, so applying from elsewhere rarely leads to an offer. Check the description for tax or right-to-work rules before you apply.