Company
Pi Security
Annual salary
$168k – $235k/yr
Location
Mission District, San Francisco, CA
Listed
22d ago
- Experience:
- 2 to 6 years
- Workplace:
- 5 days a week on-site in San Francisco, CA
- Equity:
- Competitive equity
Open to people who live in the US, Canada or the UK
We can only put candidates forward for this role from those three countries for now. See the contract AI training jobs open in more countries.
On-site in Mission District, San Francisco, CA.
Send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role.
Apply for a referral →The recruiter emails you before anything happens and may suggest other jobs that suit you better.
About this role
Pi is building an agentic product security platform for teams that need to secure software at the speed they build it. We connect security knowledge scattered across code, tickets, documents, incidents, reviews, and people. This institutional security memory helps teams triage faster, remediate in context, prevent recurring vulnerability classes, and bring security guardrails into everyday engineering work.
We’re looking for an AI Engineer to build reliable agents, improve our AI infrastructure, and turn advances in LLMs into useful security features.
What You'll Do
Develop and maintain scalable AI agents, pipelines and services.
Build and improve infrastructure for running, evaluating, and monitoring AI agents.
Work with product and security teams to design and deliver AI-powered security features.
Evaluate advances in LLMs and AI agents and apply them where they improve the product
Common questions
How do I apply for the AI Engineer (US) role at Pi Security? +
Use the Apply for a referral button on this page to send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role. They'll email you to check you're interested, then put you forward for this job or others that suit you better. It's free.
What does this Pi Security role pay? +
The listing gives $168k – $235k/yr.
Is this role remote? +
The listing gives the location as Mission District, San Francisco, CA. 5 days a week on-site in San Francisco, CA.
AI Engineer interview prep
Questions AI Engineer interviewers often ask. They aren't taken from this listing.
How do you decide which evaluation metric to optimize for a classification model when the classes are imbalanced?
Accuracy is misleading on imbalanced data because a model can score well by always predicting the majority class. I look at precision, recall, and F1 for the minority class specifically, and pick the metric that matches the real cost of false positives versus false negatives in the deployment context, since those costs are rarely symmetric in practice.
What is your process for debugging a deep learning model that trains fine but performs poorly at inference time?
I start by checking for a train/inference mismatch: different preprocessing, batch normalization behaving differently in eval mode, or data leakage during training that inflated validation scores. Comparing the exact input pipeline used at training time against the one used at inference usually surfaces the discrepancy before I need to touch the model architecture itself.
How do you approach feature selection when working with high-dimensional tabular data?
I start with correlation and mutual information analysis to drop obviously redundant features, then use a tree-based model's feature importance as a first pass filter. From there, recursive feature elimination with cross-validation tells me where the marginal value of additional features drops off, which keeps the final feature set both smaller and more robust to overfitting.
Explain how you would detect and address data drift in a production machine learning pipeline.
I monitor the statistical distribution of input features and model outputs over time, using something like population stability index or KL divergence against a reference window. When drift crosses a threshold, I retrain on recent data rather than the full historical set, and I keep an alert in place so drift is caught before it shows up as a quality regression.
Related roles
Browse all referral roles
Salaried roles at startups, AI companies and established businesses, all filled by referral. One application covers every role.
See all referral roles →