Skip to content
aitrainer.work - AI Training Jobs Platform
Engineering Full-Time
Pi Security

AI Engineer (US)

Pi Security • Mission District, San Francisco, CA

Company

Pi Security

Annual salary

$168k – $235k/yr

Location

Mission District, San Francisco, CA

Listed

22d ago

Experience:
2 to 6 years
Workplace:
5 days a week on-site in San Francisco, CA
Equity:
Competitive equity

On-site in Mission District, San Francisco, CA.

Send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role.

Apply for a referral →

The recruiter emails you before anything happens and may suggest other jobs that suit you better.

About this role

Pi is building an agentic product security platform for teams that need to secure software at the speed they build it. We connect security knowledge scattered across code, tickets, documents, incidents, reviews, and people. This institutional security memory helps teams triage faster, remediate in context, prevent recurring vulnerability classes, and bring security guardrails into everyday engineering work.

We’re looking for an AI Engineer to build reliable agents, improve our AI infrastructure, and turn advances in LLMs into useful security features.

What You'll Do

  • Develop and maintain scalable AI agents, pipelines and services.

  • Build and improve infrastructure for running, evaluating, and monitoring AI agents.

  • Work with product and security teams to design and deliver AI-powered security features.

  • Evaluate advances in LLMs and AI agents and apply them where they improve the product

Common questions

How do I apply for the AI Engineer (US) role at Pi Security? +

Use the Apply for a referral button on this page to send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role. They'll email you to check you're interested, then put you forward for this job or others that suit you better. It's free.

What does this Pi Security role pay? +

The listing gives $168k – $235k/yr.

Is this role remote? +

The listing gives the location as Mission District, San Francisco, CA. 5 days a week on-site in San Francisco, CA.

AI Engineer interview prep

Questions AI Engineer interviewers often ask. They aren't taken from this listing.

How do you decide which evaluation metric to optimize for a classification model when the classes are imbalanced?

Accuracy is misleading on imbalanced data because a model can score well by always predicting the majority class. I look at precision, recall, and F1 for the minority class specifically, and pick the metric that matches the real cost of false positives versus false negatives in the deployment context, since those costs are rarely symmetric in practice.

What is your process for debugging a deep learning model that trains fine but performs poorly at inference time?

I start by checking for a train/inference mismatch: different preprocessing, batch normalization behaving differently in eval mode, or data leakage during training that inflated validation scores. Comparing the exact input pipeline used at training time against the one used at inference usually surfaces the discrepancy before I need to touch the model architecture itself.

How do you approach feature selection when working with high-dimensional tabular data?

I start with correlation and mutual information analysis to drop obviously redundant features, then use a tree-based model's feature importance as a first pass filter. From there, recursive feature elimination with cross-validation tells me where the marginal value of additional features drops off, which keeps the final feature set both smaller and more robust to overfitting.

Explain how you would detect and address data drift in a production machine learning pipeline.

I monitor the statistical distribution of input features and model outputs over time, using something like population stability index or KL divergence against a reference window. When drift crosses a threshold, I retrain on recent data rather than the full historical set, and I keep an alert in place so drift is caught before it shows up as a quality regression.

See all 10 AI Engineer interview questions →

Related roles

Pi Security

Browse all referral roles

Salaried roles at startups, AI companies and established businesses, all filled by referral. One application covers every role.

See all referral roles →