Staff / Senior Staff Machine Learning Engineer, Inference (US)
Inworld • South Bay Area
Company
Inworld
Annual salary
$300k – $430k/yr
Location
South Bay Area
Listed
115d ago
- Experience:
- 6 to 20 years
- Workplace:
- This role can either be onsite in Mountain View, CA, or remote for candidates in the US, Canada, Germany, Switzerland, United Kingdom, or Serbia, strong preference for Bay Area candidates who can work on-site in Mountain View, CA
- Visa:
- Visa sponsorship available
- Equity:
- Competitive equity
Open to people who live in the US, Canada or the UK
We can only put candidates forward for this role from those three countries for now. See the contract AI training jobs open in more countries.
Remote (listed in South Bay Area). Visa sponsorship available.
Send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role.
Apply for a referral →The recruiter emails you before anything happens and may suggest other jobs that suit you better.
What you need
- A year ago, reliably working agentic systems and sub-second multimodal inference at scale barely existed. Nobody has a decade of experience here. So we're not screening for a resume template, we're looking for strong people from varied backgrounds who learn fast, thrive in ambiguity, and can show us what they've built, broken, and understood.
Experience We Find Useful
You don't need all of this. But you need enough to make a case.
Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
Model Acceleration. Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding.
High-Performance Systems. Proficiency in C++, CUDA, Rust, or highly optimized Python. You know how to profile code and squeeze every ounce of performance out of NVIDIA GPUs.
Distributed Systems & Scaling. Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections.
**Public work. **Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups.
Full-cycle ownership. You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production.
Background. PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.
Who Thrives Here
You don’t need a roadmap to start walking; you’re comfortable picking a direction and building the map as you go.
You believe engineering isn't finished until it’s shipped and stable. You have a bias for impact over purely theoretical optimizations.
You don't just ship code; you obsess over the why. You’re the first to question an architecture if you think there’s a better way to solve the core latency or throughput problem.
You aren't satisfied with "the PM said so." You thrive on deep context and want to understand the fundamental logic behind every decision we make.
Common questions
How do I apply for the Staff / Senior Staff Machine Learning Engineer, Inference (US) role at Inworld? +
Use the Apply for a referral button on this page to send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role. They'll email you to check you're interested, then put you forward for this job or others that suit you better. It's free.
What does this Inworld role pay? +
The listing gives $300k – $430k/yr.
Is this role remote? +
Yes, the listing is remote. This role can either be onsite in Mountain View, CA, or remote for candidates in the US, Canada, Germany, Switzerland, United Kingdom, or Serbia, strong preference for Bay Area candidates who can work on-site in Mountain View, CA. Visa sponsorship available.
Machine Learning Engineer interview prep
Questions Machine Learning Engineer interviewers often ask. They aren't taken from this listing.
How do you choose between a simpler interpretable model and a more complex model that performs marginally better?
The decision depends on how the model's output is used downstream. If a human needs to act on and justify individual predictions, like in lending or healthcare, interpretability often outweighs a small accuracy gain. If the model feeds an automated system where explainability isn't a hard requirement, I'll take the complexity if the performance gain is meaningful and validated, not just noise.
Explain how you would set up a proper train, validation, and test split for a time series problem.
Random splitting leaks future information into training for time series, so I split chronologically instead, training on the earliest period, validating on the next, and testing on the most recent. Any cross-validation also needs to respect time order, using something like rolling-origin validation, rather than standard k-fold, which would otherwise let the model see the future during validation.
What statistical test would you use to determine if an A/B test result is significant, and what assumptions does it rely on?
For a conversion-rate comparison, a two-proportion z-test or chi-squared test is standard, and it assumes independent observations and a large enough sample size for the normal approximation to hold. If the sample is small or the metric isn't binary, I'd switch to a more appropriate test, since applying the wrong test's assumptions is a common source of false confidence in A/B results.
How do you approach hyperparameter tuning efficiently when the model is expensive to train?
I use a coarse random search first to identify the promising region of the hyperparameter space, since grid search wastes compute on combinations unlikely to matter. From there, Bayesian optimization narrows in on the best configuration with far fewer expensive training runs than an exhaustive search would require, which matters a lot when each run is costly.
Related roles
Browse all referral roles
Salaried roles at startups, AI companies and established businesses, all filled by referral. One application covers every role.
See all referral roles →