Member of Technical Staff - ML & Inference Infra (US)
NextToken • San Francisco Bay Area
Company
NextToken
Annual salary
$200k – $350k/yr
Location
San Francisco Bay Area
Listed
32d ago
- Experience:
- 1 to 8 years
- Workplace:
- In person, 5 days a week in Mountain View, CA. Open to candidates in Palo Alto and San Francisco
- Equity:
- 0.1, 2%
Open to people who live in the US, Canada or the UK
We can only put candidates forward for this role from those three countries for now. See the contract AI training jobs open in more countries.
On-site in San Francisco Bay Area.
Send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role.
Apply for a referral →The recruiter emails you before anything happens and may suggest other jobs that suit you better.
About this role
NextToken builds specialized models and learning systems for production AI agents. We believe every company will develop intelligence of its own: models that understand its work, embody its judgment, and improve through experience. We’re building the post-training methods and infrastructure to make that happen, working directly with companies on real production tasks. Our work spans reinforcement learning, environments for training and evaluating agents, and continual learning, all aimed at making models more capable, faster, and cheaper to run.
Founded by Nitish Kulkarni and Alankar Jain, who led search infrastructure and RL post-training for Google’s AI Overviews and AI Mode, NextToken brings research and production engineering together in a small team. As an early technical hire, you’ll work directly with the founders, shape our research and systems, and see your work deployed in real-world products.
What you'll do
- Take broad ownership over our serving and training infrastructure for specialized language models across experimentation and production.
- Profile and improve latency, throughput, GPU utilization, and cost under realistic workloads.
- Optimize batching, caching, memory use, and model execution; evaluate techniques such as quantization and speculative decoding where useful.
- Support inference for evaluations, data generation, and RL rollouts, and build reliable model deployment and monitoring workflows.
- Work closely with the founders and research team to allocate GPU capacity between training and serving, and to deploy post-trained models into customer environments.
What you need
- Hands-on experience building or optimizing LLM inference systems, with clear ownership of performance or reliability improvements.
- Strong systems engineering and debugging skills, including understanding of GPU memory and compute constraints.
- Experience with serving frameworks such as vLLM, SGLang, or TensorRT-LLM, or comparable infrastructure.
- Ability to measure tradeoffs between model quality, latency, throughput, and cost, and choose practical solutions.
- Comfort taking a system from prototype to production in a small team. CUDA/Triton, distributed inference, multi-model serving, and training infrastructure experience are valuable additions. We welcome engineers whose depth spans both model execution and production systems.
Common questions
How do I apply for the Member of Technical Staff - ML & Inference Infra (US) role at NextToken? +
Use the Apply for a referral button on this page to send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role. They'll email you to check you're interested, then put you forward for this job or others that suit you better. It's free.
What does this NextToken role pay? +
The listing gives $200k – $350k/yr.
Is this role remote? +
The listing gives the location as San Francisco Bay Area. In person, 5 days a week in Mountain View, CA. Open to candidates in Palo Alto and San Francisco.
Related roles
Browse all referral roles
Salaried roles at startups, AI companies and established businesses, all filled by referral. One application covers every role.
See all referral roles →