Software Engineer - ML/LLM Inference (San Francisco) Job at Alldus, San Francisco, CA

QklONjVjd1lCZ0JTd1UwdkZveWtGcEFaUWc9PQ==
  • Alldus
  • San Francisco, CA

Job Description

Get AI-powered advice on this job and more exclusive features.

Direct message the job poster from Alldus

Principal Recruitment Consultant | AI & Machine Learning | Co-organizer of the AI in Action Podcast

My client is searching for a talented engineer to work on ML/LLM inference and serving. They specialize in developing next-gen LLM fine-tuning and inference engines.

We are seeking a talented and motivated Software Engineer specializing in Machine Learning (ML) and Large Language Model (LLM) inference to join our dynamic ML Inference team. In this role, you will bridge the gap between AI/ML research and systems programming to build and enhance our next-generation LLM Inference Engine. You will play a crucial role in optimizing the performance, scalability, and efficiency of our LLM serving systems.

Key Responsibilities:

Develop and Enhance Inference Engine:

  • Design, implement, and optimize the next-generation LLM Inference Engine.
  • Integrate the latest LLM inference techniques from research to enhance latency and throughput.

Performance Optimization:

  • Conduct deep performance optimizations across multiple layers of the technology stack, including PyTorch, C++, and CUDA.
  • Analyze and improve system performance to meet the demands of various use cases.
  • Work closely with customers to understand specific performance requirements and optimize solutions accordingly.
  • Provide technical expertise and support to ensure successful deployment and operation of inference systems.

Technical Leadership:

  • Define the roadmap and technical vision for the inference stack.
  • Lead initiatives to drive innovation and maintain the competitive edge of our inference technologies.

Infrastructure Development:

  • Collaborate with partner teams to build and maintain scalable, multi-replica serving infrastructure.
  • Ensure the reliability and scalability of LLM serving systems to handle increasing workloads.

Qualifications:

Technical Skills:

  • Proficiency in systems programming languages such as C++.
  • Strong experience with machine learning frameworks, particularly PyTorch.
  • Expertise in GPU programming and CUDA for performance optimization.
  • Solid understanding of AI/ML concepts, especially related to large language models.

Experience:

  • Proven experience in developing and optimizing ML/LLM inference systems.
  • Demonstrated ability to integrate research advancements into production systems.
  • Experience with performance tuning and profiling across various technology stacks.
  • Experience with vLLM

Seniority level

  • Seniority level

    Mid-Senior level

Employment type

  • Employment type

    Full-time

Job function

  • Industries

    Staffing and Recruiting and Software Development

Referrals increase your chances of interviewing at Alldus by 2x

Inferred from the description for this job

San Francisco, CA $130,000.00-$238,000.00 3 days ago

San Francisco, CA $40,000.00-$70,000.00 2 weeks ago

San Francisco, CA $145,000.00-$230,000.00 5 days ago

Full-Stack Software Engineer (Jr/Mid level)

San Francisco, CA $220,000.00-$350,000.00 4 hours ago

San Francisco, CA $150,000.00-$230,000.00 2 months ago

San Francisco, CA $150,000.00-$176,000.00 2 months ago

San Francisco, CA $99,500.00-$200,000.00 1 day ago

San Francisco, CA $130,000.00-$140,000.00 2 days ago

San Francisco, CA $120,000.00-$190,000.00 8 months ago

San Francisco, CA $125,000.00-$175,000.00 1 month ago

Software Engineer, Frontend (All Levels)

San Francisco, CA $150,000.00-$220,000.00 1 hour ago

San Francisco, CA $56.25-$173,000.00 2 weeks ago

San Francisco, CA $176,000.00-$250,000.00 2 weeks ago

Alameda, CA $130,000.00-$160,000.00 4 weeks ago

San Francisco, CA $150,000.00-$283,000.00 2 weeks ago

San Francisco, CA $150,000.00-$300,000.00 5 days ago

San Francisco, CA $165,000.00-$165,000.00 2 years ago

San Francisco, CA $140,000.00-$280,000.00 7 months ago

San Francisco, CA $140,000.00-$180,000.00 1 month ago

San Francisco, CA $130,000.00-$185,000.00 2 months ago

San Francisco, CA $99,500.00-$200,000.00 1 day ago

San Francisco, CA $150,500.00-$269,200.00 2 days ago

San Francisco, CA $100,000.00-$200,000.00 1 year ago

San Francisco, CA $120,000.00-$200,000.00 2 years ago

San Francisco, CA $150,000.00-$250,000.00 9 months ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr

Job Tags

Full time,

Similar Jobs

IDR Healthcare

Travel Certified Surgical First Assistant / Scrub Tech - $1,984 per week Job at IDR Healthcare

 ...IDR Healthcare is seeking a travel Surgical First Assistant for a travel job in Manchester, New Hampshire. Job Description & Requirements ~ Specialty: Surgical First Assistant ~ Discipline: Allied Health Professional ~ Start Date: 08/18/2025~ Duration: 13... 

BJ's Wholesale Club

SR. Account Manager, Retail Media Job at BJ's Wholesale Club

 ...multiple projects at same time Self-sufficient, solution orientated and strong problem-solving skills Experienced in Microsoft Suite (Word, Excel, PowerPoint) and Outlook Faster learner being able to easily adapt within an ever changing work environment... 

B.E.L. Associates, Inc.

Non-invasive Cardiologist in Beautiful SW NY - $50K Sign on Bonus! Visa Sponsorship Available Job at B.E.L. Associates, Inc.

 ...incentives. Benefits include five weeks of vacation and one week CME. Live on a beautiful lake and enjoy the culture of the region. About an hour from Buffalo, NY. J1 and H1b candidates are urged to apply. Minimum Salary Range: $500,000 | Maximum Salary Range: $541,666... 

AutoMaximizer, Inc.

Mobile App Developer iOS & Android Job at AutoMaximizer, Inc.

 ...Job Description Job Description This is a remote position. Mobile App Developer iOS & Android Mobile App Developer iOS & Android Mobile App Developer iOS & Android Mobile App Developer iOS & Android Requirements Mobile App Developer iOS & Android... 

US Tech Solutions, Inc.

Social Media Escalations & Crisis Response Manager Job at US Tech Solutions, Inc.

Duration: 8 Months Contract(Hybrid)Job Description: The Social Media Manager is responsible for leading crisis management efforts across...  ...process and contextual awareness on potential emergent issues.Experience: Experience managing cross-functional or cross-team...