Senior Software Engineer - Systems

Boson AI

Apply to this job
Santa Clara HQ on site Until 9/7/2026 3+ years exp H-1B sponsor history First posted July 9, 2026 Last posted July 9, 2026
Job description
About Boson AI: At Boson AI, we are not just building AI solutions; we are pioneering the future of enterprise AI. Driven by a passion for cutting-edge AI research, particularly in the transformative areas of large language models and agentic systems, our mission is to tackle the most complex real-world problems for businesses and unlock significant value. We are a dynamic and collaborative team of researchers and engineers who thrive on pushing the boundaries of what's possible, dedicated to delivering high-quality, reliable products that seamlessly integrate into the fabric of enterprise workflows and set new industry standards.
 
About the Role: Build and operate the core platform behind Boson's model APIs and agentic products. You'll own the infrastructure that every Boson agent runs on — API serving, state management, data pipelines, context retrieval, and execution runtime — and make it fast, reliable, and easy for product teams to build on.

Responsibilities

  • Own and evolve the core platform infrastructure: API serving layer, state management, policy enforcement engine, and execution runtime for agentic workflows.
  • Design and operate high-throughput, low-latency distributed services that back our model API products — including request routing, load management, rate limiting, and multi-tenant isolation.
  • Build and maintain downstream data pipelines (ETL/ELT) for API logs, usage analytics, and billing — ensuring data correctness, freshness, and queryability at scale.
  • Develop production-grade internal SDKs and libraries with clean APIs, strong type safety, and clear contracts that product teams can build on confidently.
  • Architect context and memory systems for conversational workloads — low-latency retrieval, caching, and integration with vector stores and retrieval pipelines.
  • Instrument end-to-end observability: define SLIs/SLOs, build structured logging and tracing, and drive reliability improvements across the platform.
  • Collaborate closely with ML and product teams to integrate model serving, voice runtime, and tooling infrastructure under tight latency and quality constraints.

Qualifications

  • 3+ years building and operating backend systems at scale — you've owned services that other teams depend on in production.
  • Strong distributed systems fundamentals: concurrency, fault tolerance, consistency tradeoffs, capacity planning.
  • Hands-on experience with data pipeline infrastructure (Kafka/Kinesis, Spark/Flink, Airflow, or similar) for log processing, analytics, or ETL workloads.
  • Track record of designing APIs and frameworks adopted by other engineering teams — you care about developer experience and long-term maintainability.
  • Proficiency in at least one systems language (Go, Rust, Java, C++) or Python in a performance-sensitive context.
  • Comfortable working across the stack: cloud infrastructure (AWS/GCP), containerized deployments (K8s), CI/CD, and production oncall.

Bonus point

  • Experience with LLM serving, agentic orchestration patterns (ReAct, planner-executor), or RAG pipelines.
  • Familiarity with emerging agent integration protocols (MCP, A2A) or orchestration frameworks (LangChain, LlamaIndex).
  • Background in real-time media systems (audio/video streaming, low-latency signaling).
  • Experience building high-stakes platform services (payments, identity, core data) where correctness and auditability are non-negotiable.

Compensation (from employer):
180000–400000 USD per year

About this role

Summary

Build and maintain core platform backend systems, APIs, data pipelines, and infrastructure.

Job title

Senior Software Engineer - Systems

Experience level

3+ years

Minimum experience

3+ years exp

Industry

software

Location requirements

Santa Clara HQ, on-site preferred

Salary

$180k–$400k

Visa sponsorship

H-1B sponsor history

Management role

No

Skills & keywords

Required skills

distributed systemsAPI designdata pipeline infrastructureGo Rust Java C++ PythonAWS GCP Kubernetes CI/CD

Preferred skills

LLM servingagent orchestrationMCP A2A protocolreal-time media systemsplatform services

Specializations

distributed systemsAPI developmentdata pipelinesperformance engineeringcloud infrastructure
Locations

Structured locations inferred from the posting.

Unknown location

On-site
Related searches