Skip to Content
GenAIAI/ML On-site

Software Engineer, Agentic Runtime

Job Overview

The Agentic Runtime team builds the low‑latency, reliable, and secure foundation that powers Glean’s AI agents and assistant experiences at scale. You’ll design and operate core runtime services for multi‑turn orchestration, tool calling, model routing, memory, streaming, and safety.

Key points

  1. Own impactful runtime problems end‑to‑end — from architecture and design to production launch and ongoing reliability.

  2. Build and evolve core services for session lifecycle, streaming responses (e.g., gRPC/WebSockets), structured tool execution, memory/state, and policy/guardrails.

  3. Design for performance, correctness, and cost: reduce p50/p95 latency, improve tail behavior, and optimize token/tool budgets.

  4. Integrate with leading LLM providers (e.g., OpenAI, Anthropic, Google Gemini) and internal evaluation frameworks to improve quality and predictability.

  5. Harden the platform with fault isolation, retries, timeouts, circuit‑breaking, backpressure, and graceful degradation.

  6. Depending on your location, the General Data Protection Regulation (GDPR), California Consumer Privacy Act (CCPA)…

  7. By submitting your application, you are agreeing to our use and processing of your data as required.

  8. US applicants and their applications are subject to arbitration of disputes as outlined in our Applicant Arbitration Agreement.

Skills

LLMsPythonJavaScriptTypeScriptJavaAWSAzureGoogle Cloud

Experience: Mid-level

Similar jobs

More open roles like this one