The Gateway to AI Innovation
Secure, observe, and control your AI applications with Gloo Gateway, the leading cloud native gateway built on Envoy
Unleash Developer Productivity. Accelerate AI Innovation
Fast Track Development
Eliminate development friction, boilerplate code, and avoidable errors in applications consuming LLM APIs with AI gateway features and guardrails across multiple LLM providers.
- Prompt templating
- Multi-LLM provider
- Prompt retries
Secure and Control
Protect applications, models, and data from inappropriate access. Ensure safe use of AI with governance controls, auditability, and visibility into consumption.
- Prompt guard & auditing
- Data exfiltration protection
- Token-weighted rate limiting
Expand and Scale
Leverage advanced AI integration patterns for data augmentation and integration with cloud-native gateway capabilities to support high-volume, zero-downtime AI connectivity.
- Retrieval Augmented Generation (RAG)
- Model fallback/failover
- Semantic caching
Get started with Gloo AI Gateway
Lab 1: Learn how to manage credentials and access control to secure AI workloads.
Lab 2: Implement prompt management and guards to improve response accuracy and protect sensitive content.
Lab 3: Learn how rate limiting and model failovers can help control token usage and build resiliency.
Lab 4: Deploy semantic caching and RAG to improve output precision and optimize API performance.
Start consuming LLMs across AI applications with Gloo AI Gateway.
Credential Management
Implement authentication and authorization controls across AI workloads.
Model Failover
Build resiliency into AI workloads and ensure uninterrupt service across LLM provider APIs.
Access Control
Learn how to configure a JWT provider using Gloo API Gateway and enforce access control for LLMs.
Prompt Guard
How to detect and block sensitive content and secure data interactions within LLM APIs.
Retrieval Augmented Generation (RAG)
Configure API credentials and refine query responses for precise outputs with RAG.
Prompt Enrichment
Set up and optimize system and user prompts to enhance LLM capabilities and responses.
Rate Limiting
Manage LLM consumption and resource allocation by rate limiting and controlled token usage within LLM APIs.
Semantic Caching
Learn how to configure Gloo AI Gateway to utilize Redis and optimize API performance.
Supercharge your cloud-native AI application starting with Gloo Gateway
Cloud Native From The Start
Implement the only enterprise API gateway built on Envoy. Run billions of transactions at a fraction of the cost with a programmable model that developers love.
Flexible, Multi-Tiered Architecture
Run Gloo Gateway as a central enterprise gateway and as a distributed micro-gateway. Reduce hair-pinning and eliminate single points of failure without sacrificing central governance and control.
Observe and Control Every Connection
API Traffic is now omnidirectional. Gloo Gateway seamlessly pairs with Istio – with or without sidecars – for complete North, South, East, and West connectivity and observability.
API Platform Ready for Center Stage
The next-generation of API platforms are not monoliths – they are composable. Gloo Gateway integrates with leading technologies like Backstage and Kubernetes Gateway API to provide modern platform engineering and cloud teams with the tools and solutions they want to use.
“Gloo checked all the boxes. API Gateway, advanced traffic routing, strong security controls, multi-cloud support and self-service for developers.”
T-Mobile
