Microsoft released a dedicated AI Gateway tier for Azure API Management in public preview, rearchitecting the control plane around models and MCP servers rather than traditional API routes. The gateway federates models from Foundry-hosted OpenAI, Anthropic, Mistral, AWS Bedrock, and Google Vertex AI, with routing on exact model name match so teams can deploy multivendor inference without managing backend proliferation. Tools are sourced from remote MCP servers, OpenAPI specs, or a built-in connector covering 1000+ SaaS applications.
Policies are now card-based in the portal instead of XML—covering token/request limits, quotas, content safety, and model fallback—and the gateway spins up in about a minute with no scale units to plan. Telemetry exports as OpenTelemetry metrics to Application Insights, Datadog, Grafana, or custom destinations. The operating model splits central governance (platform teams approve models and tools) from team self-service (app teams build against published assets without routing changes back to the center). A gateway-scoped access key reaches every model and tool, which creates risk: teams familiar with API Management's per-API keys lose that blast-radius boundary.
The preview is qualified: no SLA, APIs and pricing unpublished, quotas on models/tools/throughput undisclosed, and the tier is available only in East US 2 and Sweden Central. Cost governance at the gateway was praised as underrated—most teams bolt on spend tracking post-incident. But the announcement leaves a critical open question: who governs the full agent lifecycle? The gateway can fail over and retry on error, but if an agent produces useful output before a non-clean completion, the annotation/preservation of that partial result is unspecified.
For architects, this is a step toward multivendor model governance but early-stage. The no-SLA, unpublished pricing, and lack of clarity on agent execution semantics mean production adoption should wait for GA and documentation. The gateway-scoped key is a material risk compared to per-API scoping; teams need explicit guidance on key rotation and compartmentalization. Watch for how Databricks' Unity AI Gateway (announced June, still beta) and AWS bedrock management stack evolve in parallel—the three clouds are now all racing on this control plane layer.