Inworld Launches Realtime Inference for Consumer AI Applications

New first-party inference service combines optimized open models, cache-aware routing, model-specific decoding, and production controls through an OpenAI-compatible API

MOUNTAIN VIEW, Calif.--(BUSINESS WIRE)--Inworld, a research lab and inference provider focused on realtime AI for consumer-facing applications, today launched Realtime Inference, a first-party service for running open models with the latency, reliability, and economics required by high-volume consumer products.

Realtime Inference is available through Inworld’s OpenAI-compatible API and the company’s Realtime Router. Developers can call Inworld-hosted models directly or combine them with more than 100 third-party models through conditional routing, provider fallback, traffic splitting, caching, and per-request observability.

The launch arrives as AI infrastructure competition moves from training models to operating them. Recent technical discussion and coverage have concentrated on inference cost, KV-cache design, speculative decoding, throughput, and production reliability. Inworld is applying those systems to a specific workload: consumer applications that repeatedly process long conversation history, return short replies, serve unpredictable traffic, and absorb the cost of every interaction.

“Cost is the first thing standing in these developers’ way, so it is the first thing we are taking down,” said Kylan Gibbs, CEO and co-founder of Inworld. “But a consumer app does not succeed on cheap inference alone. These teams need help with growth and distribution. We are building toward a model where a small team can come to Inworld and get the technology, the economics, and the support to build a real consumer business.”

Built around the consumer interaction

Consumer AI products often send prompts that are substantially longer than their replies. The system may repeatedly process instructions, memories, safety rules, and conversation history, while the user expects the next turn to begin immediately. One visible response can also require several model calls for intent, safety, retrieval, ranking, memory, tools, and speech.

Realtime Inference is designed around the completed interaction rather than an isolated token. The service includes:

• Inworld-optimized open models from model families including Gemma, DeepSeek, GLM, Qwen, and MiMo, with current availability and prices published in the Inworld model catalog.
• Cache-aware serving that reuses repeated prompt context and supports lower cached-input rates on eligible models.
• Model-specific decoding and precision work designed to improve first-token latency, streaming pace, and throughput without bypassing quality evaluations.
• Conditional routing and failover through Realtime Router, including routes based on language, plan, intent, session depth, or other request metadata.
• Production observability for model selection, attempt chains, time to first token, token cost, cache behavior, and routing reason, with data exports to product analytics systems.

Production evidence with visible measurement boundaries

Inworld’s launch measurements include production observations, traffic replays, fault tests, and operation-level benchmarks. The company is labeling those evidence classes separately rather than treating them as interchangeable.

In one seven-day production comparison for Gemma 4 26B, requests per serving instance increased from 2.6 to 4.6 while median time to first token improved from 178 milliseconds to 161 milliseconds. In a replay of 5,000 production requests for GLM 5.3 Flash, cache-aware routing increased the cache-hit rate from 25 percent to 47 percent.

Over one 30-day period, GLM 5.3 Flash completed more than 20 million requests with 98 counted failures, a 99.9995 percent completion rate for that model and measurement window. Inworld defines counted failures for this result as server errors, timeouts, and capacity rejections; client errors and cancellations are excluded.

These results are first-party measurements tied to the named model, workload, and test. Developers should repeat performance tests from their own backend region using their own prompts, concurrency, and quality requirements.

Designed for products where success increases infrastructure pressure

Status, Wishroll’s AI social simulation, scaled from prototype to one million users in 19 days. The product uses Inworld’s model and routing infrastructure to support a consumer experience with persistent context and frequent generation.

“We scaled from prototype to 1 million users in 19 days with over 20x cost reduction,” said Fai Nur, CEO of Status.

The reduction reflects application and infrastructure work together, including prompt design, model selection, application-specific evaluations, routing, load testing, and serving cost. It should not be read as a token-price comparison alone.

Realtime Inference can be used independently or alongside Inworld’s Realtime TTS, Realtime STT, Realtime API, Router, and managed compute.

Available now

Realtime Inference is available today through Inworld’s developer platform. Developers can browse current models, providers, context windows, and input, output, and cache-read prices at inworld.ai/models. Technical documentation and quickstarts are available at docs.inworld.ai/router/introduction.

About Inworld

Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. The company builds first-party speech models, serves text, image, video, and decision models, and operates the infrastructure behind modular APIs for speech recognition, voice generation, routing, inference, realtime interactions, and compute. Inworld helps developers meet the quality, latency, reliability, and cost requirements of high-use consumer AI products.