JioStar: A Multilingual AI Assistant for 500 Million Users

Key Takeaways

1JioStar launched a video discovery and cricket statistics AI assistant in eight weeks, integrated with JioHotstar

2The experience works in over 15 languages and targets more than 500 million users

3The company emphasizes continuous evaluation, safeguards, human expertise, and monitoring of model effectiveness

4Technical trade-offs focus on accuracy, latency, and cost, with particular attention to voice

Why it matters : This feedback provides insights into the success factors for a conversational AI at a very large scale, from design to production.

The chief architect of JioStar describes the production rollout of a video discovery assistant and cricket statistics tool, built in eight weeks and available in over 15 languages. The key elements include a continuous evaluation approach, safeguards, support from domain experts, and monitoring of model effectiveness to operate at the scale of 500 million users.

Validation, Safeguards, and Experts at the Core of Reliability

JioStar structured the evaluation into two components: an evaluation framework and a validation layer, aiming to evolve the system before and during production. In the production phase, a feedback loop transforms user behaviors and failures into new evaluation cases, a mechanism deemed challenging to implement. Domain experts, including sports magazine writers, were involved to anticipate the demands of discerning users, with JioStar believing that in-depth human expertise is necessary beyond a simple prompt. The initial design phase involved defining the scope and the questions the system should not answer, establishing safeguards from the outset. Validation relies on a maker-checker scheme, with one model generating the response and a validating LLM pursuing a competing objective, raising the question of how to evaluate the validator itself. Despite the acceleration of developments, JioStar emphasizes that human judgment and validation remain essential, highlighting the need for safeguards and clear accountability assigned to the right individuals.

Technical Trade-offs and Production Monitoring

The design was guided by a trade-off between accuracy, latency, and cost. Voice plays a central role, with over 60% of users preferring it to text, necessitating relevant, rapid, and economically viable responses. In hindsight, Vijay Seshadri would invest earlier in production monitoring focused on model efficiency. Classic infrastructure metrics like P99 latency remain necessary, but JioStar considers them insufficient unless complemented by specific monitoring of model effectiveness in real-world situations.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Streamlined Organization and Support from OpenAI

The project began with a small team of data scientists and engineers gathered around a clearly defined problem. Engineers deployed by OpenAI worked with the team to identify priority success factors. For evaluation, the team started with an initial dataset and used models to increase the variety of cases. Codex helped accelerate the generation of evaluation sets and system updates.

Use Cases Covering Entertainment and Cricket

The assistant targets both entertainment viewers and cricket fans, with differing needs. The former may make general queries or describe a movie without knowing the title, while the latter look for specific events like wicket falls, sixes, or matches associated with statistical performances, and experts seek ball-by-ball statistics. The experience combines conversational interaction with visual presentation, following an approach described as a voice-visual interface by JioStar.

Multilingual Scope and Tight Timeline

The assistant, integrated into JioHotstar via the OpenAI API, was designed and launched in about eight weeks. Work began in October and November, targeting a major cricket tournament scheduled for February, resulting in a timeline of approximately two months. The experience supports over 15 languages, adding an extra constraint to a tight schedule.

Scaling Requirements and Steering Advice

JioStar's platforms serve over 500 million consumers and have already supported up to 72.5 million concurrent viewers for sporting events. At the OpenAI Executive Summit in Singapore, chief architect Vijay Seshadri shared his vision for content discovery and recommendations for scaling in enterprise settings. JioStar observes a gap between technological advancements and operational reality, encouraging a focus on outcomes rather than activity, as input metrics like lines of code generated do not alone demonstrate value creation. For Seshadri, AI should serve to simplify processes, workflows, and software systems. The company also emphasizes that human skills and agency are often underestimated: a tool like Codex can amplify the impact of a talented engineer, but without the necessary judgment to validate production systems, the risk also increases.