Advertising is a key source of revenue in most businesses. It certainly forms a key pillar for JioHotstar. There is so much technology hidden under the surface. In this blog, we will cover some of the complexity that gets unlocked when we render an ad on our content. Every 30-second ad break is a high-stakes performance involving millions of calculations, real-time bidding, and surgical precision.

When you open JioHotstar to watch a live sports event or simply a movie, you see a Fence* ad (static display) or a Preroll* or a Midroll* Ad (video). What you don’t see is the “Ad Decision” engine firing at millisecond speeds to decide: Why this ad? Why now? And why you?

Speak Like an Adtech Pro

Getting your foot inside the door, we need to understand some jargon!

• CPM (Cost Per Mille): The cost an advertiser pays for every 1,000 impressions.
• CPC — Cost per Click
• SSAI (Server-Side Ad Insertion): Ads are “stitched” into the video stream on the server. At JioHotstar scale, this often happens at a Cohort level where we are grouping similar users to a balanced personalization of Ads.
• CSAI (Client Side Ad Insertion): The frontend client app(player) interrupts the video stream based on cue markers to show an ad. These cue markers are known well in advance when the user plays any content.
• SGAI (Server-Guided Ad Insertion): The server “guides” the player on what to do, but the player performs the final insertion. This allows for 1:1 personalization at a live event scale where we do not have the cue markers in advance.
• Programmatic vs. Direct: Direct is a handshake deal (e.g., “I want 1M views on an Ad”). Programmatic is an automated auction where we ask a 3rd-party SSP (Supply Side Platform) for an ad in real-time.

Some common Ad Formats on JioHotstar

Preroll Ad: The video ad you see before the actual content video starts.

Midroll Ad: The video ad you see while in between the content.

Fence Ad: The static display ad you see below the player on the app.

The Demand Puzzle

Imagine you’re at your local market where you see two establishments side-by-side:

1. A global smartphone brand launching its latest flagship wants 10 million impressions across India to drive nationwide awareness.
2. A Local Boutique: Wants 1000 clicks (CPC) specifically from users in Indiranagar, Bengaluru.

Both want to run ads, but their goals, scale, targeting strategy, and optimization signals are fundamentally different. One is optimizing for reach[how many different types of customers saw it] and visibility[how many unique geographies was this seen in] at national scale. The other is optimizing for precision, locality, and more measurable performance. Our system must juggle thousands of these conflicting demands simultaneously.

This leads us to the ultimate engineering challenge: The Selection.

Zeroing In: The Ad Decision Logic

How do we pick 2–3 ads out of thousands for a 30-second ad “pod”? At JioHotstar, we use a very intricate implementation of waterfall tiers approach mixed with our PID and SHALE pacing algorithms. These calculations are all performed and delivered in under 100ms, even during massive concurrency spikes like the final over of an IPL match.

During the Live sports events, the ads services get a very spiky traffic giving little to no time for the services to react to organic scale-up if we are not prepared in advance. The peaks are observed during ad breaks while the API traffic becomes more predictable and lower during the innings break(20:40 hrs — 21:10 hrs) as we see it in the graph above.

1. The Relevance

Out of 10,000 potential candidate ads, which specific ads are actually eligible for a female user, on a premium Android phone device, watching a comedy movie, in Indiranagar, Bengaluru?

Every request involves a realtime filtering of ads across geography, user affinity, device signals and advertiser constraints, etc. Within a few milliseconds, the system eliminates everything that doesn’t qualify and what remains is a smaller candidate pool.

2. The Experience

Nobody wants to watch the same brand ad again and again till they learn it by heart! This is why the ad server then must check for the Frequency caps (FCaps) across the combination of millions of users and thousands of ads via some distributed cache.

3. The Ad Pacing and Time Interval Capping

During a single IPL match where millions are watching it live, how do we stop the ad delivery when it hits its 1M impression target? How do we ensure we do not deliver all the 10M ad impressions within the first few days of a month-long campaign which was intended to create a prolonged reach on the masses?

This is where the math gets heavy. We use pacing algorithms (like PID or SHALE) to tackle these problems and ensure a uniform distribution of ad impressions throughout the campaign period. This also helps in preventing Over-delivery(OD) and Under-delivery(UD) of ads.

3. The Priority Decisioning

Out of 10 Ads that are qualifying for that one spot, which one brings out the most value or is the most important to be selected?

We resolve this through a Tiered Waterfall approach, where selection then proceeds tier by tier.

• Higher priority campaigns are evaluated first.
• Lower tiers are considered only if the ad pod is not filled completely.
• Any leftover duration is filled with in-house promos.

If the selected ad is a direct deal, we immediately return the VAST response. But if it’s a programmatic ad, then we depend on third party SSP demand where fill rate uncertainty (guaranteed vs open exchange) becomes a factor.

The Programmatic Challenge: Fill Rate

Once we’ve ranked our eligible ads, the final step is fulfillment. If the winner is a Direct Deal Ad, we serve it immediately. But if it’s a Programmatic Ad, we’re essentially asking an external auction house (SSP): “Hey, do you have an ad for this user right now?”

This is where the Fill Rate comes in. It’s the probability of actually getting an ad back. This becomes the make-or-break metric. Think of it as a reservation vs. a walk-in at a busy restaurant.

Programmatic Guaranteed (The Reservation): We’ve pre-negotiated this. The advertiser has committed to the volume of ads, so the Fill Rate is usually 80% to 100%.

Open Exchange (The Walk-in): We throw the request out to the highest bidder in the open market. Because it depends on real-time competition and prices, the Fill Rate can plummet to 1–2%.

Closing the Loop: Attribution

The journey doesn’t end when the ad finishes. A plethora of ad tracker events are fired every second for qualifications, selections, impressions, and clicks from the moment the request lands on the server to the point when the ad is shown to the users on the app’s UI.

Our big-data pipelines process millions of these events every second, converting raw telemetry into accurate delivery metrics and advertiser dashboards. But delivery metrics alone aren’t enough. Advertisers ultimately care about outcomes.

This is where attribution comes in.

Click-Through Attribution (CTA)

Click-Through Attribution measures conversions that happen after a user clicks on an ad and later completes an action such as installing an app, signing up, or making a purchase.

User clicks -> User converts -> Conversion is credited to the ad.

View-Through Attribution (VTA)

This is like a subconscious win. Many users don’t click an ad during a tense IPL match (because FOMO!), but they might buy that product or install that app the next day. VTA tracks users who saw the ad and later converted, proving that the ad worked even without a click.

VTA captures the impact of brand exposure which helps when we need a large-scale awareness campaigns for some new product of a big brand.

Conclusion

So the next time you see a 30-second ad on JioHotstar, remember: you aren’t just watching a video. You are witnessing the finale of a sub 100-millisecond symphony!

It’s a world where algorithms balance the scales of supply and demand, where SGAI stitches a personalized experience for millions concurrently, and where big-data pipelines turn billions of pings into meaningful business insights.

For the customer, it’s just a break in the game, but for us, it’s one big complex dance.

Do you want to participate in building systems that handle intricate sub millisecond decisions? Do check out open roles if you want to build for millions of customers, who will use features that you build!

Journey of an Ad Request: The Hidden Engineering was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

Generative AI offers a path to content creation at speeds and scales that traditional pipelines cannot match. One high-value use case: making existing IP characters-sports presenters, show hosts, brand ambassadors -deliver new scripts in their own likeness, without a reshoot, within IP & contract limitations.

The premise is simple: take a reference image, provide audio, and generate a video where the character speaks naturally. In practice, this is harder than it sounds. Base proprietary video models like Kling, Veo often struggle with lip-sync quality, identity drift, and motion consistency.

To address this, we built a production pipeline for character-consistent talking-head generation using LoRA (Low-Rank Adaptation), a lightweight fine-tuning method that adapts a base model to a specific character without retraining the entire network. This makes character adaptation practical to train, store, and deploy across multiple IP characters at scale.

Why LoRA?

LoRA works by freezing the original model weights and injecting small, trainable low-rank matrices into specific layers. Instead of updating a weight matrix W directly, LoRA decomposes the update into two smaller matrices: ΔW = A × B, where A and B have a rank r far smaller than the original dimensions. For rank 32 (our configuration), each adapted layer gains only ~0.1% additional parameters-producing a ~400MB adapter file vs. 41GB for the full model. Multiple character adapters can be swapped at inference without reloading the base model.

LoRA is well-established for image generation, but video introduces harder challenges: the adapter must preserve identity across dozens of frames (not just one), motion and structure are entangled across diffusion noise levels, and video LoRAs train on far fewer samples (15–20 clips vs. hundreds of images), making overfitting the dominant risk.

This blog covers how we fine-tuned two video architectures-Wan 2.1 and LTX-2-with LoRA, the data preparation that made it possible, and the results that determined which approach works at production scale.

Model Training

Preparing Training Data for Character LoRAs

The quality of a character LoRA depends on the quality of its training data, so our pipeline begins with a data preparation stage that converts raw footage into curated, LoRA-ready datasets. The system takes two inputs: source video footage and a clear reference image of the target IP character. As shown in Figure 1, the pipeline moves from scene preparation and character filtering to captioning, latent precomputation, and LoRA training.

Raw videos are first split into scenes using PySceneDetect, with very short clips removed to avoid noisy transitions and montage fragments. We then use InsightFace / RetinaFace to detect faces and extract embeddings, which are matched against the reference image using cosine similarity. This retrieval stage achieved about 85% recall in identifying clips containing the target character. Only clips where the target character appears consistently are retained, even in multi-speaker or edited footage.

The retained clips are classified by shot type to preserve viewpoint balance and framing diversity. This helps the LoRA generalize better instead of overfitting to a single dominant angle.

The most important step is captioning, where each curated clip is described using Qwen2.5-Omni/Gemini Flash and assigned a unique trigger word for the target IP character. As illustrated in Figure 1, these captioned clips are then precomputed into reusable video latents and text embeddings before entering LoRA training.

At inference time, the trained adapter is invoked through the same character trigger, making the workflow reusable across prompts and scalable across multiple IP characters.

Model Selection

We focused on Wan 2.1 and LTX-2 because they were the strongest open-weight candidates in our evaluation, offering image-to-video quality closest to proprietary models according to leaderboard ranking while also supporting character-centric generation and LoRA-based adaptation in a practical training pipeline.

These two models also gave us useful architectural contrast. Wan 2.1 was a strong initial candidate for character adaptation, while LTX-2 offered a more flexible and production-oriented inference stack. Comparing them helped us assess quality, controllability, and operational fit for scalable deployment.

Experiment 1: Wan 2.1

Our first LoRA experiment was on Wan 2.1 I2V 14B. Training data was manually curated clips from out IP character catalogue.

Single-Stage LoRA (Baseline): Single-stage LoRA trains one adapter across the full diffusion timestep range from 0 to 1000. This means the same adapter has to learn structure, motion, and fine detail together.

The limitation is that these aspects of generation are controlled by different timestep regions during denoising. As a result, the adapter learns a compromise rather than specializing effectively for any one of them.

Observations: Over-articulated lips, robotic motion, visible jumps between clips, and identity drift.

Three-Stage LoRA (Improved): Three-stage LoRA splits training into three adapters across different time-step ranges, allowing structure, motion, and fine detail to be learned separately, as shown in Table 1.

Three separate LoRA files are loaded at inference with the low-noise adapter boosted to 1.3x scale for sharper lip definition. We also tuned inference: audio guidance 5.0 → 3.5 (curbs over-articulation), motion frames 9 → 15 (smoother transitions), text guidance 1.0 → 0.8 (better identity), sample steps 40 → 50 (higher quality).

Observations: Mouth opening, clip transitions, and identity preservation all improved significantly. Though, lip movements remained unconvincing for production and motion consistency was weak. Wan 2.1 had reached its architectural ceiling.

Experiment 2: LTX-2

We then moved to LTX-2, a diffusion-based video generation model designed for efficient, high-quality image-to-video synthesis. Its architecture separates visual compression, text conditioning, and denoising into a cleaner training and inference stack, making it easier to adapt with LoRA for character-specific generation. In practice, this gave us a more flexible and production-friendly setup than Wan 2.1, with better support for stable identity preservation and higher-resolution outputs through two-stage upsampling from 960×544 to 1920×1152.

Unlike Wan 2.1, LTX-2 works well with a simpler single-stage LoRA setup. That reduces training complexity, simplifies checkpoint management, and makes it easier to operationalize across multiple IP characters. The pipeline keeps LoRA rank and optimization settings broadly aligned with Wan 2.1 for consistency, but the model itself requires less intervention to reach strong results.

Training Configuration (LTX-2):

Overfitting Behavior: As shown in Figure 2, character features begin to emerge around 1k–3k steps, recognition becomes consistent by 3k–6k, and identity generalizes more reliably to unseen scenes in the 6k–9k range. Beyond 9k steps, the model begins to memorize training poses, causing outputs to become stiff and overly specific.

Based on this behavior, checkpoints were saved every 3k steps and evaluated on unseen prompts, with the best production checkpoints typically selected from the 6k–9k step range.

Observations: In general, the fine-tuned LTX-2 outputs showed stronger identity retention, more natural lip-sync, and smoother motion than the Wan 2.1 variants. It was also more reliable in carrying the character into unseen scenes, making it the strongest model in our pipeline from a production standpoint.

Results

Model Comparison: Wan 2.1 vs. LTX-2 vs. Veo 3.1 vs. Kling 3.0

To contextualize the LoRA results, we compared outputs from both fine-tuned models against Veo 3.1 and Kling 3.0, which was used without LoRA since it does not support adapter injection. All evaluations were conducted on IP characters from our catalog across both seen training scenes and unseen contexts. The quantitative and qualitative comparison across models is summarized in Table 3.

As shown in Table 3, LTX-2 provided the best balance of controllability, visual fidelity, and operational simplicity. It preserved identity more reliably across unseen contexts, produced more natural lip-sync, and fit more cleanly into a reusable character generation workflow. Wan 2.1 remained a viable supported path, but it required more specialized tuning and delivered less consistent results. Veo 3.1 and Kling 3.0 demonstrated the strength of a large base model, but without LoRA support it was not a practical solution for reusable, character-specific generation across a catalog.

Conclusion

We built a LoRA fine-tuning pipeline that produces character-consistent talking head videos for JioHotstar IP characters at production quality. The core lessons: architecture matters more than tuning-Wan 2.1’s three-stage noise-level specialization improved results but could not overcome its architectural ceiling, while LTX-2 delivered larger gains with a simpler single-stage LoRA.

Data quality compounds-viewpoint symmetry and shot-type diversity in training data consistently improved generalization across characters. Character selection shapes everything downstream-direct-to-camera monologue with clean audio produces the most reliable training and evaluation signal.

In a follow-up post, we will cover our LLM-as-a-Judge evaluation framework and training data duration optimization experiments.

Do you want to contribute to the development of Indic language models? Do check out open roles if you want to build for millions of customers, who will use features that you build!

Consistent Generative Video: Building Video Models That Remember was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

As JioHotstar serves a vast, diverse audience, generative AI is crucial for fast, scalable content creation across languages and formats, capabilities that traditional methods cannot match. Content systems are rapidly shifting to generative-first architectures, synthesising video, audio, and narrative programmatically.

However, progress is uneven: visual generation is advancing rapidly, while audio generation lags, especially for Indic languages, which lack the mature datasets and investment seen in English speech.

Under realistic production workloads in cinematic contexts, existing speech systems struggle to preserve emotional expressiveness, maintain pronunciation stability, and deliver consistent outputs across repeated generations.

These failures persist even on widely adopted commercial platforms, making audio the primary bottleneck in deploying AI-driven content pipelines alongside increasingly capable video systems.

Why Indic Audio Breaks at Scale

Speech generation systems that appear stable in demos often degrade under production-scale workloads. For Indic languages, these failures surface earlier and more acutely due to four structural constraints:

• Lack of emotion-rich training data: Indic speech datasets are dominated by neutral narration. Unlike English, which benefits from large emotionally diverse corpora and mature conditioning techniques, Indic models struggle to produce emotionally aligned speech in character-driven settings.
• Insufficiency of prompt-based control: Prompting influences tone superficially but cannot enforce consistency or repeatability. Without explicit modeling of emotion, language-specific phonetics, and stability constraints, prompt-level approaches fail production reliability requirements.
• Linguistic complexity and code-switching: Hindi speech relies heavily on prosody, vowel duration, and rhythmic stress. Most systems lack language-aware mechanisms to resolve these variations, producing inconsistent or unnatural Hinglish output.

How was this Solved?

We adopt a modular methodology that treats expressive speech as a core modelling challenge. Our approach combines carefully selected open-source architectures with targeted adaptation strategies for Hindi and Hinglish, augmented by language-aware preprocessing, emotion-centred data curation, and evaluation frameworks designed for production-scale content pipelines.

Architecture Selection

We evaluated three dominant architectural classes.

Non-autoregressive TTS models (e.g., IndicF5) offer low latency and stable pronunciation, but their explicit factorisation of prosody into auxiliary predictors limits expressiveness emotional variation collapses toward the training mean, producing perceptually flat speech.

Latent diffusion models (e.g., FishAudio) operate in compressed acoustic latent space and can capture global structure, but their effectiveness depends on pretraining coverage; lacking Indic pretraining, FishAudio showed high inference variability and was excluded from further benchmarking.

LLM-backed generative architectures emerged as the strongest candidate. These systems frame speech synthesis as conditional token generation, with text and reference audio jointly guiding the production of speech tokens.

Crucially, by learning prosody and rhythm implicitly through sequence modelling rather than through explicit auxiliary predictors, they are better positioned to capture character-specific expressive patterns making them the ideal foundation for character-consistent, emotionally aligned Hindi and Hinglish audio generation.

As shown in Fig. 1, Chatterbox-class systems consume two primary inputs at inference time: the target text and a short reference audio clip. The reference audio is encoded by a Speech Tokenizer into discrete speech tokens and a speaker embedding, capturing voice timbre and coarse speaking characteristics.

These, combined with tokenized text, form the conditioning context for a large causal transformer (Text-to-Token, T3), which autoregressively predicts speech tokens. By conditioning jointly on text, speaker embeddings, reference speech tokens, and previously generated tokens, the T3 model captures long-range dependencies across linguistic content, pacing, and prosody allowing expressive patterns to emerge implicitly.

The predicted tokens are then decoded into continuous acoustic representations and converted to a waveform by a neural vocoder.

Key scientific advantages over alternatives:

(1) Implicit prosody modeling: rhythm, pacing, and emotional variation are learned through sequence modeling of speech tokens, not factored into explicit predictors that collapse under sparse expressive data.

(2) Strong conditioning pathways: text, style, and character embeddings are integrated early and jointly, preserving conditioning signals throughout generation and reducing drift across inference runs.

We conducted controlled experiments on Hindi scripts from our catalog evaluated model behavior across key dimensions. Metrics were computed over multiple inference runs to assess stability and repeatability rather than best-case performance. FishAudio was excluded due to the absence of Indic pre-training data.

Why Off-the-Shelf Inference Was Insufficient

Even with Chatterbox pre-trained on Hindi, zero-shot inference exposed a clear gap between voice timbre matching and expressive speaking style reproduction.

Generated speech consistently lacked character-specific prosody, rhythm, and emotional delivery producing perceptually monotonic outputs.

The root cause lies in the architecture’s conditioning design: reference audio provides coarse speaker and acoustic cues, while the T3 model learns an averaged prosody distribution across speakers during large-scale pre-training. This favours generalisation and stability but suppresses idiosyncratic speaking styles.

These effects are amplified for Hindi and Hinglish, where meaning and intent rely heavily on timing, stress, and intonation. In practice, zero-shot inference produced weak emotional alignment, pronunciation inconsistencies for mixed-language text, and high variability across runs establishing the need for targeted fine-tuning to explicitly encode expressive and language-specific priors.

Data Preparation: Speech Enhancement and Emotion Annotation

Noisy source audio is cleaned through a sequential pipeline: HTDemucs (speech isolation) → VoiceFixer (reverb/artifact removal) → DeepFilter (enhancement), or resemble-enhance as a single-pass alternative then gate-filtered via DNSMOS and loudness normalized.

For emotion labeling, audio segments are diarised, transcript-aligned, and classified by Qwen and Gemini against a predefined emotion taxonomy. Gemini was selected as the final annotator after outperforming Qwen on expressive classes (Happy/Energetic/Expressive) on a held-out validation set, directly translating to richer prosodic diversity in the training data.

Drawing on in-house archives and the enhancement pipeline, we curated a cross-lingual (Hindi + English) dataset spanning hundreds of speakers, deliberately designed to expose the model to code-switching and diverse prosodic patterns:

Fine-Tuning Strategy

Fine-tuning was applied selectively to the Text-to-Token (T3) generative model, while keeping the speech tokenizer, decoder, and vocoder frozen. This design choice is scientifically deliberate: the T3 model needs to learn how the character speaks — pace, emphasis, and emotional cadence — without altering who the character sounds like.

However, since the voice identity is encoded in the frozen tokenizer and decoder, approximately one hour of expressive speech is sufficient to shift the T3’s conditional token distribution toward character-specific prosodic patterns, without triggering identity drift.

Training used the AdamW optimizer with a conservative learning rate to preserve pre-trained weights, a cosine learning rate schedule with a short warm-up for training stability, and gradient clipping to prevent unstable updates.

The training objective focused solely on predicting audio tokens from text inputs. Early stopping was implemented against validation loss to halt training before overfitting.

As shown in Fig.3, after 3–4 epochs, training loss continued declining toward zero while validation loss began rising, a clear overfitting signal, corroborated by a rise in gradient norm. The early stopping mechanism successfully arrested training before severe overfitting, saving the best-performing checkpoint and ensuring the final model generalizes well to unseen data.

Results and Observations

Fine-tuning outcomes were evaluated using quantitative metrics and systematic qualitative audits. Quantitative metrics captured pronunciation accuracy, emotional alignment, long-form consistency, and speaker similarity across multiple inference runs. Qualitative audits identified failure modes not fully reflected by aggregate metrics, particularly for Hinglish and domain-specific content.

The most scientifically significant result is the +27% gain in Speaker Similarity despite the frozen decoder and vocoder. This confirms that T3 fine-tuning successfully learned character-specific prosodic priors without destabilizing voice identity — validating the selective fine-tuning hypothesis.

Emotion Alignment improved by 19% and Pronunciation Accuracy by 23%, with Mean Opinion Score (MOS) rising from 3.3 to 4.2 (approaching the 11labs benchmark of 4.3). The following issues were observed during our extensive testing, which were mitigated as described:

Issue-1: Gibberish Artifacts in Short Phrase Generation

Observation: For utterances under five seconds, the fine-tuned model occasionally generated non-linguistic or gibberish speech toward the end of the audio. This was reproducible for a subset of short phrases and was not observed in the pre-trained base model.

Interpretation: Short sequences provide fewer autoregressive steps, increasing sensitivity to distribution shifts introduced during fine-tuning. The model overshoots termination boundaries or produces unstable token transitions near sequence end, a failure mode specific to the fine-tuned distribution.

Mitigation: Root cause analysis identified the failure at the generation boundary level. Targeted fixes were applied during data curation (ensuring short-phrase samples had clean terminal boundaries) and at the generation stage (constraining end-of-sequence token behavior). After applying these mitigations, gibberish artefacts were no longer observed in short-phrase inference.

Issue-2: Pronunciation Errors in Domain-Specific and Hinglish Vocabulary

Observation: Despite overall natural speech quality, domain-specific terms and code-switched vocabulary were mispronounced even when the base model had been pre-trained on Hindi and English appeared sporadically in training audio.

Interpretation: The root cause is ambiguity in text-to-phoneme alignment under code-switching. The model had largely seen fully Hindi or fully English contexts during training, whereas inference-time inputs (e.g., cricket commentary) contained tightly interleaved Hindi and English tokens, leading to unstable pronunciation for certain words.

Mitigation: A pre-inference transcript normalisation strategy using a lightweight lexicon mapping replaced consistently mispronounced words with acoustically preferred alternatives before inference:

This normalization produced clear improvements in pronunciation consistency without affecting overall fluency.

Conclusion

By combining LLM-assisted emotion annotation with targeted T3 fine-tuning on enhanced expressive data, we achieved ~20% improvement in Emotion Alignment, ~27% in Speaker Similarity, and MOS rising from 3.3 to 4.2, approaching commercial benchmarks while maintaining stability across repeated inference runs.

This establishes a reliable foundation for character-driven Hindi and Hinglish audio generation at scale. In the next blog, we extend this work to regional Indic languages, tackling even steeper challenges in data scarcity and linguistic diversity.

Do you want to contribute to the development of Indic language models? Do check out open roles if you want to build for millions of customers, who will use features that you build!

Beyond Imitation: Teaching AI to Speak Like a Character in Hindi was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the world of live sports streaming, there is no “undo” button. Live events bring out emotions in viewers and any minor buffering or failure can cause disappointment in our customers, especially if it happens in crunch moments of the live event.

Our Live video validation has gradually moved from being fully manual to islands of automation. We had the tools to check manifests and APIs, but we lacked a cohesive brain to run them. Here we talk about how we moved from manual intervention to an automated, cohesive, real-device validation ecosystem.

Manual “QA” Failed at Scale

Before we built our current pipeline, our validation process was a series of stretch efforts that simply couldn’t keep up with the scale of modern streaming. Manual QA can work for a single video and small number of matches, but when we look at streaming a live event in 12 languages, 5 camera angles, 74 matches over 2 months, automation is the only way to reliably run a QA process. At peak, JioHotstar (JHS) has seen close to 55 concurrent live streams, each with their unique stream variations.

• The Orchestration Gap: We started automated stream checks early on, but stream checks were started manually. If an engineer forgot to trigger a script for a specific regional feed, that feed went unmonitored.
• The “Orphaned” Test Problem: We would often find validation scripts running for hours after a match ended because there was no central logic to tell them to stop. Automation is wonderful but cloud costs are real too.
• The Silent Failure: A manifest might look perfect on a server (200 OK), but the actual player on a Smart TV might be stuck on a buffering wheel due a media sequence mismatch or due to ads related discontinuity tags or a DRM mismatch.

We didn’t just need a bigger QA team; we needed an orchestrator for our automation, a deeper video validation mechanism and real devices.

Phase 1: The Steering (Video Validation Orchestrator)

It’s not enough for a manual or automated test to certify that a live event is working by testing for a couple of seconds. The automation needs to follow the live event’s lifecycle. The same video stream needs to be continuously validated throughout the event.

Apart from languages and camera angles we discussed earlier, JioHotstar’s video encoding systems also generate many video variants based on device (e.g. TV and mobile get different videos), CDN (what if a video is working on one part and failed on another), advanced video and audio experience like Dolby Vision, Dolby 5.1, HDR10, 4K etc. These video streams can fail independently and hence must be tested independently too.

Instead of manual triggers, the Orchestrator treats our Content Metadata System (CMS) as the absolute source of truth and also validates all the video variants.

How the “Steering” Works

The Orchestrator operates on a Discovery-and-Dispatch model:

• State Discovery: It polls the metadata provider to maintain a real-time map of every scheduled match, regional feed, and alternative camera angle (like VR360). We are already working on consuming content publish/un publish events to make this process even better.
• Automated Lifecycle Management: It automatically “spins up” validation workers before “match-start” and, more importantly, “spins them down” once the content is unpublished to prevent resource leakage.
• Parallel Execution: For every live event, the Orchestrator dispatches two distinct, concurrent validation strategies; server-side (Video Validator) and real-device (Play Portal)

Phase 2: Dual-Layer Validation

To ensure total coverage, we split our focus between the “plumbing” of the stream and the actual “eyes” of the user.

Strategy A: Video Validator (Deep-Stream Server Side Inspection)

This is our Server-Side watchdog. It doesn’t look at the UI; it looks at the raw data packets and manifests to identify backend failures before they reach the player.

• Manifest Integrity: Checks for discontinuities, missing segments, and media sequence drift.
• Ad-Tech Health: Validates SSAI (Server-Side Ad Insertion) markers and ensures CUE-IN/CUE-OUT timings are structurally sound.
• CDN Load-Balancing: Verifies that all CDN edges (Akamai, Cloud front, JIO etc.) are serving identical, healthy chunks

Video Validator service is the first one to catch any issues with CDNs, encoders, and SSAI manifest stitchers. Any errors caught here are shown on the dashboard (covered later) and relevant alerts are raised for our teams.

Our players and CDNs are tuned to retry and recover from intermittent failures (like the ones in the diagram below), but we still need to know and understand the root cause when these failures happen.

Strategy B: Play Portal (Real-Device Synthetic User Simulation)

While the Validator checks the plumbing, Play Portal acts as the eyes. It launches automated playback sessions on real devices (Android Mobile, iPhones, TVs, Web Browsers) to ensure content is actually playable.

• Playback Handshake: Does the player initialize and reach “playing” state within the SLA?
• UI Overlay Checks: All expected audio languages are listed in the player and is playable
• Buffering: Any rebuffering or other esoteric scenarios where video frames get stuck are detected here.

The Pulse of the Stadium on a Dashboard

When we monitor at this scale, the Grafana dashboards stop looking like lines of code and start looking like a heartbeat. We can actually track the stadium’s atmosphere through our validation logs:

1. The “Toss” Mini-Spike: 30 minutes before the first ball, our Orchestrator triggers a surge of validation workers. The dashboard “wakes up” as we see the first successful playback handshakes across 12 regional languages.
2. The First Ball Explosion: As the first ball is bowled, request volume explodes. Our Kubernetes HPA (Horizontal Pod Autoscaler) kicks in, spinning up more Video Validator pods to handle the massive concurrency.
3. The Middle-Overs Lull: During the middle overs, we look for long-running stability — monitoring for “Silent Drifts” or memory leaks in our real-device probes that only appear after 60 minutes of playback.
4. The Second Innings “Re-Sync”: As the chase begins and a new wave of viewers joins, we ensure mid-roll ad markers are firing correctly, catching “Empty VAST” errors before they impact revenue.
5. The Pre-Death Overs Ramp: In the final 5 overs, we are often using the last ounce of CDN bandwidth available in the country. Validation here hits the same intensity that the match is experiencing.
6. The Show Continues(Match End): The match may end but our viewers are still here to find out who got the player of the match trophy, our validators continue to hum after the match.
7. The Sudden Drop (Unpublish), Traffic drops, the Orchestrator detects the “Unpublished” state, and begins the graceful scale-down of our workers.

Engineering Takeaway: Watching this cycle isn’t just about “uptime.” It’s about Elasticity. Our system needs to be as dynamic as the game itself — expanding its “lungs” during the death overs and breathing out when the stadium goes dark.

Real-World Impact: Detection in Action

Our automated validations start much before toss and this has saved us from major regional outages by catching issues before we published matches for the audience:

• Manifest 404s: Caught ingest gaps where regional feeds hadn’t reached the CDN edge, allowing for a fix before the first viewer clicked play. We spin up 100s of encoders for every match when one fails, it’s detected using multiple mechanisms including Video Validator.
• SSAI / SGAI Failures: Identified invalid cues during high-traffic windows, preventing “black screen” ad breaks.
• Media Sequence Drift: Detected sync issues across different bitrates that would have caused infinite buffering for users. This detection led to our player teams strengthening media sequence handling.

The Takeaway: Building Confidence, Not Just Code

Moving to an automated stream validation model wasn’t just a technical upgrade; it was the only way to operate to ensure that we caught problems before hearing about it from social media.

By combining Server-Side Validation with Client-Side Automation, we created a safety net that covers the entire “journey of a pixel.”

We no longer hope the stream is working; we know it is, from the moment it leaves the stadium to the moment it renders in the fan’s living room.

Are you interested in ensuring you build video quality agents? Do check out open roles if you want to build for millions of customers, who will use features that you build!

Beyond Manual Checks: How We Built a 24/7 “Watchdog” for Live Sports Streaming was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

AI video generation has rapidly moved from a mere novelty to a practical tool, allowing users to easily create brief video clips from text prompts. While generating a single, impressive video is straightforward, the true difficulty lies in scaling this process to produce consistent, repeatable, and production-grade content.

This post contends that the inability to generate high-quality, production-grade AI content at scale is due to systemic failures in the current approach, not limitations in the core AI models themselves.

The Requirements of Production Quality

Achieving production-grade video demands a system capable of handling several stringent requirements:

• Volume: Generating a high quantity of short videos, not just isolated clips
• Complexity: Supporting multiple distinct scenes within a single video
• Consistency: Reliably reusing and maintaining characters, locations, and objects across all content
• Collaboration: Facilitating team workflows rather than being restricted to individual use
• Predictability: Ensuring stable cost structures and reliable delivery timelines
• Compliance: Adhering to brand guidelines, safety regulations, and intellectual property rights

It is against these rigorous standards that the inherent flaws of existing AI video systems become obvious. The core challenges preventing production-scale AI content are detailed below:

#1: Prompt-Centric Systems Cannot Scale

While prompting is an excellent interface for exploration, it proves fragile for production work. The prompt-based approach suffers because:

• Consistency is solely dependent on the user’s conceptualisation and is not encoded within the system
• Output quality fluctuates based on the specific user who wrote the prompt
• Re-running an identical prompt offers no guarantee of the same outcome
• Operational knowledge remains uncodified, failing to be formally integrated into the system

As teams and projects expand, prompts inevitably become:

• Unnecessarily long
• Managed through error-prone copy-and-paste practices
• Subject to small, undocumented alterations
• Difficult to systematically analyze and maintain

At scale, this leads to prompt drift, where video outputs increasingly diverge from the original creative intent, despite the seemingly consistent prompts. Prompting is fundamentally an improvisational tool, not a structured system contract.

#2: Continuity Failures: Drifting Characters, Mutating Scenes, Disappearing Objects

In professional video, continuity is critical: a character’s appearance and voice must be uniform, locations must not subtly change without authorization, and props must persist as intended.

Yet, in most AI video generation:

• Characters are fully regenerated for every instance
• Locations are merely implied, lacking systematic enforcement mechanisms
• Objects are treated as incidental details instead of continuously tracked entities

This results in a gradual breakdown of realism, manifesting as:

• Shifts in facial structure
• Subtle changes in wardrobe
• Erratic transformation of background environments
• Minor but noticeable deviations in voice from the baseline

Human viewers instantly perceive these inconsistencies, even if computational models score the outputs favorably. Consistency is not a minor visual detail; it is a fundamental cognitive contract with the audience.

#3: Quality Control Is Manual and Subjective

The quality assurance for most AI video generation relies heavily on:

• Human visual inspection
• Subjective assessment (“This looks acceptable”)
• Creative intuition, lacking objective metrics

This method works only at low volumes. At production scale, it immediately introduces problems such as:

• Inconsistent judgment across human reviewers
• Unpredictable drift in quality standards
• The inadvertent release of suboptimal content
• The incorrect rejection of high-quality outputs

If output quality cannot be measured objectively, it cannot be reliably enforced. Production systems must integrate objective evaluation as a primary, foundational capability, not a secondary add-on.

#4: Regenerations Lead to Unpredictable Cost

When an error or undesirable outcome occurs in current AI video generation, the standard solution is simply:

“Try generating it again.”

AI video generation is inherently resource-intensive, particularly in terms of GPU utilization. While cost is a minor concern at low volumes, it becomes a critical issue at production scale. The common cost challenges include:

• “Retry storms”: Cycles of repeated, unsuccessful retries
• Unconstrained and potentially endless regeneration attempts
• Over-reliance on expensive, high-specification models
• A lack of correlation between production expenditure and final output quality

A Necessary Shift in Mental Model

To overcome these systemic failures, effective production systems must prioritize:

• The system architecture first
• Consistency and continuity first
• Evaluation and quality control first

To operate effectively at scale, AI video systems require a fundamental conceptual shift:

This transformation necessitates a rigorous architectural approach, not merely computational workarounds.

Subsequent articles in this series will explore the crucial paradigm shifts required to establish AI video as a viable asset for professional teams.

Are you interested in AI Video Generation? Do check out open roles if you want to build for millions of customers, who will use features that you build!

AI Video Generation at Scale — Prompts Don’t Scale was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

If you’ve ever streamed a high-stakes match on JioHotstar, the experience probably felt like a simple “tap and play”. It’s a seamless transition from the app icon to the stadium; a flick of a finger and you’re right there, cheering for your players to hit another six. Beneath that simple play button lies one of the most aggressive engineering challenges in the world.

At Hotstar, we navigate a “Tsunami” of traffic across one of the most complex network landscapes in the world. During events like the IPL or a high-stakes ODI match, we don’t just manage millions of users - we manage millions of unique network realities. If you thought just placing a Content Delivery Network (CDN) makes the magic happen — read on.

The Illusion of the Monolithic Network

Traditionally, CDN traffic management happens at the network provider or state level. For years, this was enough. But at our scale, “good enough” is the enemy of the audacious. A live event requires upwards of 60–80 Tbps of network bandwidth for streaming. To put things into perspective, it’s like instantly grabbing the entire 4K movie collection from JioHotstar (hundreds of blockbusters). Now imagine finishing all those downloads in just one second ….and then doing it again the next second and the next. That is the relentless pace of the tidal wave we face during a major event.

However, India’s network isn’t a monolith. It is a mosaic of fiber, 4G, 5G and fluctuating bandwidth that changes from one street to the next. A user on a 5G connection in South Delhi faces a vastly different network topology than a user on a local ISP in rural Rajasthan.

When you are only as good as the video you deliver, you realize that macro-level routing hits a ceiling. To provide the best Quality of Service (QoS), we needed to treat every state, every city, and eventually every cohort, as a unique routing decision.

The Solution: The QoS Routing Manager

To solve the “Last Mile” problem, we built a real-time observability and orchestration engine: the QoS Routing Manager.

The mission of this service is simple but massive: Observe crucial video metrics at a granular level and adjust traffic weights dynamically to ensure every user is mapped to the best possible CDN for their specific location.

The Granular Cohort

Instead of routing by “Maharashtra” or “Jio,” we segment users into Cohorts. A cohort is a group of people who share a common characteristic over a given period. Currently for us cohort is a specific combination of geographical, network and business categorisation i.e ASN-Country-State-City-UserType.

This allows us to detect if a specific provider is facing issues in a specific city, even if their national health looks perfect. By slicing the data this way, we can bypass localized congestion before it affects the broader user base

The Power of The Scoring Logic

The QoS Routing Manager pulls real-time performance data from our sophisticated in-house OLAP analytical processing beast - called ARGUS, which serves as our eyes and ears when it comes to video performance across the platform. It collects heartbeat data from the devices, processes them and provides the telemetry to our service.

We map every metric - Playback Failure Rate (PFR), Rebuffering, and RTT Latency - into normalized scores using a Linear Piecewise Scoring function. This allows us to define “Severity Buckets” (Ideal, Baseline, Sev3, Sev2, Sev1) based on direct business impact.

To determine which CDN “wins” for a specific cohort, we calculate a Cumulative Health Score based on a specific precedence order:

Cumulative Score= ( X * PFR{score} ) + ( Y * Rebuffer{score} ) + ( Z * RTT{score} )

We weigh PFR most heavily to ensure that “reachability” is the absolute priority, followed closely by the “fluidity” of the stream (Rebuffering) and the “snappiness” of the connection (RTT).

Filtering the Noise: The Power of EWMA

Raw network telemetry is inherently noisy. A momentary 4G tower hand-off in Jammu or transient packet loss in Bangalore can look like a critical failure in a 10-second window.

If our routing engine reacted to every micro-spike, we would introduce dangerous volatility, creating a “jittery” experience where users are constantly bounced between CDNs. We needed a way to smooth out the true performance trend from the momentary noise.

To achieve this, we don’t pass raw metrics directly into our scoring engine. Instead, we pass all incoming telemetry through an Exponentially Weighted Moving Average (EWMA) filter.

The formula we use is:

EWMA{t} = α * x_t + ( 1 — α ) * EWMA{t-1}

where EWMA{t} is the new smoothed value, x_t is the raw input score, and EWMA{t-1} is the previous smoothed history.

We tune our smoothing factor α to approximately 0.6. In practical terms, this means our engine ensures recent metrics have a stronger influence on the final score while keeping the last 5 scores to have significance

Only once the metrics are smoothed by EWMA do we move to the routing.

Two-Phase Capacity Steering

A high score isn’t the only requirement for routing. We must balance “Customer Joy” with “Infrastructure Dynamics”. For a media streaming entity, the biggest tradeoff is quality and available network bandwidth.

It’s like a bridge where every viewer wants to drive a wide luxury bus (High Quality), but the physical lanes (Bandwidth) are finite; during a peak surge, there simply isn’t enough pavement to let everyone drive a bus at once without the bridge failing, so you have to balance the vehicle size just to keep everyone moving.

Our engine keeps track of the used bandwidth on the CDNs and employs two distinct strategies:

• Pre-Threshold Guardrail (< X% Utilization): We preemptively throttle CDNs that are on track to hit their capacity limits too early, even if they are performing well.
• Uniform Exhaustion (>= X% Utilization): During a surge, we shift logic to ensure all CDNs exhaust their capacity at the same time, squeezing every possible megabit out of our infrastructure.

Safety Gates: Resilience Over Risk

In a system of this scale, “no update” is better than a “bad update.” We built a defense-in-depth approach covering both data integrity and routing logic.

1. ASN Constraints (Hard Binding): Physics and contracts still matter. Some CDNs only have presence on specific networks. Before any scoring happens, the system applies hard constraints to ensure we never route a cohort to a CDN that physically cannot serve it.
2. Volatility Dampening (Max Deviation): To prevent wild swings in traffic that could de-stabilize the network, we cap the maximum percentage change allowed in a single iteration (e.g., a CDN cannot gain or lose more than y% share in one minute).
3. The “Warm-Up” Floor (Minimum Weight): We never let a functional CDN drop to 0% traffic. We enforce a configurable floor (typically 5%). This keeps CDN caches warm and DNS paths active, ensuring that if we need to fail-back to them instantly during a crisis, they are ready to take the load immediately.

The Audacious Impact

The results of moving to granular, cohort-based management has been significant. During a recent T20 Match, we ran a A/B rollout of the QoS Routing Manager with only ASN-State cohorts. For the treatment group:

• Playback Failure Rate (PFR) improved by a staggering 11%.
• Rebuffering and RTT Latency both saw a 2% improvement.

In our world of 50M+ concurrent users, an 11% improvement in PFR represents millions of users who stayed connected to the game instead of seeing a loading spinner. This is how we ensure that whether you are in a high-rise in Chennai or a village in Sikkim, you enjoy each and every six with best in class quality.

The work continues to build on the millions of datapoints that stream in that allow us to steer all our customer sessions to a stable viewing experience!

Are you interested in solving high-concurrency challenges at the edge? Do check out open roles if you want to build for millions of customers, who will use features that you build!

Orchestrating JioHotstar Traffic: The Difference Between a Loading Spinner and a Winning Six was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

Client Side Ad Insertion(CSAI) levels up using multi-period DASH in the Android ecosystem. This is our journey to upgrade ExoPlayer to leverage multi-period DASH.

Introduction

Multi-period DASH is a variant of DASH format that offers significant advantages, such as the ability to insert dynamic content like disclaimers, dub cards, and ad breaks without re-encoding the entire video. This flexibility is crucial for delivering personalized and localized content to millions of users across diverse regions.

While the DASH standard natively supports multi-period manifests, the Android ExoPlayer ecosystem (specifically the AdsMediaSource component) was designed with a single-period assumption. This missing piece in the puzzle meant we couldn't support Client-Side Ad Insertion (CSAI) on these modern streams out of the box.

This post details our engineering journey: identifying the constraints, evaluating architectural alternatives, and ultimately redesigning ExoPlayer’s Ad handling to support multi-period content seamlessly.

Context

The Evolution of DASH Content

JioHotstar relied on single-period DASH for entertainment content, where the entire stream including ads was encapsulated within a single period. This approach limited flexibility: inserting disclaimers, dub cards, and to some extent, dynamic ad breaks required re-encoding the entire video, which was both time-consuming and resource-intensive.

Consider a common localization scenario: the same title is launched across multiple regions with different primary languages. Audio dubs may be recorded by different artists per market, so dub-card credits vary by region; likewise, legal and regulatory requirements differ, driving region-specific disclaimers.

Under a single-period workflow, each of these variations would necessitate a distinct transcode of the full asset. If we target ~20 regions, that implies ~20 full transcodes, leading to long time-to-market, high cost, and significant operational complexity — super unscalable.

Multi-period DASH addresses these challenges by dividing content into multiple periods, each representing a distinct segment (e.g., disclaimer, main content, ad break). This modularity enables seamless insertion of additional content without re-encoding, significantly reducing operational overhead. In practice, we can attach region-specific disclaimers as a lightweight period and swap dub-card credits per locale without touching the main essence, keeping a single mezzanine and avoiding redundant full transcodes.

The Challenge of Ad Monetization

While multi-period DASH brought flexibility, it also introduced a critical challenge to our Ad insertion mechanism for Video on Demand(VOD) content. ExoPlayer, our Android media player, lacked native support for client-side ad insertion in multi-period DASH streams.

Root Cause

ExoPlayer’s code restricted ad playback to single-period content, for Ads supported users, Exoplayer uses AdsMediaSource for Ads playback and tracking. The AdsMediaSource does not allow Multi period contents with Ads. It throws IllegalArgumentException if the content has more than one period.

The crash happens as soon as AdsMediaSource object is created, irrespective of whether there are actual ads inserted in the stream or not.

Digging into the library code, we found explicit assertions guarding against complexity:

// Logic inside AdsMediaSource / SinglePeriodAdTimeline 
Assertions.checkState(periodCount == 1);

An immediate thought here would be to upgrade to latest AndroidX Media3 package, unfortunately, it had the same issue.

Beyond just this assertion, the architecture of AdsLoader and AdPlaybackState was built around a "Shared State" model. In a multi-period timeline (e.g., Period 0: Logo -> Period 1: Movie), Exoplayer applied the same AdPlaybackState to every period.

Because of this shared state we couldn’t just remove the assertion and move on with life, there were problems after that like a Preroll scheduled at 0s would try to play at the start of every period (Logo start, Movie start etc.), or Preroll will not play at ingress points like Continue watching from app.

Key Findings

ExoPlayer’s native multi-period DASH support has no client-side ad insertion. assert(periodCount == 1) assertions crash playback outright on multi-period streams (related discussion).

Four problems followed from that root constraint:

AdPlaybackState is designed for single-period timelines — applied to multi-period content, ads repeat across periods.

Cue-point alignment breaks at period boundaries — ad breaks trigger early, late, or not at all.

Preroll handling requires explicit edge-case logic: play once on cold start, suppress on re-entry from Continue Watching and equivalent ingress points.

Backward compatibility forced a split serving strategy — single-period DASH for older clients, multi-period for newer ones.

Extending AdsMediaSource to handle this required architectural changes to ExoPlayer's ad handling layer. Rollout was phased, with playback failure rate, buffer times, and ad impressions as the primary watch metrics.

Insights

These findings underscored the need for a flexible, scalable approach to ad insertion in multi-period DASH content. Addressing ExoPlayer’s limitations and rethinking AdPlaybackState and cue-point handling laid the groundwork for a solution balancing technical feasibility and user experience.

Methodology

To address the challenges of enabling ads on multi-period DASH content in ExoPlayer, the team adopted a systematic and iterative approach:

• Audited ExoPlayer’s DASH manifest handling and ad timeline management to locate all single-period assumptions.
• Prototyped assertion removal on multi-period manifests; used observed failures to surface edge cases early.
• Built custom AdPlaybackState logic to split the main state into period-specific instances, scoping each ad to its designated period.
• Implemented cue-point realignment relative to each period’s start time.
• Replaced SinglePeriodAdTimeline with a custom MultiPeriodAdTimeline.
• Gated multi-period DASH behind a version check; legacy clients continue on single-period.
• Tested preroll/midroll, cross-period seeking, and platform variants (Android, Android TV, Fire TV).
• Phased rollout starting with select content; monitored failure rates, buffer times, and ad impressions before expanding.

The Solution: A Custom MultiPeriodAdTimeline

Two quick options were ruled out early — removing the assertion and upgrading to Media3 (covered in root cause).

Three turnkey approaches were evaluated:

• Separate players for ads and content
• Client-side playlist stitching of single-period content and ad clips
• ClippingMediaSource + ConcatenatingMediaSource to simulate a single timeline

All three introduce buffering at player or content switches, lose AdsMediaSource capabilities (timeline management, seek handling), and scatter implementation complexity across multiple teams.

Solution : MultiPeriodAdTimeline

The chosen path: replace SinglePeriodAdTimeline with a custom MultiPeriodAdTimeline.

First attempt — apply AdPlaybackState only to the content period; treat disclaimer and credits as ad-free.

Simple to implement. Two problems:

• Pre-roll plays after the disclaimer, not at stream start
• No ads in the credits period — breaks standard behavior where the last cue-point fires on a direct seek to end

Second attempt — create dedicated AdPlaybackState per period (logo, content, dub card), each including a pre-roll to handle Continue Watching entry points.

Works, but hardcodes period sequence and count on the client. Any structural change to the stream breaks compatibility. Streaming team loses the freedom to modify period structure independently.

Breaking the Monolith

We took a step back and tried to generalize the solution. The biggest challenge was the “Shared State.” We handled all periods equally. We still have single main AdPlaybackState with cue-points as if we had single period content.

Internally its duplicated and transformed specially for each period. Any modifications happen on the main AdPlaybackState and are mirrored internally when it is updated.

We decided to keep indexing of ad breaks in AdPlaybackStates for each period the same. In each period will be all the cue-points in the same indices, but some will be ignored (skipped). Each period will have this transformations for each cue-point.

• Subtract start position of each period — times are relative to period start — some cue-points may end up being negative, but that is fine for us.
• Mark cue-points after the period end as skipped. These will be played in the following periods

That is it. A lot of investigation and iterations crystalized in few simple rules. After the transformation, the cue-point positions were as intended and all ad breaks are triggered even across period boundaries. From user experience, there is no difference between multi period and single period content. In the end single period is only a special case of multi period content now.

The Indexing Problem

ExoPlayer identifies ad groups by index. If we simply removed “non-relevant” ad groups from a period’s state, the indices would shift, causing the player to play the wrong ad or crash.

The Fix: We kept the Ad Group count constant across all periods.

• If Ad Group #1 belongs to Period 1, but we are currently configuring Period 0, we mark Ad Group #1 as SKIPPED in Period 0's state using AdPlaybackState.withSkippedAdGroup().
• This ensures adGroupIndex 1 always refers to the same logical ad break, regardless of which period is currently active.

Handling Continuity

We had to ensure that seeking across period boundaries didn’t re-trigger ads. Using the global bookmarking map, if a user watches an ad in Period 1 and seeks back to Period 0, the player knows that logic “Ad Break #X” is already played.

Backward Compatibility

We served multi-period DASH only to newer app versions; legacy versions continued with single-period DASH.

Summary

The implementation of ads on multi-period DASH content in ExoPlayer yielded the following outcomes:

Technical Achievements

• Seamless Ad Playback: Ads are now played at the correct times, without repetition, even across period boundaries.
• Accurate Cue-point Handling: Dynamic adjustment of cue points ensures ads trigger precisely as intended.
• Robust Backward Compatibility: Users on older app versions experience no disruption, as they continue to receive single-period DASH.
• Performance Metrics: Key metrics such as start lag, buffering, ad impressions, and playback failure rates remained within acceptable thresholds throughout the rollout.

User Experience

• Dynamic Content Delivery: The platform can now insert disclaimers, dub cards, and localized content dynamically, enhancing personalization.
• Uninterrupted Viewing: Users experience smooth transitions between content and ads, with no playback disruptions.

Business Impact

• Uninterrupted Monetization: No impact on monetization as we modernized our media stack.
• Operational Efficiency: Reduced need for re-encoding and streamlined content pipelines have lowered operational overhead.
• Cost Savings: Re-encoding a large media library like JioHotstar’s would have been cost-prohibitive. With this solution, no separate encoding is needed for existing content.

Conclusion

“Simple” features like adding a 5-second logo or disclaimer often hide iceberg-sized engineering challenges. By diving deep into the internals of ExoPlayer and rethinking how AdPlaybackStates are managed, we turned a hard constraint into a flexible capability.

If you are facing a similar issue, we have proposed these changes to be merged in upstream on Media3 Github repo here.

This project reinforced a key lesson for us: sometimes the best way to move forward isn’t to work around the platform (multi-player), but to improve the platform itself!

Want to dig into player internals and contribute back to projects like ExoPlayer? Do check out open roles if you want to build for millions of customers, who will use features that you build!

Monetisation : Multi-period DASH x ExoPlayer was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

We discuss building strong eval frameworks as part of our Generative AI studio to ensure that generated video maintains a high quality bar.

Introduction

As JioHotstar scales to serve one of the world’s largest streaming audiences, generative AI(GenAI) offers a powerful opportunity to accelerate content creation, adapt stories across languages and regions, and unlock creative workflows that traditional pipelines cannot match.

Customers evaluate AI-generated video with the same standards they apply to premium productions; any drift in character identity, structural deformity, abrupt scene shift, or unsafe visual undermines realism instantly.

Generative models, being probabilistic, introduce such inconsistencies naturally, and at our scale even rare defects accumulate into meaningful quality gaps. Ensuring stable characters, coherent locations, and safe content therefore becomes a scientific challenge central to customer acceptance.

To address this, we built a Validation Framework that operates as a first-class component of the generation pipeline. This closed-loop system enforces quality, safety, and consistency at the same cadence as creation, enabling generative video to meet production-grade expectations on the JioHotstar platform.

System Overview: The Validation Layer

The video generation process starts with editorial scripts, from which the system extracts characters, locations, accessories, and context. These entities drive keyframe generation, which expands into video clips and assembles into the final video.

The Validation Framework (Fig. 1) spans all stages of this process and operates synchronously within the workflow. It evaluates intermediate outputs, flags issues early, and triggers targeted regeneration or parameter adjustments to maintain quality. Brand detection and safety checks act as hard gates, while other modules guide regeneration to preserve consistency and visual integrity.

Each module governs a specific quality dimension, character consistency, deformity, scene continuity, brand safety, content appropriateness, or story coherence and emits go/no-go signals that control progression. The system logs all validation outcomes and samples them for periodic human review to support ongoing calibration.

Together, these modules form the framework’s control surface, defining the quality of generative video. Most components operate at production readiness, while long-range story and concept continuity remains an active area of experimentation as we continue refining metrics and validation strategies.

Character Consistency

Character consistency ensures that a character’s visual identity remains stable across all frames and scenes. Generative models, being stochastic, can drift subtly in facial features, proportions. These deviations break temporal realism and make the sequence unusable for production.

To quantify consistency, we represent each character through frame-level embeddings and compare them against a reference “hero character” image approved by the creative team. This hero image serves as the canonical visual anchor for that character.

We use an ensemble of independent facial similarity models (e.g., Buffalo-L, Antelope-v2, FaceNet, etc.), each fine-tuned for intra-character identity matching. Rather than representing a face with a single global embedding, these models extract multiple localised facial feature vectors corresponding to stable semantic regions of the face (such as eyes, nose bridge, jawline, and facial contours).

Sampling multiple localized descriptors improves robustness to pose changes, partial occlusion, lighting variation, and expression drift failure modes that are common in video generation but underrepresented in still-image similarity tasks. Each character, k, is represented in form of embeddings as f_hero^(k)

For a given frame i, each model m​ produces an embedding f_i^(k,m)​. We compute similarity using cosine similarity, which measures angular alignment in embedding space and remains invariant to feature magnitude.

This property is critical because generative models can alter contrast, illumination, and texture intensity without changing identity. Distance-based metrics (Euclidean or Manhattan) distance are sensitive to these magnitude shifts and empirically produce unstable thresholds across frames.

We also experimented with Jaccard similarity on facial embeddings but observed weaker alignment with human-in-loop evaluations. The similarity between the generated frame for a character k and its corresponding hero image for a model m, S_i^(k,m) is computed as:

Each model has a threshold m calibrated on human-labeled data. The binary decision per model m for each character k is:

The final consistency decision C uses an ensemble aggregation across all models (N) and characters, tuned for recall maximization:

where, m​ represents each model’s reliability weight and defines ensemble sensitivity.

This design ensures that every potential inconsistency is captured, even if one model fails, prioritizing recall over precision. Minor false positives are filtered through, ensuring consistency over short frame windows rather than isolated detections. Thresholds m and m are periodically re-calibrated using human-in-loop feedback. Annotators review flagged segments, refine labels, and feed corrections back into the validation loop. Thus, by fusing diverse fine-tuned models and anchoring every comparison to the hero reference, the system maintains stable character identity across the generation pipeline, regardless of lighting or scene transition.

For, side-profile consistency, which often reveals identity drift missed in frontal views. We generate strict left and right 90-degree profile references from the approved hero image, excluding frontal or angled views. We store the front, left, and right profiles as canonical anchors and compare generated frames against them to validate identity across viewing angles. This check surfaces profile-specific inconsistencies early and triggers regeneration when needed.

Character Deformity Detection

Character deformities break visual realism immediately. Generative models can produce warped limbs, misaligned joints, or anatomically implausible body structures when spatial constraints fail during sampling. To detect these failures reliably, we built a deformity-classification pipeline grounded in curated abnormality data and a trained YOLO-family detector.

We first evaluated general-purpose multimodal models such as Gemini and Qwen-VL on deformity detection. These models achieved only ≈40% recall on our internal human-annotated deformity dataset, and they consistently failed to detect subtle or multi-region structural distortions. This baseline confirmed that deformity detection requires a dedicated model trained on explicit abnormality signals. To detect anatomical distortions reliably, we built a deformity-detection module optimized for high recall and early intervention during video generation.

• Used Tencent’s Distortion dataset (Predicting Distortion in Real-World Human Images) as the base corpus and curated it using human-in-loop review to improve label reliability.
• Applied a segmentation-guided cleaning pipeline to remove annotations outside the human region, discard low-confidence samples pseg​, and filter out deformity regions below a minimum area threshold Amin​; segmentation masks also provided body-part bounding boxes.
• Trained a YOLO-family detector on the curated dataset to localize and classify deformities across full-body and body-part crops, explicitly optimizing for high recall., Achieving ~35% recall lift on the same human-annotated evaluation set in comparison to Gemini and Qwen.
• Integrated the detector across all generation stages to flag anatomical failures early and trigger regeneration or parameter adjustments before outputs propagate downstream.

Location Consistency

Location consistency ensures that generated scenes remain faithful to the scripted description and stable across multiple scenes within the same location. This is extremely important since drift in layout, lighting, spatial structure, or persistent objects breaks continuity and degrades perceived quality.

Consistency with scripted location and object constraints: During script parsing, LLMs extract structured location descriptions that capture spatial layout, environmental cues, lighting intent, and object-level constraints. These descriptions define both the expected set of objects and a subset of mandatory elements whose presence must be preserved.

We evaluate object presence using vision language models (VLMs) and flag in case if required objects are missing. In parallel, we assess overall scene fidelity by aligning text embeddings derived from the location description with visual scene embeddings extracted from generated keyframes using our VLM stack (e.g., Gemini, Qwen).

We calibrate similarity thresholds through human-in-loop (HIL) evaluation, selecting cutoffs that best correlate with human judgments of scene correctness. Frames that fall below the calibrated threshold indicate semantic or structural violations and trigger regeneration or parameter adjustment. Figure 5 illustrates how alignment scores reflect adherence to scripted location and object constraints.

Scene Visualization: Interior. Police interrogation room. Dim overhead lighting. Metal table bolted to the floor. Two chairs facing each other. One-way mirror. No windows.

Consistency across scenes within the same location: For locations that recur across multiple scenes, we treat the first validated keyframe as the one achieving the highest text–image alignment score as the location anchor. Using our vision–language model (VLM) stack, we extract scene-level embeddings from subsequent frames and compare them against this anchor via cosine similarity to detect structural and stylistic drift. This formulation allows us to enforce consistency in spatial layout, lighting characteristics, and persistent background elements when the same room, street, or set reappears at different points in the video. Figure 6 illustrates this anchor-based consistency check.

LLMs provide strong priors when generating detailed location descriptions, but they do not yet reliably detect subtle spatial or geometric inconsistencies in generated frames. We are therefore exploring additional approaches, including embedding-based scene classifiers, layout-consistency models, and structure-aware validators, to strengthen this module. These efforts aim to convert location consistency into a fully measurable and enforceable dimension within the validation framework.

Brand Logo Detection/Safety Checks

Brand-logo/Safety violations act as hard safety gates in the generation pipeline. The system blocks any output containing such elements and triggers regeneration until the output passes all safety checks. The validation loop operates as follows: the detector scans each generated unit, flags any violation, the system regenerates the content with adjusted constraints, and the detector re-evaluates the regenerated output. If repeated attempts fail, the system escalates the case for manual review. This loop ensures no flagged safety issue propagates downstream.

We enforce these safety dimensions through prompt-level controls and automated detection. During script parsing, LLMs generate structured content descriptions, including required or disallowed visual categories. These descriptions guide the image-generation model to avoid branded items and unsafe content.

We curated datasets for both tasks: brand/logo samples across categories such as laptops, consumer electronics, apparel, etc. and safety violations across violence, child-abuse, racism, hate symbols, and other violation types.

Using these datasets, we optimized prompts and detection thresholds for high recall, achieving >95% recall on internal evaluations. When the detector identifies a brand or NSFW instance, the system regenerates the output with stricter constraints to remove the violation.

Engineering Controls to Prevent Recurrence of Violations

Context-Aware Decoding:
We add structured negative constraints that suppress brand names, logos, or unsafe categories during generation. These constraints adjust the decoding trajectory of the image-generation model and reduce the probability of producing forbidden visual elements.

Adaptive Prompt Rewriting:
When a violation is detected, the system rewrites the prompt by tightening constraints, clarifying allowed content, and removing ambiguous phrasing. These rewritten prompts condition the regeneration step and help eliminate repeated violations across attempts.

Summary and Future Directions

This work presents a validation framework for generative video that operates as a first-class component of the generation pipeline. By integrating validation directly into the workflow, the system detects and corrects character inconsistency, anatomical deformities, location drift, and safety violations during generation.

The framework applies recall-first validation, multi-model similarity checks, vision–language alignment, and thresholds calibrated through human feedback to convert qualitative notions of visual quality into enforceable signals. This design prevents narrative-breaking errors from propagating while allowing controlled creative variation.

Next, we will formalize a unified evaluation and metrics layer that measures video, audio, lip-sync, and temporal consistency and supports systematic optimization.

We will also extend the framework to address long-range scene and concept continuity, enforcing coherence across scenes, episodes, and story arcs. These extensions will complete the quality stack required to deploy generative video systems at production scale.

Want to be at the forefront of generative video in India? Do check out open roles if you want to build for millions of customers, who will use features that you build!

Gen AI Video — Building Scalable Validation Framework was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

Recap

In Part 1, we shared the strategy — why we had to move away from CocoaPods, how we evaluated our options, and the phased migration plan we designed to modernize 60 pods over 2 quarters without disrupting feature development.

We bring it all home and share our learnings and scars.

Phase 1: Proving That SPM Could Coexist

Big bang approaches for a product that supports over a billion Indians is not on the menu. We could not stop the train and we had to upgrade while we ran as fast as we were.

Swift Package Manager (SPM) had to live alongside CocoaPods without disrupting day-to-day development.

This was not negotiable. It was also about building confidence gradually.

Our dependency graph was already complex, and CocoaPods was deeply embedded in how the app was built, tested, and shipped. Introducing a second dependency manager into that ecosystem was risky. If coexistence failed, everything downstream would be compromised.

So we treated Phase 1 as a controlled experiment.

Defining Success

We were explicit about what success looked like:

• Developers should not need to change their workflows
• CI pipelines must remain stable
• No runtime regressions
• CocoaPods must remain fully functional

Choosing the Right First Dependencies

We intentionally avoided internal SDKs in this phase. Instead, we chose third-party libraries that already had mature SPM support and met three criteria: widely used in the app, minimal customization, and no deep runtime coupling with internal frameworks or development pods.

This allowed us to isolate SPM behavior without risking business-critical flows. By migrating only a handful of carefully selected dependencies, we could observe real-world behavior without destabilizing the system.

Invisible to Developers

One of the most important constraints we enforced was developer invisibility.

Developers continued to open the same workspace, build using the same schemes, run tests the same way, and rely on the same CI signals. There were no new scripts to run, no new commands to remember, no changes to onboarding docs.

SPM dependencies resolved automatically by Xcode in the background. If someone hadn’t been told we were testing SPM, they wouldn’t have noticed.

Phase 2: Internal Pods and Binary Distribution

Phase 1 inspired confidence and we ramped up and doubled down. Internal pods were where things got interesting.

These weren’t isolated third-party libraries. They were shared across multiple apps, some across Android, and were actively developed. Moving them required more than a format change — it required rethinking how we distribute internal code.

The Promise of Binary Targets

Moving our internal pods to SPM binary targets felt like a natural evolution. Using .xcframework with .binaryTarget allowed us to distribute prebuilt artifacts instead of rebuilding large internal modules every time.

.binaryTarget(
    name: "CoreSDK",
    url: "https://example.com/CoreSDK.xcframework.zip",
    checksum: "..."
)

This allowed us to preserve encapsulation, reduce build times, and decouple SDK evolution from app builds.

But this phase also exposed one of our biggest challenges.

The Problem: Binary Targets and Private Repositories

Very quickly, we ran into a problem that wasn’t obvious from the documentation.

Swift Package Manager assumes that binary artifacts are publicly accessible. When SPM encounters a binaryTarget(url:), it attempts to download the zip file, verifies the checksum, and caches the artifact locally. What it does not do is authenticate.

That assumption works fine for open-source packages hosted publicly — but completely breaks down for private internal SDKs.

We hosted our .xcframework.zip files as GitHub release assets inside private repos. Everything looked correct: URL was valid, checksum matched, artifact was present. Yet builds consistently failed:

Failed to download binary artifact
The requested URL returned error: 404

The file existed. The problem was subtle but critical: SPM was making unauthenticated HTTP requests to private URLs. GitHub correctly responded with 404, not 401, masking the real issue.

Why This Was a Big Deal

This wasn’t just an inconvenience — it had architectural implications:

• Developers couldn’t resolve packages locally
• CI pipelines failed deterministically
• Binary targets became unusable for internal SDKs
• The entire binary-distribution strategy was at risk

At this point, we had to pause and ask: Can SPM actually work for private, enterprise-scale binary distribution?

Exploring Workarounds

We explored multiple approaches, each with trade-offs:

Making repositories public — Immediately ruled out. Internal SDKs contain proprietary logic.

Embedding tokens in URLs — Technically possible, but unacceptable. Security risk, tokens leak via logs, impossible to rotate safely.

Git LFS — Workable, but introduced large repository sizes, slower clones, and additional tooling overhead. Didn’t scale well for frequent SDK releases.

Artifact repositories (S3/Nexus) — Viable, but required additional infrastructure, credential management, and URL signing logic.

We wanted something simpler for GitHub-hosted binaries.

The Solve: .netrc Authentication

The solution came from understanding how SPM downloads binaries.

SPM relies on standard system networking under the hood. That means it respects .netrc credentials, just like curl or git. By configuring authentication at the system level, we could allow SPM to fetch private binaries without changing a single line of Package.swift.

# ~/.netrc
machine github.com
login GITHUB_USERNAME
password GITHUB_PERSONAL_ACCESS_TOKEN

Once this file was present, SPM successfully authenticated, binary artifacts downloaded correctly, checksums validated as expected, and builds became deterministic again.

Most importantly, this worked identically on developer machines and CI runners.

Hardening : Making It CI-Friendly

On CI, we injected the .netrc file securely at runtime using secrets:

echo "machine github.com login $GITHUB_USER password $GITHUB_TOKEN" >> ~/.netrc
chmod 600 ~/.netrc

This gave us no credentials in source control, easy token rotation, and clear audit boundaries. It also aligned well with GitHub Actions and self-hosted runners.

What This Unlocked

Solving this problem unlocked the full potential of SPM binary targets:

• Internal SDKs could be versioned independently
• App builds became significantly faster
• SDK releases became predictable artifacts
• Teams consumed binaries without worrying about source-level coupling

Onwards!

Phase 3: Development Pods → Swift Packages

Development pods were not just dependencies — they were living parts of the app, evolving alongside features, touched daily by multiple teams. They were also deeply intertwined with how our codebase was structured.

The Challenge: Living Code, Not Just Dependencies

Our development pods served a very specific purpose: they allowed teams to iterate on shared modules without releasing binaries, supported rapid local changes and debugging, and encoded architectural boundaries inside the Podfile.

Over time, they became like an extension of the app — not just dependencies.

Replacing them with Swift packages meant answering a hard question: Can we retain the same developer experience without CocoaPods doing the heavy lifting for us?

Preserving the Existing Structure

One of our biggest concerns was accidentally reshaping the codebase. We did not want to flatten modules, merge responsibilities, or introduce artificial package boundaries.

Instead, we followed a strict rule: every development pod becomes a Swift package with the same conceptual boundaries.

That meant one pod became one package, with the same folder structure, same ownership, and same responsibility. This discipline paid off later when debugging regressions and onboarding developers.

Language Boundaries: Objective-C and Swift

Swift Package Manager supports Objective-C, but it does not allow multiple languages within the same target. A single target cannot contain both Swift and Objective-C sources.

Several of our development pods relied on exactly that — Swift and Objective-C files coexisting within the same logical module, with bridging handled implicitly by the build system. Under SPM, this was no longer possible.

To move forward, we restructured these modules intentionally. Objective-C code was moved into dedicated package targets with explicitly defined public headers. Swift targets then depended on these Objective-C targets through clear, declared dependencies.

This refactoring made language boundaries explicit and removed implicit bridging behavior. While it required effort, it resulted in a cleaner dependency graph and clearer ownership across modules.

Platform Boundaries: XIBs and Multi-Platform Packages

Legacy XML Interface Builder (XIB)s introduced a different kind of challenge — one tied closely to how Swift Package Manager treats multi-platform packages.

Under CocoaPods, platform-specific behavior could be handled through Podfile logic or build configurations. This allowed development pods to bundle UI resources that behaved differently across iOS and tvOS without that distinction being explicit in the code.

Swift Package Manager takes a stricter approach. Packages are inherently multi-platform, and resources declared in a package are shared across all supported platforms. There is no native way to conditionally include or exclude resources based on platform within the same target.

Several development pods contained XIBs that were platform-specific and loaded conditionally at runtime. Once these pods became Swift packages, those assumptions no longer held.

Rather than fragmenting packages or introducing complex runtime branching, we made a deliberate architectural decision: we moved most XIB-based UI into code.

This shift reduced reliance on resource bundling, eliminated fragile platform assumptions, and aligned better with the multi-platform model encouraged by Swift Package Manager — even though it required more upfront refactoring.

Build Behavior: Replacing Post-Install Scripts

One aspect we underestimated initially was how much logic lived outside the code itself.

Over the years, our CocoaPods setup had accumulated substantial scripting inside the post_install block. These scripts handled modifying build settings across targets, injecting compiler and linker flags, adjusting deployment targets, patching generated project settings, code generation for modules, and handling branding assets.

CocoaPods made this convenient because it centralized these changes in one place. But once we moved away from CocoaPods, that implicit behavior disappeared immediately.

Swift Package Manager does not offer an equivalent of a post_install hook. That forced us to confront an important reality: a lot of critical build behavior was hidden in scripts that developers rarely looked at.

To preserve correctness without reintroducing global magic, we deliberately moved this logic closer to where it actually mattered. Most essential scripting was redistributed into explicit Xcode build phases, scoped to the relevant app or framework targets. In some cases, we replaced scripts entirely by fixing the underlying configuration rather than patching it at build time.

This shift had two important effects: build behavior became more visible and discoverable, and changes were scoped to specific targets instead of being applied globally.

Resources, Flags, and Conditional Logic

Resources — Assets that were automatically bundled by CocoaPods now had to be declared explicitly:

.target(
    name: "UserProfileKit",
    resources: [
        .process("Resources")
    ]
)

This forced us to audit every resource and validate runtime access paths — something CocoaPods had silently handled for years.

Build Settings — CocoaPods’ pod_target_xcconfig allowed us to inject compiler and linker settings easily. SPM requires these to be expressed explicitly:

swiftSettings: [
    .define("ENABLE_LOGGING", .when(configuration: .debug))
]

For edge cases, we used .unsafeFlags — but sparingly and deliberately. This made us more intentional about what each module actually required.

Conditional Inclusion — In CocoaPods, it was common to conditionally include pods based on build configurations. SPM does not support conditional dependencies for custom build configurations.

This forced a shift in thinking. Instead of conditionally including dependencies, we moved toward conditionally usingthem via compile-time flags and feature gates:

#if ENABLE_EXPERIMENTAL_FEATURE
// Feature-specific code
#endif

This change improved clarity, even though it required refactoring.

Keeping Development Fast

A common fear with moving development pods to SPM is slower iteration. We paid close attention to this.

Local packages were referenced via relative paths, changes reflected immediately in the app, and the debugging experience remained intact. From a developer’s perspective, very little changed — which was exactly what we wanted.

CI and Testing Implications

Moving development pods affected CI in subtle ways. Test targets needed explicit dependency declarations, schemes had to be updated, and build order changed slightly.

This surfaced hidden assumptions in our pipelines — but fixing them made CI more robust and predictable.

The Emotional Reality

This phase took time. It required patience. And it touched a lot of code.

But it also marked a turning point. By the end of Phase 3, CocoaPods was no longer central to our architecture, Swift packages were no longer “new,” and the system felt cleaner and more explicit.

We stopped thinking in terms of pods and started thinking in terms of modules with explicit contracts.

The Surprises We Didn’t See Coming

We expected friction. We anticipated refactoring. What we didn’t fully anticipate was how many assumptions CocoaPods had been quietly absorbing for us over the years — assumptions that surfaced only once Swift Package Manager forced everything into the open.

These challenges didn’t appear all at once. They emerged gradually, often at inconvenient moments, and almost always in places we thought were already “done.”

Duplicate Symbols at Runtime

One of the more subtle issues we encountered was related to duplicate symbols — but not in the way they typically present themselves. These were not build-time or linker failures. Builds succeeded, targets launched normally, and at a glance everything appeared to be working.

The problems surfaced only at runtime.

Because Swift Package Manager builds packages as static libraries by default, the same dependency could end up being embedded multiple times through different dependency paths. This resulted in multiple instances of what was expected to be a single module being present in the process.

At runtime, this caused the wrong instance of certain symbols to be referenced:

• Global state would appear to reset or diverge
• Dependency injection would resolve to unexpected instances
• Mocks would not behave as expected

In some cases, the only signal we had was a vague runtime warning:

objc[12345]: Class MySharedService is implemented in both
/path/to/App.app/App and /path/to/AppTests.xctest/AppTests.
One of the two will be used. Which one is undefined.

Nothing crashed. Nothing failed to launch. But from that point onward, behavior was undefined.

The challenge was not detecting the issue, but diagnosing it. Since nothing failed during build or launch, the failures initially looked like flaky tests or logical bugs rather than a dependency problem.

Resolving this required a careful audit of how dependencies were introduced across app, framework, and test targets. We had to ensure that shared modules were linked exactly once and that dependency graphs were consistent across targets.

This was a strong reminder that test targets are not passive consumers of the app binary. They are independent bundles with their own runtime environment.

Builds That Succeeded but Crashed

Some of the most difficult issues gave us the least amount of signal.

The app compiled successfully. There were no compiler errors. There were no linker warnings. And yet, the application crashed immediately at runtime.

These failures didn’t surface during build because, from the compiler’s perspective, everything was valid. All symbols were present, all dependencies resolved, and the binary was produced without complaint. The problem only emerged once the app launched and the runtime attempted to load and resolve those symbols.

dyld: Symbol not found: _$s15MySharedModule16CriticalServiceC11sharedInstanceACvgZ
  Referenced from: /Applications/App.app/App
  Expected in: /Applications/App.app/Frameworks/MySharedModule.framework/MySharedModule

dyld: Library not loaded: @rpath/MySharedModule.framework/MySharedModule
  Reason: image not found

Diagnosing these issues required stepping outside the usual compile–link–run mental model. We had to inspect the final app binary, verify which frameworks and libraries were actually embedded, and confirm that runtime search paths and linkage settings were aligned with how the dependencies were built.

Saying Goodbye to Slather

One of the bigger surprises had nothing to do with compilation, linking, or dependency resolution. It was code coverage.

For years, we had relied on Slather as our coverage tool. It was stable, familiar, and deeply integrated into our CI pipelines. We assumed it would continue to work as we moved dependencies to Swift Package Manager.

That assumption turned out to be wrong.

Slather does not support pure Swift packages as first-class citizens. It expects coverage to be generated from Xcode projects or workspaces, not standalone packages. As more of our codebase moved into Swift packages, coverage for package-based modules simply disappeared.

The only way to keep Slather working would have been to create artificial “host” projects whose sole purpose was to run package tests and collect coverage. That approach went directly against our goal of making systems simpler.

At that point, it became clear that the problem wasn’t Swift Package Manager — it was the tooling around it.

We switched to an xcresult-based coverage pipeline, parsing coverage directly from Xcode's native test result bundles. This aligned us with the direction Apple was already taking. Coverage became more accurate, easier to reason about, and independent of how the code was packaged.

Phase 4: Removing CocoaPods

By the time we reached this phase, CocoaPods was no longer doing much work.

All third-party dependencies had already moved to Swift Package Manager. Internal SDKs were consumed as binary targets. Development pods had been fully replaced with package-based modules.

And yet, CocoaPods was still there — quietly present, still wired into the system.

Removing it was less about technical effort and more about confidence.

Knowing When We Were Ready

For several weeks, CocoaPods existed in the repository almost as a safety net. During this period, we monitored build stability across all configurations and regions, CI performance and reliability, developer onboarding, and QA cycles without CocoaPods involvement.

Only after we had multiple successful releases — without touching CocoaPods at any stage — did we decide it was time to move on.

The Final Delete

The final step was straightforward: we removed the Podfile, the CocoaPods-generated .xcworkspace, and all remaining CocoaPods-related files and scripts.

There was no disruption, no rollback, and no follow-up fixes.

The system simply continued to work — without CocoaPods.

The day we merged that PR felt like crossing a finish line we’d been racing toward for six months.

What This Journey Taught Us

By the time CocoaPods was fully removed, the migration itself had stopped feeling like the most important outcome.

What mattered more was how the journey reshaped the way we think about our codebase, our tooling, and the systems that support day-to-day development.

This wasn’t just a dependency migration. It was a gradual recalibration of engineering discipline.

Tooling Should Fade Into the Background

The best tooling is the kind you don’t think about.

CocoaPods had accumulated years of scripts, configuration overrides, and implicit behavior. Swift Package Manager, in contrast, forced us to be explicit — but once configured, it largely disappeared into the background.

When developers no longer need to remember setup steps or debug dependency resolution, cognitive load drops. Productivity rises not because things move faster, but because there’s less to manage.

Migration Is About Trust, Not Speed

One of the most important decisions we made was not rushing.

By migrating in phases and allowing CocoaPods and SPM to coexist, we preserved trust: trust from developers that their workflows wouldn’t break, trust from QA that releases wouldn’t destabilize, trust from leadership that the migration wouldn’t impact delivery.

The time spent validating coexistence and waiting through release cycles was not overhead — it was risk mitigation.

CI/CD Is Part of the Architecture

Several challenges — especially around binary targets, coverage, and caching — forced us to acknowledge something we had underweighted before:

CI/CD is not infrastructure glue. It’s part of the system design.

Solving problems like authenticated binary downloads or deterministic package resolution required thinking beyond Xcode and into the pipeline itself. Once we did, CI became more reliable, more predictable, and easier to maintain.

Closing Thoughts

CocoaPods played a critical role in helping us scale our iOS and tvOS ecosystem at a time when the platform needed it most. It gave us structure, enabled modularization, and supported years of rapid development. For that, it deserves recognition.

Swift Package Manager represents where the Apple ecosystem is heading. It aligns more closely with Swift itself, integrates natively with Xcode, and encourages explicit, predictable dependency management. Adopting it wasn’t just a response to CocoaPods’ sunset — it was an investment in long-term maintainability and clarity.

The migration was not quick, and it was never meant to be. We approached it deliberately, prioritizing stability over speed and confidence over convenience. By moving in phases, preserving existing workflows, and validating each step through real release cycles, we ensured that modernization didn’t come at the cost of ongoing development.

Six months. Two quarters. Sixty pods. Zero broken releases. One transformed dependency management system.

If you’re standing at a similar crossroads, our advice is simple but hard-earned:

Move with intent. Move carefully. And never break development in the process.

We’re hiring for our client teams! Do check out open roles if you want to build it right, while you build for millions of customers, who will use features that you build!

Modernizing Dependency Management: Beyond CocoaPods (Part 2 — The Execution) was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.

CocoaPods formed the spine of our iOS build workflow. In this two part blog, we’re sharing our journey to transparently switch away from CocoaPods to SPM, minus the drama, but plus all the scars!

Where We Started

CocoaPods was at the heart of our development workflow — for everything. Dependency resolution, configuration overrides, CI integration — all became tightly coupled to CocoaPods. It stopped being a convenience layer and became part of our infrastructure.

Then came the announcement that changed everything.

CocoaPods’ trunk would become read-only in December 2026.

This wasn’t just an ecosystem update. It was a tectonic shift. A read-only trunk meant no new pod versions, no straightforward path to adopt upstream fixes, and increasing exposure to unpatched issues over time. Any disruption here wouldn’t just affect builds — it would impact active feature development, CI stability, and release confidence across teams.

So when the announcement landed, the question wasn’t “Should we move?” That decision had effectively been made for us.

What followed was one of the most ambitious infrastructure initiatives we’ve undertaken — a six-month effort, touching every corner of our codebase, and ultimately transforming how we build and ship our apps.

This was open-heart surgery on a system that powered millions of app sessions every day — performed while the patient was still running around

This is that story.

Choices, choices..

Before committing to any solution, we took a deliberate step back and evaluated the dependency management landscape as a whole. Since we had to move, we wanted to make as much of a future proof choice as possible.

Several factors mattered deeply to us:

• Scalability — Handle the size and complexity of our codebase, while remaining adaptable.
• Learning curve — It couldn’t slow teams down or force widespread workflow changes.
• Future-proofing — We wanted to align with the direction the Apple ecosystem was moving.
• Tooling integration — It had to work well with project generation and tooling we already relied on.

With those constraints in mind, we evaluated the available options.

Carthage

Carthage is lightweight, decentralized, and intentionally avoids modifying Xcode projects — qualities that align well with engineering simplicity. For smaller projects or teams with straightforward dependency needs, it can be an excellent choice.

However, managing binary distribution at scale, supporting internal SDKs, handling complex build configurations, and maintaining consistent tvOS support required more manual orchestration than we were comfortable with. Over time, this would have shifted operational complexity from tooling into team workflows.

Carthage wasn’t a bad fit universally; it just wasn’t the right fit for our ecosystem.

Bazel and Buck

We also explored Bazel and Buck — not dependency managers in the traditional sense, but complete build systems. Both are powerful and proven at massive scale, offering deterministic builds, strong caching, and sophisticated dependency graphs.

However, adopting either would have meant replacing our entire build system, moving away from Xcode’s native build model, and introducing a steep learning curve for developers whose daily workflows are deeply tied to Xcode.

For our teams, the cost of that transition far outweighed the benefits.

Swift Package Manager (SPM)

Swift Package Manager wasn’t perfect. It lacked some of the flexibility we were used to, and certain features required workarounds. But it had two qualities that ultimately mattered most.

First, it was native — integrated directly with Xcode, aligned with Swift’s evolution, and benefiting from ongoing investment by Apple. Second, it allowed us to preserve our existing project structure while gradually modernizing it. We could migrate incrementally, validate changes in production, and avoid large-scale rewrites.

That balance — modernization without disruption — was the turning point. Swift Package Manager positioned us to evolve with the platform, rather than constantly working around it.

Replacing a beating heart…

This was open-heart surgery on a system that powered millions of app sessions every day — performed while the patient was still running around!

At the time of the migration, our ecosystem included:

• ~60 iOS/tvOS engineers actively shipping features
• ~30 SDETs maintaining extensive automation and test infrastructure
• A multi-language codebase (Swift, Objective-C, and supporting tooling) spanning millions of lines of code
• 60+ pods in total — external dependencies, internal SDKs, and local development pods under constant iteration
• Multiple markets(India, International) with distinct configurations

Every one of those engineers relied on CocoaPods behaving in very specific, sometimes undocumented ways. Every automation script, every CI pipeline, every local development workflow had assumptions baked in about how dependencies resolved, how builds were structured, and how artefacts were produced.

We made extensive use of pre_install and post_install hooks. We maintained multiple build configurations for India and international markets — involving conditional linking, selective dependency inclusion, and market-specific code paths. Our CI pipelines were tightly coupled to CocoaPods-generated projects in ways that weren't always visible until something broke. This was organisational Jenga, only, we couldn’t let that tower fall, or even shake!

Easy, right?

Core Constraint: Do Not Break Development

From day one, we aligned on one non-negotiable rule:

This migration must not block ongoing development.

This constraint shaped everything.

Teams were actively shipping features. Local pods and internal SDKs were under constant iteration. CI pipelines had to remain stable across multiple environments. We couldn’t afford a “big bang” migration that froze development, forced widespread workflow changes, or introduced uncertainty into release cycles. Migration had to occur in phases with both systems co-existing.

This wasn’t the cleanest approach — maintaining two dependency systems in parallel added complexity. It was the safest path forward. It meant teams could keep shipping while we rebuilt the foundation underneath them.

Our Migration Strategy

Once we committed to a phased migration, we needed a structure that balanced safety with forward momentum. Each phase had to deliver real progress, while still preserving the ability to ship features without disruption.

We broke the journey into four deliberate stages:

Phase 1: Proving That SPM Could Coexist

Before touching any critical paths, we focused on coexistence. Swift Package Manager was introduced alongside CocoaPods — not as a replacement, but as a parallel system. This phase was about validation.

Phase 2: Internal Pods and Binary Distribution

With coexistence proven, we moved to internal SDKs and binary dependencies. These were more controlled, lower-churn modules, making them ideal candidates for early migration. This phase helped us establish patterns for versioning, distribution, and consumption at scale.

This is where we built the playbook that would carry us through the harder phases ahead.

Phase 3: Development Pods → Swift Packages

This was the crucible.

Development pods were deeply intertwined with active feature work, often containing mixed-language code and UI resources. Migrating these required structural changes — not just mechanical conversions — and forced us to confront some of Swift Package Manager’s core constraints head-on.

This phase stretched across quarters, required constant coordination with feature teams, and tested every assumption we’d made about the migration strategy.

This is where patience mattered more than speed.

Phase 4: Removing CocoaPods

Only after the system had fully stabilized under SPM did we execute the final step: removing CocoaPods entirely. By this point, CocoaPods was no longer a dependency — it was technical debt waiting to be deleted.

The day we removed the Podfile from the repository felt like crossing a finish line we’d been racing toward for six months!

The Payoff: Measurable Wins

This wasn’t change for change’s sake. After two quarters of sustained effort, the results were tangible and significant.

Immediate Impact

• Automation stability improved — test targets became less sensitive to implicit CocoaPods behavior
• Dependency graphs became explicit — making ownership, impact analysis, and refactoring significantly easier
• CI pipelines became more predictable — fewer pod-related cache invalidations and mysterious failures
• Build times improved by 2x — especially in incremental builds, due to better dependency isolation and binary targets wherever possible
• App startup time reduced by 200–300ms — SPM’s cleaner dependency loading eliminated redundant framework initialization at launch. The migration also allowed us to adopt the latest linker (previously blocked due to crashes on older OS versions), which improved dynamic library load times and static linking performance.

Long-Term Gains

Just as importantly, we fundamentally reduced our risk profile. Dependency management stopped being a fragile layer propped up by scripts, conventions, and tribal knowledge. It became something the platform itself understood — native, supported, and evolving with the ecosystem.

We went from dreading the CocoaPods deprecation deadline to being ahead of it by a year.

Six months. Two quarters. Sixty engineers. Sixty-plus pods. Zero feature freezes. One satisfying Podfile deletion.

The Scars — Part 2

In Part 2, we’ll go deep into how these phases were executed in practice.

We’ll cover the real issues we encountered along the way — mixed Objective-C and Swift targets, XIBs in development pods, CI assumptions that quietly broke, and the architectural decisions we had to make to move forward safely.

This is where theory met reality — and where the migration truly earned its scars!

We’re hiring for our client teams! Do check out open roles if you want to build it right, while you build for millions of customers, who will use features that you build!

Dependency Management: Our Journey Beyond CocoaPods (Part 1 — The Strategy) was originally published in JioHotstar on Medium, where people are continuing the conversation by highlighting and responding to this story.