Loading the next video before you swipe

In a vertical short-video feed, you swipe up and the next video plays. If that takes even a moment, you notice the black screen or the spinner, and you notice it on every swipe. That’s a lot of swipes.

Once you’ve swiped, the player can’t make the network go any faster. So the only real option is to load things before you swipe. By the time a page settles, part of its video should already be on the phone.

I also wanted to know whether it was actually working. Debug builds log how long it takes from a page settling to its first frame on screen. The app also sends each video’s startup time to analytics, and if a start takes five seconds or more, it reports the player’s bandwidth estimate as well. That makes it easier to tell a slow connection apart from a slow app.

The first player was pretty simple. On every swipe it swapped in the new video, prepared it, and started playing. To get ahead of the swipe, I added Media3’s DefaultPreloadManager next to it. The next video would load its first three seconds, while the one after it and the previous one would fetch their manifests and pick their tracks.

The problem was the “next to it” part. The preload manager and the player were built separately, and Media3’s preloading guide says the player has to be built from the same builder as the preload manager so they can share components. Mine wasn’t. I couldn’t simply give them the same LoadControl either, because that only works if they run on the same playback thread. So each one had its own, and whatever the preload manager loaded never made it to the player. You’d swipe, and the player would download those same three seconds again.

So I scaled it back to manifests only. That still warmed up the connection, so the player could skip the DNS, TCP, and TLS handshakes, and the whole thing used less than 2 MB of memory. Better than nothing, but the video itself still only started downloading after the swipe.

Looking back, that’s the part I’d warn people about. Preloading only helps if the player ends up using what you preloaded. Sounds obvious, but it wasn’t obvious enough to me the first time.

About a week later I rewrote the player. Now there’s only one ExoPlayer in the app, and it’s built by the same DefaultPreloadManager.Builder that builds the preload manager. They share the playback thread, the load control, and the bandwidth meter, so the preloaded data can actually reach the player.

The feed itself is the player’s playlist. Every item in it is the MediaSource the preload manager gives back for that video, which means anything that’s been preloaded is already sitting inside the item the player is about to play. A swipe is just a seek to that index, plus telling the preload manager where we are now. Simplified, it looks something like this:

val builder = DefaultPreloadManager.Builder(context, statusControl)
    .setMediaSourceFactory(cachedMediaSourceFactory)
    .setLoadControl(shortsLoadControl)

// Same builder for both, so the player can use what was preloaded
val player = builder.buildExoPlayer()
val preloadManager = builder.build()

fun moveWindowTo(index: Int) {
    statusControl.currentIndex = index
    preloadManager.setCurrentPlayingIndex(index)
    preloadManager.invalidate()
}

// The feed is the playlist, and each item is the preload manager's source
feed.forEachIndexed { index, item -> preloadManager.add(item, index) }
player.setMediaSources(feed.mapNotNull { preloadManager.getMediaSource(it) })
player.prepare()
moveWindowTo(0)

fun onPageSettled(index: Int) {
    player.seekTo(index, 0L)
    moveWindowTo(index)
}

When the next page of the feed comes in, the new items get added to the end of the playlist, and the video that’s currently playing doesn’t get prepared again. Having one player pays off somewhere else too. If you go from the feed to another screen that plays the same video, like a detail page, it just keeps playing, since it’s the same player and the same item. Only the items around it change.

Something that’s easy to miss is that the builder creates two track selectors from the same factory, one for the player and one for the preload manager, and they can’t share a single instance. So every change to quality, audio language, or subtitles has to go to both. If they don’t match, the player can end up picking different tracks than the ones that were preloaded, and that preload goes to waste.

To decide how much to prepare for each video, the preload manager asks a TargetPreloadStatusControl, based on how far that video is from the one that’s playing. Media3’s guide has an example to start from. Mine leans forward, since that’s the way people mostly swipe:

A quick note on those levels. “Manifest” means the preload manager fetches and parses the manifest. “Track selection” goes one step further and picks which video and audio tracks to use, but still doesn’t load any video. “First 4 s” means four seconds of actual video in memory, ready to play.

The window doesn’t wait for the swipe to finish, either. During a fling, the pager already knows which page it’s going to land on, so the feed calls moveWindowTo for that page right away, just without the seek. The preloading shifts to that page while the pager is still scrolling, and the thumbnails around it get requested at the same time.

Preloading also spends data on videos people might never watch, so I kept the window small. At most eight seconds of video gets loaded outside the one that’s playing, and nothing bigger than 1280 pixels on the long edge. The playing video buffers 5 to 30 seconds ahead instead of Media3’s default of 50, and it starts as soon as half a second is buffered. Everything the player and the preload manager download goes through a 200 MB disk cache too, so swiping back to a video reads it from disk instead of downloading it again.

The very first video is the hardest one, since there’s no earlier swipe to hide the loading behind. So part of that work happens during app start.

While the app is creating its session, the player gets built in parallel. Once the app knows it isn’t blocked, by a forced update for example, it fetches the first page of the feed in the background. I keep that page for up to 60 seconds, and the feed uses it for its first load instead of calling the API again. The first video on that page goes straight to the preload manager, so it starts getting its manifest and tracks ready before the feed even shows up.

There’s one catch with that saved page. It was fetched for whoever was signed in at the time, so it gets thrown away when someone signs in or out, or when their profile changes in a way that affects what they’re allowed to watch. Otherwise you could open the app and get a first video that was picked for a different account, or for an older version of your own profile.

Even with all this, a page can sometimes appear before its video has a frame ready, like after a really fast fling. So every page shows its thumbnail until the video’s first frame is on screen. Worst case, you see a still image for a moment before it starts moving, instead of a black screen.

The thumbnails are prefetched around the page the feed is heading to, and funnily enough, that prefetch had the same problem as attempt one. About two months later, a teammate found that it had never actually worked. It went through a different image loader than the feed, and its requests ended up with different cache keys, so the page never used the prefetched images. The blank page it was meant to prevent had been there the whole time.

What makes bugs like these annoying is that nothing looks broken. Nothing crashes and there are no errors. The requests go out just fine, and then nothing uses what comes back. So these days I check where a preload actually ends up instead of assuming it works. For video, that’s the time from a page settling to its first frame. For anything cached, it’s making sure the screen reads the exact same cache entry the prefetch wrote.

If you’re building something like this with Media3, start with the player built from the preload manager. It would’ve saved me the whole first attempt.