Exploring Methods for Implementing iOS HLS Cache

Exploring Methods for Implementing iOS HLS Cache

How to achieve caching while playing m3u8 streaming video files using AVPlayer

photo by Mihis Alex

[2023/03/12] Update

• The next article, “Comprehensive Guide to Implementing Local Cache with AVPlayer”, teaches you how to achieve AVPlayer caching

I have open-sourced my previous implementation, and those in need can use it directly.

• Customizable cache strategy, you can use PINCache or others…
• Externally, you only need to call the make AVAsset factory, input the URL, and the AVAsset will support caching
• Implemented data flow strategy using Combine
• Wrote some tests

About

HTTP Live Streaming (HLS) is a streaming media network transmission protocol based on HTTP proposed by Apple.

For example, when playing music, in a non-streaming situation, we use mp3 as the music file. The larger the file, the longer it takes to download completely before it can be played. HLS, on the other hand, splits a file into multiple small files, playing as it reads. So, once the first segment is received, playback can start without downloading the entire file!

The .m3u8 file records the bitrate, playback order, time, and other information of these segmented .ts small files. It can also provide encryption and decryption protection, low-latency live streaming, etc.

Example of an .m3u8 file (aviciiwakemeup.m3u8):

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:9.900411,
aviciiwakemeup–00001.ts
#EXTINF:9.900400,
aviciiwakemeup–00002.ts
#EXTINF:9.900411,
aviciiwakemeup–00003.ts
#EXTINF:9.900411,
.
.
.
#EXTINF:6.269389,
aviciiwakemeup-00028.ts
#EXT-X-ENDLIST

*EXT-X-ALLOW-CACHE has been deprecated in iOS≥ 8/Protocol Ver.7. Whether this line is present or not is meaningless.

Goal

For a streaming media service, Cache is extremely important; because each audio file can range from a few MBs to several GBs. If every replay requires fetching the file from the server again, it would be very taxing on the server’s loading, and the traffic costs are \(\). Having a Cache layer can save a lot of money for the service, and users won’t have to waste bandwidth and time re-downloading; it’s a win-win mechanism (but remember to set limits/clear periodically to avoid filling up the user’s device).

Problem

In the past, when not dealing with streaming, handling mp3/mp4 was straightforward: download the file to the device before playing, and start playback only after the download is complete. Since the file has to be fully downloaded before playback anyway, we might as well use URLSession to download the file and then feed the local file path (file://) to AVPlayer for playback. Alternatively, the formal way is to use AVAssetResourceLoaderDelegate to cache the downloaded data in the delegate methods.

For streaming, the idea is also quite straightforward: first read the .m3u8 file, then parse the information inside, and cache each .ts file. However, implementing this turned out to be more complicated than I imagined, which is why this article exists!

For playback, we still use iOS AVFoundation’s AVPlayer directly. There is no difference in operation between streaming and non-streaming files.

Example:

let url: URL = URL(string: "https://zhgchg.li/aviciiwakemeup.m3u8")
var player: AVPlayer = AVPlayer(url: url)
player.play()

2021–01–05 Update:

We decided to revert to using mp3 files, so we can directly use AVAssetResourceLoaderDelegate for implementation. For detailed implementation, refer to “AVPlayer Streaming Cache Implementation”.

Implementation Solutions

Several solutions to achieve our goal and the issues encountered during implementation.

Solution 1. AVAssetResourceLoaderDelegate ❌

The first thought was to follow the same approach as with mp3/mp4 files: use AVAssetResourceLoaderDelegate to cache .ts files in the delegate methods.

Unfortunately, this approach doesn’t work because we can’t intercept the download request information for .ts files in the delegate. This is confirmed in this Q&A and the official documentation.

For AVAssetResourceLoaderDelegate implementation, refer to “AVPlayer Streaming Cache Implementation”.

Solution 2.1 URLProtocol Intercept Requests ❌

URLProtocol is a method I recently learned. All requests based on the URL Loading System (URLSession, API calls, image downloads, etc.) can be intercepted to modify the Request and Response before returning them, making it seem like nothing happened. For more on URLProtocol, refer to this article.

Using this method, we planned to intercept AVFoundation AVPlayer’s requests for .m3u8 and .ts files. If there is a local cache, return the cached data directly; otherwise, send the request out. This would achieve our goal.

Again, unfortunately, this approach doesn’t work either because AVFoundation AVPlayer’s requests are not on the URL Loading System, so we can’t intercept them. *Some say it works on the simulator but not on the actual device

Solution 2.2 Force it into URLProtocol ❌

Based on Solution 2.1, a brute-force method: if I change the request URL to a custom scheme (e.g., streetVoiceCache://), AVFoundation won’t be able to handle this request and will throw it out, allowing our URLProtocol to intercept and do what we want.

let url: URL = URL(string: "streetVoiceCache://zhgchg.li/aviciiwakemeup.m3u8?originScheme=https")
var player: AVPlayer = AVPlayer(url: url)
player.play()

URLProtocol will intercept streetVoiceCache://zhgchg.li/aviciiwakemeup.m3u8?originSchme=https, at this point, we just need to restore it to the original URL, then send a URLSession request to fetch the data and handle the cache ourselves here; the .ts file requests in the m3u8 will also be intercepted by URLProtocol, and similarly, we can handle the cache ourselves here.

Everything seemed perfect, but when I excitedly Build-Run the APP, Apple slapped me in the face:

Error: 12881 “CoreMediaErrorDomain custom url not redirect”

It doesn’t accept the Response Data for the .ts file Request I provided. I can only use the urlProtocol:wasRedirectedTo method to redirect to the original Https request to play normally, even if I download the .ts file locally and then redirectTo that file:// file; it still doesn’t accept it. Checking the official forum revealed that this approach is not allowed; .m3u8 can only originate from Http/Https (so even if you put the entire .m3u8 and all segmented files .ts locally, you can’t use file:// to play with AVPlayer), and .ts cannot use URLProtocol to provide Data.

fxxk…

Solution 2.2–2 Same as Solution 2.2 but with Solution 1 AVAssetResourceLoaderDelegate to implement ❌

Implementation is the same as Solution 2.2, feeding AVPlayer a custom Scheme to enter AVAssetResourceLoaderDelegate; then we handle it ourselves.

Same result as 2.2:

Error: 12881 “CoreMediaErrorDomain custom url not redirect”

Official forum gave the same answer.

It can be used for decryption processing (refer to this article or this example) but still cannot achieve Cache functionality.

Solution 3. Reverse Proxy Server ⍻ (Feasible, but not perfect)

This method is the most commonly suggested solution when looking for ways to handle HLS Cache; it involves setting up an HTTP Server on the APP to act as a Reverse Proxy Server.

The principle is simple, set up an HTTP Server on the APP, assuming it’s on port 8080, the URL would be http://127.0.0.1:8080/; then we can handle the incoming Requests and provide Responses.

Applying this to our case, change the request URL to: http://127.0.0.1:8080/aviciiwakemeup.m3u8?origin=http://zhgchg.li/

In the HTTP Server’s Handler, intercept and handle *.m3u8, when a Request comes in, it will enter our Handler, and we can do whatever we want, control what Data to Response, and the .ts files will also come in; here we can implement our desired Cache mechanism.

For AVPlayer, it’s just a standard http://.m3u8 streaming audio file, so there won’t be any issues.

For a complete implementation example, refer to:

Because I also referred to this example, I also used GCDWebServer for the Local HTTP Server part. Additionally, there is a newer Telegraph available for use. ( CocoaHttpServer hasn’t been updated for a long time, so it’s not recommended anymore)

Looks good! But there’s a problem:

Our service is music streaming rather than a video playback platform. In many cases, users switch music in the background; will the Local HTTP Server still be there then?

GCDWebServer’s documentation states that it will automatically disconnect when entering the background and automatically resume when returning to the foreground. However, you can disable this mechanism by setting the parameter GCDWebServerOption_AutomaticallySuspendInBackground:false.

But in practice, if no requests are sent for a period of time, the server will still disconnect (and the status will be incorrect, still showing as isRunning), which feels like it was killed by the system. After delving into the HTTP Server approach, I found that the underlying layer is based on sockets. According to the official documentation on socket services, this issue cannot be resolved. The system will suspend it when there are no new connections in the background.

*There are some convoluted methods found online… like sending a long request or continuously sending empty requests to ensure the server is not suspended by the system in the background.

All of the above applies to the app being in the background. When in the foreground, the server is very stable and won’t be suspended due to idleness, so there’s no such issue!

Since it relies on other services, even if there are no issues in the development environment, it is recommended to implement a rollback mechanism in actual applications (AVPlayer.AVPlayerItemFailedToPlayToEndTimeErrorKey notification); otherwise, if the service crashes, the user will be stuck.

So it's not perfect...

Solution 4. Use the HTTP Client’s caching mechanism ❌

Our .m3u8/.ts files’ Response Headers all provide Cache-Control, Age, eTag… these HTTP Client Cache information. Our website’s cache mechanism works perfectly on Chrome, and the new official Protocol Extension for Low-Latency HLS preliminary specification also mentions that cache-control headers can be set for caching.

But in practice, AVFoundation AVPlayer does not have any HTTP Client Caching effect, so this route is also a dead end! Pure wishful thinking.

Solution 5. Do not use AVFoundation AVPlayer to play audio files ✔

Implement audio file parsing, caching, encoding, and playback functionality yourself.

This is too hardcore, requiring very deep technical skills and a lot of time; not researched.

Here is an open-source player for reference: FreeStreamer. If you really choose this solution, it’s better to stand on the shoulders of giants and directly use third-party libraries.

Solution 5-1. Do not use HLS

Same as Solution 5, too hardcore, requiring very deep technical skills and a lot of time; not researched.

Solution 6. Convert .ts segment files to .mp3/.mp4 files ✔

Not researched, but indeed feasible. However, it sounds complicated, having to process the downloaded .ts files, convert them individually to .mp3 or .mp4 files, and then play them in order or compress them into one file or something. It just doesn’t sound easy to do.

Interested parties can refer to this article.

Solution 7. Download the complete file before playing ⍻

This method cannot be precisely called “caching while playing.” It actually involves downloading the entire audio file content before starting playback. If it is .m3u8, as mentioned in Solution 2.2, it cannot be directly downloaded and played locally.

To implement this, you need to use the iOS ≥ 10 API AVAssetDownloadTask.makeAssetDownloadTask, which will actually package the .m3u8 into .movpkg and store it locally for user playback.

This is more like offline playback rather than caching.

Additionally, users can view and manage the downloaded packaged audio files from “Settings” -> “General” -> “iPhone Storage” -> APP.

Below is the downloaded video section

For detailed implementation, refer to this example:

Conclusion

The exploration journey above took almost a whole week, going around in circles, almost driving me crazy. Currently, there is no reliable and easy-to-deploy method.

If there are new ideas, I will update!

References

• iOS Audio Playback (Nine): Caching While Playing
• StyleShare/HLSCachingReverseProxyServer

If you have any questions or comments, feel free to contact me.

===

本文中文版本

===

This article was first published in Traditional Chinese on Medium ➡️ View Here