Research on Preloading and Caching Page and File Resources in iOS WKWebView
Study on improving page loading speed by preloading and caching resources in iOS WKWebView.
{:target="_blank"}](/assets/5033090c18ba/1*KACJYJkLfa2u5iKYJlJb2Q.jpeg)
Photo by Antoine Gravier
Background
For some reason, I have always been quite connected to “Cache”. I have previously been responsible for researching and implementing the “ iOS HLS Cache Implementation Journey “ and “ Comprehensive Guide to Implementing Local Cache Functionality in AVPlayer “ for AVPlayer; Unlike streaming caching, which aims to reduce playback traffic, this time the main task is to improve the loading speed of In-app WKWebView, which also involves research on preloading and caching in WKWebView; However, to be honest, the scenario of WKWebView is more complex. Unlike AVPlayer, which streams audio and video as one or more continuous Chunk files, only file caching is needed, WKWebView not only has its own page files but also imported resource files ( .js, .css, font, image…) which are rendered by the Browser Engine to present the page to the user. There are too many aspects in between that the App cannot control, from network to frontend page JavaScript syntax performance, rendering methods, all of which require time.
This article is only a study on the feasibility of iOS technology, and it may not be the final solution. In general, it is recommended that frontend developers start from the frontend to achieve a significant effect, please optimize the time it takes for the first content to appear on the screen (First Contentful Paint) and improve the HTTP Cache mechanism. On the one hand, it can speed up the Web/mWeb itself, affect the speed of Android/iOS in-app WebView, and also improve Google SEO ranking.
Technical Details
iOS Restrictions
According to Apple Review Guidelines 2.5.6:
Apps that browse the web must use the appropriate WebKit framework and WebKit JavaScript. You may apply for an entitlement to use an alternative web browser engine in your app. Learn more about these entitlements.
Apps can only use the WebKit framework provided by Apple (WKWebView) and are not allowed to use third-party or modified WebKit engines. Otherwise, they will not be allowed on the App Store; starting from iOS 17.4, to comply with regulations, the EU region can use other Browser Engines after obtaining special permission from Apple.
If Apple doesn’t allow it, we can’t do it either.
[Unverified] Information suggests that even the iOS versions of Chrome and Firefox can only use Apple WebKit (WKWebView).
Another very important thing to note:
WKWebView runs on a separate thread outside the main app thread, so all requests and operations do not go through our app.
HTTP Cache Flow
The HTTP protocol includes a Cache protocol, and the system has already implemented a Cache mechanism in all components related to the network (URLSession, WKWebView…). Therefore, the Client App does not need to implement anything, and it is not recommended for anyone to create their own Cache mechanism. Directly following the HTTP protocol is the fastest, most stable, and most effective approach.
The general operation process of HTTP Cache is as shown in the diagram above:
- Client initiates a request.
- Server responds with Cache strategy in the Response Header. The system URLSession, WKWebView, etc., will automatically cache the response based on the Cache Header, and subsequent requests will also automatically apply this strategy.
- When requesting the same resource again, if the cache has not expired, the response will be directly retrieved from local cache in memory or disk and sent back to the app.
- If the content has expired (expiration does not mean invalid), a real network request is made to the server. If the content has not changed (still valid even if expired), the server will respond with 304 Not Modified (Empty Body). Although a network request is made, it is basically a millisecond response with no Response Body, resulting in minimal traffic consumption.
- If the content has changed, new data and Cache Header will be provided again.
In addition to local cache, there may also be network caches on Network Proxy Servers or along the way.
Common HTTP Response Cache Header parameters:
1
2
3
4
5
expires: RFC 2822 date
pragma: no-cache
# Newer parameters:
cache-control: private/public/no-store/no-cache/max-age/s-max-age/must-revalidate/proxy-revalidate...
etag: XXX
Common HTTP Request Cache Header parameters:
1
2
If-Modified-Since: 2024-07-18 13:00:00
IF-None-Match: 1234
In iOS, network-related components (URLSession, WKWebView…) handle HTTP Request/Response Cache Headers automatically and manage caching, so we do not need to handle Cache Header parameters ourselves.
For more detailed information on how HTTP Cache works, refer to “Understanding the Progressive Understanding of HTTP Cache Mechanism by Huli”.
iOS WKWebView Overview
Returning to iOS, since we can only use Apple WebKit, we can only explore ways to achieve preloading and caching through methods provided by Apple’s WebKit.
The image above provides an overview of all Apple iOS WebKit (WKWebView) related methods introduced by ChatGPT 4o, along with brief explanations. The green section pertains to methods related to data storage.
Sharing a few interesting methods:
- WKProcessPool: Allows sharing of resources, data, cookies, etc., among multiple WKWebViews.
- WKHTTPCookieStore: Manages WKWebView Cookies, cookies between WKWebViews, or URLSession Cookies within the app.
- WKWebsiteDataStore: Manages website cache files. (Read-only information and clearing)
- WKURLSchemeHandler: Registers custom Handlers to process unrecognized URL Schemes by WKWebView.
- WKContentWorld: Manages injected JavaScript (WKUserScript) scripts in groups.
- WKFindXXX: Controls page search functionality.
- WKContentRuleListStore: Implements content blockers within WKWebView (e.g., ad blocking).
Feasibility Study of Preloading Cache for iOS WKWebView
Improving HTTP Cache ✅
As introduced in the previous section on the HTTP Cache mechanism, we can ask the Web Team to enhance the HTTP Cache settings for the activity pages. On the client iOS side, we only need to check the CachePolicy setting, as everything else has been taken care of by the system!
CachePolicy Settings
URLSession:
1
2
3
let configuration = URLSessionConfiguration.default
configuration.requestCachePolicy = .useProtocolCachePolicy
let session = URLSession(configuration: configuration)
URLRequest/WKWebView:
1
2
3
4
var request = URLRequest(url: url)
request.cachePolicy = .reloadRevalidatingCacheData
//
wkWebView.load(request)
- useProtocolCachePolicy: Default, follows default HTTP Cache control.
- reloadIgnoringLocalCacheData: Does not use local cache, loads data from the network every time (but allows network, Proxy cache…).
- reloadIgnoringLocalAndRemoteCacheData: Always loads data from the network, regardless of local or remote cache.
- returnCacheDataElseLoad: Uses cached data if available, otherwise loads data from the network.
- returnCacheDataDontLoad: Only uses cached data, does not make a network request if no cached data is available.
- reloadRevalidatingCacheData: Sends a request to check if the local cache is expired, if not expired (304 Not Modified), uses cached data, otherwise reloads data from the network.
Setting Cache Size
App-wide:
1
2
3
4
5
let memoryCapacity = 512 * 1024 * 1024 // 512 MB
let diskCapacity = 10 * 1024 * 1024 * 1024 // 10 GB
let urlCache = URLCache(memoryCapacity: memoryCapacity, diskCapacity: diskCapacity, diskPath: "myCache")
URLCache.shared = urlCache
Individual URLSession:
1
2
3
4
5
6
let memoryCapacity = 512 * 1024 * 1024 // 512 MB
let diskCapacity = 10 * 1024 * 1024 * 1024 // 10 GB
let cache = URLCache(memoryCapacity: memoryCapacity, diskCapacity: diskCapacity, diskPath: "myCache")
let configuration = URLSessionConfiguration.default
configuration.urlCache = cache
Additionally, as mentioned earlier, WKWebView runs on a separate thread outside the main thread of the app, so the cache of URLRequest, URLSession is not shared with WKWebView.
How to Use Safari Developer Tools in WKWebView ?
Check if local Cache is being used.
Enable Developer Features in Safari:
Enable isInspectable in WKWebView:
1
2
3
4
5
func makeWKWebView() -> WKWebView {
let webView = WKWebView(frame: .zero)
webView.isInspectable = true // is only available in ios 16.4 or newer
return webView
}
Add webView.isInspectable = true to WKWebView to use Safari Developer Tools in Debug Build versions.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
p.s. This is my test WKWebView project opened separately
Set a breakpoint at `webView.load`.
**Start Testing:**
Build & Run:
When the execution reaches the breakpoint at `webView.load`, click "Step Over".
Go back to Safari, select "Develop" in the toolbar -> "Simulator" -> "Your Project" -> "about:blank".
- Since the page has not started loading, the URL will be about:blank.
- If about:blank does not appear, go back to XCode and click the "Step Over" button again until it appears.
Developer tools corresponding to the page will appear:
Return to XCode and click "Continue Execution":
Go back to Safari, and in the developer tools, you can see the resource loading status and full developer tools functionality (components, storage space debugging, etc.).
**If there is HTTP Cache for network resources, the transmitted size will display as "Disk":**
You can also view cache information by clicking inside.
#### Clear WKWebView Cache
```swift
// Clean Cookies
HTTPCookieStorage.shared.removeCookies(since: Date.distantPast)
// Clean Stored Data, Cache Data
let dataTypes = WKWebsiteDataStore.allWebsiteDataTypes()
let store = WKWebsiteDataStore.default()
store.fetchDataRecords(ofTypes: dataTypes) { records in
records.forEach { record in
store.removeData(
ofTypes: record.dataTypes,
for: records,
completionHandler: {
print("clearWebViewCache() - \(record)")
}
)
}
}
Use the above method to clear cached resources, local data, and cookie data in WKWebView.
However, improving HTTP Cache only achieves caching (faster on subsequent visits), and preloading (first visit) will not be affected. ✅
Improve HTTP Cache + WKWebView Preload Entire Page 😕
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class WebViewPreloader {
static let shared = WebViewPreloader()
private var _webview: WKWebView = WKWebView()
private init() { }
func preload(url: URL) {
let request = URLRequest(url: url)
Task { @MainActor in
webview.load(request)
}
}
}
WebViewPreloader.shared.preload("https://zhgchg.li/campaign/summer")
After improving HTTP Cache, the second time loading WKWebView will be cached. We can preload all the URLs in the list or homepage in advance to have them cached, making it faster for users when they enter.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
> **_After testing, it is theoretically feasible; but the performance impact and network traffic loss are too significant_** _; Users may not even go to the detailed page, but we preload all pages to feel a bit like shooting in the dark._
> _Personally, I think it is not feasible in reality, and the disadvantages outweigh the benefits, cutting off one's nose to spite one's face. 😕_
### Enhance HTTP Cache + WKWebView Preload Pure Resources 🎉
Based on the optimization method above, we can combine the HTML Link Preload method to preload only the resource files \(e.g. \.js, \.css, font, image...\) that will be used in the page, allowing users to directly use cached resources after entering without initiating network requests to fetch resource files.
> **_This means I am not preloading everything on the entire page, I am only preloading the resource files that the page will use, which may also be shared across pages; the page file \.html is still fetched from the network and combined with the preloaded files to render the page._**
Please note: We are still using HTTP Cache here, so these resources must also support HTTP Cache, otherwise, future requests will still go through the network.
```xml
<!DOCTYPE html>
<html lang="zh-tw">
<head>
<link rel="preload" href="https://cdn.zhgchg.li/dist/main.js" as="script">
<link rel="preload" href="https://image.zhgchg.li/v2/image/get/campaign.jpg" as="image">
<link rel="preload" href="https://cdn.zhgchg.li/assets/fonts/glyphicons-halflings-regular.woff2" as="font">
<link rel="preload" href="https://cdn.zhgchg.li/assets/fonts/Simple-Line-Icons.woff2?v=2.4.0" as="font">
</head>
</html>
Common supported file types:
- .js script
- .css style
- font
- image
The Web Team will place the above HTML content in the path agreed upon with the App, and our WebViewPreloader will be modified to load this path, so that WKWebView will parse <link> preload resources and generate caches while loading.
1
2
3
WebViewPreloader.shared.preload("https://zhgchg.li/campaign/summer/preload")
// or all in one
WebViewPreloader.shared.preload("https://zhgchg.li/assets/preload")
After testing, a good balance between traffic loss and preloading can be achieved . 🎉
The downside is that maintaining this cache resource list is necessary, and web optimization for page rendering and loading is still required; otherwise, the perceived time for the first page to appear will still be long.
URLProtocol ❌
Additionally, considering our old friend URLProtocol, all requests based on URL Loading System (URLSession, openURL…) can be intercepted and manipulated.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class CustomURLProtocol: URLProtocol {
override class func canInit(with request: URLRequest) -> Bool {
// Determine if this request should be handled
if let url = request.url {
return url.scheme == "custom"
}
return false
}
override class func canonicalRequest(for request: URLRequest) -> URLRequest {
// Return the request
return request
}
override func startLoading() {
// Handle the request and load data
// Change to a caching strategy, read files locally first
if let url = request.url {
let response = URLResponse(url: url, mimeType: "text/plain", expectedContentLength: -1, textEncodingName: nil)
self.client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed)
let data = "This is a custom response!".data(using: .utf8)!
self.client?.urlProtocol(self, didLoad: data)
self.client?.urlProtocolDidFinishLoading(self)
}
}
override func stopLoading() {
// Stop loading data
}
}
// AppDelegate.swift didFinishLaunchingWithOptions:
URLProtocol.registerClass(CustomURLProtocol.self)
Abstract idea is to secretly send URLRequest -> URLProtocol -> download all resources by yourself in the background, user -> WKWebView -> Request -> URLProtocol -> respond with preloaded resources.
Same as mentioned earlier, WKWebView runs on a separate thread outside the main thread of the app, so URLProtocol cannot intercept requests from WKWebView.
But I heard that using dark magic seems possible, not recommended, it may lead to other issues (rejection during review).
This path is blocked ❌.
WKURLSchemeHandler 😕
Apple introduced a new method in iOS 11, which seems to compensate for the inability of WKWebView to use URLProtocol. However, this method is similar to AVPlayer’s ResourceLoader, only system-unrecognized schemes will be handed over to our custom WKURLSchemeHandler for processing.
The abstract idea remains the same in the background, where WKWebView secretly sends Request -> WKURLSchemeHandler -> download all resources by yourself, user -> WKWebView -> Request -> WKURLSchemeHandler -> respond with preloaded resources.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import WebKit
class CustomSchemeHandler: NSObject, WKURLSchemeHandler {
func webView(_ webView: WKWebView, start urlSchemeTask: WKURLSchemeTask) {
// Custom handling
let url = urlSchemeTask.request.url!
if url.scheme == "custom-scheme" {
// Change to caching strategy, read file locally first
let response = URLResponse(url: url, mimeType: "text/html", expectedContentLength: -1, textEncodingName: nil)
urlSchemeTask.didReceive(response)
let html = "<html><body><h1>Hello from custom scheme!</h1></body></html>"
let data = html.data(using: .utf8)!
urlSchemeTask.didReceive(data)
urlSchemeTask.didFinish()
}
}
func webView(_ webView: WKWebView, stop urlSchemeTask: WKURLSchemeTask) {
// Stop
}
}
let webViewConfiguration = WKWebViewConfiguration()
webViewConfiguration.setURLSchemeHandler(CustomSchemeHandler(), forURLScheme: "mycacher")
let customURL = URL(string: "mycacher://zhgchg.li/campaign/summer")!
webView.load(URLRequest(url: customURL))
- Because http/https are schemes that the system can handle, we cannot customize the handling of http/https; you need to switch the scheme to one that the system does not recognize (e.g.,
mycacher://). - All paths in the page must use relative paths to automatically append
mycacher://for our Handler to capture. - If you do not want to change http/https but still want to access http/https requests, you can only resort to dark magic, not recommended, as it may lead to other issues (rejection during review).
- Cache page files and respond by yourself, Ajax, XMLHttpRequest, Fetch requests used in the page may be blocked by CORS Same-Origin Policy (because it will send requests from mycacher:// to http://zhgchg.li/xxx, different origins), requiring a decrease in website security to use.
- You may need to implement your own Cache Policy, such as when to update? How long is it valid? (similar to what HTTP Cache does).
Overall, while theoretically feasible, the implementation requires a huge investment; it is not cost-effective and difficult to scale and maintain stability 😕
Feeling that the WKURLSchemeHandler method is more suitable for handling web pages with large resource files that need to be downloaded, declaring a custom scheme to be processed by the app to render the web page cooperatively.
Bridging WKWebView network requests to be sent by the app 🫥
Change WKWebView to use the interface defined by the app (WkUserScript) instead of Ajax, XMLHttpRequest, Fetch, for the app to request resources.
This example is not very helpful because the first screen appears too slow, not the subsequent loading; and this method will cause a deep and strange dependency relationship between Web and App 🫥
Starting from Service Worker ❌
Due to security issues, only Apple’s own Safari app supports it, WKWebView does not support it❌.
WKWebView Performance Optimization 🫥
Optimize to improve the performance of loading views in WKWebView.
WKWebView itself is like a skeleton, and the web page is the flesh. After researching, optimizing the skeleton (e.g. reusing WKProcessPool) has limited effect, possibly a difference of 0.0003 -> 0.000015 seconds.
Local HTML, Local Resource Files 🫥
Similar to the Preload method, but instead of putting the active page in the App Bundle or fetching it remotely at startup.
Putting the entire HTML page may also encounter CORS same-origin issues; it feels like using the “Improve HTTP Cache + WKWebView Preload pure resources” method instead; putting it in the App Bundle only increases the App Size, fetching it remotely is WKWebView Preload 🫥
Frontend Optimization Approach 🎉🎉🎉
{:target="_blank"}](/assets/5033090c18ba/1*Y3nDpbc4aEd0wg7Enk4k8A.png)
Reference wedevs optimization suggestions, the frontend HTML page is expected to have four loading stages, from loading the page file (.html) at the beginning to First Paint (blank page), then to First Contentful Paint (rendering the page skeleton), then to First Meaningful Paint (adding page content), and finally to Time To Interactive (allowing user interaction).
Using our page for testing; browsers, WKWebView will first request the page body .html and then load the required resources, while building the screen for the user according to the program instructions. Comparing with the article, it is found that the page stages only go from First Paint (blank) to Time To Interactive (First Contentful Paint only has the Navigation Bar, which should not count much…), missing the intermediate stages of rendering for the user, thus extending the overall waiting time for the user.
And currently, only resource files have HTTP Cache settings, not the page body.
Additionally, you can refer to Google PageSpeed Insights for optimization suggestions, such as compression, reducing script size, etc.
Because the core of in-app WKWebView is still the web page itself; therefore, adjusting from the frontend web page is a very effective way to make a big difference with a small adjustment. 🎉🎉🎉
Improving User Experience 🎉🎉🎉
A simple implementation, starting from the user experience, adding a Loading Progress Bar, not just showing a blank page to confuse the user, let them know that the page is loading and where the progress is.🎉🎉🎉
Conclusion
The above is some ideation research on feasible solutions for WKWebView preloading and caching. The technology is not the biggest issue, the key is still the choice, which ways are most effective for users with the lowest development cost. Choosing these ways may achieve the goal directly with minor changes; choosing the wrong way will result in a huge investment of resources and may be difficult to maintain and use in the future.
There are always more solutions than difficulties, sometimes it’s just a lack of imagination.
Maybe there are some legendary combinations that I haven’t thought of, welcome everyone to contribute.
References
WKWebView Preload Pure Resource🎉 Solution can refer to the following video
The author also mentioned the method of WKURLSchemeHandler.
The complete Demo Repo in the video is as follows:
iOS Old Driver Weekly
The sharing about WkWebView in the Old Driver Weekly is also worth a look.
Chat
Long-awaited return to writing long articles related to iOS development.
If you have any questions or suggestions, feel free to contact me.
===
===
This article was first published in Traditional Chinese on Medium ➡️ View Here
