Real-World Codable Decoding Issues (Part 2)

Real-World Codable Decoding Issues (Part 2)

Handling Response Null Fields Reasonably, No Need to Always Rewrite init decoder

Photo by Zan

Introduction

Following the previous article “Real-World Codable Decoding Issues”, as development progresses, new scenarios and problems have emerged. Hence, this part continues to document the encountered situations and research insights for future reference.

The previous part mainly solved the JSON String -> Entity Object Decodable Mapping. Once we have the Entity Object, we can convert it into a Model Object for use within the program, View Model Object for handling data display logic, etc. On the other hand, we need to convert the Entity into NSManagedObject to store it in local Core Data.

Main Issue

Assume our song Entity structure is as follows:

struct Song: Decodable {
    var id: Int
    var name: String?
    var file: String?
    var coverImage: String?
    var likeCount: Int?
    var like: Bool?
    var length: Int?
}

Since the API EndPoint may not always return complete data fields (only id is guaranteed), all fields except id are Optional. For example, when fetching song information, a complete structure is returned, but when liking a song, only the id, likeCount, and like fields related to the change are returned.

We hope that whatever fields the API Response contains can be stored in Core Data. If the data already exists, update the changed fields (incremental update).

But here lies the problem: After Codable Decoding into an Entity Object, we cannot distinguish between “the data field is intended to be set to nil” and “the Response did not provide it”

A Response:
{
  "id": 1,
  "file": null
}

For A Response and B Response, the file is null, but the meanings are different; A intends to set the file field to null (clear the original data), while B intends to update other data and simply did not provide the file field.

A developer in the Swift community proposed adding a null Strategy similar to date Strategy in JSONDecoder, allowing us to distinguish the above situations, but there are currently no plans to include it.

Solution

As mentioned earlier, our architecture is JSON String -> Entity Object -> NSManagedObject, so when we get the Entity Object, it is already the result after decoding, and there is no raw data to operate on; of course, we can use the original JSON String for comparison, but it would be better not to use Codable in that case.

First, refer to the previous article to use Associated Value Enum as a container to hold values.

enum OptionalValue<T: Decodable>: Decodable {
    case null
    case value(T)
    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        if let value = try? container.decode(T.self) {
            self = .value(value)
        } else {
            self = .null
        }
    }
}

Using generics, T is the actual data field type; .value(T) can hold the decoded value, and .null represents that the value is null.

struct Song: Decodable {
    enum CodingKeys: String, CodingKey {
        case id
        case file
    }
    
    var id: Int
    var file: OptionalValue<String>?
    
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        
        self.id = try container.decode(Int.self, forKey: .id)
        
        if container.contains(.file) {
            self.file = try container.decode(OptionalValue<String>.self, forKey: .file)
        } else {
            self.file = nil
        }
    }
}

var jsonData = """
{
    "id":1
}
""".data(using: .utf8)!
var result = try! JSONDecoder().decode(Song.self, from: jsonData)
print(result)

jsonData = """
{
    "id":1,
    "file":null
}
""".data(using: .utf8)!
result = try! JSONDecoder().decode(Song.self, from: jsonData)
print(result)

jsonData = """
{
    "id":1,
    "file":"https://test.com/m.mp3"
}
""".data(using: .utf8)!
result = try! JSONDecoder().decode(Song.self, from: jsonData)
print(result)

The example is simplified to only include id and file data fields.

The Song Entity implements its own decoding method, using the contains(.KEY) method to determine whether the response includes the field (regardless of its value). If it does, it decodes it into OptionalValue; within the OptionalValue Enum, it decodes the actual value we want. If the value is successfully decoded, it is placed in .value(T); if the value is null (or decoding fails), it is placed in .null.

1. When the response includes the field and value: OptionalValue.value(VALUE)
2. When the response includes the field and the value is null: OptionalValue.null
3. When the response does not include the field: nil

This way, we can distinguish whether the field is provided or not, and when writing to Core Data, we can determine whether to update the field to null or not update this field at all.

Other Research — Double Optional ❌

Optional!Optional! is quite suitable for handling this scenario in Swift.

struct Song: Decodable {
    var id: Int
    var name: String??
    var file: String??
    var converImage: String??
    var likeCount: Int??
    var like: Bool??
    var length: Int??
}

1. When the response provides the field & value: Optional(VALUE)
2. When the response provides the field & the value is null: Optional(nil)
3. When the response does not provide the field: nil

However… Codable JSONDecoder Decode handles both Double Optional and Optional with decodeIfPresent, treating both as Optional without special handling for Double Optional; so the result remains the same as before.

Other Research — Property Wrapper ❌

Initially, it was thought that Property Wrapper could be used for elegant encapsulation, such as:

@OptionalValue var file: String?

But before delving into the details, it was found that Codable Property fields marked with Property Wrapper require the API response to have that field, otherwise, a keyNotFound error will occur, even if the field is Optional. ?????

There is also a discussion thread on the official forum regarding this issue… It is estimated that this will be fixed in the future.

Therefore, when choosing packages like BetterCodable or CodableWrappers, consider the current issue with Property Wrapper.

Other Problem Scenarios

1. API Response uses 0/1 to represent Bool, how to Decode?

import Foundation

struct Song: Decodable {
    enum CodingKeys: String, CodingKey {
        case id
        case name
        case like
    }
    
    var id: Int
    var name: String?
    var like: Bool?
    
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        self.id = try container.decode(Int.self, forKey: .id)
        self.name = try container.decodeIfPresent(String.self, forKey: .name)
        
        if let intValue = try container.decodeIfPresent(Int.self, forKey: .like) {
            self.like = (intValue == 1) ? true : false
        } else if let boolValue = try container.decodeIfPresent(Bool.self, forKey: .like) {
            self.like = boolValue
        }
    }
}

var jsonData = """
{
    "id": 1,
    "name": "告五人",
    "like": 0
}
""".data(using: .utf8)!
var result = try! JSONDecoder().decode(Song.self, from: jsonData)
print(result)

Extending the previous section, we can initialize Decode in init and decode it into int/Bool, then assign it ourselves. This way, we can extend the original fields to accept 0/1/true/false.

2. Don’t want to rewrite the init decoder every time

If you don’t want to create your own Decoder, you can override the original JSON Decoder to add more functionality.

We can extend KeyedDecodingContainer and define public methods ourselves. Swift will prioritize executing the methods we redefine under the module, overriding the original Foundation implementation.

This affects the entire module.

And it’s not a true override, you can’t call super.decode, and be careful not to call yourself (e.g., decode(Bool.Type, forKey) in decode(Bool.Type, forKey)).

There are two decode methods:

• decode(Type, forKey:) handles non-Optional data fields
• decodeIfPresent(Type, forKey:) handles Optional data fields

Example 1. The main issue mentioned earlier can be directly extended:

extension KeyedDecodingContainer {
    public func decodeIfPresent<T>(_ type: T.Type, forKey key: Self.Key) throws -> T? where T : Decodable {
        //better:
        switch type {
        case is OptionalValue<String>.Type,
             is OptionalValue<Int>.Type:
            return try? decode(type, forKey: key)
        default:
            return nil
        }
        // or just return try? decode(type, forKey: key)
    }
}

struct Song: Decodable {
    var id: Int
    var file: OptionalValue<String>?
}

Since the main issue is with Optional data fields and Decodable types, we override the decodeIfPresent<T: Decodable> method.

It is speculated that the original implementation of decodeIfPresent returns nil if the data is null or the response does not provide it, without actually running decode.

So the principle is simple: as long as the Decodable Type is OptionValue<T>, it will try to decode regardless, allowing us to get different state results. But actually, not judging the Decodable Type also works, meaning all Optional fields will try to decode.

Example 2. Problem scenario 1 can also be extended using this method:

extension KeyedDecodingContainer {
    public func decodeIfPresent(_ type: Bool.Type, forKey key: KeyedDecodingContainer<K>.Key) throws -> Bool? {
        if let intValue = try? decodeIfPresent(Int.self, forKey: key) {
            return (intValue == 1) ? (true) : (false)
        } else if let boolValue = try? decodeIfPresent(Bool.self, forKey: key) {
            return boolValue
        }
        return nil
    }
}

struct Song: Decodable {
    enum CodingKeys: String, CodingKey {
        case id
        case name
        case like
    }
    
    var id: Int
    var name: String?
    var like: Bool?
}

var jsonData = """
{
    "id": 1,
    "name": "告五人",
    "like": 1
}
""".data(using: .utf8)!
var result = try! JSONDecoder().decode(Song.self, from: jsonData)
print(result)

Conclusion

Codable has been used in various tricky ways, some of which are quite convoluted because Codable’s constraints are too strong, sacrificing much of the flexibility needed in real-world development. In the end, you might even start to question why you chose Codable in the first place, as the advantages seem to diminish…

References

• You might not need to override the init(from:) method

Review

• Summary of Decode Issues Encountered in Real-World Use of Codable (Part 1)

If you have any questions or feedback, feel free to contact me.

===

本文中文版本

===

This article was first published in Traditional Chinese on Medium ➡️ View Here