iOS: Insuring Your Multilingual Strings!

iOS: Insuring Your Multilingual Strings!

Using SwifGen & UnitTest to ensure the safety of multilingual operations

Photo by Mick Haupt

Problem

Plain Text Files

iOS handles multilingual support through Localizable.strings plain text files, unlike Android which uses XML format for management. This means there is a risk of accidentally corrupting or missing language files during daily development. Additionally, multilingual errors are not detected at Build Time and are often only discovered after release when users from a specific region report issues, significantly reducing user confidence.

A previous painful experience involved forgetting to add ; in Localizable.strings due to being too accustomed to Swift. This caused all subsequent strings in a particular language to break after release. An urgent hotfix was needed to resolve the issue.

If there is an issue with multilingual support, the Key will be displayed directly to the user

As shown above, if the DESCRIPTION Key is missing, the app will directly display DESCRIPTION to the user.

Inspection Requirements

• Ensure the correct format of Localizable.strings (each line must end with ;, valid Key-Value pairs)
• All multilingual Keys used in the code must have corresponding definitions in Localizable.strings
• Each language in Localizable.strings must have corresponding Key-Value records
• Localizable.strings must not have duplicate Keys (otherwise, Values may be accidentally overwritten)

Solution

Using Swift to Write a Comprehensive Inspection Tool

The previous approach was to “ Use Swift to Write Shell Scripts Directly in Xcode! “ referencing the Localize 🏁 tool to develop a Command Line Tool in Swift for external multilingual file inspection. The script was then placed in Build Phases Run Script to perform checks at Build Time.

Advantages: The inspection program is injected externally, not dependent on the project. It can be executed without XCode or building the project, and can pinpoint the exact line in a file where the issue occurs. Additionally, it can perform formatting functions (sorting multilingual Keys A-Z).

Disadvantages: Increases Build Time (~+3 mins), process divergence, and scripts are difficult to maintain or adjust according to project structure. Since this part is not within the project, only the person who added this inspection knows the entire logic, making it hard for other collaborators to touch this part.

Interested readers can refer to the previous article. This article mainly introduces how to achieve all the inspection functions of Localizable.strings through XCode 13 + SwiftGen + UnitTest.

XCode 13 Built-in Build Time Check for Localizable.strings File Format Correctness

After upgrading to XCode 13, it comes with a built-in Build Time check for the Localizable.strings file format. The check is quite comprehensive, and besides missing ;, it will also catch any extra meaningless strings.

Using SwiftGen to Replace the Original NSLocalizedString String Base Access Method

SwiftGen helps us convert the original NSLocalizedString String access method to Object access, preventing typos and missing Key declarations.

SwiftGen is also a Command Line Tool; however, this tool is quite popular in the industry and has comprehensive documentation and community resources for maintenance. There is no need to worry about maintenance issues after introducing this tool.

Installation

You can choose the installation method according to your environment or CI/CD service settings. Here, we will use CocoaPods for a straightforward installation.

Please note that SwiftGen is not really a CocoaPod; it will not have any dependencies on the project’s code. Using CocoaPods to install SwiftGen is simply to download this Command Line Tool executable.

Add the swiftgen pod to the podfile:

pod 'SwiftGen', '~> 6.0'

Init

After pod install, open Terminal and cd to the project directory

/L10NTests/Pods/SwiftGen/bin/swiftGen config init

Initialize the swiftgen.yml configuration file and open it

strings:
  - inputs:
      - "L10NTests/Supporting Files/zh-Hant.lproj/Localizable.strings"
    outputs:
      templateName: structured-swift5
      output: "L10NTests/Supporting Files/SwiftGen-L10n.swift"
      params:
        enumName: "L10n"

Paste and modify it to fit your project’s format:

inputs: Project localization file location (it is recommended to specify the localization file of the DevelopmentLocalization language)

outputs: output: The location of the converted swift file params: enumName: Object name templateName: Conversion template

You can use swiftGen template list to get the list of built-in templates

flat v.s. structured

The difference is that if the Key style is XXX.YYY.ZZZ, the flat template will convert it to camelCase; the structured template will convert it to XXX.YYY.ZZZ object according to the original style.

Pure Swift projects can directly use the built-in templates, but if it is a Swift mixed with OC project, you need to customize the template:

flat-swift5-objc.stencil:

// swiftlint:disable all
// Generated using SwiftGen — https://github.com/SwiftGen/SwiftGen

{% if tables.count > 0 %}
{% set accessModifier %}{% if param.publicAccess %}public{% else %}internal{% endif %}{% endset %}
import Foundation

// swiftlint:disable superfluous_disable_command file_length implicit_return

// MARK: - Strings

{% macro parametersBlock types %}{% filter removeNewlines:"leading" %}
  {% for type in types %}
    {% if type == "String" %}
    _ p{{forloop.counter}}: Any
    {% else %}
    _ p{{forloop.counter}}: {{type}}
    {% endif %}
    {{ ", " if not forloop.last }}
  {% endfor %}
{% endfilter %}{% endmacro %}
{% macro argumentsBlock types %}{% filter removeNewlines:"leading" %}
  {% for type in types %}
    {% if type == "String" %}
    String(describing: p{{forloop.counter}})
    {% elif type == "UnsafeRawPointer" %}
    Int(bitPattern: p{{forloop.counter}})
    {% else %}
    p{{forloop.counter}}
    {% endif %}
    {{ ", " if not forloop.last }}
  {% endfor %}
{% endfilter %}{% endmacro %}
{% macro recursiveBlock table item %}
  {% for string in item.strings %}
  {% if not param.noComments %}
  {% for line in string.translation|split:"\n" %}
  /// {{line}}
  {% endfor %}
  {% endif %}
  {% if string.types %}
  {{accessModifier}} static func {{string.key|swiftIdentifier:"pretty"|lowerFirstWord|escapeReservedKeywords}}({% call parametersBlock string.types %}) -> String {
    return {{enumName}}.tr("{{table}}", "{{string.key}}", {% call argumentsBlock string.types %})
  }
  {% elif param.lookupFunction %}
  {# custom localization function is mostly used for in-app lang selection, so we want the loc to be recomputed at each call for those (hence the computed var) #}
  {{accessModifier}} static var {{string.key|swiftIdentifier:"pretty"|lowerFirstWord|escapeReservedKeywords}}: String { return {{enumName}}.tr("{{table}}", "{{string.key}}") }
  {% else %}
  {{accessModifier}} static let {{string.key|swiftIdentifier:"pretty"|lowerFirstWord|escapeReservedKeywords}} = {{enumName}}.tr("{{table}}", "{{string.key}}")
  {% endif %}
  {% endfor %}
  {% for child in item.children %}
  {% call recursiveBlock table child %}
  {% endfor %}
{% endmacro %}
// swiftlint:disable function_parameter_count identifier_name line_length type_body_length
{% set enumName %}{{param.enumName|default:"L10n"}}{% endset %}
@objcMembers {{accessModifier}} class {{enumName}}: NSObject {
  {% if tables.count > 1 or param.forceFileNameEnum %}
  {% for table in tables %}
  {{accessModifier}} enum {{table.name|swiftIdentifier:"pretty"|escapeReservedKeywords}} {
    {% filter indent:2 %}{% call recursiveBlock table.name table.levels %}{% endfilter %}
  }
  {% endfor %}
  {% else %}
  {% call recursiveBlock tables.first.name tables.first.levels %}
  {% endif %}
}
// swiftlint:enable function_parameter_count identifier_name line_length type_body_length

// MARK: - Implementation Details

extension {{enumName}} {
  private static func tr(_ table: String, _ key: String, _ args: CVarArg...) -> String {
    {% if param.lookupFunction %}
    let format = {{ param.lookupFunction }}(key, table)
    {% else %}
    let format = {{param.bundle|default:"BundleToken.bundle"}}.localizedString(forKey: key, value: nil, table: table)
    {% endif %}
    return String(format: format, locale: Locale.current, arguments: args)
  }
}
{% if not param.bundle and not param.lookupFunction %}

// swiftlint:disable convenience_type
private final class BundleToken {
  static let bundle: Bundle = {
    #if SWIFT_PACKAGE
    return Bundle.module
    #else
    return Bundle(for: BundleToken.self)
    #endif
  }()
}
// swiftlint:enable convenience_type
{% endif %}
{% else %}
// No string found
{% endif %}

The above provides a template collected from the internet and customized to be compatible with Swift and Objective-C. You can create a flat-swift5-objc.stencil file and paste the content or click here to download the .zip.

If you use a custom template, you won’t use templateName, but instead declare templatePath:

swiftgen.yml:

strings:
  - inputs:
      - "L10NTests/Supporting Files/zh-Hant.lproj/Localizable.strings"
    outputs:
      templatePath: "path/to/flat-swift5-objc.stencil"
      output: "L10NTests/Supporting Files/SwiftGen-L10n.swift"
      params:
        enumName: "L10n"

Specify the templatePath to the location of the .stencil template in the project.

Generator

After setting it up, you can manually run in Terminal:

/L10NTests/Pods/SwiftGen/bin/swiftGen

Execute the conversion. After the first conversion, manually drag the converted result file (SwiftGen-L10n.swift) from Finder into the project so the program can use it.

Run Script

In the project settings -> Build Phases -> + -> New Run Script Phases -> paste:

if [[ -f "${PODS_ROOT}/SwiftGen/bin/swiftgen" ]]; then
  echo "${PODS_ROOT}/SwiftGen/bin/swiftgen"
  "${PODS_ROOT}/SwiftGen/bin/swiftgen"
else
  echo "warning: SwiftGen is not installed. Run 'pod install --repo-update' to install it."
fi

This way, the generator will run and produce the latest conversion results every time the project is built.

How to use in CodeBase?

L10n.homeTitle
L10n.homeDescription("ZhgChgLi") // with arg

With Object Access, there will be no typos, and keys used in the code but not declared in the Localizable.strings file will not occur.

However, SwiftGen can only generate from a specific language, so it cannot prevent the situation where a key exists in the generated language but is forgotten in other languages. This situation can only be protected by the following UnitTest.

Conversion

Conversion is the most challenging part of this issue because a project that has already been developed extensively uses NSLocalizedString. Converting it to the new L10n.XXX format is complex, especially for sentences with parameters String(format: NSLocalizedString. Additionally, if Objective-C is mixed in, you must consider the different syntax between Objective-C and Swift.

There is no special solution; you can only write a Command Line Tool yourself. Refer to the previous article on using Swift to scan the project directory and parse NSLocalizedString with Regex to write a small tool for conversion.

It is recommended to convert one scenario at a time, ensuring it can build before converting the next one.

• Swift -> NSLocalizedString without parameters
• Swift -> NSLocalizedString with parameters
• Objective-C -> NSLocalizedString without parameters
• Objective-C -> NSLocalizedString with parameters

Use UnitTest to check for missing or duplicate keys in each language file compared to the main language file

We can write UniTest to read the contents of the .strings file from the Bundle and test it.

Read .strings from Bundle and convert to object:

class L10NTestsTests: XCTestCase {
    
    private var localizations: [Bundle: [Localization]] = [:]
    
    override func setUp() {
        super.setUp()
        
        let bundles = [Bundle(for: type(of: self))]
        
        //
        bundles.forEach { bundle in
            var localizations: [Localization] = []
            
            bundle.localizations.forEach { lang in
                var localization = Localization(lang: lang)
                
                if let lprojPath = bundle.path(forResource: lang, ofType: "lproj"),
                   let lprojBundle = Bundle(path: lprojPath) {
                    
                    let filesInLPROJ = (try? FileManager.default.contentsOfDirectory(atPath: lprojBundle.bundlePath)) ?? []
                    localization.localizableStringFiles = filesInLPROJ.compactMap { fileFullName -> L10NTestsTests.Localization.LocalizableStringFile? in
                        let fileName = URL(fileURLWithPath: fileFullName).deletingPathExtension().lastPathComponent
                        let fileExtension = URL(fileURLWithPath: fileFullName).pathExtension
                        guard fileExtension == "strings" else { return nil }
                        guard let path = lprojBundle.path(forResource: fileName, ofType: fileExtension) else { return nil }
                        
                        return L10NTestsTests.Localization.LocalizableStringFile(name: fileFullName, path: path)
                    }
                    
                    localization.localizableStringFiles.enumerated().forEach { (index, localizableStringFile) in
                        if let fileContent = try? String(contentsOfFile: localizableStringFile.path, encoding: .utf8) {
                            let lines = fileContent.components(separatedBy: .newlines)
                            let pattern = "\"(.*)\"(\\s*)(=){1}(\\s*)\"(.+)\";"
                            let regex = try? NSRegularExpression(pattern: pattern, options: [])
                            let values = lines.compactMap { line -> Localization.LocalizableStringFile.Value? in
                                let range = NSRange(location: 0, length: (line as NSString).length)
                                guard let matches = regex?.firstMatch(in: line, options: [], range: range) else { return nil }
                                let key = (line as NSString).substring(with: matches.range(at: 1))
                                let value = (line as NSString).substring(with: matches.range(at: 5))
                                return Localization.LocalizableStringFile.Value(key: key, value: value)
                            }
                            localization.localizableStringFiles[index].values = values
                        }
                    }
                    
                    localizations.append(localization)
                }
            }
            
            self.localizations[bundle] = localizations
        }
    }
}

private extension L10NTestsTests {
    struct Localization: Equatable {
        struct LocalizableStringFile {
            struct Value {
                let key: String
                let value: String
            }
            
            let name: String
            let path: String
            var values: [Value] = []
        }
        
        let lang: String
        var localizableStringFiles: [LocalizableStringFile] = []
        
        static func == (lhs: Self, rhs: Self) -> Bool {
            return lhs.lang == rhs.lang
        }
    }
}

We defined a Localization to store the parsed data, find lproj from the Bundle, then find .strings from it, and then use regular expressions to convert multilingual sentences into objects and put them back into Localization for subsequent testing.

Here are a few things to note:

• Use Bundle(for: type(of: self)) to get resources from the Test Target
• Remember to set the STRINGS_FILE_OUTPUT_ENCODING of the Test Target to UTF-8, otherwise, reading the file content using String will fail (the default is Binary)
• The reason for using String to read instead of NSDictionary is that we need to test for duplicate Keys, and using NSDictionary will overwrite duplicate Keys when reading
• Remember to add the .strings File to the Test Target

TestCase 1. Test for duplicate Keys in the same .strings file:

func testNoDuplicateKeysInSameFile() throws {
    localizations.forEach { (_, localizations) in
        localizations.forEach { localization in
            localization.localizableStringFiles.forEach { localizableStringFile in
                let keys = localizableStringFile.values.map { $0.key }
                let uniqueKeys = Set(keys)
                XCTAssertTrue(keys.count == uniqueKeys.count, "Localized Strings File: \(localizableStringFile.path) has duplicated keys.")
            }
        }
    }
}

Input:

Result:

TestCase 2. Compare with DevelopmentLocalization language to check for missing/redundant Keys:

func testCompareWithDevLangHasMissingKey() throws {
    localizations.forEach { (bundle, localizations) in
        let developmentLang = bundle.developmentLocalization ?? "en"
        if let developmentLocalization = localizations.first(where: { $0.lang == developmentLang }) {
            let othersLocalization = localizations.filter { $0.lang != developmentLang }
            
            developmentLocalization.localizableStringFiles.forEach { developmentLocalizableStringFile in
                let developmentLocalizableKeys = Set(developmentLocalizableStringFile.values.map { $0.key })
                othersLocalization.forEach { otherLocalization in
                    if let otherLocalizableStringFile = otherLocalization.localizableStringFiles.first(where: { $0.name == developmentLocalizableStringFile.name }) {
                        let otherLocalizableKeys = Set(otherLocalizableStringFile.values.map { $0.key })
                        if developmentLocalizableKeys.count < otherLocalizableKeys.count {
                            XCTFail("Localized Strings File: \(otherLocalizableStringFile.path) has redundant keys.")
                        } else if developmentLocalizableKeys.count > otherLocalizableKeys.count {
                            XCTFail("Localized Strings File: \(otherLocalizableStringFile.path) has missing keys.")
                        }
                    } else {
                        XCTFail("Localized Strings File not found in Lang: \(otherLocalization.lang)")
                    }
                }
            }
        } else {
            XCTFail("developmentLocalization not found in Bundle: \(bundle)")
        }
    }
}

Input: (Compared to DevelopmentLocalization, other languages lack the declaration Key)

Output:

Input: (DevelopmentLocalization does not have this Key, but it appears in other languages)

Output:

Summary

Combining the above methods, we use:

• The new version of XCode to ensure the correctness of the .strings file format ✅
• SwiftGen to ensure that the CodeBase does not reference multilingual content incorrectly or without declaration ✅
• UnitTest to ensure the correctness of multilingual content ✅

Advantages:

• Fast execution speed, does not slow down Build Time
• Maintained by all iOS developers

Advanced

Localized File Format

This solution cannot be achieved, and the original Command Line Tool written in Swift is still needed. However, the Format part can be done in git pre-commit; if there is no diff adjustment, it will not be done to avoid running once every build:

#!/bin/sh

diffStaged=${1:-\-\-staged} # use $1 if exist, default --staged.

git diff --diff-filter=d --name-only $diffStaged | grep -e 'Localizable.*\.\(strings\|stringsdict\)$' | \
  while read line; do
    // do format for ${line}
done

.stringdict

The same principle can be applied to .stringdict

CI/CD

swiftgen does not need to be placed in the build phase, as it runs every build, and the code appears only after the build is complete. It can be changed to generate the command only when there are adjustments.

Clearly identify which Key is wrong

The UnitTest program can be optimized to output clearly which Key is Missing/Redundant/Duplicate.

Use third-party tools to completely free engineers from multilingual work

As mentioned in the previous talk “ 2021 Pinkoi Tech Career Talk — High-Efficiency Engineering Team Unveiled “, in large teams, multilingual work can be separated through third-party services, reducing the dependency on multilingual work.

Engineers only need to define the Key, and multilingual content will be automatically imported from the platform during the CI/CD stage, reducing the manual maintenance phase and making it less prone to errors.

Special Thanks

Wei Cao , iOS Developer @ Pinkoi

For any questions or comments, feel free to contact me.

===

本文中文版本

===

This article was first published in Traditional Chinese on Medium ➡️ View Here