Encode nil value as null with JSONEncoder

Question

I'm using Swift 4's JSONEncoder. I have a Codable struct with an optional property, and I'd like this property to show up as null value in the produced JSON data when the value is nil. However, JSONEncoder discards the property and does not add it to the JSON output. Is there a way to configure JSONEncoder so that it preserves the key and sets it to null in this case?

Example

The code snippet below produces {"number":1}, but I'd rather like it to give me {"string":null,"number":1}:

struct Foo: Codable {
  var string: String? = nil
  var number: Int = 1
}

let encoder = JSONEncoder()
let data = try! encoder.encode(Foo())
print(String(data: data, encoding: .utf8)!)

Solution 1

Yes, but you'll have to write your own encode(to:) implementation, you can't use the auto-generated one.

struct Foo: Codable {
    var string: String? = nil
    var number: Int = 1

    func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(number, forKey: .number)
        try container.encode(string, forKey: .string)
    }
}

Encoding an optional directly will encode a null, like you're looking for.

If this is an important use case for you, you may consider opening a defect at bugs.swift.org to ask for a new OptionalEncodingStrategy flag to be added on JSONEncoder to match the existing DateEncodingStrategy, etc. (See below why this is likely impossible to actually implement in Swift today, but getting into the tracking system is still useful as Swift evolves.)

---

Edit: To Paulo's questions below, this dispatches to the generic encode<T: Encodable> version because Optional conforms to Encodable. This is implemented in Codable.swift this way:

extension Optional : Encodable /* where Wrapped : Encodable */ {
    @_inlineable // FIXME(sil-serialize-all)
    public func encode(to encoder: Encoder) throws {
        assertTypeIsEncodable(Wrapped.self, in: type(of: self))

        var container = encoder.singleValueContainer()
        switch self {
        case .none: try container.encodeNil()
        case .some(let wrapped): try (wrapped as! Encodable).__encode(to: &container)
        }
    }
}

This wraps the call to encodeNil, and I think letting stdlib handle Optionals as just another Encodable is better than treating them as a special case in our own encoder and calling encodeNil ourselves.

Another obvious question is why it works this way in the first place. Since Optional is Encodable, and the generated Encodable conformance encodes all the properties, why does "encode all the properties by hand" work differently? The answer is that the conformance generator includes a special case for Optionals:

// Now need to generate `try container.encode(x, forKey: .x)` for all
// existing properties. Optional properties get `encodeIfPresent`.
...

if (varType->getAnyNominal() == C.getOptionalDecl() ||
    varType->getAnyNominal() == C.getImplicitlyUnwrappedOptionalDecl()) {
  methodName = C.Id_encodeIfPresent;
}

This means that changing this behavior would require changing the auto-generated conformance, not JSONEncoder (which also means it's probably really hard to make configurable in today's Swift....)

Solution 2

Here's an approach that uses a property wrapper (requires Swift v5.1):

@propertyWrapper
struct NullEncodable<T>: Encodable where T: Encodable {
    
    var wrappedValue: T?

    init(wrappedValue: T?) {
        self.wrappedValue = wrappedValue
    }
    
    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        switch wrappedValue {
        case .some(let value): try container.encode(value)
        case .none: try container.encodeNil()
        }
    }
}

Sample usage:

struct Tuplet: Encodable {
    let a: String
    let b: Int
    @NullEncodable var c: String? = nil
}

struct Test: Encodable {
    @NullEncodable var name: String? = nil
    @NullEncodable var description: String? = nil
    @NullEncodable var tuplet: Tuplet? = nil
}

var test = Test()
test.tuplet = Tuplet(a: "whee", b: 42)
test.description = "A test"

let data = try JSONEncoder().encode(test)
print(String(data: data, encoding: .utf8) ?? "")

Output:

{
  "name": null,
  "description": "A test",
  "tuplet": {
    "a": "whee",
    "b": 42,
    "c": null
  }
}

Full implementation here: https://github.com/g-mark/NullCodable

Solution 3

I am using this enum to control the behavior. It was required by our backend:

public enum Tristate<Wrapped> : ExpressibleByNilLiteral, Encodable {

/// Null
case none

/// The presence of a value, stored as `Wrapped`.
case some(Wrapped)

/// Pending value, not none, not some
case pending

/// Creates an instance initialized with .pending.
public init() {
    self = .pending
}

/// Creates an instance initialized with .none.
public init(nilLiteral: ()) {
    self = .none
}

/// Creates an instance that stores the given value.
public init(_ some: Wrapped) {
    self = .some(some)
}

public func encode(to encoder: Encoder) throws {
    var container = encoder.singleValueContainer()
    switch self {
        case .none:
            try container.encodeNil()
        case .some(let wrapped):
            try (wrapped as! Encodable).encode(to: encoder)
        case .pending: break // do nothing
    }
}

}

typealias TriStateString = Tristate<String>
typealias TriStateInt = Tristate<Int>
typealias TriStateBool = Tristate<Bool>

/// Test

struct TestStruct: Encodable {
var variablePending: TriStateString?
var variableSome: TriStateString?
var variableNil: TriStateString?

}

    /// Structure with tristate strings:
    let testStruc = TestStruct(/*variablePending: TriStateString(),*/ // pending, unresolved
                               variableSome: TriStateString("test"), // some, resolved
                               variableNil: TriStateString(nil)) // nil, resolved

    /// Make the structure also tristate
    let tsStruct = Tristate<TestStruct>(testStruc)

    /// Make a json from the structure
    do {
        let jsonData = try JSONEncoder().encode(tsStruct)
        print( String(data: jsonData, encoding: .utf8)! )
    } catch(let e) {
        print(e)
    }

/// Output

{"variableNil":null,"variableSome":"test"}

// variablePending is missing, which is a correct behaviour

Solution 4

Here is an approach we have used in a project. Hope it helps.

struct CustomBody: Codable {
    let method: String
    let params: [Param]

    enum CodingKeys: String, CodingKey {
        case method = "method"
        case params = "params"
    }
}

enum Param: Codable {
    case bool(Bool)
    case integer(Int)
    case string(String)
    case stringArray([String])
    case valueNil
    case unsignedInteger(UInt)
    case optionalString(String?)

    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        if let x = try? container.decode(Bool.self) {
            self = .bool(x)
            return
        }
        if let x = try? container.decode(Int.self) {
            self = .integer(x)
            return
        }
        if let x = try? container.decode([String].self) {
              self = .stringArray(x)
              return
          }
        if let x = try? container.decode(String.self) {
            self = .string(x)
            return
        }
        if let x = try? container.decode(UInt.self) {
            self = .unsignedInteger(x)
            return
        }
        throw DecodingError.typeMismatch(Param.self, DecodingError.Context(codingPath: decoder.codingPath, debugDescription: "Wrong type for Param"))
    }

    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        switch self {
        case .bool(let x):
            try container.encode(x)
        case .integer(let x):
            try container.encode(x)
        case .string(let x):
            try container.encode(x)
        case .stringArray(let x):
            try container.encode(x)
        case .valueNil:
            try container.encodeNil()
        case .unsignedInteger(let x):
            try container.encode(x)
        case .optionalString(let x):
            x?.isEmpty == true ? try container.encodeNil() : try container.encode(x)
        }
    }
}

And the usage is something like this.

RequestBody.CustomBody(method: "WSDocMgmt.getDocumentsInContentCategoryBySearchSource", params: [.string(legacyToken), .string(shelfId), .bool(true), .valueNil, .stringArray(queryFrom(filters: filters ?? [])), .optionalString(sortMethodParameters()), .bool(sortMethodAscending()), .unsignedInteger(segment ?? 0), .unsignedInteger(segmentSize ?? 0), .string("NO_PATRON_STATUS")])

View more solutions

Comments

[Paulo Mattos]

Very well written question ;) You clearly stated what you want and the current result you are getting. If only your fellow hackers would follow this style...

[Paulo Mattos]

Would you care to show/link which encode overload would match the optional string property? And isn’t using encodeNil(forKey:) a better approach here (readability wise)?

[Rob Napier]

@PauloMattos Edited.

[Paulo Mattos]

Thanks for the write up Rob! Gonna (slowly) digest all this and come back with more questions ;) For now, I am guessing when conditional conformance (finally!) lands the Optional encodable implementation gonna be a lot safer...

[Peterdk]

I created a Swift bugreport since I am needing this functionality. Feel free to add your thoughts on it there, in case you need this too. bugs.swift.org/browse/SR-9232

[Dario Scoppelletti]

You should replace by ``` @propertyWrapper struct NullEncodable<T>: Encodable where T: Encodable { var wrappedValue: T? func encode(to encoder: Encoder) throws { var container = encoder.singleValueContainer() switch wrappedValue { case .some(let value): try container.encode(value) case .none: try container.encodeNil() } } } ``` in order to use any configuration applied to JSONEncoder.

[mredig]

I absolutely love this solution and have an update for it: add ``` init(wrappedValue: T?) { self.wrappedValue = wrappedValue } ``` to the wrapper type so that implicit struct initializers don't throw a fit.

[mredig]

Found a couple more tricks! I posted them on a gist since they are too much to contain in a non-formatted comment here... gist.github.com/mredig/f6d9efb196a25d857fe04a28357551a6 - feel free to update your answer from it!

[Steven Grosmark]

@mredig apparently great minds think alike! That's what I have in the full implementation here: github.com/g-mark/NullCodable

[ChipsAndBits]

@ Steven Grosmark It doesn't seem to work while decoding when missing a key. It throws DecodingError.keyNotFound. If I remove the @NullEncodable wrapper, then it works fine. Any thoughts?

[Slavik Voloshyn]

Elegant solution 🧐 @StevenGrosmark And also thanks for supporting SPM!

[Steven Grosmark]

@ChipsAndBits Good point. To achieve this, you would need to extend KeyedDecodingContainer to emulate decodeIfPresent (because although the wrapped value is optional, the property wrapper is never itself optional). I updated the repo at github.com/g-mark/NullCodable.

[Steven Grosmark]

Thanks, @SlavikVoloshyn!

[mnl]

Love this solution!!!

[jpulikkottil]

Could you add its decodable function ? [I mean could you make Tristate confirms to Codable protocol]