Format floats with standard json module

Question

I am using the standard json module in python 2.6 to serialize a list of floats. However, I'm getting results like this:

>>> import json
>>> json.dumps([23.67, 23.97, 23.87])
'[23.670000000000002, 23.969999999999999, 23.870000000000001]'

I want the floats to be formated with only two decimal digits. The output should look like this:

>>> json.dumps([23.67, 23.97, 23.87])
'[23.67, 23.97, 23.87]'

I have tried defining my own JSON Encoder class:

class MyEncoder(json.JSONEncoder):
    def encode(self, obj):
        if isinstance(obj, float):
            return format(obj, '.2f')
        return json.JSONEncoder.encode(self, obj)

This works for a sole float object:

>>> json.dumps(23.67, cls=MyEncoder)
'23.67'

But fails for nested objects:

>>> json.dumps([23.67, 23.97, 23.87])
'[23.670000000000002, 23.969999999999999, 23.870000000000001]'

I don't want to have external dependencies, so I prefer to stick with the standard json module.

How can I achieve this?

Solution 1

Note: This does not work in any recent version of Python.

Unfortunately, I believe you have to do this by monkey-patching (which, to my opinion, indicates a design defect in the standard library json package). E.g., this code:

import json
from json import encoder
encoder.FLOAT_REPR = lambda o: format(o, '.2f')
    
print(json.dumps(23.67))
print(json.dumps([23.67, 23.97, 23.87]))

emits:

23.67
[23.67, 23.97, 23.87]

as you desire. Obviously, there should be an architected way to override FLOAT_REPR so that EVERY representation of a float is under your control if you wish it to be; but unfortunately that's not how the json package was designed:-(.

Solution 2

import simplejson
    
class PrettyFloat(float):
    def __repr__(self):
        return '%.15g' % self
    
def pretty_floats(obj):
    if isinstance(obj, float):
        return PrettyFloat(obj)
    elif isinstance(obj, dict):
        return dict((k, pretty_floats(v)) for k, v in obj.items())
    elif isinstance(obj, (list, tuple)):
        return list(map(pretty_floats, obj))
    return obj
    
print(simplejson.dumps(pretty_floats([23.67, 23.97, 23.87])))

emits

[23.67, 23.97, 23.87]

No monkeypatching necessary.

Solution 3

Really unfortunate that dumps doesn't allow you to do anything to floats. However loads does. So if you don't mind the extra CPU load, you could throw it through the encoder/decoder/encoder and get the right result:

>>> json.dumps(json.loads(json.dumps([.333333333333, .432432]), parse_float=lambda x: round(float(x), 3)))
'[0.333, 0.432]'

Solution 4

If you're using Python 2.7, a simple solution is to simply round your floats explicitly to the desired precision.

>>> sys.version
'2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)]'
>>> json.dumps(1.0/3.0)
'0.3333333333333333'
>>> json.dumps(round(1.0/3.0, 2))
'0.33'

This works because Python 2.7 made float rounding more consistent. Unfortunately this does not work in Python 2.6:

>>> sys.version
'2.6.6 (r266:84292, Dec 27 2010, 00:02:40) \n[GCC 4.4.5]'
>>> json.dumps(round(1.0/3.0, 2))
'0.33000000000000002'

The solutions mentioned above are workarounds for 2.6, but none are entirely adequate. Monkey patching json.encoder.FLOAT_REPR does not work if your Python runtime uses a C version of the JSON module. The PrettyFloat class in Tom Wuttke's answer works, but only if %g encoding works globally for your application. The %.15g is a bit magic, it works because float precision is 17 significant digits and %g does not print trailing zeroes.

I spent some time trying to make a PrettyFloat that allowed customization of precision for each number. Ie, a syntax like

>>> json.dumps(PrettyFloat(1.0 / 3.0, 4))
'0.3333'

It's not easy to get this right. Inheriting from float is awkward. Inheriting from Object and using a JSONEncoder subclass with its own default() method should work, except the json module seems to assume all custom types should be serialized as strings. Ie: you end up with the Javascript string "0.33" in the output, not the number 0.33. There may be a way yet to make this work, but it's harder than it looks.

Solution 5

Here's a solution that worked for me in Python 3 and does not require monkey patching:

import json

def round_floats(o):
    if isinstance(o, float): return round(o, 2)
    if isinstance(o, dict): return {k: round_floats(v) for k, v in o.items()}
    if isinstance(o, (list, tuple)): return [round_floats(x) for x in o]
    return o


json.dumps(round_floats([23.63437, 23.93437, 23.842347]))

Output is:

[23.63, 23.93, 23.84]

It copies the data but with rounded floats.

View more solutions

Comments

[Nelson]

This solution does not work in Python 2.7 using Python's C version of the JSON encoder.

[Matthew Schinckel]

Those look like strings, not numbers.

[mikepurvis]

I like this solution; better integration, and works with 2.7. Because I am building up the data myself anyway, I eliminated the pretty_floats function and simply integrated it into my other code.

[Nelson]

Another approach for Python 2.6 using JSONEncoder.iterencode and pattern matching can be seen at github.com/migurski/LilJSON/blob/master/liljson.py

[Lincoln B]

Hopefully this makes passing around your floats more lightweight - I like how we can avoid messing with the JSON classes which can suck.

[Guido van Rossum]

However you do this, use something like %.15g or %.12g instead of %.3f .

[Rory Hart]

I found this snippet in a junior programmer's code. This would have created a very serious but subtle bug if it had not been caught. Can you please place a warning on this code explaining the global implications of this monkey patching.

[Jeff Kaufman]

It's good hygiene to set it back when you're done: original_float_repr = encoder.FLOAT_REPR encoder.FLOAT_REPR = lambda o: format(o, '.2f') print json.dumps(1.0001) encoder.FLOAT_REPR = original_float_repr

[Aryeh Leib Taurog]

Incidentally, format(.3, '.2f') seems to be an order of magnitude slower than "%.2f" % .3.

[Jan Šimbera]

Does not work for me on standard Python 2.7.3 on win64. The first list is formatted normally, but after that the behavior flips to the default and all consecutive lists are formatted with the default behavior.

[konrad]

Please note that the original_float_repr solution is not threadsafe.

[Anonymous]

Thank you, this is really helpful suggestion. I didn't know about the parse_float kwarg!

[Brent Faust]

The simplest suggestion here that also works in 3.6.

[Nico Schlömer]

As others have pointed out, this is no longer working in at least Python 3.6+. Add a few digits to 23.67 to see how .2f is not respected.

[shaneb]

Note the phrase "don't mind the extra CPU load". Definitely do not use this solution if you have a lot of data to serialize. For me, adding this alone made a program doing a non-trivial calculation take 3X longer.

[Guglie]

In Python3 it gives "Map object is not JSON serializable" error, but you can resolve converting the map() to a list with list( map(pretty_floats, obj) )

[Azat Ibrakov]

@Guglie: that's because in Python 3 map returns iterator, not a list

[Tomasz Gandor]

Looks neat, but seems to not work on Python 3.6. In particular, I didn't see a FLOAT_REPR constant in the json.encoder module.

[naught101]

Yes, this is a quite silly example... why use 2 digit rounding if you're only feeding it two significant-digit numbers?

[szali]

Doesn't work for me (Python 3.5.2, simplejson 3.16.0). Tried it with %.6g and [23.671234556, 23.971234556, 23.871234556], it still prints the whole number.

[faridSam]

This doesn't work with the built-in Python3 json package, which doesn't use __repr__().

[Daniel F]

This does not work if you need a precision of 6 decimals or so.

[MrR]

so you need to get floats in and out of numpy ndarrays for this to work?

[proski]

For a simple monkey-patch solution that works with Python 3, see my answer at stackoverflow.com/questions/54370322

[Victor Schröder]

I was tempted to do something like this, but this solution has a MAJOR drawback: it will affect everything that looks like a float, even if it is in the middle of a string or, even worse, in a key of the JSON representation of the data. It could be used only for very specific kinds of data when you are 100% sure you won't find anything looking like a float anywhere else.

[Matt]

I would tend to agree, I like this solution in the end stackoverflow.com/a/69056325/5125264

[Victor Schröder]

In fact, the solution mentioned by @Matt above is what I ended up using, but that solution has another major drawback, because it applies the format globally. In case we need different precision for different parts of the JSON serialized string, that won't work...