How to deal with Number precision in Actionscript?

Question

I have BigDecimal objects serialized with BlazeDS to Actionscript. Once they hit Actionscript as Number objects, they have values like:

140475.32 turns into 140475.31999999999998

How do I deal with this? The problem is that if I use a NumberFormatter with precision of 2, then the value is truncated to 140475.31. Any ideas?

Solution 1

This is my generic solution for the problem (I have blogged about this here):

var toFixed:Function = function(number:Number, factor:int) {
  return Math.round(number * factor)/factor;
}

For example:

trace(toFixed(0.12345678, 10)); //0.1

• Multiply 0.12345678 by 10; that gives us 1.2345678.
• When we round 1.2345678, we get 1.0,
• and finally, 1.0 divided by 10 equals 0.1.

Another example:

trace(toFixed(1.7302394309234435, 10000)); //1.7302

• Multiply 1.7302394309234435 by 10000; that gives us 17302.394309234435.
• When we round 17302.394309234435 we get 17302,
• and finally, 17302 divided by 10000 equals 1.7302.

---

Edit

Based on the anonymous answer below, there is a nice simplification for the parameter on the method that makes the precision much more intuitive. e.g:

var setPrecision:Function = function(number:Number, precision:int) {
 precision = Math.pow(10, precision);
 return Math.round(number * precision)/precision;
}

var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9
trace(setPrecision(number,2)); //Result is 10.98
trace(setPrecision(number,3)); //Result is 10.988 and so on

N.B. I added this here just in case anyone sees this as the answer and doesn't scroll down...

Solution 2

i've used Number.toFixed(precision) in ActionScript 3 to do this: http://livedocs.adobe.com/flex/3/langref/Number.html#toFixed%28%29

it handles rounding properly and specifies the number of digits after the decimal to display - unlike Number.toPrecision() that limits the total number of digits to display regardless of the position of the decimal.

var roundDown:Number = 1.434;                                             
// will print 1.43                                                        
trace(roundDown.toFixed(2));                                              

var roundUp:Number = 1.436;                                               
// will print 1.44                                                        
trace(roundUp.toFixed(2));                                                

Solution 3

I ported the IBM ICU implementation of BigDecimal for the Actionscript client. Someone else has published their nearly identical version here as a google code project. Our version adds some convenience methods for doing comparisons.

You can extend the Blaze AMF endpoint to add serialization support for BigDecimal. Please note that the code in the other answer seems incomplete, and in our experience it fails to work in production.

AMF3 assumes that duplicate objects, traits and strings are sent by reference. The object reference tables need to be kept in sync while serializing, or the client will loose sync of these tables during deserialization and start throwing class cast errors, or corrupting the data in fields that don't match, but cast ok...

Here is the corrected code:

public void writeObject(final Object o) throws IOException {
    if (o instanceof BigDecimal) {
        write(kObjectType);
        if(!byReference(o)){   // if not previously sent
            String s = ((BigDecimal)o).toString();                  
            TraitsInfo ti = new TraitsInfo("java.math.BigDecimal",false,true,0);
            writeObjectTraits(ti); // will send traits by reference
            writeUTF(s);
            writeObjectEnd();  // for your AmfTrace to be correctly indented
        }
    } else {
            super.writeObject(o);
        }
}

There is another way to send a typed object, which does not require Externalizable on the client. The client will set the textValue property on the object instead:

TraitsInfo ti = new TraitsInfo("java.math.BigDecimal",false,false,1);           
ti.addProperty("textValue");
writeObjectTraits(ti);
writeObjectProperty("textValue",s);

In either case, your Actionscript class will need this tag:

[RemoteClass(alias="java.math.BigDecimal")]

The Actionscript class also needs a text property to match the one you chose to send that will initialize the BigDecimal value, or in the case of the Externalizable object, a couple of methods like this:

public  function writeExternal(output:IDataOutput):void {       
    output.writeUTF(this.toString());
}
public  function readExternal(input:IDataInput):void {          
    var s:String = input.readUTF();
    setValueFromString(s);
}

This code only concerns data going from server to client. To deserialize in the other direction from client to server, we chose to extend AbstractProxy, and use a wrapper class to temporarily store the string value of the BigDecimal before the actual object is created, due to the fact that you cannot instantiate a BigDecimal and then assign the value, as the design of Blaze/LCDS expects should be the case with all objects.

Here's the proxy object to circumvent the default handling:

public class BigNumberProxy extends AbstractProxy {

    public BigNumberProxy() {
        this(null);
    }

    public BigNumberProxy(Object defaultInstance) {
        super(defaultInstance);
        this.setExternalizable(true);

        if (defaultInstance != null)
           alias = getClassName(defaultInstance);
    }   

    protected String getClassName(Object instance) {
        return((BigNumberWrapper)instance).getClassName();
    }

    public Object createInstance(String className) {
        BigNumberWrapper w = new BigNumberWrapper();
        w.setClassName(className);
        return w;
    }

    public Object instanceComplete(Object instance) {
    String desiredClassName = ((BigNumberWrapper)instance).getClassName();
    if(desiredClassName.equals("java.math.BigDecimal"))
        return new BigDecimal(((BigNumberWrapper)instance).stringValue);
    return null;
}

    public String getAlias(Object instance) {
        return((BigNumberWrapper)instance).getClassName();
    }

}

This statement will have to execute somewhere in your application, to tie the proxy object to the class you want to control. We use a static method:

PropertyProxyRegistry.getRegistry().register(
    java.math.BigDecimal.class, new BigNumberProxy());

Our wrapper class looks like this:

public class BigNumberWrapper implements Externalizable {

    String stringValue;
    String className;

    public void readExternal(ObjectInput arg0) throws IOException, ClassNotFoundException {
        stringValue = arg0.readUTF();       
    }

    public void writeExternal(ObjectOutput arg0) throws IOException {
        arg0.writeUTF(stringValue);     
    }

    public String getStringValue() {
        return stringValue;
    }

    public void setStringValue(String stringValue) {
        this.stringValue = stringValue;
    }

    public String getClassName() {
        return className;
    }

    public void setClassName(String className) {
        this.className = className;
    }

}

Solution 4

Surprisingly the round function in MS Excel gives us different values then you have presented above. For example in Excel

Round(143,355;2) = 143,36

So my workaround for Excel round is like:

public function setPrecision(number:Number, precision:int):Number {
precision = Math.pow(10, precision);

const excelFactor : Number = 0.00000001;

number += excelFactor;

return (Math.round(number * precision)/precision);
}

Solution 5

guys, just check the solution:

            protected function button1_clickHandler(event:MouseEvent):void
            {
                var formatter:NumberFormatter = new NumberFormatter();
                formatter.precision = 2;
                formatter.rounding = NumberBaseRoundType.NEAREST;
                var a:Number = 14.31999999999998;

                trace(formatter.format(a)); //14.32
            }

View more solutions