BigDecimal precision problem

Posted by paxman356 on Mon, 25 Oct 2021 03:12:37 +0200

BigDecimal precision problem

Recently, I encountered a problem when displaying the price of the project. The display of a commodity with a price of 99999.999 in the shopping cart is 100000.00.

The reason is that in the original project code, BigDecimal format is used, but when the front end needs to be displayed, String type is used.

In the process of type conversion, there is a loss of precision. So I went down to do some research on the high-precision display of BigDecimal.

To sum up, there are two main ways to convert BigDecimal to String, and the output of the two methods is inconsistent for the price of 999999.999. The specific codes are as follows:

BigDecimal price = new BigDecimal("999999.999");  //price information 

//Method 1
String format1 = String.format("%.2f", price); 
System.out.println("stringFormat:  "+format1); 	// 1000000.00

//Method 2
String format2 = price.setScale(2, RoundingMode.DOWN).toPlainString();
System.out.println("scale:  "+ format2); 	// 999999.99

String.format mode

The first method, String.format, is the code originally used in the project. To find out, go back to the format source code and see the code as follows:

public Formatter format(Locale l, String format, Object ... args) {
  ensureOpen();
  int last = -1;
  int lasto = -1;
  FormatString[] fsa = parse(format); // First, parse the special identifier in the string
  for (int i = 0; i < fsa.length; i++) {
    FormatString fs = fsa[i]; 
    int index = fs.index(); // Gets the subscript of the corresponding identifier
    try {
      switch (index) {
        case -2:  
          fs.print(null, l);
          break;
        case -1:  
          if (last < 0 || (args != null && last > args.length - 1))
            throw new MissingFormatArgumentException(fs.toString());
          fs.print((args == null ? null : args[last]), l);
          break;
        case 0:  
          lasto++;
          last = lasto;
          if (args != null && lasto > args.length - 1)
            throw new MissingFormatArgumentException(fs.toString());
          fs.print((args == null ? null : args[lasto]), l); // critical code
          break;
        default:  
          last = index - 1;
          if (args != null && last > args.length - 1)
            throw new MissingFormatArgumentException(fs.toString());
          fs.print((args == null ? null : args[last]), l);
          break;
      }
    } catch (IOException x) {
      lastException = x;
    }
  }
  return this;
}

You can see that the key method is actually fs.print(). Go back here and check the source code of print. You can see that the print source code will first make a judgment according to the current type. Here we use BigDecimal, so we will go to the printFloat method.

    if (dt) {
        printDateTime(arg, l);
        return;
    }
    switch(c) {
    case Conversion.DECIMAL_INTEGER:
    case Conversion.OCTAL_INTEGER:
    case Conversion.HEXADECIMAL_INTEGER:
        printInteger(arg, l);
        break;
    case Conversion.SCIENTIFIC:
    case Conversion.GENERAL:
    case Conversion.DECIMAL_FLOAT:
    case Conversion.HEXADECIMAL_FLOAT: 
        printFloat(arg, l); // The method will come here
        break;
    case Conversion.CHARACTER:
    case Conversion.CHARACTER_UPPER:
        printCharacter(arg);
        break;
    case Conversion.BOOLEAN:
        printBoolean(arg);
        break;
    case Conversion.STRING:
        printString(arg, l);
        break;
    case Conversion.HASHCODE:
        printHashCode(arg);
        break;
    case Conversion.LINE_SEPARATOR:
        a.append(System.lineSeparator());
        break;
    case Conversion.PERCENT_SIGN:
        a.append('%');
        break;
    default:
        assert false;
    }
}

After tracing into the printFloat method, you can see that further, the printFloat method will judge the type of the object currently to be processed and select the appropriate method for processing.

private void printFloat(Object arg, Locale l) throws IOException {
  if (arg == null)
    print("null");
  else if (arg instanceof Float)
    print(((Float)arg).floatValue(), l);
  else if (arg instanceof Double)
    print(((Double)arg).doubleValue(), l);
  else if (arg instanceof BigDecimal)
    print(((BigDecimal)arg), l); // Key methods
  else
    failConversion(c, arg);
}

Trace back to the corresponding method of BigDecimal and check the corresponding print method. You can see that the key method is actually print(), so we continue to trace back to check the corresponding code.

private void print(BigDecimal value, Locale l) throws IOException {
  if (c == Conversion.HEXADECIMAL_FLOAT)
    failConversion(c, value);
  StringBuilder sb = new StringBuilder();
  boolean neg = value.signum() == -1; // Determines whether the current number symbol is positive or negative
  BigDecimal v = value.abs(); // Take its absolute value
  leadingSign(sb, neg); // Set the first symbol, '+' or '(' or '-'
  print(sb, v, l, f, c, precision, neg); // Key methods
  trailingSign(sb, neg); // Remove redundant symbols
  a.append(justify(sb.toString()));
}
private void print(StringBuilder sb, BigDecimal value, Locale l,Flags f, char c, int precision, boolean neg)throws IOException{
  if(c == Conversion.SCIENTIFIC){
    ......
  } else if (c == Conversion.DECIMAL_FLOAT) {
    int prec = (precision == -1 ? 6 : precision); // It refers to the precision that needs to be reserved in our current setting. In the example, 2 decimal places are reserved.
    int scale = value.scale(); // scale refers to the precision range of the current number. In the example, it is 3 decimal places, so it is 3.

    if (scale > prec) { // If the current number of digits is greater than the number of digits to keep
      int compPrec = value.precision();//Here, comPrec refers to the number of digits of the whole number, in the example, 9
      if (compPrec <= scale) {
        value = value.setScale(prec, RoundingMode.HALF_UP);
      } else {
        compPrec -= (scale - prec); 
        // Recalculate the number of reduced digits, and then call the BigDecimal method to rearrange the digits.
        // The BigDecimal method itself adopts RoundMode.HalfUp, so carry will occur after construction, resulting in different results.
        value = new BigDecimal(value.unscaledValue(), 
                               scale,
                               new MathContext(compPrec));
      }
    }
    BigDecimalLayout bdl = new BigDecimalLayout(
      value.unscaledValue(),
      value.scale(),
      BigDecimalLayoutForm.DECIMAL_FLOAT);

    char mant[] = bdl.mantissa();
    int nzeros = (bdl.scale() < prec ? prec - bdl.scale() : 0);

    if (bdl.scale() == 0 && (f.contains(Flags.ALTERNATE) || nzeros > 0))
      mant = addDot(bdl.mantissa());

    mant = trailingZeros(mant, nzeros);

    localizedMagnitude(sb, mant, f, adjustWidth(width, f, neg), l);
  }
}

So far, we understand the disadvantage of the first method, that is, it is the RoundMode.halfUp method adopted by default, which will automatically carry and lead to data deviation.

price.setScale mode

The second method is the repaired method. The advantage of this method is that you can customize the carry and rounding strategies to get more logical results. The specific source code is as follows:

public BigDecimal setScale(int newScale, int roundingMode) {
  if (roundingMode < ROUND_UP || roundingMode > ROUND_UNNECESSARY)
    throw new IllegalArgumentException("Invalid rounding mode");
  int oldScale = this.scale;
  if (newScale == oldScale)        // If the old and new digits are consistent, it will be returned directly
    return this;
  if (this.signum() == 0)            // 0 returns any number of digits
    return zeroValueOf(newScale);
  if(this.intCompact!=INFLATED) { // If the current value does not exceed - 2 ^ 63 (the upper limit of long type), the calculation is performed
    long rs = this.intCompact; //rs is the current value without decimal point, eg. 999.99 = = > 99999 
    if (newScale > oldScale) { 
      // If the new reserved digit is greater than the original digit
        int raise = checkScale((long) newScale - oldScale);
        if ((rs = longMultiplyPowerTen(rs, raise)) != INFLATED) {
          return valueOf(rs,newScale);
        }
        BigInteger rb = bigMultiplyPowerTen(raise);
        return new BigDecimal(rb, INFLATED, newScale, (precision > 0) ? precision + raise : 0);
      } else {	
      	// Otherwise, calculate the number of bits to be discarded and create a new BigDecimal object accordingly
        int drop = checkScale((long) oldScale - newScale);
        if (drop < LONG_TEN_POWERS_TABLE.length) {
          // Key methods
          return divideAndRound(rs, LONG_TEN_POWERS_TABLE[drop], newScale, roundingMode, newScale);
        } else {
          return divideAndRound(this.inflated(), bigTenToThe(drop), newScale, roundingMode, newScale);
        }
      }
  } else {
    if (newScale > oldScale) {
      int raise = checkScale((long) newScale - oldScale);
      BigInteger rb = bigMultiplyPowerTen(this.intVal,raise);
      return new BigDecimal(rb, INFLATED, newScale, (precision > 0) ? precision + raise : 0);
    } else {
      int drop = checkScale((long) oldScale - newScale);
      if (drop < LONG_TEN_POWERS_TABLE.length)
        return divideAndRound(this.intVal, LONG_TEN_POWERS_TABLE[drop], newScale, roundingMode,
                              newScale);
      else
        return divideAndRound(this.intVal,  bigTenToThe(drop), newScale, roundingMode, newScale);
    }
  }
}

Then we catch up with the divideAndRound method,

private static BigDecimal divideAndRound(long ldividend, long ldivisor, int scale, int roundingMode,
                                         int preferredScale) {
  int qsign; // Symbol
  long q = ldividend / ldivisor; // About to the corresponding digit, eg. 999999999 / 10 = = > 9999999 
  if (roundingMode == ROUND_DOWN && scale == preferredScale)
    //If it is a rounding scheme, the corresponding value can be returned directly.
    return valueOf(q, scale); // This method creates a BigDecimal object with the corresponding value. 
  long r = ldividend % ldivisor; 
  qsign = ((ldividend < 0) == (ldivisor < 0)) ? 1 : -1;
  if (r != 0) {
    boolean increment = needIncrement(ldivisor, roundingMode, qsign, q, r);
    return valueOf((increment ? q + qsign : q), scale);
  } else {
    if (preferredScale != scale)
      return createAndStripZerosToMatchScale(q, scale, preferredScale);
    else
      return valueOf(q, scale);
  }
}

summary

To sum up, the String.format method uses the Round_HalfUp method by default to create the corresponding object, resulting in the rounding of the number. The setScale method can freely select the corresponding rounding method, which is more flexible in general. In combination with the toplanstring method, all required functions can be well realized.

Topics: Java Back-end