Go如何精确计算小数-Decimal研究

##1 浮点数为什么不精确
先看两个case

// case1: 135.90*100 ====
// float32
var f1 float32 = 135.90
fmt.Println(f1 * 100) // output:13589.999
// float64
var f2 float64 = 135.90
fmt.Println(f2 * 100) // output:13590

浮点数在单精度下, 135.9*100即出现了偏差, 双精度下结果正确.

// case2: 0.1 add 10 times ===
// float32
var f3 float32 = 0
for i := 0; i < 10; i++ {
f3 += 0.1
}
fmt.Println(f3) //output:1.0000001
// float64
var f4 float64 = 0
for i := 0; i < 10; i++ {
f4 += 0.1
}
fmt.Println(f4) //output:0.9999999999999999

0.1加10次, 这下无论是float32和float64都出现了偏差.

为什么呢, Go和大多数语言一样, 使用标准的IEEE754表示浮点数, 0.1使用二进制表示结果是一个无限循环数, 只能舍入后表示, 累加10次之后就会出现偏差.

##2 数据库是怎么做的
MySQL提供了decimal(p,d)/numberlic(p,d)类型的定点数表示法, 由p位数字(不包括符号、小数点)组成, 小数点后面有d位数字, 占p+2个字节, 计算性能会比double/float类型弱一些.

##3 Go代码如何实现Decimal
在GitHub搜decimal, star数量比较多的是TiDB里的MyDecimal和ithub.com/shopspring/decimal的实现.

  • shopspring的Decimal实现比较简单, 思路是使用十进制定点数表示法, 有多少位小数就小数点后移多少位, value保存移之后的整数, exp保存小数点后的数位个数, number=value*10^exp, 因为移小数点后的整数可能很大, 所以这里借用标准包里的math/big表示这个大整数. exp使用了int32, 所以这个包最多能表示小数点后有32个十进制数位的情况.

    Decimal结构体的定义如下

    // Decimal represents a fixed-point decimal. It is immutable.
    // number = value * 10 ^ exp
    type Decimal struct {
    value *big.Int
    // NOTE(vadim): this must be an int32, because we cast it to float64 during
    // calculations. If exp is 64 bit, we might lose precision.
    // If we cared about being able to represent every possible decimal, we
    // could make exp a *big.Int but it would hurt performance and numbers
    // like that are unrealistic.
    exp int32
    }
  • TiDB里的MyDecimal定义位于github.com/pingcap/tidb/util/types/mydecimal.go, 实现比shopspring的Decimal复杂多了, 也更底层(不依赖math/big), 性能也更好(见下面的benchmark). 其思路是:
    digitsInt保存数字的整数部分数字个数, digitsFrac保存数字的小数部分数字个数, resultFrac保存计算及序列化时保留至小数点后几位, negative标明数字是否为负数, wordBuf是一个定长的int32数组(长度为9), 数字去掉小数点的主体保存在这里, 一个int32有32个bit, 最大值为(2**31-1)2147483647(10个十进制数), 所以一个int32最多能表示9个十进制数位, 因此wordBuf 最多能容纳9*9个十进制数位.

    // MyDecimal represents a decimal value.
    type MyDecimal struct {
    digitsInt int8 // the number of *decimal* digits before the point.
    digitsFrac int8 // the number of decimal digits after the point.
    resultFrac int8 // result fraction digits.
    negative bool
    // wordBuf is an array of int32 words.
    // A word is an int32 value can hold 9 digits.(0 <= word < wordBase)
    wordBuf [maxWordBufLen]int32
    }

看看这两种decimal类型在文首的两个case下的结果, 同时跑个分.

main_test.go

package main
import (
"testing"
"github.com/shopspring/decimal"
"github.com/pingcap/tidb/util/types"
"log"
)
var case1String = "135.90"
var case1Bytes = []byte(case1String)
var case2String = "0"
var case2Bytes = []byte("0")
func ShopspringDecimalCase1() decimal.Decimal {
dec1, err := decimal.NewFromString(case1String)
if err != nil {
log.Fatal(err)
}
dec2 := decimal.NewFromFloat(100)
dec3 := dec1.Mul(dec2)
return dec3
}
func TidbDecimalCase1() *types.MyDecimal {
dec1 := new(types.MyDecimal)
err := dec1.FromString(case1Bytes)
if err != nil {
log.Fatal(err)
}
dec2 := new(types.MyDecimal).FromInt(100)
dec3 := new(types.MyDecimal)
err = types.DecimalMul(dec1, dec2, dec3)
if err != nil {
log.Fatal(err)
}
return dec3
}
func ShopspringDecimalCase2() decimal.Decimal {
dec1, err := decimal.NewFromString(case2String)
if err != nil {
log.Fatal(err)
}
dec2 := decimal.NewFromFloat(0.1)
for i := 0; i < 10; i++ {
dec1 = dec1.Add(dec2)
}
return dec1
}
func TidbDecimalCase2() *types.MyDecimal {
dec1 := new(types.MyDecimal)
dec1.FromString(case2Bytes)
dec2 := new(types.MyDecimal)
dec2.FromFloat64(0.1)
for i := 0; i < 10; i++ {
types.DecimalAdd(dec1, dec2, dec1)
}
return dec1
}
// case1: 135.90*100 ====
func BenchmarkShopspringDecimalCase1(b *testing.B) {
for i := 0; i < b.N; i++ {
ShopspringDecimalCase1()
}
b.Log(ShopspringDecimalCase1()) // output: 13590
}
func BenchmarkTidbDecimalCase1(b *testing.B) {
for i := 0; i < b.N; i++ {
TidbDecimalCase1()
}
b.Log(TidbDecimalCase1()) // output: 13590.00
}
// case2: 0.1 add 10 times ===
func BenchmarkShopspringDecimalCase2(b *testing.B) {
for i := 0; i < b.N; i++ {
ShopspringDecimalCase2()
}
b.Log(ShopspringDecimalCase2()) // output: 1
}
func BenchmarkTidbDecimalCase2(b *testing.B) {
for i := 0; i < b.N; i++ {
TidbDecimalCase2()
}
b.Log(TidbDecimalCase2()) // output: 1.0
}
BenchmarkShopspringDecimalCase1-8 2000000 664 ns/op 340 B/op 10 allocs/op
BenchmarkTidbDecimalCase1-8 20000000 99.2 ns/op 48 B/op 1 allocs/op
BenchmarkShopspringDecimalCase2-8 300000 5210 ns/op 4294 B/op 111 allocs/op
BenchmarkTidbDecimalCase2-8 3000000 517 ns/op 83 B/op 3 allocs/op

可见两种实现在上面两个case下表示准确, TiDB的decimal实现的性能高于shopspring的实现, 堆内存分配次数也更少.

分享到 评论

DockerContainer下gdb无法正常工作的解决办法

昨天想在Mac上使用gdb调试一个Linux下编译的动态链接库, 以正常选项启动一个docker container, 运行gdb却发现如下错误提示.

warning: Error disabling address space randomization: Operation not permitted
Cannot create process: Operation not permitted
During startup program exited with code 127.
(gdb)

在google搜索结果里第6个才找到正确答案, https://www.google.com/search?safe=off&q=docker+gdb+warning%3A+Error+disabling+address+space+randomization%3A+Operation+not+permitted+Cannot+create+process%3A+Operation+not+permitted+During+startup+program+exited+with+code+127&oq=docker+gdb+warning%3A+Error+disabling+address+space+randomization%3A+Operation+not+permitted+Cannot+create+process%3A+Operation+not+permitted+During+startup+program+exited+with+code+127, 原来是docker run中的一个不太常用的选项, docker run –privileged, 加上即可.

于是找官方文档查看此选项的解释, 了解到: 默认docker是以受限模式下运行container, 如不能在container中运行再运行一个docker, 不能访问宿主机上的真实设备, /dev/, gdb无法访问真实的内存设备.
>

Runtime privilege and Linux capabilities

--cap-add: Add Linux capabilities
--cap-drop: Drop Linux capabilities
--privileged=false: Give extended privileges to this container
--device=[]: Allows you to run devices inside the container without the --privileged flag.

By default, Docker containers are “unprivileged” and cannot, for example, run a Docker daemon inside a Docker container. This is because by default a container is not allowed to access any devices, but a “privileged” container is given access to all devices (see the documentation on cgroups devices).

When the operator executes docker run –privileged, Docker will enable access to all devices on the host as well as set some configuration in AppArmor or SELinux to allow the container nearly all the same access to the host as processes running outside containers on the host. Additional information about running with –privileged is available on the Docker Blog.

If you want to limit access to a specific device or devices you can use the –device flag. It allows you to specify one or more devices that will be accessible within the container.

>$ docker run --device=/dev/snd:/dev/snd ...
>
分享到 评论