IDL double与float的精度检验

pro test

  compile_opt idl2

  a = 100.000000002

  b = 100.000000000

  help, a

  help, b

  if a gt b then $

    print, '100.000000002 gt 100.000000000'

  if a eq b then $

    print, '100.000000002 eq 100.000000000'

  if a lt b then $

    print, '100.000000002 lt 100.000000000'

  a = 100.000000002d

  b = 100.000000000d

  help, a

  help, b

  if a gt b then $

    print, '100.000000002d gt 100.000000000d'

  if a eq b then $

    print, '100.000000002d eq 100.000000000d'

  if a lt b then $

    print, '100.000000002d lt 100.000000000d'

  print, ''

  a = double(100.000000002)

  b = double(100.000000000)

  help, a

  help, b

  if a gt b then $

    print, 'double(100.000000002) gt double(100.000000000)'

  if a eq b then $

    print, 'double(100.000000002) eq double(100.000000000)'

  if a lt b then $

    print, 'double(100.000000002) lt double(100.000000000)'

  print, ''

  a = 100.000000002*1.0d

  b = 100.000000000*1.0d

  help, a

  help, b

  if a gt b then $

    print, '100.000000002*1.0d gt 100.000000000*1.0d'

  if a eq b then $

    print, '100.000000002*1.0d eq 100.000000000*1.0d'

  if a lt b then $

    print, '100.000000002*1.0d lt 100.000000000*1.0d'

  print, ''

 

  a = double('100.1234567892')

  b = double('100.1234567890')

  help, a

  help, b

  if a gt b then $

    print, '100.1234567892*1.0d gt 100.1234567890*1.0d'

  if a eq b then $

    print, '100.1234567892*1.0d eq 100.1234567890*1.0d'

  if a lt b then $

    print, '100.1234567892*1.0d lt 100.1234567890*1.0d'

  print, ''

end

 

运行结果

A               FLOAT     =       100.000

B               FLOAT     =       100.000

100.000000002 eq 100.000000000

 

A               DOUBLE    =        100.00000

B               DOUBLE    =        100.00000

100.000000002d gt 100.000000000d

 

A               DOUBLE    =        100.00000

B               DOUBLE    =        100.00000

double(100.000000002) eq double(100.000000000)

 

A               DOUBLE    =        100.00000

B               DOUBLE    =        100.00000

100.000000002*1.0d eq 100.000000000*1.0d

 

A               DOUBLE    =        100.12346

B               DOUBLE    =        100.12346

str100.1234567892 gt str100.1234567890

 

初始的数据如果不是double型的，后面转为double型是没有用的，因为精度已经丢失

从a=100.000000002 可知，虽然是小数点后9位， 但IDL认为是float型

从a=100.000000002d 可知，加上d之后，IDL认为是double型

从a = double('100.1234567892') 可知，读入一个位数很多的字符串，将其转为double，可以最大限度的保持后续计算的精度