1.94.2: Resize dialog: Wrong rounding

[Karl02]

When resizing an image via Image -> Resize, selecting "Keep ratio" (i.e. preserve the original aspect ratio) and entering a new value for width or height, the respective other value is calculated automatically, but not correctly rounded to the nearest integer value.

[Peter2]

. ..entering a new value for width or height...

Which units - cm, pixel, inch?

Peter

[Karl02]

I meant the Screen Size in pixels.

Furthermore, for the Screen Size in percent no real values are accepted, while for the Print Size in percent this is possible.

[Peter2]

...the respective other value is calculated automatically, but not correctly rounded to the nearest integer value.

Please give an example.

...Furthermore, for the Screen Size in percent no real values are accepted, while for the Print Size in percent this is possible.

Yes - confirmed.

Peter

[Karl02]

Example: Lets assume you have an image of size 900 x 600 pixels (width x height) and want to resize it to a width of 250 pixels. If you want to preserve the original aspect ratio, the new height should be 250 x 600 / 900 = 166 2/3 pixels. This must be rounded to an integer value, and the nearest integer value is 167. However, xnview rounds to 166 pixels.

[JohnFredC]

In some cases (though perhaps not XnView), proper rounding requires a method that is statistically sound. The usual method is to round UP when the least significant digit is even, and DOWN when the least significant digit is odd.

Or vice versa!

[Karl02]

I think the standard solution is to round up for fractions >= 0.5 and round down else. This can be done by adding 0.5 and afterwards cutting off the fraction.

[JohnFredC]

Gadzooks, I mispoke

You are correct about every circumstance except when the least significant digit is 5. It is easily demonstrated that the method for (>=0.5) overstates proportionally to 0.5*n at the limit, where n=the number of rounded instances.

The statistically correct rule, restated (my changes in red):

If the least significant digit is...

...less than 5 then round down

else
...greater than 5 then round up

else
...equal to 0.5 then:

"round UP when the second to least significant digit is even, and DOWN when the second to least significant digit is odd, or vice versa."

Sorry for the confusion. Some times "ex cathedra" is all the way from the bell tower to the cold hard and unforgiving ground.

Statistical accuracy at the limit is irrelevant to XnView, though...

[Karl02]

In my opinion it is always better to round up for fractions >0.5 (e.g. 0.51, 0.52, etc.) and round down for fractions <0.5 (e.g. 0.49, 0.48 ), because the rounded value will be closer to the desired value (i.e. the rounding error will be smaller). The probability for a fraction to be exactly =0.500000 is actually rather small, because this is a single point and no interval (or a very small interval in a computer), so it should not matter statistically if you round up or down. However, to have a predictable and consistent behaviour, it would be better to round always in one direction. I think it is usual to round up in that case.

[JohnFredC]

If you deal with financials in software (my profession), you'll quickly find that rounding everything up in the manner you suggest in your last sentence can get you fired.

[helmut]

From what I can see you both (Karl02 and JohnFredC) have suggested the same rounding, except for the detail if there the resulting number should have more than just one digit.

Some samples:

Rounding to 0 digit:
0.498 -> 0
0.543 -> 1

Rounding to 1 digit:
0.498 -> 0.5
0.543 -> 0.5

Rounding to 2 digit:
0.498 -> 0.50
0.543 -> 0.54

[Karl02]

I think I understand now what John meant: If the fraction is exactly equal to 0.50000000..., then the error would be the same for rounding up or down (+/- 0.5). Therefore, you could in one half of the cases round up and in the other half of the cases round down, depending on the digit on the left side of the 5, e.g.:

Code: Select all

             Error
0.500 -> 0   -0.5
1.500 -> 2   +0.5
2.500 -> 2   -0.5
3.500 -> 4   +0.5
4.500 -> 4   -0.5
5.500 -> 6   +0.5
6.500 -> 6   -0.5
7.500 -> 8   +0.5
8.500 -> 8   -0.5
9.500 ->10   +0.5

Now the rounding errors should eliminate each other statistically if you round many times.

[JohnFredC]

Yes karl02.. that is exactly what I meant. Thanks.

Hence the even up/odd down (or vice versa) rule I mentioned. The even/odd criteria applies to the second least significant digit only when the least significant digit is 5.

When large populations of numbers are considered, the number of instances where the least significant digit is 5 will approach 10% (1/10) at the limit, so it is a straight-forward calculation to estimate the accumulating error of any rounding rule. The even/odd rule approaches 0 error at the limit, as illustrated by karl2's sample chart.

It's one thing to stand in front of a group of people at a marker board and illustrate the point with a marker and verbal demonstration. It's another to write out the explanation in a forum.

My posts in this thread are self-evidently not my best. My apologies.

[Prof. Jewell]

Yes karl02.. that is exactly what I meant. Thanks.

Hence the even up/odd down (or vice versa) rule I mentioned. The even/odd criteria applies to the second least significant digit only when the least significant digit is 5.

When large populations of numbers are considered, the number of instances where the least significant digit is 5 will approach 10% (1/10) at the limit, so it is a straight-forward calculation to estimate the accumulating error of any rounding rule. The even/odd rule approaches 0 error at the limit, as illustrated by karl2's sample chart.

It's one thing to stand in front of a group of people at a marker board and illustrate the point with a marker and verbal demonstration. It's another to write out the explanation in a forum.

My posts in this thread are self-evidently not my best. My apologies.

I don't know if you guys would be interested or not, but the procedure that you are discussing had a name in the 19th century (back before computers) which is a bit ironic. Then it was called "the computer's rule." Back then a "computer" was a human being.