1.94.2: Resize dialog: Wrong rounding

Bugs found in XnView Classic. Please report only one bug per topic!

Moderators: helmut, XnTriq, xnview

User avatar
Karl02
Posts: 134
Joined: Mon Sep 03, 2007 1:00 pm
Location: Germany

1.94.2: Resize dialog: Wrong rounding

Post by Karl02 »

When resizing an image via Image -> Resize, selecting "Keep ratio" (i.e. preserve the original aspect ratio) and entering a new value for width or height, the respective other value is calculated automatically, but not correctly rounded to the nearest integer value.
-- Karl
User avatar
Peter2
XnThusiast
Posts: 1280
Joined: Thu Nov 24, 2005 3:07 pm
Location: CH

Re: xnview v1.94.2: Resize dialog: Wrong rounding

Post by Peter2 »

Karl02 wrote:. ..entering a new value for width or height...
Which units - cm, pixel, inch?

Peter
XnView 2.50.2 German, Win 10
User avatar
Karl02
Posts: 134
Joined: Mon Sep 03, 2007 1:00 pm
Location: Germany

Post by Karl02 »

I meant the Screen Size in pixels.

Furthermore, for the Screen Size in percent no real values are accepted, while for the Print Size in percent this is possible.
-- Karl
User avatar
Peter2
XnThusiast
Posts: 1280
Joined: Thu Nov 24, 2005 3:07 pm
Location: CH

Post by Peter2 »

Karl02 wrote:...the respective other value is calculated automatically, but not correctly rounded to the nearest integer value.
Please give an example.
Karl02 wrote:...Furthermore, for the Screen Size in percent no real values are accepted, while for the Print Size in percent this is possible.
Yes - confirmed.

Peter
XnView 2.50.2 German, Win 10
User avatar
Karl02
Posts: 134
Joined: Mon Sep 03, 2007 1:00 pm
Location: Germany

Post by Karl02 »

Example: Lets assume you have an image of size 900 x 600 pixels (width x height) and want to resize it to a width of 250 pixels. If you want to preserve the original aspect ratio, the new height should be 250 x 600 / 900 = 166 2/3 pixels. This must be rounded to an integer value, and the nearest integer value is 167. However, xnview rounds to 166 pixels.
-- Karl
User avatar
JohnFredC
XnThusiast
Posts: 2010
Joined: Wed Mar 17, 2004 8:33 pm
Location: Sarasota Florida

Post by JohnFredC »

In some cases (though perhaps not XnView), proper rounding requires a method that is statistically sound. The usual method is to round UP when the least significant digit is even, and DOWN when the least significant digit is odd.

Or vice versa! 8)
John
User avatar
Karl02
Posts: 134
Joined: Mon Sep 03, 2007 1:00 pm
Location: Germany

Post by Karl02 »

:?:
I think the standard solution is to round up for fractions >= 0.5 and round down else. This can be done by adding 0.5 and afterwards cutting off the fraction.
-- Karl
User avatar
JohnFredC
XnThusiast
Posts: 2010
Joined: Wed Mar 17, 2004 8:33 pm
Location: Sarasota Florida

Post by JohnFredC »

Gadzooks, I mispoke :( :(

You are correct about every circumstance except when the least significant digit is 5. It is easily demonstrated that the method for (>=0.5) overstates proportionally to 0.5*n at the limit, where n=the number of rounded instances.

The statistically correct rule, restated (my changes in red):

If the least significant digit is...

...less than 5 then round down

else
...greater than 5 then round up

else
...equal to 0.5 then:

"round UP when the second to least significant digit is even, and DOWN when the second to least significant digit is odd, or vice versa."

Sorry for the confusion. Some times "ex cathedra" is all the way from the bell tower to the cold hard and unforgiving ground.

Statistical accuracy at the limit is irrelevant to XnView, though...
John
User avatar
Karl02
Posts: 134
Joined: Mon Sep 03, 2007 1:00 pm
Location: Germany

Post by Karl02 »

In my opinion it is always better to round up for fractions >0.5 (e.g. 0.51, 0.52, etc.) and round down for fractions <0.5 (e.g. 0.49, 0.48 ), because the rounded value will be closer to the desired value (i.e. the rounding error will be smaller). The probability for a fraction to be exactly =0.500000 is actually rather small, because this is a single point and no interval (or a very small interval in a computer), so it should not matter statistically if you round up or down. However, to have a predictable and consistent behaviour, it would be better to round always in one direction. I think it is usual to round up in that case.
-- Karl
User avatar
JohnFredC
XnThusiast
Posts: 2010
Joined: Wed Mar 17, 2004 8:33 pm
Location: Sarasota Florida

Post by JohnFredC »

If you deal with financials in software (my profession), you'll quickly find that rounding everything up in the manner you suggest in your last sentence can get you fired.
John
User avatar
helmut
Moderator
Posts: 8499
Joined: Sun Oct 12, 2003 6:47 pm
Location: Frankfurt, Germany

Post by helmut »

From what I can see you both (Karl02 and JohnFredC) have suggested the same rounding, except for the detail if there the resulting number should have more than just one digit.

Some samples:

Rounding to 0 digit:
0.498 -> 0
0.543 -> 1

Rounding to 1 digit:
0.498 -> 0.5
0.543 -> 0.5

Rounding to 2 digit:
0.498 -> 0.50
0.543 -> 0.54
User avatar
Karl02
Posts: 134
Joined: Mon Sep 03, 2007 1:00 pm
Location: Germany

Post by Karl02 »

I think I understand now what John meant: If the fraction is exactly equal to 0.50000000..., then the error would be the same for rounding up or down (+/- 0.5). Therefore, you could in one half of the cases round up and in the other half of the cases round down, depending on the digit on the left side of the 5, e.g.:

Code: Select all

             Error
0.500 -> 0   -0.5
1.500 -> 2   +0.5
2.500 -> 2   -0.5
3.500 -> 4   +0.5
4.500 -> 4   -0.5
5.500 -> 6   +0.5
6.500 -> 6   -0.5
7.500 -> 8   +0.5
8.500 -> 8   -0.5
9.500 ->10   +0.5
Now the rounding errors should eliminate each other statistically if you round many times.
-- Karl
User avatar
JohnFredC
XnThusiast
Posts: 2010
Joined: Wed Mar 17, 2004 8:33 pm
Location: Sarasota Florida

Post by JohnFredC »

Yes karl02.. that is exactly what I meant. Thanks. :)

Hence the even up/odd down (or vice versa) rule I mentioned. The even/odd criteria applies to the second least significant digit only when the least significant digit is 5.

When large populations of numbers are considered, the number of instances where the least significant digit is 5 will approach 10% (1/10) at the limit, so it is a straight-forward calculation to estimate the accumulating error of any rounding rule. The even/odd rule approaches 0 error at the limit, as illustrated by karl2's sample chart.

It's one thing to stand in front of a group of people at a marker board and illustrate the point with a marker and verbal demonstration. It's another to write out the explanation in a forum.

My posts in this thread are self-evidently not my best. My apologies.
John
Prof. Jewell

avoiding rounding errors in long calculations.

Post by Prof. Jewell »

JohnFredC wrote:Yes karl02.. that is exactly what I meant. Thanks. :)

Hence the even up/odd down (or vice versa) rule I mentioned. The even/odd criteria applies to the second least significant digit only when the least significant digit is 5.

When large populations of numbers are considered, the number of instances where the least significant digit is 5 will approach 10% (1/10) at the limit, so it is a straight-forward calculation to estimate the accumulating error of any rounding rule. The even/odd rule approaches 0 error at the limit, as illustrated by karl2's sample chart.

It's one thing to stand in front of a group of people at a marker board and illustrate the point with a marker and verbal demonstration. It's another to write out the explanation in a forum.

My posts in this thread are self-evidently not my best. My apologies.
I don't know if you guys would be interested or not, but the procedure that you are discussing had a name in the 19th century (back before computers) which is a bit ironic. Then it was called "the computer's rule." Back then a "computer" was a human being. :shock: