Page 1 of 1

There is a clear anomaly in the compression of reversible webp in XnViewMP1.6.0 and later.

Posted: Wed Apr 17, 2024 10:59 am
by abobong
XnviewMP1.7.1 64bit
Windows 11
cpu Ryzen 9 5950X 32 threads
gpu gtx 1080
memory 32G ddr2666 operating in settings

Reproduction method:
・Compress each of the prepared images with webp. Prepare the latest XnviewMP (64bit) and XnviewMP1.5.5 (64bit), and set each to reversible (Compression method 6).
When comparing both versions, the compression rate of the latest version of webp is always lower.

I will state the conclusion at the beginning.

・There is a clear anomaly in the compression of reversible webp in XnViewMP1.6.0 and later. The cause is almost certainly in the latest xnviewmp.exe.
・If we have narrowed down the file causing the problem to this extent, I would like it to be recognized as a bug that should be resolved.
(Since it's not open source, we can't touch the program source, so all we can do is ask the programmer to tackle it.)


This is a repeated request.

・For the smallest images, there is only a 4-byte difference, but for webp images exceeding 6000 pixels in both width and height, the capacity increases by nearly 50 kilobytes per file in the latest XnViewMP.
・I have been using the reversible webp image of XnViewMP for a long time and plan to continue using it in the future, so I don't want this problem to be left unattended any longer. (The same bug exists in XnConvert.)

・The function [Keep original file when encoded result is larger] is very wonderful. However, the abnormal compression rate of webp becomes a bottleneck, so I still can't switch to the latest version.

・Until now, I thought there was a problem with libwebp (a plugin from an external source). However, this is a clear bug caused by xnviewmp.exe.

・I would like to request that the compression rate of reversible webp be improved. I think I will continue to make the same request regularly until the problem is resolved. (I can't wait for years for other wise men to present the same problem.)

-------------------------------------------------------------

The following is the process that led to this point.

This is a continuation from the content of the previous article. Please refer to it. (The same issue has been continuing since version 1.6.0.)
viewtopic.php?f=62&t=46805&p=195696#p195696
updated to XnviewMP1.7.1.
The version I am comparing with is XnviewMP1.5.5, which is the version just before the problem occurred.


001.png
001.png (6.64 KiB) Viewed 1344 times
[001.png]
-Basic Settings-
sumple.png
sumple.png (96 Bytes) Viewed 1344 times
[sumple.png]
-Sample-



*What I tried*

1.
・Swap the xnviewmp.exe of XnviewMP1.7.1 and XnviewMP1.5.5.
XnViewMP1.7.1 embedded with the xnviewmp.exe of XnViewMP1.5.5 (confirmed to start. This will be referred to as [A].)
XnViewMP1.5.5 embedded with the xnviewmp.exe of XnViewMP1.7.1 (unable to start. This will be referred to as [a].)
・If you convert the sample image to webp with Compression method 6 in [A] and it is generated at 30 bytes, it means that a normal file is being created.

*Results*
Since a 30-byte webp image was generated, it can be hypothesized that there was an anomaly in the xnviewmp.exe of XnViewMP1.7.1, which was the only one excluded from the combination of [A].


・However, if we consider the possibility of abnormalities in files other than xnviewmp.exe in XnViewMP1.7.1, we need to test more combinations.

・When converting webp with Compression method 6 using XnViewMP1.7.1, it will always generate a 34-byte file. However, for operation verification, I will gradually reconfigure to older dlls.

From here on, it's a detour to derive the answer, so if you're in a hurry to conclude, you can skip to Experiment 4. It's okay.


2.
・Delete the subfolder of XnViewMP1.7.1, copy all the subfolders and the files contained within from XnviewMP1.5.5 to XnViewMP1.7.1, and check if it can be launched.

*Results*
I was able to convert to a 34-byte webp image with Compression method 6, so considering the results of Experiment 1, I can conclude that there is no problem with the plugins contained in the subfolder of XnViewMP1.7.1.


3.
・Overwrite each dll in the same hierarchy as xnviewmp.exe in XnViewMP1.7.1 with the dll in XnViewMP1.5.5, and check the operation.
Check if the sample can be converted to a webp image using Compression method 6.
・If the application fails to launch as a result of overwriting the dll, it will not be possible to verify its operation, so consider it as a candidate for suspicious files.
・dll files newly added from XnviewMP1.5.5 to XnViewMP1.7.1 will also be considered as suspicious file candidates.
test3 image.png
*Results*

・DLL that no longer launches after overwriting with older versions
[Qt5Widgets.dll, Qt5Gui.dll, Qt5Gui.dll, Qt5Core.dll]

・DLL that produces an error message and doesn't launch after overwriting with older versions
[Qt5Network.dll, mdk.dll]

・Newly added DLLs
[libssl-3-x64.dll, libcrypto-3-x64.dll [*Note 1]]



・The above eight dlls cannot be swapped with the dlls of the same name in XnviewMP1.5.5. In other words,
it is not possible to verify the operation of each individual file.
For convenience, let's call these eight DLL files [β].



・However, as of Experiment 1, since [A] was operating normally during the sample image conversion, the suspicion on [β] is weak.

・If you can even launch [a] combined with [β], you will be able to narrow down the suspicion to only xnviewmp.exe of XnViewMP1.7.1.
This is because there were no abnormalities in [β] during the experiment with [A].

(*Note 1: Tried to replace libcrypto-1_1-x64.dll and libssl-1_1-x64.dll included in XnViewMP1.5.5, but it did not launch.)


4.
We will combine Experiment 3 [β] (eight specific DLL files included in XnViewMP1.7.1) with [a] (XnViewMP1.7.1's xnviewmp.exe imported into XnViewMP1.5.5 that cannot be launched) from Experiment 1 and confirm if the program can be launched.
We will also convert the sample image.

*Results*
We confirmed that the program can be launched and also converted the sample image to webp format using Compression method 6 with 34 bytes. Therefore, [β] can be ruled out as the cause of malfunction. (*Note 2)
We will refer to this combination as [a'].

・We imported XnviewMP1.5.5's xnviewmp.exe into [a'] and converted the sample image to webp format using Compression method 6. As a result, a 30-byte webp image was generated.
This led us to conclude that there is an issue with XnViewMP1.7.1's xnviewmp.exe.

([*Note 2] It was not possible to replace [libcrypto-1_1-x64.dll , libssl-1_1-x64.dll] from XnviewMP1.5.5 with [libssl-3-x64.dll , libcrypto-3-x64.dll] from XnViewMP1.7.1.
It can be assumed that this is a compatibility issue between libss and libcrypto, and is not directly related to the issue at hand.)

-------------------------------------------------------------

*Extra* [What will happen in the actual work process?]
・I will compress JPEG images, which are not yet processed for storage, into webp format and test how much the difference will be with various settings.
・I will convert approximately 112 files of JPG images, with an average size of 8MB and a resolution of around 5200x7200 pixels, into the webp file format. (Assuming that all images can be compressed to reduce their file size.)
・Since I was processing it for personal use, it includes two sumple.webp files (totaling 60 bytes), and it is displayed as 114 files on the property.


original.png
original.png (12.95 KiB) Viewed 1344 times
Original
941 megabytes, 987,006,086 bytes)



ver 1.5.5 q-6.png
ver 1.5.5 q-6.png (12.91 KiB) Viewed 1344 times
XnViewMP1.5.5 Compression method 6
824 megabytes, 864,052,787 bytes, compression time 3 minutes
・Since it doesn't take much time in my environment, I use the highest compression setting as a standard.



ver 1.7.1 q-6.png
ver 1.7.1 q-6.png (17.66 KiB) Viewed 1344 times
Compression method 6 on XnViewMP 1.7.1 (the latest version under scrutiny):
829 MB or 870,092,247 bytes compressed in 1 minute.
・The compression time is similar to XnviewMP1.5.5's Compression method 5,
but the compression ratio is about the same as XnviewMP1.5.5's Compression method 2.


ver 1.5.5 q-5.png
ver 1.5.5 q-5.png (12.9 KiB) Viewed 1344 times
XnViewMP1.5.5 Conpression method 5
825 megabytes, 865,604,663 bytes, compression time 1 minute
・The performance of time and compression ratio is very good.


[A].png
[A].png (12.91 KiB) Viewed 1344 times
Experiment 1. [A] Compression method 6 (XnViewMP1.5.5's xnviewmp.exe embedded in XnviewMP1.7.1)
824 megabytes, 864,250,281 bytes, compression time 2 minutes [One more sample image of 30 bytes was mistakenly included]
・This is a combination of XnViewMP1.5.5 with the latest plugin, but there was almost no difference compared to [XnViewMP1.5.5 Compression method 6].
・However, I feel that the compression time has been slightly shortened.
・This [A] performance is a simulation of the latest version of XnViewMP without any bugs.



*General review*
・Although it's rough, the results have been further refined with the addition of the compression time element.
・What became clear as a result was that there was not much difference in the compression processing time between XnViewMP1.7.1 Compression method 6 and XnViewMP1.5.5 Compression method 5.
・In addition, the compression ratio of XnViewMP1.7.1 Compression method 6 is significantly lower than that of XnViewMP1.5.5 Compression method 5, so this can be recognized as a clear bug, not a malfunction.
・Can we really ignore a bug that has been so clearly identified?
(All of them seem to have no significant difference in compression time, but it only appears that way because 32 threads are being processed simultaneously.)

Please actively consider fixing this in the next version.



Dear Developer,

Re: There is a clear anomaly in the compression of reversible webp in XnViewMP1.6.0 and later.

Posted: Sun Apr 28, 2024 1:52 pm
by xnview
Nothing has changed between 1.5.5 & 1.6.0 in the writing function. 1.6.0 and later now use LibWebP 1.3.2

Re: There is a clear anomaly in the compression of reversible webp in XnViewMP1.6.0 and later.

Posted: Mon Apr 29, 2024 1:31 pm
by abobong
As you pointed out, there is little difference in performance of libwebp itself compared to the old version. However, the current reversible WebP settings in XnviewMP are designed to sacrifice maximum compression ratio in order to significantly improve speed.

If this is an intentional customization, making irreversible adjustments is not a good solution. In the future, with faster PC environments becoming more widespread and compression speeds also increasing, settings that prioritize speed may lose their importance. Limiting the maximum compression ratio at this point may not benefit end users in the future.

Re: There is a clear anomaly in the compression of reversible webp in XnViewMP1.6.0 and later.

Posted: Wed May 01, 2024 2:59 pm
by xnview
do you have same thing when use cwebp from 1.3.2?

Re: There is a clear anomaly in the compression of reversible webp in XnViewMP1.6.0 and later.

Posted: Fri May 03, 2024 6:35 am
by abobong
Here are the results of running cwebp with the libwebp1.3.2 GUI:
(I am unable to handle Command Prompt.)

For the files compared in this case, compressing with the libwebp1.3.2 GUI showed even higher compression rates (approximately 821MB, although I have no screenshot as proof). Since the number of threads that can be processed simultaneously is different, I did not measure the required time.

However, since I was unable to reproduce the same results as XnviewMP even when changing the settings for cwebp, I decided that my results might not be impartial and refrained from submitting them.

As a side note, I re-encoded the 821MB file that was output from cwebp with Compression Method 6 using XnviewMP1.7.7.1 under the same conditions, and the resulting file size was 824MB. This is almost the same as Compression Method 6 of XnViewMP1.5.5, but I have not investigated whether this is just a coincidence.

While it may be easy to classify the discovered phenomenon as a bug from a user's standpoint, it can be difficult to fully prove the phenomenon along with its cause. I apologize for any careless statements made regarding this matter.