1.8.6 Infrequent Bug

[abobong]

Sometimes encoded images are not generated.

In multi-encode mode, if 'delete encoded images' is on, the original is deleted even if the copy wasn't made.

(In the logs, it's shown as: "Processing successful 0 bytes (-100%)")

Suggestion: If a 0-byte or 0x0x0 image is generated, it should be considered a failure and skipped, regardless of encoding success.

[xnview]

in batch convert? do you a way to reproduce?

[abobong]

Batch conversion, highly sporadic, low reproducibility.
However, treating 0 byte or 0x0x0 data generation as a collective error might prevent issues.
Problem is very hard to detect.（Difficult to connect to the problem even if discovered later.）

Request: Could batch convert handle 0 byte or 0x0x0 data generation as a group of errors?

・Normal : 0 bytes 0x0x0 → Creation failed. Skipped and counted as an error.

・this case: 0 bytes 0x0x0 (-100%) → Treated as creation success, original size comparison is displayed.

------

Batch convert, très sporadique, faible reproductibilité.
Traiter 0 byte/0x0x0 en erreur collective pourrait éviter des problèmes. 
Le problème est très difficile à détecter. (Difficile de relier au problème même s'il est découvert plus tard.)

Ne pourrait-on pas traiter toute sortie de données 0 byte ou 0x0x0 comme une erreur ?

・Normal : 0 bytes 0x0x0 → Création échouée. Ignorée et comptabilisée comme une erreur.
・Ce cas : 0 bytes 0x0x0 (-100 %) → Traitée comme une création réussie, la comparaison de la taille originale est affichée.

[xnview]

Ne pourrait-on pas traiter toute sortie de données 0 byte ou 0x0x0 comme une erreur ?

・Normal : 0 bytes 0x0x0 → Création échouée. Ignorée et comptabilisée comme une erreur.
・Ce cas : 0 bytes 0x0x0 (-100 %) → Traitée comme une création réussie, la comparaison de la taille originale est affichée.

Ces cas ne sont pas traités comme erreurs?

[abobong]

Quoi qu'il en soit, le fait de ne pas avoir pu conserver les captures d'écran ou les journaux est de ma faute.

[abobong]

We've had a development in this issue, and fortunately, we were able to capture the moment the bug occurred. Although we've experienced actual damage, the fact that the problem has become clear is a silver lining.

el.png (15.13 KiB) Viewed 428 times

The relevant part of the log is as follows:
E:\～　～\069.webp : 保存(save)
・・・・・・・・・・・・・・・・・・・・・3.02MBー〉0バイト(byte)(-100%)
Other Japanese examples are... 読み込み中(Loading...) , 読み込み(Load Image)

In this case, not even a 0-byte jxl file is generated, and the original image is moved to the trash.

While identifying the root cause is our ultimate goal, our top priority at this point is to implement a preventative measure against file loss due to misrecognition. Therefore, ensuring these files are not sent to the trash is of utmost importance.

The "'delete encoded images' is on" feature has also caused normal file deletions due to unexpected encoder behavior in other instances. Details: viewtopic.php?p=204974#p204974

Therefore, we strongly propose adding the following behavior:

If a 0-byte file is saved during file generation.
In the event that 'delete encoded images' is enabled and the generated file is 0 bytes, the original image ought not to be deleted irrespective of the outcome of the encoding process.

Investigating the bug at this stage is like repeatedly trying to identify the cause while there's a risk of data loss, which is far too dangerous.

Implementing this is an unavoidable necessary step and should be considered an important investment for the "'delete encoded images' is on" feature.

[xnview]

it's always when output is jpeg-xl?

[abobong]

This symptom only occurs during the encoding of jpg xl.
To my great dismay, the current situation is incredibly accidental. However, the reason I was somehow able to discover this is because once this symptom appears in a file folder, it continues to produce the same error for the remaining unprocessed files. In other words, I realized it because I witnessed the moment the file encoding logs started accelerating abnormally.

[xnview]

do you use 'multiple cpu core'?

[abobong]

Yes. I am allocating 28 out of 32 threads.

[xnview]

with 4 or without threads, are you able to reproduce?

[abobong]

I haven't tested it yet, and I'm hesitant to because success would mean losing files. Debugging is time-consuming since it involves real-world operation. If we had preventative measures, it would be easier for others to take over when they encounter the same problem. With prevention in place, error reports would simply state that file compression occasionally fails. Preventing file loss should be the priority.

[abobong]

I tried encoding with 2000 images per thread for five sets, and 10000 images with four threads for one set.

I didn't encounter any particular issues. It was clearly faster for the 2000 images with one thread to finish the measurement compared to the 10000 images with four threads.

My hypothesis is that this error becomes more likely to occur when the PC is under heavy load.

[abobong]

Dear Developers,
Regarding the data loss issue during JPEG-XL conversion, I believe two events are occurring:

1.Encoding Error: Rarely, the JPEG-XL encoding process does not complete normally, and the file is not generated (although the logs indicate that a 0-byte file was successfully generated).
2.Deletion Function Based on Post-Encoding File Size: If the post-encoding file size is smaller than the original file, the original file is deleted.
Currently, I assume the developers are primarily concerned with investigating the cause of the encoding error (1).
However, the fundamental cause of this data loss accident is that the deletion function (2) is being applied even to files that were not even generated by the encoding error (displayed as 0 bytes in the logs).
Therefore, resolving (1) first does not justify neglecting (2). It is necessary to modify the behavior of (2) as appropriate and improve its functionality.
In addition, urgently trying to resolve (1) without first addressing (2) is extremely difficult and involves a significant risk of data loss with each investigation. This is completely unreasonable.
If the same error occurs and a user continues to use the software without noticing it, what will their saved folders look like five years from now? Their precious family photos that were encoded may have disappeared.The problem with this error is that it is very difficult to notice until it is actually discovered.

[xnview]

i've sent you a PM