0.94 and 0.94.1 CRASH with 'double free or corruption (out)' [Linux]

[xnviocumo]

XnView MP Linux
Version 0.94 64bits (Nov 13 2019)
Libformat version 7.33
Ubuntu 18.04


XnView 0.94 crashes frequently with 'double free or corruption (out)'

Effect:

Since the upgrade to 0.94, XnView frequently (though not always) crashes when using or trying to use the Batch Convert tool. The more obvious error displayed is "double free or corruption (out)", although there are other warning/error messages also captured in the console. Please see here the full logs of running xnview from the console.

This is just the part of the logs starting exactly from the moment I click Menu -> Tools -> Batch Convert :

Code: Select all

ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
### ThumbLoaderThread :: setFileList(0)
### ThumbLoaderThread :: setFileList() wait
######ThumbLoaderThread :: run : 350de00 
######ThumbLoaderThread :: forever : 350de00 
 ** start condition wait
### ThumbLoaderThread :: setFileList() after wait
ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
### ThumbLoaderThread :: setFileList ok
ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
### ThumbLoaderThread :: setFileList(0)
ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
### ThumbLoaderThread :: setFileList ok
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
### ThumbLoaderThread :: setFileList(0)
ThumbLoaderThread :: stop()
### ThumbLoaderThread :: stop() ok
### ThumbLoaderThread :: setFileList ok
QSettings::endGroup: No matching beginGroup()
libpng warning: iCCP: known incorrect sRGB profile
QFileSystemWatcher::addPaths: list is empty
double free or corruption (out)
Aborted

To reproduce:

Important: Note that it doesn't necessarily crash 100% of the times, but it is very frequent, so much that it is not usable. I have noted it mostly in Batch Convert and have logged it, but I have also noted it in other operations, but I haven't logged them. In some cases, it does crash 100% of the times, especially after it has crashed the first time. With that said:

1. Enter a directory with several dozens of pictures (see my logs, in this case 74 jpg files).
2. Either select one, few, many or all files (it also happens without selecting any files!)
3. Select Menu -> Tools -> Batch Convert...

Actual behavior (bug): XnView crashes (see logs) exhibiting 'double free or corruption (out)' at the very last line before "Aborted". There are other messages that seem wrong too.

Expected behavior: Batch Convert tool launches and allow us to use it normally, without ever crashing.

Important additional information:

• Normally the problem happens after successfully having used Batch Convert a few times, until it crashes for example during the conversion process or at the begining of the conversion. Then it becomes unusable, crashing as soon as one selects "Batch Convert" from the main menu.

• In the example in my logs, the directory has only 74 jpg "normal" files, no hidden files, no any other kind of files.

• The problem happens with other files and other directories too, not only this one.

• There are no png files at all, tested with Linux 'file' command.
  However, in the logs there are many warning: 'libpng warning: iCCP: known incorrect sRGB profile', which is an old issue identified as related to versions of libpng > 1.6, and as far as I have been investigating is supposed to not be critical. Still, I find very strange because there are no png files involved.

• There is also the error "101 Unregistered schema namespace URI" happening one time for each of the files listed in the directory. In the example logs, 74 pics, and 74 times this error. Is this related to this bug: https://dev.exiv2.org/issues/0000640 ?

• All file names are without spaces or special characters.

• The original directory did have spaces in the names, so I copied it to /tmp/211874/
  and the problem is identical with or without spaces in the path or file names.

• The file sizes are between 182 KiB and 931 KiB, average is about500KiB

• With the

  Code: Select all

  file

  command in linux, this is the output of one of the average pictures:

  Code: Select all

  $ file pic_211874046.jpg 
  pic_211874046.jpg: JPEG image data, JFIF standard 1.02, resolution (DPI), density 72x72, segment length 16, Exif Standard: [TIFF image data, little-endian, direntries=10, manufacturer=NIKON CORPORATION, model=NIKON D3S, orientation=upper-left, xresolution=162, yresolution=170, resolutionunit=2, software=Adobe Photoshop CS4 Windows, datetime=2010:10:15 09:41:49], progressive, precision 8, 1065x1600, frames 3

  All the others are very similar to this, diferences are sizes.

• With imagemagick's

  Code: Select all

  identify

  command, this is the typical output:

  Code: Select all

  $ identify pic_211874046.jpg
  pic_211874046.jpg JPEG 1065x1600 1065x1600+0+0 8-bit sRGB 575KB 0.030u 0:00.019
  $ identify pic_211874054.jpg
  pic_211874054.jpg JPEG 1600x1065 1600x1065+0+0 8-bit sRGB 364KB 0.030u 0:00.030

imagemagick Version: 8:6.9.7.4+dfsg-16ubuntu6.8
libpng16-config version: 1.6.34

[Edit: Added "and 0.94.1" to the subject, because the problem also affects 0.94.1]

[xnview]

do you use 'actions'? is it possible to have these 74 jpeg files? No problem with previous version?
do you use 'core' option?
could you post a screenshot of 'output' tab?

[xnviocumo]

do you use 'actions'?

Yes. I had been using one: "Level". Sometimes I was using a "scritp" of three actions: a resize, autolevels and sharpen, that I had saved as a script (see below).

is it possible to have these 74 jpeg files?

That particular set I would prefer not, but if you give me a little time I will try to get another group of pics where I also get problems. I am not totally convinced that it is related to specific folder or pics, but it makes sense to send you perhaps a zip file with a folder that has failed for me. Let me try and experiment more. It is also nasty that at some point it crashes always if I select "Batch Convert" without having any folder with pictures selected. This means, I open XnView and it's by default in the root directory '/' because I don't have it to remember last directory; so without doing anything else, if I select Tools -> Batch Convert, then it crashes. I have to experiment it more to see if this is reproducible all the time.

Does this happens to you? What is the expected behavior if one tries to open Batch Convert on '/' where there are no pictures at all? Error message or it should open and let us browse files?

Yesterday I couldn't get Batch Convert to work at all anymore, with or without any folder with pictures selected or with or without any pictures selected. Today, after crashing a couple of times, I managed to select a few random pictures and selected "Batch Convert" and voilà, there you have below the two screenshots, below. As I am writing this, I have not pressed "Convert" yet.

No problem with previous version?

As far as I can remember, I didn't have problems with the previous version (0.93). I was not using Batch Convert so frequently either, but I did use it.

do you use 'core' option?

Yes, the "Use CPU Cores" version is checked, "Use CPU Cores: 4 ". I think this was like this by default when I upgraded to 0.94 because I don't think I checked that, at least not in 0.94. I may be wrong. I know that in 0.93 whenever I tried the cores, I got an error that I don't remember, something like I couldn't use that option if I had selected something else, but it didn't bored me, so I just didn't use the cores option, I thought I would see some other day what configuration needed to be changed.
Do you think I should try without this option checked?

could you post a screenshot of 'output' tab?

See the output tab and also the actions tab:

output tab

batch_conv-output_tab.jpg (48.35 KiB) Viewed 976 times

actions tab

batch_conv-actions.jpg (58.73 KiB) Viewed 976 times

[xnview]

but it makes sense to send you perhaps a zip file with a folder that has failed for me. Let me try and experiment more. It is also nasty that at some point it crashes always if I select "Batch Convert" without having any folder with pictures selected. This means, I open XnView and it's by default in the root directory '/' because I don't have it to remember last directory; so without doing anything else, if I select Tools -> Batch Convert, then it crashes. I have to experiment it more to see if this is reproducible all the time.

yes please

Yes, the "Use CPU Cores" version is checked, "Use CPU Cores: 4 ". I think this was like this by default when I upgraded to 0.94 because I don't think I checked that, at least not in 0.94. I may be wrong. I know that in 0.93 whenever I tried the cores, I got an error that I don't remember, something like I couldn't use that option if I had selected something else, but it didn't bored me, so I just didn't use the cores option, I thought I would see some other day what configuration needed to be changed.
Do you think I should try without this option checked?

yes please

[xnviocumo]

Thank you, Pierre.
Ok, I will do as mentioned. Please allow me a little of time, I want to make those experiments to see if I can provide you something more for you to work with. I know it's difficult with not so much information that what I have currently. I am not sure if I can dedicate time today for this, but I'll update you as soon as I have something useful.
Thanks!

[xnviocumo]

The problem also affects the version 0.94.1.

On a long session of troubleshooting and investigation, I captured debugging information that I think is quite relevant. Odd enough, the place where I found this XnView debug information, is in the ~/.xsession-errors file.

In fact, this is creating a silent problem: XnView is writing a huge amount of debugging logs --not session errors, as one would expect-- to ~/.xsession-errors and this file will grow to a ridiculously big size, most especially on systems that are not shutdown for long time (e.g. just doing "Suspend" or "hibernate"). I got my entire /home partition completely full with this file growing crazy since many days. That's quite bad and important, but a separate topic, though. But back to the first point:

XnView 0.94.1 crashed again in the same fashion as described before, but this time I was very carefully tracing the origins of the writes made to ~/.xsession-errors, for which I wrote a script and set up the experiment to catch XnView's "contribution" when it crashes. The following debug lines are from the last milliseconds before the crash. Note especially, the last 5 lines, which include the fatal 'double free or corruption (out)', but this time with more context:

Code: Select all

######### CategoryModel :: checkAndSave()
MyView::set LOAD FULL /some/path/to/netwpic_904014.jpg
BitmapLoadThread :: stop()
LOAD INFO </some/path/to/netwpic_904014.jpg>
BitmapLoadThread: load preview 160 120  (0)
## BitmapLoadThread :: load 160 120 
## Bitmap :: load 2048 
###### LOAD BITMAP </some/path/to/netwpic_904014.jpg>
######### CategoryModel :: checkAndSave()
  ## BitmapLoadThread :: loaded 170 256 
BitmapLoadThread: load full
## BitmapLoadThread :: load -1 -1 
## Bitmap :: load 0 
###### LOAD BITMAP </some/path/to/netwpic_904014.jpg>
## MyView::onLoaded </some/path/to/netwpic_904014.jpg> </some/path/to/nelibpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
QSettings::endGroup: No matching beginGroup()
libpng warning: iCCP: known incorrect sRGB profile
QFileSystemWatcher::addPaths: list is empty
double free or corruption (out)
Aborted

What I was doing with the program to create the issue, was extremely simple:

1. Open XnView by clicking its icon. By default, it lands on my root partition.
2. Selected a dir with pics, in Browser view.
3. Selected a group of about 25 pics
4. Pressed <Ctrl><U> to open Batch Convert, and in that moment, XnView 0.94.1 crashed, as shown in the debugging info of those last milliseconds.

Additional notes at this point:

During the approximately 40 seconds that took to complete those four steps, XnView wrote almost 10000 lines in ~/.xsession-errors, including many "libpng warning" (per `fatrace`, the origin is the XnView process, though the complaint is from libpng) and an enormous amount of debugging, `printf` details of everything XnView is doing. Every time I run XnView and just start clicking around, the xsession-errors file grows exaggerately just by XnView's contribution. It is not the only process that writes there, of course, but this contribution is just not sane.

In Linux, the uncontrolled growth of ~/.xsession-errors is an old problem that is debated since years and usually the blame falls on the applications that for some reason end up using this incorrectly (there is a lot of information and debate online about this problem), which forces users to create mechanisms to cripple the original intended functionality of the ~/.xsession-errors file. I am not an expert on this particular topic, so my opinion is not relevant in this issue. But that is what I have learned in many hours of research and experimentation. Not knowing any better, I will also have to cripple the xsession-errors functionality because I can't afford the uncontrolled growth of of it.

That said, I have not yet done the tests I want to do, time is limited (oh, crap! it's 5AM!) and I had to understand what was going on with the lack of space in my /home partition, and I also found a couple of other unrelated sources of trouble, so now I am definitely much more happier!

But above all, and most importantly: Thank you so much for this brilliant piece of software!

[xnview]

That said, I have not yet done the tests I want to do, time is limited (oh, crap! it's 5AM!) and I had to understand what was going on with the lack of space in my /home partition, and I also found a couple of other unrelated sources of trouble, so now I am definitely much more happier!

was you able to make tests?