MP v0.72 x64 windows memory fault.

*** Please report new bugs here! ***

Moderator: Dreamer

Post Reply
CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

MP v0.72 x64 windows memory fault.

Post by CameronD » Thu Feb 26, 2015 5:31 am

I have win 7 x64 with 8GB physical RAM and a fixed 8.5GB Paging space permanently allocated.

Recently XnViewMP was killed by windows after doing almost nothing, and Windows gave me a warning about running low in memory, even though task manager continually showed physical memory running at around 3 to 3.5GB free and another 2GB or so "standby" - containing known data but available if necessary.

The Event viewer showed a message from Resource-Exhaustion-Detector:

Code: Select all

Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: xnview.exe (3372) consumed 13125148672 bytes, firefox.exe (8012) consumed 309899264 bytes, ... etc
Note - this is 13GB!

So I started the Resource Monitor from the task manager page, and watched while I ran XnViewMP.
When only the file browser window was open and I was wandering between folders the "Commit" value crept uop from around 25MB to over 100MB and seemed to stabilise.
The Working set looked OK, going up and down in the 100MB to 300MB range.

As soon as I open a single image window in XnView the commit value jumps by 4GB, and sometimes to 8GB.
As I select left or right to go though the images, the commit value bounces around in multiples of 4GB.

When I kill the image window and return to the browser the commit value remains at 4GB until I select a different folder at which time it drops back to the 100MB region.

This is quite reproducible at the moment.

Hmm, I thought this sounded familiar - I reported this a year ago with V0.64: http://newsgroup.xnview.com/viewtopic.php?f=62&t=29312 but then it seemed to be associated with rebuilding thumbnails.

This is not a memory leak as such, because the program eventually frees the memory. It looks like a 32-bit value of -1 is suddenly interpreted as a 64-bit unsigned value.
Last edited by CameronD on Thu Feb 26, 2015 1:03 pm, edited 1 time in total.

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Thu Feb 26, 2015 6:16 am

If I use up a bit more RAM, by running a 1GB VM, then it seems to notice that the 3rd increment is not available and only bumps the commit from 8GB to a bit under 10 GB.

So I suppose that means it must get killed when another program increases its own memory allocation.

Update: On a freshly booted system the problem is not as reproducible.
If I simply open an image from the browser and then slowly go forwards and back through the images then the Commit charge remains around the 100MB value. If I scroll quickly through the images then it will occasionally jump by 4GB, and rarely to 8GB but as I scroll through again then it will eventually drop back to less than 100MB.

User avatar
xnview
Author of XnView
Posts: 32081
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP v0.72 x64 windows memory fault.

Post by xnview » Thu Feb 26, 2015 1:04 pm

CameronD wrote: As soon as I open a single image window in XnView the commit value jumps by 4GB, and sometimes to 8GB.
As I select left or right to go though the images, the commit value bounces around in multiples of 4GB.
The opened folder contains a lot of files?
Pierre.

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Fri Feb 27, 2015 12:10 am

xnview wrote:The opened folder contains a lot of files?
That is not a factor.
Sometimes only 4 files. I can trigger the bug when the browser is in details view (no thumbnails). The times that I was scrolling rapidly through to trigger it the folder might have had 100 images.

One important point is that you need to be using something other than task manager to see the issue. I presume you have better tools in your software development kit, but for anybody else, the simplest way is to
  • type "Resource monitor" into the start menu search window
  • select the memory tab
  • run xnviewMP, find xnview.exe in the list and tick the checkbox on the left - this keeps xnview at the top of the list
  • Then watch the Commit values as you scroll through images.
  • It only samples once a second, and the graph on the right shows the previous minute's results, with big jumps as the 4GB mallocs and releases happen.
I just restarted XnViewMP, selected a folder with 5 images in the browser window, opened the first image and it made an 8GB malloc.
The monitor sampling interval is too slow to show if it was two mallocs of 4GB each.
When I closed the full image tab it released 4GB.

I can also trigger the bug in the browser window alone by selecting a single image and it displays in the preview window. Then use the up/down arrows to select different images and eventually it will show the 4GB jump.

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Fri Feb 27, 2015 12:35 am

I just tried to reproduce it on a laptop, which has 6GB RAM and could not see any evidence of a problem.
I cannot think of any significant differences other than RAM capacity, as much the same software is installed on both.

User avatar
xnview
Author of XnView
Posts: 32081
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP v0.72 x64 windows memory fault.

Post by xnview » Fri Feb 27, 2015 10:23 am

CameronD wrote:I just tried to reproduce it on a laptop, which has 6GB RAM and could not see any evidence of a problem.
I cannot think of any significant differences other than RAM capacity, as much the same software is installed on both.
same image files?
Pierre.

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Fri Feb 27, 2015 3:29 pm

xnview wrote:same image files?
No, but it happens on so many different images that I do not think that is important.

I will try again tomorrow, with two more PCs. and the same images.

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Sat Feb 28, 2015 2:18 am

I think I have found the common factor. I can reproduce on:
  • desktop: win 7 7GB RAM
  • laptop: win 7 8GB RAM
  • another desktop: Win 8.1 16GB RAM
If I enable "use ICC profile for monitor" then the 4GB jumps happen. As soon as I disable this then XnView behaves properly.

Both win 7 machines have custom monitor profiles from the i1display Pro as the system profile.
The win 8.1 machine has no custom profiles installed.
The Win 8.1 machine showed the bug in two situations:
  • use custom profile was ticked, but profile name was blank (accidental test)
  • use system profile (default sRGB)
On the win 8.1 machine, XnView could be made to allocate up to 25GB for itself, by scrolling rapidly through the images.

User avatar
JohnFredC
XnThusiast
Posts: 2010
Joined: Wed Mar 17, 2004 8:33 pm
Location: Sarasota Florida

Re: MP v0.72 x64 windows memory fault.

Post by JohnFredC » Sat Feb 28, 2015 4:43 pm

I just experienced this problem. Only the Browser was open, and only a few (but very large) files in the folder. The thumbs for the files in the folder had already been created.

The MP session had been running for over 24 hours and had been open at the same folder for most of them, but just sitting idle.

Windows 8.1.
John

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Sun Mar 01, 2015 1:09 am

JohnFredC wrote:... just sitting idle.
Yes, it seems to require another program allocating memory to trigger the windows warning. MP will happily gobble up nearly all available memory, but does not complain if it cannot get more.

User avatar
xnview
Author of XnView
Posts: 32081
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP v0.72 x64 windows memory fault.

Post by xnview » Mon Mar 09, 2015 9:26 am

CameronD wrote:As soon as I disable this then XnView behaves properly.
could you send me the profile used, and an image file to test?
do you use 'Read ahead' or 'cache behind'?
Pierre.

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Mon Mar 09, 2015 3:12 pm

xnview wrote:
CameronD wrote:As soon as I disable this then XnView behaves properly.
could you send me the profile used, and an image file to test?
The default sRGB profiles in Windows 7 and Win 8.1 will cause the problem.

The images that cause problems all seem to have have embedded ICC profiles - either the standard sRGB provided by Canon, or photoshop's AdobeRGB (1998).
I have tested images from about 6 cameras. Any that were taken straight from the camera do not have an embedded ICC profile, but just provide a bit of colourspace info in the EXIF header.
If I use software to process from raw images then ICC profiles become embedded - also if I process in photoshop then sometimes they also get an embedded profile.
It seems to be likely to happen (but not always) for any image that XnView shows with the little "ICC" icon in the Info panel of "details" view.
I will do a bit more experimenting before sending you test images.
do you use 'Read ahead' or 'cache behind'?
I am not sure what you are asking here. Is this XnView setting, or OS?

User avatar
xnview
Author of XnView
Posts: 32081
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP v0.72 x64 windows memory fault.

Post by xnview » Tue Mar 10, 2015 7:59 am

CameronD wrote:
do you use 'Read ahead' or 'cache behind'?
I am not sure what you are asking here. Is this XnView setting, or OS?
Settings>View>Misc
Pierre.

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Tue Mar 10, 2015 2:09 pm

xnview wrote:
CameronD wrote:
do you use 'Read ahead' or 'cache behind'?
I am not sure what you are asking here. Is this XnView setting, or OS?
Settings>View>Misc
Under cache: both "read one image ahead" and "keep current image" are enabled.

I disabled both and it made no difference.

CameronD
Posts: 303
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP v0.72 x64 windows memory fault.

Post by CameronD » Fri Mar 20, 2015 3:07 pm

I have emailed some small test images.

I can reproduce the problem with folders containing single images. The big memory allocation occurs either when I select an image to show a preview, or double click to open the full view.

Each image had the profile applied by Photoshop CS.

sRGB, Adobe RGB and AppleRGB all show the memory problem.

I also tried two greyscale profiles but neither gave any problem.

Post Reply