M1 Mac Excessive Memory usage on Batch Convert

*** Please report new bugs here! ***

Moderators: helmut, XnTriq, xnview, Dreamer

John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

Upon performing a Batch Convert of some 500 images, I noticed excessive usage of memory. If I convert too many images, I eventually get a message asking to force quit an app to create more memory. Below is an screenshot of the memory usage:
Attachments
Screenshot 2021-10-28 at 21.20.31.jpg
Last edited by John_Matthews on Thu Oct 28, 2021 7:44 pm, edited 1 time in total.
John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

I should have added that it ended up using 34GB of memory and the ONLY way to lower the memory usage was to quit the app.
User avatar
xnview
Author of XnView
Posts: 46255
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by xnview »

which formats in input and output?
Pierre.
John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

There were multiple formats: ORF, RW2, jpg
Outputs were all jpg

Xnview was successful, just with a limited number of batch converts and I had to quit to liberate that memory.
I'm fairly confident you can recreate this on any M1 mac running Big Sur.
User avatar
xnview
Author of XnView
Posts: 46255
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by xnview »

I can't reproduce, do you use multi core? If you convert jpg into jpg, same problem?
Pierre.
John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

Hello Pierre,
I think this happens only with RAW files. I tried with only jpg and was unable to reproduce it (multicore selected).

Here are my exact settings for the RAW export:
Attachments
Screenshot 2021-11-01 at 14.20.35.jpeg
Screenshot 2021-11-01 at 14.20.16.jpeg
Screenshot 2021-11-01 at 14.20.05.jpeg
Screenshot 2021-11-01 at 14.19.50.jpeg
Screenshot 2021-11-01 at 14.19.41.jpeg
Screenshot 2021-11-01 at 14.19.29.jpeg
John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

I just retested with multicore unselected and batch convert exhibited the same behavior. My thought is that somehow the loop is not clearing memory after each save. The end result is a memory creep that never clears unless xnview is closed. After converting enough files, OSX will prompt "Force quit" to free up resources. This is what happened to me after converting about 1000 raw files.

Another behavior questionable behavior is that when you manipulated images in the "Actions" tab, there's also a memory creep problem, slowly taking up more and more memory after about 20 minutes of manipulating raw files. Perhaps, it's related?

Attached is a screen shot result of having only 1 core doing the work. I stopped the processing after about 20 minutes. Even after stopping it, Activity Monitor is show xnview taking considerable memory and never releasing it. :(
Attachments
Screenshot 2021-11-01 at 14.51.20.jpg
User avatar
xnview
Author of XnView
Posts: 46255
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by xnview »

John_Matthews wrote: Mon Nov 01, 2021 1:54 pm I just retested with multicore unselected and batch convert exhibited the same behavior. My thought is that somehow the loop is not clearing memory after each save. The end result is a memory creep that never clears unless xnview is closed. After converting enough files, OSX will prompt "Force quit" to free up resources. This is what happened to me after converting about 1000 raw files.
Is it possible to send me a raw file?
Pierre.
John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

Well, I have thousands of RAW files... which one? I really don't think it's a specific type of RAW file.

Anyway, here's a link to some RAW ORF files on dpreview:

https://www.dpreview.com/sample-galleri ... e-gallery/

Look on the right-hand side and you'll see a way to download the RAW image of the selected image.
User avatar
xnview
Author of XnView
Posts: 46255
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by xnview »

is it the 1.94 version?
and what do you have in 'load format settings' > RAW?
Pierre.
John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

I have:

XnView MP MacOSX
Version 0.99.1 64bits (Sep 23 2021)
Libformat version 7.95

I could not find "Load format settings".
User avatar
xnview
Author of XnView
Posts: 46255
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by xnview »

John_Matthews wrote: Mon Nov 01, 2021 3:26 pm I could not find "Load format settings".
So it's in Settings>Formats
Pierre.
John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

Here are my setting > Formats:
Attachments
Screenshot 2021-11-01 at 16.36.52.jpeg
Screenshot 2021-11-01 at 16.36.39.jpeg
Screenshot 2021-11-01 at 16.36.28.jpeg
User avatar
xnview
Author of XnView
Posts: 46255
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by xnview »

i can't reproduce, are you able to reproduce with ORF (from link)? With 50 of your RAWs?
Pierre.
John_Matthews
Posts: 14
Joined: Wed Oct 27, 2021 3:27 am

Re: M1 Mac Excessive Memory usage on Batch Convert

Post by John_Matthews »

I don't think it'll make much of a difference if I use the ORF files from the link or ORF files from my drive. The problem isn't really visible with 50 ORF files. It's more like 500 ORF files. Memory management is different on M1 machines. Are you using a M1 mac? Maybe that's it.

I can make a screen cast of the problem but first I might do a reinstall of xnview... making sure it's vanilla as possible. I want this to work. I'm certain I can't be the only one have this problem. What could it be on my machine with ONLY this app?
Post Reply