Force thumbnail building on directories with thousands of files

[jellen]

Hi there-

We are trying to use XnView MP to look at directories with many files (10,000+) . I am generating these images from a scientific instrument.

They are in .png format, so I do not believe there is any embedded thumbnail as there would be with .jpg.

Once I render the images, they don't change. So what I would like to do is be able to open XnView, point it at a root directory of a project, and tell it to 'render thumbnails' or whatever it needs to do overnight. There are many options under settings that look applicable, but they don't seem to make any difference. For example, under settings->thumbnail, there's an option for 'Create Thumbnails for whole folder', but whether or not I have that checked, it doesn't seem to make a difference. There's an option for 'create high quality thumbnail', and I have turned that off, but I don't know that it helps go any faster.

Under settings->catalog, I have enabled the catalog. I have increased the memory usage for database engine from 50MiB to 500MiB. I have 'enable thumbnail caching' turned on. I tried setting the 'base path of your pictures', but that doesn't seem to do it (or maybe I haven't waited long enough yet).

When I look at the cached directory listing, it immediately says that one particular directory has 34,187 files, and the size of the cache is 131.85 MiB. However, when I scroll around that folder, XnView is definitely still rendering thumbnails, because they do not all display immediately, there is a lot of waiting. But that number never goes up. So is that just the size of all the thumbnails once they are eventually rendered? What else can I do to try to scroll through a very large folder faster?

Ultimately, it is not very useful to scroll through 30,000+ images to look for anything. I would prefer to be able to filter these images by different queries ( as requested here: viewtopic.php?f=110&t=36957 ). But until then, looking at large directories is the only choice.

Any suggestions?

Thanks.

[xnview]

you have settings>Catalog>Rebuild thumbnails (for all folders already in the catalog)

[icancomplain]

I know this is an old post, but I have the same need and am unsure how to go about it. I want all the thumbnails to be built ahead of time for fast browsing, but they only build when I go into that subdirectory. I have a "Pictures" directory with hundreds of subdirectories. I could add each subdirectory one-by-one to the "Catalog" settings area, but that would take forever. You can't select-all on the subdirectories. I've set the base directory to "Pictures" and I've also put it on the "included" list. Thank you for your time.

Daniel

[xnview]

I know this is an old post, but I have the same need and am unsure how to go about it. I want all the thumbnails to be built ahead of time for fast browsing, but they only build when I go into that subdirectory. I have a "Pictures" directory with hundreds of subdirectories. I could add each subdirectory one-by-one to the "Catalog" settings area, but that would take forever. You can't select-all on the subdirectories. I've set the base directory to "Pictures" and I've also put it on the "included" list. Thank you for your time.

Do you use same database on multiple PC? If no, no need to use 'base path'
If you use 'add folder' in settings>Catalog?

[icancomplain]

Do you use same database on multiple PC? If no, no need to use 'base path'
If you use 'add folder' in settings>Catalog?

Not used on multiple PC's. I will remove the 'base path'. The "Pictures" directory shows in the list on the "Catalog" settings tab and I guess the subdirectories that I have gone into. None of the other subdirectories are there. I don't know if they are supposed to or if there is some recursive box I'm supposed to check instead. I'm not at my computer right now, but will be in an hour or two to try again. Thank you.

[icancomplain]

I think it's generating the thumbnails now.

This is why it's confusing...

On the Settings->Catalog tab in the list of directories, I can see /Volumes/PicturesDP/Pictures/ which is the directory I want. It's showing up there maybe just because I clicked on it manually in the Folder explorer area or for some other reason. However, that is not a formal "import" which seems to be necessary to get it to create the thumbnails. So to my untrained eye, I'm basically having to add the Pictures directory to the Catalog list a second time, which is counter-intuitive. It probably shouldn't show there the first time, like maybe you should be required to "import" to begin with for it to show up there. To me, it would be great to visualize a folder tree with checkboxes or icons or something showing you which folders you have imported and which ones you don't. It would be the easiest way for a user to see what is there and you could check or uncheck a directory so you don't have to see it.

[rafalio]

What seems to work for me for generating thumbnails.
* In normal Browser view of Xnview, navigate in the tree to the folder which has all the files/folders you want thumbnails for.
* Right-click: "Show all files (recursive)"
* Depending on how many files you have, this might take a while. You should see a progress bar on the bottom right.

Don't navigate away from this view and let it finish. At least this works for me (xnview mp on mac)

[lucem]

So to my untrained eye, I'm basically having to add the Pictures directory to the Catalog list a second time

That fixed it for me! Thank you!

[Paran902]

I'm having the same problem with thumbnail generation.

Looks like the only way to import the folder with hundreds of subfolders is to click "Show all files".
But it doesn't work for hundreds of thousands of images, as XnView uses ALL memory and crashes.

:O

Do you have any idea how to generate thumbnails in such case?

[MacAhi]

I'm having the same problem with thumbnail generation.

Looks like the only way to import the folder with hundreds of subfolders is to click "Show all files".
But it doesn't work for hundreds of thousands of images, as XnView uses ALL memory and crashes.

:O

Do you have any idea how to generate thumbnails in such case?

I'M STUPID
I'll leave the entire text below this in case anyone else is as confused as I was. I always wished there was a way to build the thumbnails in the background, and never realized that there is. Thanks to this thread, I finally discovered it in the Catalog using the [Import folder] option. And it doesn't blow out the memory!

A status bar would be a really nice addition to this.

(Ok, I'm a bit slow some times...)

===========================
I agree with this. Memory management would be the one area in this otherwise fantastic tool that I think needs to be addressed. I have another tool which looks for duplicates. It can process 500k files and never go over .6G or so, while XnView will use >20G, not release it, and sometimes crash the system.

That's why whenever I need to build thumbs for many thousands of files, I often have to divide them into separate folders and do them one-at-a-time and close and restart the program often in order to free up the memory.

When I use "Show files in subfolders", I make sure to load Activity Monitor, set on Memory and keep a close eye on it so I don't crash my system.

Attaching an example:
I just scanned a mixed media folder with subfolders containing ~48k items.
It didn't crash, but the memory usage went up to 23G, the screenshot I'm attaching is 5 minutes after completion of the scan and you can see that the memory still isn't released. I can click on another folder and most of that will be released, but I need to close the program and relaunch it in order to get a full release.

Update:
I realized that it's not the building of the thumbnails that's causing the memory blowout, it's just loading them. The previous 48k folder was itself a subfolder. After I built that and several others I built it from the parent (~89k mixed-media files). The memory use went significantly higher, (27G) and didn't release. I closed and reopened the program, went to that folder and selected "Show files in subfolders" at which point, the already built thumbnails were loaded into memory. Screenshots attached: