Loading database in order to search

Ask for help and post your question on how to use XnView MP.

Moderators: XnTriq, helmut, xnview

Post Reply
Geoff
Posts: 15
Joined: Wed Nov 04, 2015 6:11 pm

Loading database in order to search

Post by Geoff »

When I open XnViewMP I cannot search anything unless I click on the little icon at the top "Show files in sub folder" - this took me a long time to find. After a completly new loading this takes several minutes presumably as the Thumbs.db file and the XnView.db are produced. Once these are there then each time you change to one of my 2 directories you have to press the icon and you see a green bar at the bottom of the screen as the files load - this takes about 15 seconds. Then if you turn off the computer and bring up XnViewMP the first loading takes 25 seconds and subsequent loadings take 15 seconds.

I understand from the forum that the programme is loading the thumbs.db first and this takes the time. I know nothing about SQlite but would it be possible to search the smaller XnView.db first and then when it finds the files that match the search it only loads those thumbnails?

Thank you,

Geoff
Geoff
Posts: 15
Joined: Wed Nov 04, 2015 6:11 pm

Re: Loading database in order to search

Post by Geoff »

I have found that after adding or modifying the IPTC caption or adding, moving or deleting files the quick search slows down as the .db files grow in size. The simplest way to speed things up is to delete the 2 .db files and restart XnViewMP which rebuilds those files. On my fast home computer this takes about 8 minutes and twice that on the museum computer but this is no problem.

The problem comes if somebody is searching and they get impatient waiting for the Thumb.db file to load as this takes about 25 seconds on the museum computer. Also this had to be reloaded each time you changed from one directory to the other. There is a reason why I have 2 directories and quite often you need to search one and then the other. The file structure is a directory called DIGITAL PHOTOS divided into 2 directories called BCAM DIGITAL PHOTOS with 15,400 images sub-divided into many descriptive sub-directories and DISPLAY AIRCRAFT BCAM with 3,800 images. The size will probably double as more photos are scanned.

Below are the results for loading this file structure on my fast home computer when changing the thumbnail size in Settings>Catalogue from "Lossy-Low quality jpeg" to "Lossless (ZIP)" The Thumb.db is about 9 times larger with the higher quality and this almost doubles the load time. The rebuild time is not much different. The Lossy-Low quality jpeg is fine for scanning the search results as you open the image file to examine it further.
----------------------------------
Lossy- Low quality jpeg
After deleting thumb and XnView databases
7min 58 sec to rebuild Thumb.db of 88,480KB and XnView.db of 20,120KB

Time after db is loaded is 8 seconds for DIGITAL PHOTOS
7 secs for BCAM DIGITAL PHOTOS
3 secs for DISPLAY AIRCRAFT BCAM

After closing XnViewMP and restart 9 sec
After turning off computer and restart for DIGITAL PHOTOS 15 secs
then all times as before
-----------------------------
Lossless (ZIP)
After deleting thumb and XnView databases
8 Min 41 sec to rebuild Thumb.db of 816,184KB and XnView.db of 20,120KB

Time after db in place is 15 seconds for DIGITAL PHOTOS
14 secs for BCAM DIGITAL PHOTOS
4 secs for DISPLAY AIRCRAFT BCAM

After closing XnViewMP and restart 15 sec
After turning off computer and restart for DIGITAL PHOTOS 26 secs
then all times as before

Geoff
User avatar
xnview
Author of XnView
Posts: 43357
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Loading database in order to search

Post by xnview »

you means able to use the search dialog?? or quick search?
Pierre.
Geoff
Posts: 15
Joined: Wed Nov 04, 2015 6:11 pm

Re: Loading database in order to search

Post by Geoff »

As I said in the second post it is on "Quick Search". The search represented by the binocular icon is accurate, and can do Boolean searches, but takes minutes and is too slow. Using version 0.86

Thank you, Pierre for an excellent programme.

Geoff
User avatar
xnview
Author of XnView
Posts: 43357
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Loading database in order to search

Post by xnview »

Geoff wrote:As I said in the second post it is on "Quick Search". The search represented by the binocular icon is accurate, and can do Boolean searches, but takes minutes and is too slow. Using version 0.86
Your problem is that 'Quick search' slow down?? But the 'Quick Search' doesn't use catalog, only loaded information.
Pierre.
Geoff
Posts: 15
Joined: Wed Nov 04, 2015 6:11 pm

Re: Loading database in order to search

Post by Geoff »

When I say the search slows down and I need to delete and rebuild the two.db database files this refers to the time it takes for the green bar to show it is fully loaded. Once everything is loaded I do not think there is any slowdown in the speed of the Quick Search which is very rapid.

If I only show the directories without pressing the icon “Show files in folder” nothing happens if I press Quick Search. I have to click on the “Show files in folder” icon and wait for all the files to be visible on the screen. The progress of the loading is shown by the green bar at the bottom right hand of the screen. The times mentioned in the 2nd post on this topic are the times for the files to be completely loaded and the “lossy-Low quality jpeg” had the fastest time.

The problem comes if people are impatient and do not let all the files load as shown by the progress of the green bar.

Also after showing all the files at the top level and you change to a directory that has sub-directories you have to press the “Show files in folder” icon again. If you go to a directory with only images you do not need to press the icon again.

Geoff
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: Loading database in order to search

Post by CameronD »

Geoff wrote:When I open XnViewMP I cannot search anything unless I click on the little icon at the top "Show files in sub folder"
Do you mean "I cannot use Quick search to search subfolders"?
That would be rather the point - "quick" only searches current folder, or displayed list. Maybe it should be called quick filter.

Why not leave the catalog DB in place and use the full search method?

You have a point about impatient users - XnViewMP should not allow you to search before it has finished loading the list. When I tried on a folder system with about 8300 files it went busy for 5 to 10 seconds and then became responsive and said it had 8300 files. But when I typed a search item it only listed about 5 of the 10 files it should have found.

I'm not sure about your comment re thumbnails. I generally list details - no thumbnails and it is still very slow loading the metadata unless that has been cached in the DB already.
Geoff
Posts: 15
Joined: Wed Nov 04, 2015 6:11 pm

Re: Loading database in order to search

Post by Geoff »

Geoff wrote:
When I open XnViewMP I cannot search anything unless I click on the little icon at the top "Show files in sub folder"
Cameron wrote
Do you mean "I cannot use Quick search to search subfolders"?
That would be rather the point - "quick" only searches current folder, or displayed list. Maybe it should be called quick filter.
Correct. If I select a subfolder "Plane 1" that has 19 images and 2 subfolders the quick search gives search results embedded in the metadata for these 19 images but not for the subfolders. On the screen the thumbnails of the 19 images are visible but the subfolders show as yellow unexpanded subfolders. I have to press "Show files in sub folder" to conduct a quick search within the subfolders.
Cameron wrote
Why not leave the catalog DB in place and use the full search method?
Thank you Success! I assume you mean do not expand the folders by pressing "Show files in sub folder"? I have tried this before and it gave no results unless I have pressed "Show files in sub folder". I always had "Use catalog" ticked as I assumed the .db files were the catalogue. When I unticked "Use catalog" I got very quick search results and could see the programme searching through each folder in the "Current pathname".

To avoid confusion I list each step in the search :-

With only the top folder showing ( or sub folder if limiting the search)
Click on the Search icon - a pair of binoculars
Leave Filename blank and make sure the folder in the box on the right hand side is the one you want to search.
"Use RegExp" is not ticked
I choose "Match any of the following conditions"
"Case sensitive" is not ticked
"Use catalog" is NOT ticked
"Recurse through subfolders" is ticked
I then choose Add IPTC:All fields and fill in the search term
Press Search and the results are found quickly and can be browsed.
To make future searches have the same conditions I save the search with a blank entry by pressing the + sign and naming it All Fields.

I admire the versatility of XnViewMP but, as in the above case, it may take a user some time to figure out what check-boxes to tick or not.

Thank you Cameron in Australia

Geoff in Canada
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: Loading database in order to search

Post by CameronD »

Geoff,
I don't understand why you need to untick the "use db", unless you have not yet scanned all the folders.
Try Settings->catalog->Add Folder and choose your top level folder.
This will add all metadata to the DB (note - nothing to do with the thumbnail db) and it should generally give a faster search.

A search on only IPTC caption or keyword will for some reason be much faster than XMP parameters or all-fields. Fastest of all is to use the catalog's "categories", which can be linked to the iptc:keywords, but you need to plan for that.

Is there any particular reason you are using an "all fields" search? Do you not have a specified system for categorising the images.
Geoff
Posts: 15
Joined: Wed Nov 04, 2015 6:11 pm

Re: Loading database in order to search

Post by Geoff »

Thanks Cameron,
I am sure I did use Settings->catalog->Add Folder on the top level folder when I originally opened up the new folders. I repeated the operation and found only a slight change in xnview.db from 20,136KB to 20,200KB.

If I tick "Use catalog"( is that the same as your "use db"? ) then I get no search results. Now even if I also have used "Show files in sub folder" I get no search results if "Use catalog" is ticked. I am using Version 0.86 64bits (Apr 25 2017)with Libformat version 7.00.

I search on IPTC-IMM and then select "All fields" from the IPTC-IMM choices and the speed of searching is completely acceptable. The only fields I enter are in the IPTC metadata for Caption, Source and Copyright. It is different than selecting All Fields before first selecting IPTC-IMM.

When I started cataloguing the images I did not, and still do not, fully understand categories and now after 25,000 images it is too late. I found the Caption could be very long and descriptive so both categories and keywords seemed irrelevant.

Now that you have told me I can search without using "Show files in sub folder" I am almost completely satisfied. The only thing I would like is for the Caption line in the IPTC-IMM info box to be able to show several lines so the complete caption shows there and you do not need to scroll or to press Ctrl + I to read all of the caption.

Thanks again for putting me on the right track.

Geoff
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: Loading database in order to search

Post by CameronD »

Geoff wrote:...The only thing I would like is for the Caption line in the IPTC-IMM info box to be able to show several lines so the complete caption shows there and you do not need to scroll or to press Ctrl + I to read all of the caption.
I have the following in Settings->Browser->tooltip (the large text entry box)

Code: Select all

<b>{Filename}</b><br>taken: {EXIF:Date Taken [d-mm H:M]}<br>mod: {Modified Date}<br>KW:{IPTC:Keywords}<br>{Size KB} Kib<br><i>{IPTC:Caption}</i>
This shows the entire caption, using auto-wrapping, when you hover the mouse over the image.
Geoff
Posts: 15
Joined: Wed Nov 04, 2015 6:11 pm

Re: Loading database in order to search

Post by Geoff »

Great! I have modified it to

{Directory}<b>{Filename}</b><br>{Size KB} Kib<br><i>{IPTC:Caption}</i>

and now see all I need. As I do not need fast loading of all thumbnails anymore I can change back to larger better quality thumbnails so that people only need to check an image at full size when they see the Caption they choose.

A very useful trick is to search for "Lancaster" and get 95 hits and then if you want those from #44 squadron just do a quick search on 44 and get 4 hits. Then when you press the grey cross to get out of quick search you are back to the 95 hits and can search for something different.

Goodnight Cameron and many thanks

Geoff
Post Reply