Feature request: Vastly increased performance with large amount of data

Bugs which are supposed to be fixed in the next test version (not available yet)

Moderators: XnTriq, xnview, Dreamer

unqltango
Posts: 52
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Tue May 05, 2020 6:22 am

Rick wrote:
Mon May 04, 2020 7:07 pm
unqltango wrote:
Mon May 04, 2020 6:29 am
The performance issue is definitely caused by the large amount of Categories (2900+ in my case). It impacts all the regular usage of the tool, for example if I just doubleclick in Total Commander on a regular, unkeyworded photo, the loading time of the photo is very slow and I have to wait several seconds before I can switch to another photo by PgUp/PgDn.

I deleted my Categories and the tool sped up again.

Conclusion: XnView can't be used for keywording in its current state, because a large amount of Categories makes the main function (open and look at pictures) very slow and problematic.
I dislike when I ask something and the forum rats reply with something like "You shouldn't want to DO that" but I have to ask if there is not a better keyword strategy that using 2900 keywords? I use several per picture and if I check say animal and child I get a different result than Human and Child the point being to get a list I can scroll through rather than just a couple of photos. Please just think about it before responding
And again there is no implied criticism of the way you work or what your needs may be.
Sorry, I'm not entirely sure what you mean by your answer.

I'm sure there are better ways to do keywording. If I'd do this professionally (as I did about 10 years ago), I would do it differently. But I do this for myself now and the requirements are different. I can use words and expressions more freely, I don't have to comply to a plethora of keywording rules, because the media I'm keywording will not be sold on stock sites. This also means I have more keywords than I usually would have.

What I wrote is accurate. XnView lags so badly because of the keywords, that it became a constant annoyance. Please don't misunderstand, I very much like XnView, IMHO the best tool in its category. I use it for some different purposes as well, like editing, converting pictures, etc. I was really happy when I learned that I can use it for keywording too, because the other tools were lacking in this area badly, compared to XnView.

Alas, the usage brought out some issues, I reported them over the months, none of them are fixed yet, but that I could live with, I learned to work around them. I'm patient in that regard, I know Pierre has a thousand other issues to work on.

But this performance issue is so bad that I have to move my keywording to an other tool (which has zero problem of handling this many keywords/media). Thankfully I can do that by exporting the keyword and media database to txt/xml and I won't lose any done work.

I will still use XnView for other things, now that I deleted my keywords from it, it is usable again.

User avatar
xnview
Author of XnView
Posts: 33205
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Wed May 06, 2020 7:08 am

unqltango wrote:
Tue May 05, 2020 6:22 am
Alas, the usage brought out some issues, I reported them over the months, none of them are fixed yet, but that I could live with, I learned to work around them. I'm patient in that regard, I know Pierre has a thousand other issues to work on.

But this performance issue is so bad that I have to move my keywording to an other tool (which has zero problem of handling this many keywords/media). Thankfully I can do that by exporting the keyword and media database to txt/xml and I won't lose any done work.

I will still use XnView for other things, now that I deleted my keywords from it, it is usable again.
Are you able to see a difference when you tag jpeg, png or other formats?
It's the fact to clean the catalog that make it usable again? The keywords will be imported when you browse folder...
Pierre.

unqltango
Posts: 52
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Wed May 06, 2020 3:36 pm

xnview wrote:
Wed May 06, 2020 7:08 am
Are you able to see a difference when you tag jpeg, png or other formats?
It's the fact to clean the catalog that make it usable again? The keywords will be imported when you browse folder...
I have a lot of different formats, but most pictures are .jpg. I didn't see any difference, but to be honest didn't look for it.

I set XnView to not import keywords from picture headers, so my Categories list is not being populated by browsing already keyworded media. I see the keywords in the Assigned set, but they don't go in the list.

When I had the 2900+ categories, doubleclicking on a picture in a folder in Total Commander loaded the low quality version of the picture, then I had to wait for several seconds before I could start moving between photos with PgUp/PgDn.

Now that I don't have the categories, the pictures load at once and I can browse them at once, zero waiting. This folder has 100+ pictures in it, all of them with 10-20 keywords in their header.

User avatar
xnview
Author of XnView
Posts: 33205
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Wed May 06, 2020 6:27 pm

unqltango wrote:
Wed May 06, 2020 3:36 pm
When I had the 2900+ categories, doubleclicking on a picture in a folder in Total Commander loaded the low quality version of the picture, then I had to wait for several seconds before I could start moving between photos with PgUp/PgDn.
Ok, i'll try to find where is the problem. Do you have a backup of catalog if i send you a test version?
Pierre.

unqltango
Posts: 52
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Wed May 06, 2020 6:31 pm

xnview wrote:
Wed May 06, 2020 6:27 pm
unqltango wrote:
Wed May 06, 2020 3:36 pm
When I had the 2900+ categories, doubleclicking on a picture in a folder in Total Commander loaded the low quality version of the picture, then I had to wait for several seconds before I could start moving between photos with PgUp/PgDn.
Ok, i'll try to find where is the problem. Do you have a backup of catalog if i send you a test version?
Thank you, but I'm OK now, as I said I moved to another tool with keywording. Please work on it only if you feel it important.

I can test it for you though, I have the categories txt and the files.xml archived.

User avatar
xnview
Author of XnView
Posts: 33205
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Thu May 14, 2020 3:15 pm

unqltango wrote:
Wed May 06, 2020 6:31 pm
I can test it for you though, I have the categories txt and the files.xml archived.
Is it possible to make a test? Reimport your categories tree, create a folder with some image files, and open one of these files in XnViewMP, do you have always the delay?
Pierre.

unqltango
Posts: 52
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Sat May 16, 2020 2:06 pm

xnview wrote:
Thu May 14, 2020 3:15 pm
unqltango wrote:
Wed May 06, 2020 6:31 pm
I can test it for you though, I have the categories txt and the files.xml archived.
Is it possible to make a test? Reimport your categories tree, create a folder with some image files, and open one of these files in XnViewMP, do you have always the delay?
I tried it with V0.96.2 64bit, Windows version. I imported in 2950+ categories (keywords) and 115000+ rows of data from xml. No issues with the imports.

No visible improvements, I still have the several seconds of unresponsiveness regularly. I get the "Not responding" message from W10, same as earlier.
The unresponsiveness is present as it was in the earlier reported use cases:
- opening any picture from Total commander - the picture appears, but I can't browse with PgUp/PgDn for several seconds
- During selecting multiple pictures which have categories
- During updating categories

BR,
Tango

User avatar
xnview
Author of XnView
Posts: 33205
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Sun May 17, 2020 11:27 am

unqltango wrote:
Sat May 16, 2020 2:06 pm
I tried it with V0.96.2 64bit, Windows version. I imported in 2950+ categories (keywords) and 115000+ rows of data from xml. No issues with the imports.
Thanks, if you import the 2950+ categories, and only 10 rows of data?
Pierre.

unqltango
Posts: 52
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Mon May 18, 2020 1:11 pm

xnview wrote:
Sun May 17, 2020 11:27 am
unqltango wrote:
Sat May 16, 2020 2:06 pm
I tried it with V0.96.2 64bit, Windows version. I imported in 2950+ categories (keywords) and 115000+ rows of data from xml. No issues with the imports.
Thanks, if you import the 2950+ categories, and only 10 rows of data?
Tested on the same version.

Imported the 2950+ categories. No issues with the import. I did NOT import any xml.

Tests:
Starting XnView by doubleclicing on a picture in Total Commander. The picture loads at once, I can browse 2-3 pictures with the mouse wheel. But then it stops and I have to wait for several seconds again.

In XnView browser, I opened a folder with dozens of already keyworded media (keywords are in IPTC/XMP). I selected several pictures at once. The Assigned set got populated at once, zero delay. I tried this in a couple of folders, same result, the Assigned set got populated with hundreds of categories, without any delay.

Next I selected about a dozen unkeyworded pictures and added some categories to them. The unresponsiveness came back at once after I clicked outside of selection (update). I added about 4 keywords to 20 pictures (so about 80 categories).

Next I quit XnView and tried the 1st test again with the TC. I have the unresponsiveness at once now, I can't even mousewheel to a new picture before I have to wait several seconds again.

So it seems while there are some areas which can be unaffected by the large amount of categories, adding even just a few of them brings back the issue.

BR,
Tango

User avatar
xnview
Author of XnView
Posts: 33205
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Mon May 18, 2020 2:06 pm

unqltango wrote:
Mon May 18, 2020 1:11 pm
Tests:
Starting XnView by doubleclicing on a picture in Total Commander. The picture loads at once, I can browse 2-3 pictures with the mouse wheel. But then it stops and I have to wait for several seconds again.
Do you have the settings to import keywords?
If you start from a clean catalog (without importing your 2950+ category tree), same problem?
Pierre.

unqltango
Posts: 52
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Mon May 18, 2020 2:37 pm

xnview wrote:
Mon May 18, 2020 2:06 pm
unqltango wrote:
Mon May 18, 2020 1:11 pm
Tests:
Starting XnView by doubleclicing on a picture in Total Commander. The picture loads at once, I can browse 2-3 pictures with the mouse wheel. But then it stops and I have to wait for several seconds again.
Do you have the settings to import keywords?
If you start from a clean catalog (without importing your 2950+ category tree), same problem?
The settings are posted on page one of this thread. Not importing categories from headers.

I only have problem if I have a lot of keywords. XnView works fine if I have only a couple of hundred. When I started keywording, everything woked normally. As the number of keywords grew, I started noticing the issues. I thought 1st that it has something to do with the database, but later I realised that it is number of keywords. It was confirmed when I deleted my categories and XnView worked fine again.

User avatar
xnview
Author of XnView
Posts: 33205
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Tue May 19, 2020 6:34 am

I've sent you a PM

If you open an image file from a folder (with only 10 files for example), same delay?
Pierre.

User avatar
xnview
Author of XnView
Posts: 33205
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Thu May 21, 2020 2:28 pm

See issue for current status and some details.
Pierre.

unqltango
Posts: 52
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Sat May 23, 2020 7:10 am

xnview wrote:
Thu May 21, 2020 2:28 pm
See issue for current status and some details.
I can confirm, XnView now works fine with large amount of Categories (in version 0.96.3, unreleased at this date).

Thank you,
Tango

Post Reply