Feature request: Vastly increased performance with large amount of data

Ideas for improvements and requests for new features in XnView MP

Moderators: XnTriq, xnview

Post Reply
unqltango
Posts: 37
Joined: Sun Sep 01, 2019 8:16 am

Feature request: Vastly increased performance with large amount of data

Post by unqltango » Sat Feb 08, 2020 9:20 am

Hi all,

I am using MP on a huge media folder, almost a TB of media in about 700 directories, mostly photos. I have 2600+ categories. The XnView.db is 91MB currently and growing slowly but steadily. I recently moved the DB and thumbs to a very fast SSD.

Still, when I work with categories, especially when selecting multiple photos to work with, MP performs extremely slow. I have to wait 2-5 seconds(!!!) to get the sum of the Assigned Categories. Another use case when I experience this slowness is when I finished adding categories to multiple pictures and I click on a new one (so basically during the updating of categories). I very often get the W10 "Not responding" warning.

Of course this slows down my work tremendously, as I have to wait a lot, doing nothing. This also leads to the issue when I click on a Category to add/remove it, but nothing happens. Thinking that I misclicked, I click it again, but I realize that I was in the "Not responding" period, so I actually clicked twice, undoing my earlier click, so I have to click it again the 3rd time. This can become annoying quickly. Anyway, I need some good news as I have tens of thousands more pictures waiting for to be categorized and I want to use MP for this. I hope I will be able to do it. I checked other tools as well, but MP has way more useful features.

I hoped that moving the DB to an SSD will solve my problems, but it didn't help. The SSD could write out whole database 5 or more times in a second, so reading/writing/modifying a few records in it should be really fast. So I'm assuming the problem is not with the DB.

I don't know what could be done, maybe some asynchronous update for categories?

BR,
Tango

jadO
Posts: 427
Joined: Wed Apr 29, 2015 6:36 am

Re: Feature request: Vastly increased performance with large amount of data

Post by jadO » Sun Feb 09, 2020 3:48 pm

I think number of categories is the problem here. Have you tried with less catgories?

unqltango
Posts: 37
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Mon Feb 10, 2020 11:17 am

jadO wrote:
Sun Feb 09, 2020 3:48 pm
I think number of categories is the problem here. Have you tried with less catgories?
Yes, when I started using MP and started building the Category list. It was fine back then.

However, the number of keywords will not go down, it will just increase in the future. Whenever I see something important on a photo for which I don't have a keyword yet, I just add it to the list. Sometimes I do some optimization, merging some keywords and such, but that is rare, so the number just goes up.

User avatar
xnview
Author of XnView
Posts: 32479
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Tue Feb 18, 2020 2:58 pm

unqltango wrote:
Sat Feb 08, 2020 9:20 am
Still, when I work with categories, especially when selecting multiple photos to work with, MP performs extremely slow. I have to wait 2-5 seconds(!!!) to get the sum of the Assigned Categories. Another use case when I experience this slowness is when I finished adding categories to multiple pictures and I click on a new one (so basically during the updating of categories). I very often get the W10 "Not responding" warning.
Is it possible to send me your category tree list?
Pierre.

unqltango
Posts: 37
Joined: Sun Sep 01, 2019 8:16 am

Re: Feature request: Vastly increased performance with large amount of data

Post by unqltango » Tue Feb 18, 2020 6:23 pm

xnview wrote:
Tue Feb 18, 2020 2:58 pm
unqltango wrote:
Sat Feb 08, 2020 9:20 am
Still, when I work with categories, especially when selecting multiple photos to work with, MP performs extremely slow. I have to wait 2-5 seconds(!!!) to get the sum of the Assigned Categories. Another use case when I experience this slowness is when I finished adding categories to multiple pictures and I click on a new one (so basically during the updating of categories). I very often get the W10 "Not responding" warning.
Is it possible to send me your category tree list?
Sorry, that is private.

I can say though that the exported file is 48KB. At the deepest it is 6 levels deep (I don't know if that matters).

Some stats about my computer, so you see that I don't use some very old pentium:
i9-9900K processor - https://youtu.be/Lr224tmu54I processor usage while selecting multiple pictures and adding categories. Some of the percentages come from the capturing tool, but the spikes are still clearly visible (@ 24 and 40 seconds), that's when XnView hangs in the "Not Responding" state. It is visible that both times the processors stay in this high usage mode for a couple of seconds.

32 GB RAM
W10 Professional

BR,
Tango

User avatar
xnview
Author of XnView
Posts: 32479
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Feature request: Vastly increased performance with large amount of data

Post by xnview » Wed Feb 26, 2020 10:27 am

unqltango wrote:
Tue Feb 18, 2020 6:23 pm
Sorry, that is private.
Without the category list, i'm not able to find the problem
Pierre.

Post Reply