Thumbnail Cache - Time to display

Ask for help and post your question on how to use XnView Classic.

Moderators: XnTriq, xnview

User avatar
Olivier_G
XnThusiast
Posts: 1423
Joined: Thu Dec 23, 2004 7:17 pm
Location: Paris, France
Contact:

Post by Olivier_G » Thu Dec 14, 2006 2:36 pm

Now the Cache panel is usable (ie: number&filesize computed in the background).

But I still see major issues with the DB:
- If I remove a folder with just 1 image in my 2GB cache, XnView will need to write/recompute a whole 2GB temporary file to remove that 5KB thumbnail... :(
- XnView is still reading 20 times each segment of the DB, which I believe has severe performance hit.

Here is a screenshot of this with FileMon when entering a folder with 3 thumbnails -> each needed segment of the DB will be read about 20 times (just 2 identical segments are highlighted here, but it continues for pages and pages like this). Also notice the "db-journal" systematic error in-between.
Image
Olivier

User avatar
xnview
Author of XnView
Posts: 31607
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Post by xnview » Thu Dec 14, 2006 2:56 pm

Olivier_G wrote:Now the Cache panel is usable (ie: number&filesize computed in the background).

But I still see major issues with the DB:
- If I remove a folder with just 1 image in my 2GB cache, XnView will need to write/recompute a whole 2GB temporary file to remove that 5KB thumbnail... :(
- XnView is still reading 20 times each segment of the DB, which I believe has severe performance hit.
Here it's the SQL DB command, not me :-)
Pierre.

User avatar
Olivier_G
XnThusiast
Posts: 1423
Joined: Thu Dec 23, 2004 7:17 pm
Location: Paris, France
Contact:

Post by Olivier_G » Thu Dec 14, 2006 4:01 pm

xnview wrote:Here it's the SQL DB command, not me :-)
There is no way in SQL-lite to remove an image from DB (+reduce DB size) without rebuilding it???

The 20x reading is a real problem with large folders, as the Disk/System can't cache everything (example: 500MB of thumbnails in RAM) -> the 20x real hit in performance probably happens there... :?
Olivier

User avatar
xnview
Author of XnView
Posts: 31607
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Post by xnview » Thu Dec 14, 2006 4:16 pm

Olivier_G wrote:
xnview wrote:Here it's the SQL DB command, not me :-)
There is no way in SQL-lite to remove an image from DB (+reduce DB size) without rebuilding it???

The 20x reading is a real problem with large folders, as the Disk/System can't cache everything (example: 500MB of thumbnails in RAM) -> the 20x real hit in performance probably happens there... :?
I don't rebuild it, it's sql lite
Pierre.

User avatar
Olivier_G
XnThusiast
Posts: 1423
Joined: Thu Dec 23, 2004 7:17 pm
Location: Paris, France
Contact:

Post by Olivier_G » Sun Dec 17, 2006 9:27 pm

I have read a bit more about SQLite and in line with the first behaviour (DB not shrinking in size), I have a more accurate question: do you now use auto_vacuum for your DB?
Olivier

User avatar
xnview
Author of XnView
Posts: 31607
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Post by xnview » Mon Dec 18, 2006 12:42 pm

Olivier_G wrote:I have read a bit more about SQLite and in line with the first behaviour (DB not shrinking in size), I have a more accurate question: do you now use auto_vacuum for your DB?
No, i use vacuum command
Pierre.

User avatar
Olivier_G
XnThusiast
Posts: 1423
Joined: Thu Dec 23, 2004 7:17 pm
Location: Paris, France
Contact:

Post by Olivier_G » Mon Dec 18, 2006 1:16 pm

xnview wrote:No, I use vacuum command
I wonder whether using 'auto_vacuum' would not allow the DB to shrink on the fly, ie: without even 'rebuilding' (contrary to how Vacuum works)... :?
If so, it would be quite interesting on a performance side.
Olivier

User avatar
Olivier_G
XnThusiast
Posts: 1423
Joined: Thu Dec 23, 2004 7:17 pm
Location: Paris, France
Contact:

Post by Olivier_G » Fri Dec 22, 2006 2:04 pm

I really wonder about the performance of auto_vacuum (doesn't rebuild DB???) vs vacuum, as it might improve things a lot... but would require to start DB from scratch again (cannot be added afterwards).
=> any feedback experience about this auto_vacuum?
Olivier

User avatar
xnview
Author of XnView
Posts: 31607
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Post by xnview » Fri Dec 22, 2006 2:10 pm

Olivier_G wrote:I really wonder about the performance of auto_vacuum (doesn't rebuild DB???) vs vacuum, as it might improve things a lot... but would require to start DB from scratch again (cannot be added afterwards).
=> any feedback experience about this auto_vacuum?
Auto_vacuum is not a good solution, very slow on add/delete entries
Pierre.

User avatar
Olivier_G
XnThusiast
Posts: 1423
Joined: Thu Dec 23, 2004 7:17 pm
Location: Paris, France
Contact:

Post by Olivier_G » Fri Dec 22, 2006 3:35 pm

xnview wrote:Auto_vacuum is not a good solution, very slow on add/delete entries
Thank you for the feedback. So much for auto_vacuum, then... :wink:

Some suggestions to reduce workload in Cache panel:
- When removing folders, don't 'vacuum' at each removal. Use a flag instead to 'vacuum' when exiting Options (an alert with 'don't show next time' would be nice). Flag would also be reset after an 'Optimize' (which should include an immediate vacuum).
- I wonder whether it's useful to display filesize of every folder. The number of thumbnails and a single general "Database size"=[filesize] would be enough, IMO.

I would like to get some feedback about those suggestions.
Olivier

Post Reply