Support WebP as internal format for thumbnails in DB Cache

Ideas for improvements and requests for new features in XnView MP

Moderators: helmut, XnTriq, xnview

Post Reply
User avatar
m.Th.
XnThusiast
Posts: 1676
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Support WebP as internal format for thumbnails in DB Cache

Post by m.Th. »

Please add support for WebP as internal format for thumbnails in thumb.db

Reasoning:
  • Much better compression than JPG/PNG.
  • Much faster compression than JPG (it supports natively all the CPU cores)
This will speedup the thumbnail creation, thumbnail reading and will save disk space.

Perhaps is better to add three levels:
  • Hi-res WebP
  • Low-res WebP
  • Loseless WebP
m. Th.

- Dark Themed XnViewMP 1.7.1 64bit on Win11 x64 -
User avatar
oops66
XnThusiast
Posts: 2005
Joined: Tue Jul 17, 2007 1:17 am
Location: France

Re: Support WebP as internal format for thumbnails in DB Cac

Post by oops66 »

+1 I support.
XnViewMP Linux X64 - Debian - X64
User avatar
m.Th.
XnThusiast
Posts: 1676
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: Support WebP as internal format for thumbnails in DB Cac

Post by m.Th. »

Seeing this one http://www.xnview.com/mantisbt/view.php?id=146 I bump the above topic. IMHO it is the right time to do it.
m. Th.

- Dark Themed XnViewMP 1.7.1 64bit on Win11 x64 -
User avatar
m.Th.
XnThusiast
Posts: 1676
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: Support WebP as internal format for thumbnails in DB Cac

Post by m.Th. »

Bump again! ;)
m. Th.

- Dark Themed XnViewMP 1.7.1 64bit on Win11 x64 -
CameronD
Posts: 311
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: Support WebP as internal format for thumbnails in DB Cac

Post by CameronD »

m.Th. wrote:[*]Much better compression than JPG/PNG.
somewhere between significantly better to, rarely, worse.
[*]Much faster compression than JPG (it supports natively all the CPU cores)
I cannot find much evidence for this. Google seem a bit coy about it, comparing PNG times but not jpeg.

I have seen a few comments, such as this where compression is much slower and decompression is 1.4x slower.
Last edited by CameronD on Wed Mar 26, 2014 8:09 am, edited 1 time in total.
User avatar
m.Th.
XnThusiast
Posts: 1676
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: Support WebP as internal format for thumbnails in DB Cac

Post by m.Th. »

CameronD wrote:
m.Th. wrote:[*]Much better compression than JPG/PNG.
somewhere between significantly better to, rarely, worse.
Except corner cases, WebP is much better at the same perceptual quality. That's why Google switched to it internally.
[*]Much faster compression than JPG (it supports natively all the CPU cores)
I cannot find much evidence for this. Google seem a bit coy about it, comparing PNG times but not jpeg.

I have seen a few comments, =http://www.igvita.com/2013/03/07/faste ... ch as this where compression is much slower and decompression is 1.4x slower.
The gain which stems from the shorter time needed to write in DB the much smaller amount of data in the case of WebP, offsets any probable time loss in compression per-se.
m. Th.

- Dark Themed XnViewMP 1.7.1 64bit on Win11 x64 -
CameronD
Posts: 311
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: Support WebP as internal format for thumbnails in DB Cac

Post by CameronD »

m.Th. wrote:
CameronD wrote:
m.Th. wrote:[*]Much better compression than JPG/PNG.
somewhere between significantly better to, rarely, worse.
Except corner cases, WebP is much better at the same perceptual quality. That's why Google switched to it internally.
Webp gives excellent results at high compression, but I would assume thumbnails should be stored at a minumim quality of Jpeg's Q=75%.
This gives 25 to 30% smaller files for the same image quality - using Google's own published data for photographic images.

Certainly, I can see a value for Google in bandwidth-constrained environments - especially for Android browsing. Mobile data can be horribly expensive in backward countries like Australia.
But the thumbnail DB does not operate under the same conditions as web pages.
m.Th. wrote:
[*]Much faster compression than JPG (it supports natively all the CPU cores)
I cannot find much evidence for this. Google seem a bit coy about it, comparing PNG times but not jpeg.

I have seen a few comments, such as this where compression is much slower and decompression is 1.4x slower.
The gain which stems from the shorter time needed to write in DB the much smaller amount of data in the case of WebP, offsets any probable time loss in compression per-se.
At this stage I cannot see any better than marginal improvement - 40% increase in decompression times versus 25 to 30% smaller images sounds to me more like a small loss.

The numbers quoted for compression are webp being 10 times as slow. Several writers have commented on the compression/quality gain coming at a cost.
If the code automatically supported multicore use then surely those tests using Google's own code would have benefited from it.
User avatar
m.Th.
XnThusiast
Posts: 1676
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: Support WebP as internal format for thumbnails in DB Cac

Post by m.Th. »

Certainly, I can see a value for Google in bandwidth-constrained environments - especially for Android browsing. Mobile data can be horribly expensive in backward countries like Australia.
But the thumbnail DB does not operate under the same conditions as web pages.
The main problem today with DAMs is what is happening when the number of photos grow.

There is no difference at small numbers. But because photo collections grow ad infinitum, having the smallest Thumbs.DB possible is a huge gain. Now we have ~600.000 photos in a MS SQL thumbs database at 60 GB. After generating (with Batch Convert) enough WebP "thumbs" of the 'same' quality (and dimensions: 300x200), getting the average size and calculating for the entire collection we'll have at the end a Thumbs.DB around of 5 (five!) GB. Nothing can beat this! 5 GB means that on a (semi)dedicated system the entire DB can live in RAM. Game over.

Also, take in account that (almost) any DB back-end stores data in pages. Hence a small difference in size between thumbs can lead (and it happens) to page overflow. Hence instead one I/O operation we'll have two. Sometimes 2x 'slower' I/O at huge number of photos is quite a lot.
At this stage I cannot see any better than marginal improvement - 40% increase in decompression times versus 25 to 30% smaller images sounds to me more like a small loss.

The numbers quoted for compression are webp being 10 times as slow. Several writers have commented on the compression/quality gain coming at a cost.
If the code automatically supported multicore use then surely those tests using Google's own code would have benefited from it.
Why don't you do your own tests?

For 64 CR2 RAW files my benchmark is the same (23 seconds) for JPEG 90 (quality) and WebP 77 (quality). And at least for my eyes the WebP 77 thumb is significantly better than JPEG 90.
m. Th.

- Dark Themed XnViewMP 1.7.1 64bit on Win11 x64 -
Post Reply