Page 1 of 1

Support WebP as internal format for thumbnails in DB Cache

Posted: Mon Jul 08, 2013 8:20 am
by m.Th.
Please add support for WebP as internal format for thumbnails in thumb.db

Reasoning:
  • Much better compression than JPG/PNG.
  • Much faster compression than JPG (it supports natively all the CPU cores)
This will speedup the thumbnail creation, thumbnail reading and will save disk space.

Perhaps is better to add three levels:
  • Hi-res WebP
  • Low-res WebP
  • Loseless WebP

Re: Support WebP as internal format for thumbnails in DB Cac

Posted: Mon Jul 08, 2013 9:47 am
by oops66
+1 I support.

Re: Support WebP as internal format for thumbnails in DB Cac

Posted: Thu Nov 14, 2013 5:23 pm
by m.Th.
Seeing this one http://www.xnview.com/mantisbt/view.php?id=146 I bump the above topic. IMHO it is the right time to do it.

Re: Support WebP as internal format for thumbnails in DB Cac

Posted: Thu Mar 20, 2014 12:52 pm
by m.Th.
Bump again! ;)

Re: Support WebP as internal format for thumbnails in DB Cac

Posted: Wed Mar 26, 2014 4:47 am
by CameronD
m.Th. wrote:[*]Much better compression than JPG/PNG.
somewhere between significantly better to, rarely, worse.
[*]Much faster compression than JPG (it supports natively all the CPU cores)
I cannot find much evidence for this. Google seem a bit coy about it, comparing PNG times but not jpeg.

I have seen a few comments, such as this where compression is much slower and decompression is 1.4x slower.

Re: Support WebP as internal format for thumbnails in DB Cac

Posted: Wed Mar 26, 2014 7:15 am
by m.Th.
CameronD wrote:
m.Th. wrote:[*]Much better compression than JPG/PNG.
somewhere between significantly better to, rarely, worse.
Except corner cases, WebP is much better at the same perceptual quality. That's why Google switched to it internally.
[*]Much faster compression than JPG (it supports natively all the CPU cores)
I cannot find much evidence for this. Google seem a bit coy about it, comparing PNG times but not jpeg.

I have seen a few comments, =http://www.igvita.com/2013/03/07/faste ... ch as this where compression is much slower and decompression is 1.4x slower.
The gain which stems from the shorter time needed to write in DB the much smaller amount of data in the case of WebP, offsets any probable time loss in compression per-se.

Re: Support WebP as internal format for thumbnails in DB Cac

Posted: Wed Mar 26, 2014 8:08 am
by CameronD
m.Th. wrote:
CameronD wrote:
m.Th. wrote:[*]Much better compression than JPG/PNG.
somewhere between significantly better to, rarely, worse.
Except corner cases, WebP is much better at the same perceptual quality. That's why Google switched to it internally.
Webp gives excellent results at high compression, but I would assume thumbnails should be stored at a minumim quality of Jpeg's Q=75%.
This gives 25 to 30% smaller files for the same image quality - using Google's own published data for photographic images.

Certainly, I can see a value for Google in bandwidth-constrained environments - especially for Android browsing. Mobile data can be horribly expensive in backward countries like Australia.
But the thumbnail DB does not operate under the same conditions as web pages.
m.Th. wrote:
[*]Much faster compression than JPG (it supports natively all the CPU cores)
I cannot find much evidence for this. Google seem a bit coy about it, comparing PNG times but not jpeg.

I have seen a few comments, such as this where compression is much slower and decompression is 1.4x slower.
The gain which stems from the shorter time needed to write in DB the much smaller amount of data in the case of WebP, offsets any probable time loss in compression per-se.
At this stage I cannot see any better than marginal improvement - 40% increase in decompression times versus 25 to 30% smaller images sounds to me more like a small loss.

The numbers quoted for compression are webp being 10 times as slow. Several writers have commented on the compression/quality gain coming at a cost.
If the code automatically supported multicore use then surely those tests using Google's own code would have benefited from it.

Re: Support WebP as internal format for thumbnails in DB Cac

Posted: Wed Mar 26, 2014 12:59 pm
by m.Th.
Certainly, I can see a value for Google in bandwidth-constrained environments - especially for Android browsing. Mobile data can be horribly expensive in backward countries like Australia.
But the thumbnail DB does not operate under the same conditions as web pages.
The main problem today with DAMs is what is happening when the number of photos grow.

There is no difference at small numbers. But because photo collections grow ad infinitum, having the smallest Thumbs.DB possible is a huge gain. Now we have ~600.000 photos in a MS SQL thumbs database at 60 GB. After generating (with Batch Convert) enough WebP "thumbs" of the 'same' quality (and dimensions: 300x200), getting the average size and calculating for the entire collection we'll have at the end a Thumbs.DB around of 5 (five!) GB. Nothing can beat this! 5 GB means that on a (semi)dedicated system the entire DB can live in RAM. Game over.

Also, take in account that (almost) any DB back-end stores data in pages. Hence a small difference in size between thumbs can lead (and it happens) to page overflow. Hence instead one I/O operation we'll have two. Sometimes 2x 'slower' I/O at huge number of photos is quite a lot.
At this stage I cannot see any better than marginal improvement - 40% increase in decompression times versus 25 to 30% smaller images sounds to me more like a small loss.

The numbers quoted for compression are webp being 10 times as slow. Several writers have commented on the compression/quality gain coming at a cost.
If the code automatically supported multicore use then surely those tests using Google's own code would have benefited from it.
Why don't you do your own tests?

For 64 CR2 RAW files my benchmark is the same (23 seconds) for JPEG 90 (quality) and WebP 77 (quality). And at least for my eyes the WebP 77 thumb is significantly better than JPEG 90.