Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC etc.)
Moderators: XnTriq, helmut, xnview
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
so which metadata will be the best?
Pierre.
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
I think that we can start the implementation with this layout:xnview wrote:so which metadata will be the best?
https://docs.google.com/spreadsheet/pub ... utput=html
Beware that you have TWO sheets there: Fields and Unique Metadata Space which is in fact the table model of the Fields.
Details in the previous posts.
m. Th.
- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
Hello,
I have a question:
... For me the Xnview.db file is becoming too big ... initially the xnview.db was designed to be splitted in two parts (the big one, for blobs and thumbnails images cache: the Thumbs.db, and a small one Xnview.db without blobs and only for the users extra datas like categories, labels colors, IPTC keywords and so) to be able to make a small backup with only users extra data and to have the search in db function faster.
... So why Xnview.db has now some blobs that consumes data space, is it really necessary?
I have a question:
... For me the Xnview.db file is becoming too big ... initially the xnview.db was designed to be splitted in two parts (the big one, for blobs and thumbnails images cache: the Thumbs.db, and a small one Xnview.db without blobs and only for the users extra datas like categories, labels colors, IPTC keywords and so) to be able to make a small backup with only users extra data and to have the search in db function faster.
... So why Xnview.db has now some blobs that consumes data space, is it really necessary?
XnViewMP Linux X64 - Debian - X64
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
Too big???oops66 wrote:Hello,
I have a question:
... For me the Xnview.db file is becoming too big ... initially the xnview.db was designed to be splitted in two parts (the big one, for blobs and thumbnails images cache: the Thumbs.db, and a small one Xnview.db without blobs and only for the users extra datas like categories, labels colors, IPTC keywords and so)
With what do you compare? Today storage options? Other databases? Which ones? (please do NOT look at Lightroom's DBs... )
Ok, you can Optimize it through Tools | Settings | Catalog | Optimize...
The main reason is exactly the topic of this thread: Large binary blobs (read: black-boxes) which holds ALL the metadata (see Images.Meta field).
However, the main target is NOT to make the db small but to make it FAST. Ok, „small” has its influence in „fast” but not it isn't the only factor.
Yep. Do it. Zip on the Db. Mine from 1,07 GB goes to @ 300 MB. I am pretty sure that when the metadata will be split, the compression ratio will be bigger. However, here the size is mostly reduced by Optimization (see above) and leaving off the noise which sometimes is put in the Meta field.to be able to make a small backup with only users extra data
Sure. This is the main reason to split the metadata from Images.Meta blob in Fields.
and to have the search in db function faster.
... So why Xnview.db has now some blobs that consumes data space, is it really necessary?
However you have a point here: I think that is better to have 3 (three) DBs: XnView.db, XnMeta.Db, and Thumbs.Db. THE fastest program on the market (AfterShotPro) has this layout (there, the DBs are called Base, Thumbs, Meta respectively).
This will solve the backup problem of the only user-generated DB (XnView.db) and will keep the speed at top.
Of course, the new DB (XnMeta.Db) will be attached in the same manner in which is used now Thumbs.db.
m. Th.
- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
( Xniew.db ... too big in relative point of view ... only half of the Thumb.db size here.)
PS-edit: It would be probably also interesting to have this third database (Xnview-user.db) reserved only for: "Not already embedded data" (into each pics) ... to have an easy way to save, check the difference, or to synchronize data, between metadata pics and this database, if required. So this Xnview-user.db must be blank if all data are already synchronized (no needed backup, because the backup already exists into each pics/jpegs).
Hello, ... In fact, that was my subliminal request toom.Th. wrote: ... However you have a point here: I think that is better to have 3 (three) DBs: XnView.db, XnMeta.Db, and Thumbs.Db.
PS-edit: It would be probably also interesting to have this third database (Xnview-user.db) reserved only for: "Not already embedded data" (into each pics) ... to have an easy way to save, check the difference, or to synchronize data, between metadata pics and this database, if required. So this Xnview-user.db must be blank if all data are already synchronized (no needed backup, because the backup already exists into each pics/jpegs).
XnViewMP Linux X64 - Debian - X64
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
Nope, sorry. We need all the (meta)data in databases (in XnMeta.Db in our case) because of Search. We need to search/filter this data. The diff. checks / sync. will be done based on Last Modified Dates.PS-edit: It would be probably also interesting to have this third database (Xnview-user.db) reserved only for: "Not already embedded data" (into each pics) ... to have an easy way to save, check the difference, or to synchronize data, between metadata pics and this database, if required. So this Xnview-user.db must be blank if all data are already synchronized (no needed backup, because the backup already exists into each pics/jpegs).
m. Th.
- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
... Basically, it's not the same last based modified date, because these not synchronized data are not already embedded to the metadata of the pics. Then a simple duplication is only needed for these few fields to be synchronized or not (colors label, categories, rattings, set comment, etc)... I don't think than it is a so big deal. If not, this third user-database is not very interesting/necessary. (so generally only few x* KB is needed instead of X*100 MB to save this "not already synchronized user data DB" )
XnViewMP Linux X64 - Debian - X64
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
Let's cool a little. We have the following facts:oops66 wrote:... Basically, it's not the same last based modified date, because these not synchronized data are not already embedded to the metadata of the pics. Then a simple duplication is only needed for these few fields to be synchronized or not (colors label, categories, rattings, set comment, etc)... I don't think than it is a so big deal. If not, this third user-database is not very interesting/necessary. (so generally only few x* KB is needed instead of X*100 MB to save this "not already synchronized user data DB" )
- all DAMs on the market have as one central feature the Real-Time Filtering on Metadata by presenting to the user the values already present in DB. Every. program. And I have installed 7-8 of them. Besides this, there are several programs which provide analytics/graphs (via a plugin like Lr) or natively (Zoner).
- the industry standard for sync between Db and Files is Real-Time sync: when you change a value (being it RCK or other/image metadata), this value is immediately written on-disk in embedded EXIF/XMP or in a XMP sidecar. Due of OS caching and slowness of human user, no performance hit is taken. This system works not only for us (which basically have only RCK values) but also for DAMs which support non-destructive editing - hence much, much, much more values (edits etc.) to write in XMP/embedded metadata.
- if one really wants an off-line sync, internally the code anyway will load the entire EXIF/metadata chunk (mind you, disks nowadays have big sectors) from file and taking in account the paged structure of SQL file, you will load in memory the entire page (many rows) even if you will read just one row or just one field. So, you will gain nothing, except code complexity. That's why off-line syncs in the way in which you want are in 99,9999% implemented by using change logs (lists of changes). However, this is an overkill for us since, anyway, we will have a metadata Db. Taking in account that this scenario is rare and the performance is the same, I say that a LMDate is enough.
m. Th.
- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
... This is already cool because technically (about Find Vs Sync): "One does not preclude the other" (a French saying) "L'un n'empêche pas l'autre".
XnViewMP Linux X64 - Debian - X64
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
Synchronisation can be either:oops66 wrote:... Basically, it's not the same last based modified date, because these not synchronized data are not already embedded to the metadata of the pics....
- required by the user
- not wanted by the user
- not possible with certain file types
For example, a user makes some changes to a selection of one or more files, the DB is changed and then the changes are immediately applied to the files. This will take a finite, but small, length of time.
The only time the DB and files should be out of sync is if the program or computer crashes in that brief time interval. If that happened then the DB itself would be unlikely to be correct either.
I cannot see much value in splitting the DB. At one extreme, which I prefer, all metadata is with the image files and the db can be safely deleted and rebuilt from file content. At the other extreme, the DB integrity would be critical, and I don't think it would be safe to lose the connection between the split DB parts.
In relation to relative speed of search, we are talking about many orders of magnitude difference. Think in the following very approximate timescales:
- less than one second: DB entry, indexed
- several seconds: DB entry, not indexed
- several more seconds: DB entry currently in the blob structure
- many minutes: not in DB - read every file for metadata
- limiting what fields get stored in DB: I think this would be quite messy to program, and prone to user choices that would be regretted later
- limiting what fields get indexed: This might offer scope for some reduction, although in the current structure there do not seem to be many indices that could be safely removed. Maybe just color and rating. I don't think that difference would be noticeable in db size
Re: Very important: Metadata V2 for 0.70 (EXIF, XMP, IPTC et
Hello,CameronD wrote:Synchronisation can be either:oops66 wrote:... Basically, it's not the same last based modified date, because these not synchronized data are not already embedded to the metadata of the pics.......
- 1-
- required by the user
2-- not wanted by the user
3-- not possible with certain file types
Yes, it's that why I think than this eventual xnview-user.db must be smaller as possible (especially for case 2 and 3) in fact, only few fields/data are concerned.
Me too when it is possible, (so excepted for case 3, via an eventual extra xnview-user.db file ?)... I prefer, all metadata is with the image files and the db can be safely deleted and rebuilt from file content...
XnViewMP Linux X64 - Debian - X64