learning metadata storage methods for category/comment

Ask for help and post your question on how to use XnView MP.

Moderators: XnTriq, helmut, xnview

Post Reply
TyMay
Posts: 5
Joined: Mon Jan 07, 2019 2:43 am

learning metadata storage methods for category/comment

Post by TyMay »

I am new to xnview and wanting to understand which types of user defined metadata are stored external without requiring embedding within each image file.
Comments are user derived and have a foot print in the "descript.ion" file for each folder known to xnview.
Comments seem to be created only through the Rclick context of thumbnails within Browser View "EditComment"
In the SearchTool Comments are searchable by the Add Menu but are called "Embedded Comments" after selection
Is that a misnomer?
I have not found a search input dialog that successfully pulls an image into the browser based on comment tag in "descript.ion"
Is it fair to say that comments are never written into the embedded metadata of an image?
These comments by sidecar file are not the same as the comment field in

CATEGORY
I have successfully forced category writes into IPTC and XMP fields but in my cataloging I would like to avoid writing into these image embedded regions
CATEGORY(Not embedded)
I notice this excellent Category Manager tool in the Browser displaying Categoriies and Sets which can be toggled on tree menu.
If I toggle the category for a thumbnail it retains the tagged setting for that thumbnail e ven after exit restart XnView despite the absence of any display of IPTC or XMP tabs in the general viewer for embedded metadata.
This leads me to believe there is at least temporary storage for Category "Manager" metadata external to the target image files.
Is this a correct deduction?
If so where/how is the category data stored (not in descripti,ion sidecar file)?
Is this a viable long term storage scheme for someone who wants to avoid writing organizational data into original image files?

Distrusting my understanding I hope for some expert explanation.and then I can learn how to avoid certain tools and menu choices and also safely configure Settings for metadata.

Ty
User avatar
m.Th.
XnThusiast
Posts: 1662
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: learning metadata storage methods for category/comment

Post by m.Th. »

Read this: https://www.xnview.com/wiki/index.php/C ... _XnView_MP

Besides that:

"descript.ion" is legacy - a very old internet standard from the age of BBSes. It has its merits tho.

The best way to persist the RCK (Rating, Colors, Keywords = Categories in XnView's terminology), comments and other metadata with the files are XMP sidecars / info.
This leads me to believe there is at least temporary storage for Category "Manager" metadata external to the target image files.
Is this a correct deduction?
Yes - but it isn't temporary at all. :)

If you don't want to save your RCK & al. with your files then just disable the corresponding settings in Tools | Settings because anyway these cataloging features are saved in the Program's database. In fact when you filter your images by eg. Categories, the program uses the DB which is waaay faster than searching on disk. You will see that using the "Categories Filter" pane in order to filter by RCK is real-time, compared with a slow search on disk.

IPTC / XMP window is targeted to the users which want to store the data IN files. In order to catalog your files via DB use "Categories" and "Category Sets" panes (you should have them already on screen - if not, then you will find them in the menu View | Data Pane)
If so where/how is the category data stored (not in descripti,ion sidecar file)?
In the program's database - Catalog in XnView's terms.

The database is pretty fast (based on SQLite) and it is managed automatically behind the scenes. There are, in fact, two databases: one for the thumbs (thumbs.db) and the other one for the metadata - XnView.db. You can see their location (for backup purposes - do not mess with them otherwise) in Tools | Settings | Integration - click on "Open the Catalog Location". You can move them on a 2ndary SSD for maximum speed, if you want.

just my 0.02c++ & HTH

If you want to ask something else, feel free :)
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
TyMay
Posts: 5
Joined: Mon Jan 07, 2019 2:43 am

Re: learning metadata storage methods for category/comment

Post by TyMay »

Thanks for the showing that the R click "Edit categories persist in the database and categories are like keywords categories in IPTC and XMP
Ihave become confused by the shared teminology. So I think that the category manager and the addition of "set definitions" is a tool just for the these categories that never leave xnview.
So if I configure the Settings Metadata to
[] import xmpt subject or keyword to DBase
does that mean that the data is brought in to DB just for purposes of fast search only
Then if I 'EDIT these IPTC?XMPcategories via the R click context menu does that persist them in Database without any write back to image?
IT appears that Write back to image is either automated via settings toggle or manually done via the "WRITE" button in the Edit dialog .
Another question" Is the write via button a single write for only the one file or for all known files which have IPTC or XMP fields living in the sqldb?

So many questions but will stop at these

thaniks Ty
User avatar
m.Th.
XnThusiast
Posts: 1662
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: learning metadata storage methods for category/comment

Post by m.Th. »

It isn't a problem to ask. :) ...but please write a little bit clearer.

I will try to guide you:

- Forget what we discussed.
- I will ask you. You will do exams now :)
- We will advance in small steps.

1. Think that we have TWO separate engines:
1.a. One which writes/reads to/from files.
1.b. One which writes/reads from Database

Understood the points above?

Have a look again at material from https://www.xnview.com/wiki/index.php/C ... _XnView_MP

2. The Engine 1.a. is depicted in "IPTC/XMP Window" section
3. The Engine 1.b. is depicted in "Categories (Keywords) Cataloging" section

Ok till now?

4. The Engine 1.a. has much more fields, but has various problems: less data quality (trust me on this), it cannot be searched quickly etc.
5. The Engine 1.b. has options in Tools | Settings | Metadata which, if enabled, allows the engine 1.b. to read/write the fields which Engine 1.b. "knows" to/from files in (almost) the same way in which Engine 1.a. does BUT - again - this applies only on the data which Engine 1.b. is aware/knows, that is Rating(Stars 1-5), Colors and Keywords/Categories (known as RCK).

Understood?

NOW the program really USES only the Engine 1.b. (and almost all the programs, in fact). It provides the Engine 1.a. as a way to put the data in/with the files IF you want to share your data with other applications / users BUT it does NOT use the Engine 1.a. - the IPTC / XMP Window, that is - in order to search etc.

So:

If you enable in Tools | Settings | Metadata - "Import XMP subject or IPTC keyword to DB Categories" this means that these fields FROM the image files will be automatically imported into DB (Engine 1.b) for search, display etc.

If you change a Category by RMB menu, in Categories pane and/or in Category Sets pane it will be changed in DB (Engine 1.b.), BUT if in Tools | Settings | Metadata "Export DB Category to XMP Subject and IPTC Keyword" is checked then this info from DB will be saved ALSO in your file. If this setting is NOT checked then this info from DB will NOT be saved to your image file. The same applies to Rating and Color - they are controlled by similar checkboxes in the same Tools | Settings | Metadata pane.

Clear till now?

Now, let's speak again about the Engine 1.a - that is "Edit IPTC/XMP Window". This engine is NOT automatic, so it needs a decisive user action by pressing the "Write" button (or similar) in order to persist the data into image file / sidecar. This window works ONLY upon the selected images at the moment of window opening. If you opened the window with one image selected you can edit the data only for one image. If you opened the window with 4 images selected, then you can edit the data for 4 images - you have two buttons "<" and ">" in order to navigate between the images and, also, you have a new button "Write to all files" in order to write to all 4 (four) files - your selection. "Write" button always applies to the current file only.

You cannot write data to all the files cataloged in your "sqldb" (as you say it) it would be very dangerous, slow and error prone, neither via Engine 1.a. (the IPTC / XMP Window) neither via Engine 1.b. - RMB Menu, Categories / Category Sets pane.

Hint: To go quickly in the Categories pane and assign categories (keywords) just press F9.

So I work like this:

a.) Select 1st image.
b.) F9
c.) write "bir" - it shows "bird", select it with the arrow keys and press Enter in order to assign this category to the current image
d.) F9 again - the focus moves to the Thumbs pane
e.) use the arrow keys to select the next image(s)
f.) F9 again - the focus moves to the Categories pane...
repeat from step c.) with the corresponding categories.
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
TyMay
Posts: 5
Joined: Mon Jan 07, 2019 2:43 am

Re: learning metadata storage methods for category/comment

Post by TyMay »

Thanks again for describing the 2 means of persistance and the 2 worlds of data/label design that live in each.
I now understand that the Write button is the immediate and manual way to do what I intend to Never do which is to write back into the tag region of an image file. Writing to image file is therefore normal off and I suppose that automatic writes are not enabled by any obscure configuration switch.
I am avoiding such writes just to be clear that the original files are unchanged in byte content or in operating system write labels. ( It is bad enough that moving files around keeps changing their "Creation date" depending on context of the copy/paste and hardware devices that are mounted by OS
So I understand my preference for avoiding image file writes means that I must limit tagging to the world of category tags. These tags appear to be fairly powerful for boolean search so that is good. But what about a single field that is short but freetext memo. Is there such a comment field that I can use that will persist only in sqldb and not in image file (and also not be mixing with the categories controlled by manager)

Than ks Ty
User avatar
m.Th.
XnThusiast
Posts: 1662
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: learning metadata storage methods for category/comment

Post by m.Th. »

I now understand that the Write button is the immediate and manual way to do what I intend to Never do which is to write back into the tag region of an image file.
Yep.
Writing to image file is therefore normal off and I suppose that automatic writes are not enabled by any obscure configuration switch.
You must check your supposition. Go to Toos | Settings | Metadata
I am avoiding such writes just to be clear that the original files are unchanged in byte content or in operating system write labels.
Perhaps is worth mentioning here that IF you would enable the above options (it seems that you don't want) the metadata is written on a separate "section" - the image itself isn't altered. If the file format supports metadata embedding (for example JPG) then the metadata is written in the same file but in a different area, separated from the image itself and if the file format doesn't support metadata embedding (most notably RAW files: CR2, NEF, ARW etc.) then the metadata is written in a separated specialized file (with the same name but with the extension XMP) called "sidecar".
So I understand my preference for avoiding image file writes means that I must limit tagging to the world of category tags.
Yes, PLUS Rating (the "Stars") and Colors (called also "Labels"). Move the mouse in the top-left corner of a thumbnail to see what I mean. The Colors' labels are customizable (see Tools | Settings | Metadata | Labels).
But what about a single field that is short but freetext memo. Is there such a comment field that I can use that will persist only in sqldb and not in image file
Nope. It can be done, the code isn't very, very complicated, but I think that Pierre (the main developer) won't do it unless upon significant community pressure because the problem isn't so much with his work as programmer but with his work as "tech support" because such a field even if "short" as you say (btw, what means "short"?) can significantly slow down the database if it will be abused. Well, I don't think that it will be overly abused but it can happen. Also, the search on this field, even if it will be significantly faster than search on disk for IPTC tags it will be significantly slower than searching for categories via Category Manager. As I said, I don't think that it will be really an issue because not many will use it, but it can happen.

Btw, why do you want such a feature?

Mind you, a free text field is the worst possible thing for data quality (searching). For example you can write for the same concept different words (synonyms) or describe the same action with antonyms, depending of the point of view which you use in the moment of cataloging (for example today I feel that is better to write "In this picture Jon gives flowers to Mary", tomorrow I'll write "Mary receives flowers from John" - if we want to search, then we must think to all possible combinations of synonyms and sometimes(!) antonyms). It is best to minimize the human factor during the cataloging phase, even if it sounds a little unnatural in order to have a higher quality of searching - I know that this isn't always possible tho.
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
Post Reply