Improvement of batch Tagging IPTC/XMP

[Gérard 91]

IPTC/XMP tagging features are quite nice.

At the moment it seems possible to add a key word without modifying other, but not to suppress one, without unifying all keywords in selected files.
It seems also possible to change all the content of a field but not to replace only a word or add one.

So it would be much better if we have some batch process to correct generic errors/mistakes in text fields (as Caption) or to change Key word (for instance to follow taxonomy evolution) .

I suggest to the benevolence of Pierre the implementation of features allowing, for selected files resulting (or not) from a query -similar as the one implemented- :
- to select concerned field
- to define word -character string- to erase (or none)
- to define word -character string- that will replace (or none)
- to define word -character string- to add
- to define position (head, end of field , or before/after an other character string)

This would be marvellous for the one, like me, who find after a while, some mistakes or lacks in IPTC/XMP fields of his pictures.

Of course this is a bulk suggestion. What do you think Pierre?

Bien cordialement
Gérard

[m.Th.]

Well, I'm not Pierre , but IMHO perhaps I could help with some little suggestions. If you think that the message is useless, just safely ignore it.

The experience in DAMs shows that the process should be a little different in order to have the desired results.

Bulk/batch data dynamics (edits) due of (human) errors, reality changes etc. are, by far, handled best in the program's Database / Catalog and from there pushed to the outside files.

There you have a centralized, spherical view of all of your keywords and metadata. There you can add, edit, merge, delete and do any other processing (for ex. assigning synonyms) because you SEE how and which the keywords really are and not how you THINK that they are. There is no guesswork involved. (Ok, there is guesswork because we are all limited humans, but let's skip this for now.)

Having a database centric management of the Catalog, makes the "batch" editing immediately transparent, intuitive and visual by the very nature of the things. For ex. changing the keyword in the Categories tree, "propagates" immediately the change to all photos. (In fact, there is no propagation because in the DB the keyword is stored by reference ID and not by name). In fact, exactly in this way XnViewMP works, if one uses the catalog.

Also the speed is much faster - several orders of magnitude compared with disks where the IPTC/XMP files reside.

The principal thing which is missing in XnViewMP is asynchronous synchronization (gee, what an expression... ) between DB and XMP/IPTC sidecars. IOW, the possibility to update from/to sidecars to/from database by pressing a button/menu option. Usually (ASP, IDImager etc.) these are implemented as two right click menu items (labeled "Update from..." and "Update to...") in the Catalog window described here: http://newsgroup.xnview.com/viewtopic.p ... 67#p113504 In some other cases (Lr) there are also some small icons on the thumbnail which shows that the Catalog info and the sidecar file are desynced and by clicking on the icon the sync is done for all the selected images.

Now XnView does this only in sync (IOW the DB is updated together with the XMP file) - if of course the corresponding setting is enabled in Tools | Settings... but this isn't enough because of many reasons. I can expand the topic upon request.

[Gérard 91]

Hello m.Th.
Thanks for you interesting post.
Of course they are many advantages to use a dB, specially a high speed for request.
But and but is important, if DB is the real image of file and not damaged *.

Independently of my opinion about my carefulness upon DB* I don't think DB will solve by itself the need of correcting some part of fields. There is a need of specific request and replace or add.
If DB do it so much the better. But for portability the best is to have informations in files first or just after.

* I really do use IPTC/XMP tag for thousands jpeg files since two years.
I have had and still have problem with the DB (lost several times informations " categories", of rating displaying labels, that where not written in files (lost of " categories" need to delete the DB
I don't use LR (perhaps I should !) for tagging only for Raw development, because of all possibilities of Xnview, speed of displaying and weight of program and DB (no need of thumbnail half size of the picture)

I read too quickly your very interesting post http://newsgroup.xnview.com/viewtopic.p ... HO#p114070 but shall do it again more "slowly" my english is not as fluent as your!
Kind regards

[m.Th.]

I will change a little the order of your text in order to answer logically:

I have had and still have problem with the DB (lost several times informations " categories", of rating displaying labels, that where not written in files (lost of " categories" need to delete the DB

This is not the DB's fault (IOW not SQLite's fault) but a known bug in XnView MP's code. ISTM that Pierre is working on that right now. Or perhaps he already fixed it?

But and but is important, if DB is the real image of file and not damaged *.

Yes, sure. This is the 'reason d'etre' of the entire system.

However, the thing can be made better by saving the info both in DB and in sidecars (in fact this is our main topic here), just that we discuss the WAY (IOW how to do it).

If DB do it so much the better. But for portability the best is to have informations in files first or just after.

Just after. (Emphasis mine) (or in parallel, to be more correct).

DB is fast. Files are slow. Hence it is much better to fill the DB, create an update log for the files and after this "the program" (the main thread) is free for work with all the data in it.

And then in a background low/medium priority thread the program will update the sidecars for the file names which are stored in the update log created at the step 1. Of course, there must be a check at the program Close event to have a look if the update thread is still working and ask the user if he wants to really terminate the program and do the update next time when he'll start the program or just wait for the thread to finish its work. Lightroom does in this way and is very safe.

Independently of my opinion about my carefulness upon DB* I don't think DB will solve by itself the need of correcting some part of fields. There is a need of specific request and replace or add.

Of course "not by itself". But one of the big advantages is that the DB will show you the available captions, keywords etc. In this way XnView, Lightroom and all the other programs work in this very moment. You can visually choose, add, organize, edit and delete your items.

Having a GUI which will provide the already entered data and guide the user leads to much higher data entry quality and search quality, because the human factor is the victim of momentary influences and will enter / search the data according with his disposition at that very moment. These are basic data quality principles. Think now what would happen if we didn't have the Categories/Keywords Tree... how we would enter / search the data and with which results. As you (and many others) said the current IPTC form is not enough and we need "some batch process to correct generic errors/mistakes"

...but this cannot be done if the form is tied directly with the sidecars because the file system is by its very nature a "distributed database" and gathering the data in one central place in order to see which are the "generic errors/mistakes" are is very costly.

That's why all the programs in our market area today fight on this central point: database management.

Based on my small experience, I proposed a small draft on how I see the metadata implemented in XnView and its relationship with sidecars here: http://newsgroup.xnview.com/viewtopic.php?f=60&t=28345

PS: Yes, Lightroom is big and slow. That's why we're here.

[Gérard 91]

Thanks a lot m.Th. for your explanation.
I tried to used categories (with trees) but twice I was obliged to all delete the DB and so I lost my work (perhaps I was wrong but don't know how to get out of problems). So I give up for a while waiting XnV MP is reliable with is DB and (also or) how to use it safely (or at least we have a scheme of the expected process). This last point also will avoid to incriminate program we wrongly use.
My Keys words are in a text file; it was nice to use in Xnview Win with which I have had no serious problem. Unhappily this one doesn't memorise rating in xmp fields.
The mistake I was talking about are spelling mistake for caption (comments) to correct or add at the bottom (or middle) of field concerning one or several files. Also replacing some Keywords (or adding but this already possible).
In my opinion, but I'm not specialist, an independent category db would be more safe. ("Pas tous les oeufs dans le même panier").
Thanks again to took time to explain.
I'm all ready going to read again composedly the thread you indicated.
If I don't find solution for my other problem here in page 2 http://newsgroup.xnview.com/viewtopic.p ... 7&start=15, I 'll shout help again!
Kind regards
Gérard

[xnview]

I tried to used categories (with trees) but twice I was obliged to all delete the DB and so I lost my work (perhaps I was wrong but don't know how to get out of problems).

By using what?

[Gérard 91]

Pierre,
I just came back home.
Sorry I am not sure to understand well your question. If you mean, how did I delete, the answer is: I deleted DB with Exp Win (out of XnView MP, because nothing to do to get out the problem with F12).
But it was with a previous version, so interest of subject probably weak
best regards

[xnview]

Sorry I am not sure to understand well your question.

What are the step to reproduce this bug?

[Gérard 91]

Pierre,
Sorry, now I can't tell how this happened, i don't remember precisely.
I can say those last weeks I used Xnview MP on a new net book (W8) during a trip. That means the laptop had a clean new install of MP (0.61). I didn't notice too big problem.
I tagged around 5000 pictures first with generic IPTC (writing in file) and after with labels (In XMP) in order to filter/select those for trash, for improvement ...
No problem noticed tagging JPEG (I've done it separately from RAW/NEF to avoid the problem of creating some file.bak).
But I noticed several crash when tagging label (note). I think one crash for ten processes. Happily I have had only few impact on the DB afterwards. When I have had, deleting the concerned folder in option menu/datatbase, resolve the problem of disappearing labels.

About the subject of the topic, I'm still interested (and I hope not to be the only one) by possibility of correcting IPTC, as I try to explain in top.
I know this wish is a supplementary job for you. Nevertheless it seems no program do it at the moment.
Hoping you can implement some thing in a new release.
Many thanks for all you 've done!

Gérard

[xnview]

About the subject of the topic, I'm still interested (and I hope not to be the only one) by possibility of correcting IPTC, as I try to explain in top.

Ok, but how to make ui for that?

[Gérard 91]

Thanks to ask for more
I'll draw something simple this evening in order to illustrate the above "raw" need .

[Gérard 91]

I tried to draw a simple proposal for the process involved. My version of XnViewMP is in French I used it such as it is.

1 Localisation of menu : in tools/metadata





2 For GUI, I included in the same frame as the one used for editing. Of course it could be simpler, if it's too difficult to have thumbnail on the right frame.
Modifying one field at a time should be easier to perform and more clear to the user. It's also sufficient for the purpose which is to correct ans not to enter data.
Adding new word/text is useless for keyword/supp category. This is already implemented and positioning such a new text as no real use..




Of course these are only proposals which need to be improved
Thanks to to consider this idea.
Gérard