On disk tag storage format

[Ethan Glasser-Camp]

Ethan Glasser-Camp <ethan.glasser.camp at gmail.com> writes:

> I've modified the script so that it would run by mangling filenames,
> which is irreversible (the original tried to encode/decode filenames
> reversibly). Then I got a little carried away, adding --verbose and
> --dry-run options as well as removing a couple trailing
> semicolons. Here's my version, in case it should interest anyone else.

Hi guys,

There was a bug in the previous version I sent. It didn't handle
unlinking tags correctly. Also, I spotted a bug in syncing to untagged
messages. Maybe I should stop using emails as version control.

---- 8< ----


-------------- next part --------------
A non-text attachment was scrubbed...
Name: linksync.py
Type: text/x-python
Size: 6574 bytes
Desc: slightly more tested this time
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20131007/ce212066/attachment-0002.py>
-------------- next part --------------

---- 8< ----

Of course, the next step is to sync using this mechanism. Rsync doesn't
really have a concept of history, which basically makes it unusable for
this purpose [1]. Unison doesn't really understand renames, so it gets
confused when you mark a message as read (which might move it from new
to cur, and definitely changes its tags). Bremner suggested
syncmaildir. Syncmaildir doesn't understand links at all. Bremner
suggested that we could use some parts of syncmaildir to implement the
tag syncing we need.

I didn't have anything else going on this weekend so I tried to
prototype the approach. It turns out to be possible to leverage some
parts of syncmaildir. I translated a bunch of smd-client into a new
program, tagsync-client, that links to messages in an existing notmuch
DB. It seems like it's possible to use it in place of the existing
smd-client by putting lines like this in your config:

SMDCLIENT=~/src/tagsync.git/tagsync-client.py
REMOTESMDCLIENT=~/src/tagsync.git/tagsync-client.py

The sequence of commands I ran:

- linksync.py to dump tags to ~/Mail/.notmuch/exported-tags
- smd-pull mail to sync ~/Mail but excluding .notmuch
- notmuch new
- smd-pull tagsync (using the above client) to sync ~/Mail/.notmuch/exported-tags
- linksync.py to pull tags from ~/Mail/.notmuch/exported-tags

syncmaildir doesn't cope well with drafts, so it might choke on that,
and it doesn't like symlinks (it thinks they're always to directories),
so be sure to run linksync with -l hard.

Here's the script. It's a work in progress; I have only tested it once in one direction.

---- 8< ----

-------------- next part --------------
A non-text attachment was scrubbed...
Name: tagsync-client.py
Type: text/x-python
Size: 19931 bytes
Desc: client script
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20131007/ce212066/attachment-0003.py>