muchsync files renames
David Mazieres
dm-list-email-notmuch at scs.stanford.edu
Sun Aug 23 13:06:20 PDT 2015
Amadeusz Żołnowski <aidecoe at aidecoe.name> writes:
> Hi David,
>
> Fist of all thank you for such elaborate answer.
>
> I have missed the paragraph about maildir.synchronize_flags somehow. I
> have it enabled. So this must be source of a problem (?).
I've only ever tested with mailder.synchronize_flags = true, because I'm
worried that if I disable it I will have a hard time re-enabling it. I
do think it is a source of inefficiency, but muchsync should still be
usable.
> Here follows steps I followed:
>
> 1. I initialized server locally with muchsync -vv. My mail is stored in
> ~/Mail on the server.
> 2. I run muchsync --init ~/mail SERVER. (Directory names do not need to
> be the same, do they?)
Confirmed that directory names do not need to be the same.
> 3. I run muchsync SERVER.
> 4. When it lasted much longer then initialization I canceled it by
> single SIGINT (^c).
Interesting. I wish I knew why this was taking much longer than running
it on the server, and whether the delay was caused by client activity or
server activity.
I don't suppose you'd be willing to make a copy of your mail database to
repeat the experiment without any risk of messing up your real maildir?
Basically what would be interesting to see is (assuming .notmuch-copy on
server is configured for this disposable copy):
# Log everything for later analysis
script
# Use new config file location for disposable copy
export NOTMUCH_CONFIG=$HOME/.notmuch-copy
# Set up a new initial database
muchsync -vvvv --init ~/test-copy SERVER -vvvv --config=.notmuch-copy
# Now initial copy is made, see if client is slow
# Is notmuch new itself slow?
notmuch new
# Is resynchronizing muchsync with notmuch slow?
muchsync -vvvv
# Now see if something weird happened on server to make notmuch new slow
ssh SERVER notmuch --config=.notmuch-copy new
# Now see if something weird happened on server to make muchsync slow
ssh SERVER muchsync -vvvv --config=.notmuch-copy
# Now finally try resynchronizing to see if this is slow
muchsync -vvvv SERVER -vvvv --config=.notmuch-copy
# Save script file
exit
Does something along these lines make sense? If you can figure out
which of these is slow (other than --init, which always will be), and
what the verbose chatter is, that would help me narrow down and identify
any problem.
> 5. I rerun muchsync SERVER and then it notified me that notmuch
> identified files names changes - more than 1000.
Were the link changes on the client (sent) or the server (received)
side?
> 6. I waited a bit and then I canceled it by SIGINT.
> 7. I run muchsync --noup SERVER. This took only seconds to finish.
So this suggests the issue is on the client side. It sounds like a bug.
I wonder if I we can just reproduce this using a public email corpus, so
we don't have to worry about people's private email.
> I suspected that muchsync at step 3 and 5 tried to push files renames
> back to server. But now I am not sure what was going on. Have I
> desynchronized file mail flags? It's hard to say if anything has broken
> for me, but I am a bit worried anyway.
>
> If I just disable maildir.synchronize_flags and rerun muchsync, will
> everything get synchronized properly?
I don't think that will change things. maildir.synchronized_flags will
make things slower, but it shouldn't affect the SUMMARY numbers you see
at the end of muchsync, other than maybe files moving from .../new to
.../cur. But presumably most of your mail files were already in cur
directories.
David
More information about the notmuch
mailing list