[PATCH 0/5] notmuch batch count
Tomi Ollila
tomi.ollila at iki.fi
Tue Jan 22 05:43:07 PST 2013
On Mon, Jan 21 2013, Jani Nikula <jani at nikula.org> wrote:
> On Wed, 16 Jan 2013, Tomi Ollila <tomi.ollila at iki.fi> wrote:
>> One could argue that we'd should send json "documents" to notmuch in
>> stdin and notmuch would output json(/sexp) "documents". That is just
>> SMOP. I bet Austin would like this solution, especially the part
>> that involves writing or integrating json parser >;).
>> I'd be happy with this 'batch' approach.
>>
>> I'll be testing this soon, but refrain from reviewing the code
>> until 0.15 is out.
>
> id:87a9s5cp38.fsf at zancas.localnet ;)
OK :D
I started testing:
First a remote script to prepare:
cat > remote-notmuch-test.sh << EOF
#!/bin/bash
printf -v ARGS '%q ' "$@" # bash feature
ssh -S '~'/.ssh/master-notmuch-remote1 -oControlMaster=no 0.1 notmuch $ARGS
EOF
chmod 755 remote-notmuch-test.sh
Note 2 things about the script:
1) '~' is in quotes so shell doesn't expand it -- ssh uses /etc/passwd
instead of environment variable to get HOME (security feature).
This can be easly tested by prefixing the command with HOME=/tmp.
2) The remote address '0.1' is ignored when control socket is active
if control socket is down, connection will fail (0.0 connects to
localhost (I'm not sure which interface in that case)).
Another "script", for emacs:
cat > remote-notmuch-test.el << EOF
(load "notmuch")
(setq notmuch-command "/home/too/remote-notmuch-test.sh")
(setq notmuch-saved-searches '(("inbox" . "tag:inbox")
("unread" . "tag:unread")
("contrib" . "tag:notmuch::contrib")
("pushed" . "tag:notmuch::pushed")
("needs-review" . "tag:notmuch::needs-review")
("moreinfo" . "tag:notmuch::moreinfo")
("stale" . "tag:notmuch::stale")
("wip" . "tag:notmuch::wip")
("bug" . "tag:notmuch::bug")
("obsolete" . "tag:notmuch::obsolete")))
(notmuch)
EOF
First ssh connection with ControlMaster needs to be started
from one terminal:
ssh -S '~'/.ssh/master-notmuch-remote1 -M remote.host.tld
Then, from another terminal, emacs:
emacs -l remote-notmuch-test.el
works, takes quite a long time -- opening 'all tags' even longer.
Let's edit wrapper script:
cat > remote-notmuch-test.sh << EOF
#!/bin/bash
printf -v ARGS '%q ' "$@" # bash feature
echo /usr/bin/time ./remote-notmuch-test.sh $ARGS >> notmuch-replay.$PPID.log
ssh -S '~'/.ssh/master-notmuch-remote1 -oControlMaster=no 0.1 notmuch $ARGS
EOF
chmod 755 remote-notmuch-test.sh
Now, after emacs -l remote-notmuch-test.el run and all tags opened,
exit emacs and run the replay script
env TIME='grepme %Us user %Ss system %P cpu %E total' /usr/bin/time sh ../notmuch-replay.1744.log
last line outputs:
grepme 0.38s user 0.40s system 1% cpu 0:48.21 total
and wc notmuch-replay.1744.log
61 240 3787 notmuch-replay.1744.log
and each of the individual command in script printed either
grepme 0.00s user 0.00s system 1% cpu 0:00.78 total or
grepme 0.00s user 0.00s system 1% cpu 0:00.79 total.
( 0.785 * 61 = 47.9 so notmuch was wasted elsewhere :)
---
Ok, now patching 2 notmuch trees, one in local and one in
remote machine and building and installing those.
Running emacs -l remote-notmuch-test.el
was fast (not in one second, but just a few).
Unsuprisingly, the "replay" script cannot be run anymore
as it requires input from stdin, therefore tuning it as follows
cat > remote-notmuch-test.sh << EOF
#!/bin/bash
printf -v ARGS '%q ' "$@" # bash feature
echo `date`: $ARGS >> notmuch-dates.$PPID.log
echo /usr/bin/time ./remote-notmuch-test.sh $ARGS >> notmuch-replay.$PPID.log
ssh -S '~'/.ssh/master-notmuch-remote1 -oControlMaster=no 0.1 notmuch $ARGS
date >> notmuch-dates.$PPID.log
EOF
chmod 755 remote-notmuch-test.sh
now, again: emacs -l remote-notmuch-test.el
then 'echo output of echo : >> notmuch-dates.2678.log'
and finally opening 'all tags'
cat notmuch-dates.2678.log
Tue 22 Jan 15:04:34 EET 2013: count
Tue 22 Jan 15:04:35 EET 2013
Tue 22 Jan 15:04:35 EET 2013: count --batch
Tue 22 Jan 15:04:36 EET 2013
Tue 22 Jan 15:04:36 EET 2013: search-tags
Tue 22 Jan 15:04:37 EET 2013
output of echo :
Tue 22 Jan 15:04:45 EET 2013: count
Tue 22 Jan 15:04:45 EET 2013
Tue 22 Jan 15:04:45 EET 2013: count --batch
Tue 22 Jan 15:04:47 EET 2013
Tue 22 Jan 15:04:47 EET 2013: search-tags
Tue 22 Jan 15:04:47 EET 2013
Tue 22 Jan 15:04:47 EET 2013: count --batch
Tue 22 Jan 15:04:49 EET 2013
3 seconds for first tags and 4 seconds for second tags (approximately :)
-- quite an improvement 48 seconds.
Also, changes work fine. Now that I forgot my laptop charger to office
I'll have to wait until tomorrow to do the core review.
(maybe the 2 'count's could be combined in elisp code ;)
>
> J.
Tomi
More information about the notmuch
mailing list