[PATCH] performance-tests: tests for renamed/copied files in notmuch new

[Tomi Ollila]

On Tue, Apr 02 2019, David Bremner wrote:

> Several people have observed that this is surprisingly slow, and we
> have a proposal to add tagging into this code path, so we want to make
> sure it doesn't imply too much of a performance hit.
> ---
>  performance-test/T00-new.sh | 30 ++++++++++++++++++++++++++++++
>  1 file changed, 30 insertions(+)
>
> I added these tests to help evaluate Michael's propesed patch. I'll send the results in a seperate email.
>
> diff --git a/performance-test/T00-new.sh b/performance-test/T00-new.sh
> index 68750129..cec28d58 100755
> --- a/performance-test/T00-new.sh
> +++ b/performance-test/T00-new.sh
> @@ -12,4 +12,34 @@ for i in $(seq 2 6); do
>      time_run "notmuch new #$i" 'notmuch new'
>  done
>  
> +manifest=$(mktemp manifestXXXXXX)
> +
> +count=0
> +total=0
> +while read -r name ; do
> +    if [ $((total % 4 )) -eq 0 ]; then
> +        echo $name >> $manifest
> +        count=$((count + 1))
> +    fi
> +    total=$((total + 1))
> +done < <(find mail -type f ! -path 'mail/.notmuch/*' )

// this comment was written last in this email, just for fun >;) //

find mail -type f ! -path 'mail/.notmuch/*' | sed -n '1~4 p' > $manifest
count=`wc $manifest`

(I'd be interested which one of the above were faster -- my suggestion
does quite a many more forks and execve's but abowe read loop 200 000 
read(2)'s and [lf]seek(2)s (and then 50 000 opens). 
well, probably no-one would notice difference...)

> +
> +while read -r name ; do
> +    mv $name ${name}.renamed
> +done <  $manifest

--------'12' -- 2 spaces above (and below...)

luckily bash read builtin does not read input byte at a time
(IIRC it read 128 bytes, then scanned for newline and then seeked
 -- in this case it can, since file was redirected -- fd is seekable)

50 000 mv(1) executions definitely take time.

perl -nle 'rename $_, "$_.renamed"' $manifest  would be significantly faster 

> +
> +time_run "new ($count mv)" 'notmuch new'
> +
> +while read -r name ; do
> +    mv ${name}.renamed $name
> +done <  $manifest
> +
> +time_run "new ($count mv back)" 'notmuch new'
> +
> +while read -r name ; do
> +    cp ${name} $name.copy
> +done <  $manifest

perl -nle 'link $_, "$_.copy"' $manifest  ?

> +
> +time_run "new ($count cp)" 'notmuch new'
> +
>  time_done
> -- 
> 2.20.1
>
> _______________________________________________
> notmuch mailing list
> notmuch at notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch