v2 of drop HTML tags when indexing

[David Bremner]

This obsoletes id:20170322112306.12060-1-david at tethera.net

Compared to the previous version:

- drop test already pushed upstream

- add diagram of new state machine

- add longer, and hopefully more illuminating comment about how the
  (new) state machines are encoded

- don't create the variable 'current', stick with the existing 'next'
  as a state index/pointer