@matrixreq/htmldiff 1.4.0 → 1.5.0

package/README.md

@@ -18,6 +18,34 @@ Quote from the original source of this fork:
 - TypeScript support.
 - Better documentation.
 
+**@matrixreq/htmldiff** is the Matrix Requirements fork of
+[node-htmldiff](https://github.com/idesis-gmbh/htmldiff.js) and adds the following on top
+of it:
+
+- Identity matching: elements with a `data-htmldiff-id` attribute are treated as atomic and
+  compared by the attribute value instead of by their content.
+- An opt-in recursive inner diff for identity-matched elements via the
+  `data-htmldiff-inner-diff` and `data-htmldiff-inner-diff-atomic-tags` attributes. See
+  *Identity matching and recursive inner diff* below for both features.
+- Atomic tags may contain nested same-named children: the atomic token ends only when the
+  tag's nesting depth returns to zero, and a stray `>` inside e.g. script content is not
+  treated as a tag boundary.
+- Tokenizer robustness: atomic tag names only match complete names (`<abbr>` is not
+  mistaken for the atomic tag `a`), and self-closing (`<div/>`) or void (`<img>`, `<br>`,
+  ...) atomic elements do not swallow the content following them.
+- Quote-aware tag parsing: `>` and `/>` inside quoted attribute values (e.g.
+  `title="a > b"`) do not end a tag prematurely, neither when tokenizing nor when
+  splitting an element for the recursive inner diff.
+- Adjacent atomic tags are wrapped in their own `<ins>`/`<del>` tags instead of being
+  combined into one.
+- Inserted tags are marked with a `data-inserted="true"` attribute.
+- Whitespace handling: repeated whitespace (except newlines) as well as `&nbsp;`/`&#160;`
+  compare as equal to regular spaces.
+- Deletions of a closing/opening tag pair (e.g. the `</p><p>` removed when two paragraphs
+  are merged) render as a proper `<del>` instead of unbalanced tags.
+- Fixed the `atomicTags` parameter of the diff function: the custom tag list was previously
+  ignored due to a broken regular expression.
+
 See also *Credits* below.
 
 ## Description
@@ -68,6 +96,55 @@ of these three parameters it will be ignored:
   `iframe,object,math,svg,script,video,head,style`.
 
 
+### Identity matching and recursive inner diff
+
+Elements can be matched across the two documents by identity instead of content by giving
+them a `data-htmldiff-id` attribute. Such an element is treated as atomic (its children are
+never diffed) and compared solely by the attribute value: two elements with the same id are
+considered equal even if their content differs, in which case the after version is rendered
+as is.
+
+To make content changes of such identity-matched elements visible, add the
+`data-htmldiff-inner-diff="true"` attribute (alongside `data-htmldiff-id`). A bare
+attribute or any value other than `false` enables the behavior;
+`data-htmldiff-inner-diff="false"` disables it:
+
+```html
+<div class="toc-entry" data-htmldiff-id="sec-1" data-htmldiff-inner-diff="true">
+  <a href="#123">1. Section name</a>
+</div>
+```
+
+When two matched atomic tokens have the same key but different content and the element has
+opted in, the element is rendered once (with the after version's opening and closing tags)
+and its inner HTML is diffed recursively, so e.g. a renamed entry shows up as
+`... <del>Old name</del><ins>New name</ins> ...` inline. Inside the recursive diff the
+default atomic tag list without `a` is used
+(`iframe,object,math,svg,script,video,head,style`): link text is diffed word by word and
+href-only changes do not produce any diff markup, while embedded content like svg stays
+atomic. To use a different atomic tags list inside the recursive diff, set the
+`data-htmldiff-inner-diff-atomic-tags` attribute on the opted-in element:
+
+```html
+<div data-htmldiff-id="sec-1" data-htmldiff-inner-diff="true"
+     data-htmldiff-inner-diff-atomic-tags="svg,iframe,a">
+  ...
+</div>
+```
+
+The value replaces the default list and has the same format as the `atomicTags` API
+parameter; an empty value means no tag name is atomic. The attribute is read from the after
+version of the element and is only consulted on elements that opted in via
+`data-htmldiff-inner-diff`.
+
+Opted-in elements nested inside other opted-in elements are diffed recursively as well;
+each nesting level requires its own `data-htmldiff-inner-diff` attribute.
+
+Limitations:
+
+- The recursion depth is capped at 10 levels as a backstop against deep
+  nesting. Opted-in elements beyond the cap are rendered as their after version.
+
 ### Example
 
 JavaScript:

package/js/htmldiff.js

@@ -68,10 +68,62 @@
      * Regular expression to check atomic tags.
      * @see function diff.
      */
-    var atomicTagsRegExp;
     // Added head and style (for style tags inside the body)
-    var defaultAtomicTagsRegExp = new RegExp('^<(iframe|object|math|svg|script|video|head|style|a)\\b');
+    // The tag name must be followed by a delimiter (not a \b word boundary): the tokenizer
+    // matches against partially read tags, and a word boundary would match at the end of an
+    // incomplete name, e.g. detecting '<abbr>' as the atomic tag 'a' while reading '<a'.
+    var defaultAtomicTagsRegExp = new RegExp('^<(iframe|object|math|svg|script|video|head|style|a)[\\s/>]');
+    var atomicTagsRegExp = defaultAtomicTagsRegExp;
     const dataHtmlDiffIdRegExp = /^<([a-z\-]+).+data-htmldiff-id=["']?((?:.(?!["']?\s+(?:\S+)=|\s*\/?[>"']))*.)["']?/;
+
+    /**
+     * Opt-in marker for the recursive inner diff. When two matched atomic tokens (typically
+     * matched by data-htmldiff-id) have equal keys but different content, an element carrying
+     * this attribute gets its inner HTML diffed recursively instead of being rendered as is.
+     * The attribute must appear in the element's opening tag. Captures the attribute value;
+     * a bare attribute or any value other than "false" enables the opt-in.
+     * @see OPS.equal and renderInnerDiff.
+     */
+    const dataHtmlDiffInnerDiffRegExp =
+        /^<[^>]*\sdata-htmldiff-inner-diff(?:\s*=\s*["']?([^"'\s/>]*)|(?=[\s/>]))/;
+
+    /**
+     * Per-element override for the atomic tags used inside a recursive inner diff. The value
+     * is a comma separated tag name list, like the atomicTags parameter of the diff function;
+     * an empty value means no tag name is atomic. Without the attribute the default list
+     * without 'a' is used, so anchors have their text content diffed word by word while
+     * embedded content like svg stays atomic. The attribute is only consulted on elements
+     * that opted in via data-htmldiff-inner-diff.
+     */
+    const dataHtmlDiffInnerDiffAtomicTagsRegExp =
+        /^<[^>]*\sdata-htmldiff-inner-diff-atomic-tags\s*=\s*["']([^"']*)["']/;
+
+    // Atomic tags used inside a recursive inner diff unless the element overrides them via
+    // data-htmldiff-inner-diff-atomic-tags: the default list without 'a'.
+    var defaultInnerDiffAtomicTagsRegExp =
+        new RegExp('^<(iframe|object|math|svg|script|video|head|style)[\\s/>]');
+
+    // Matches no tag at all: used when data-htmldiff-inner-diff-atomic-tags is empty.
+    const noAtomicTagsRegExp = /^<(?!)/;
+
+    // The number of currently active recursive inner diffs. Recursion is governed per
+    // element (each nesting level requires its own data-htmldiff-inner-diff attribute), so
+    // the depth is naturally bounded by the nesting of opted-in elements; the cap is only a
+    // backstop against pathologically deep documents.
+    var innerDiffDepth = 0;
+    const maxInnerDiffDepth = 10;
+
+    /**
+     * Builds the atomic tags regular expression from a comma separated tag name list.
+     *
+     * @param {string} atomicTags Comma separated list of tag names, e.g. 'head,script,style'.
+     *
+     * @return {RegExp} The regular expression matching the start of those tags.
+     */
+    function buildAtomicTagsRegExp(atomicTags){
+        // Require a delimiter after the name (see defaultAtomicTagsRegExp on why not \b).
+        return new RegExp('^<(' + atomicTags.replace(/\s*/g, '').replace(/,/g, '|') + ')[\\s/>]');
+    }
     
     /**
      * Checks if the current word is the beginning of an atomic tag. An atomic tag is one whose
@@ -126,6 +178,20 @@
         return /^\s*<[^>]+\/>\s*$/.test(token);
     }
 
+    /**
+     * Checks if a tag name is an HTML void element. Void elements cannot have content and can skip a
+     * closing tag, so an atomic element with a void tag name ends with its opening tag -
+     * with or without the XML style '/>'.
+     *
+     * @param {string} tag The tag name to check.
+     *
+     * @return {boolean} True if the tag name is a void element.
+     */
+    function isVoidTagName(tag){
+        return /^(area|base|br|col|embed|hr|img|input|link|meta|param|source|track|wbr)$/
+            .test(tag);
+    }
+
     /**
      * Checks if a token can be wrapped inside a tag.
      *
@@ -189,15 +255,71 @@
         var currentWord = '';
         var currentAtomicTag = '';
         var currentAtomicTagDepth = 0;
+        // The quote character of the attribute value currently being read, or null. Quoted
+        // attribute values may contain any character including '>', so no tag boundary or
+        // atomic tag detection applies inside them.
+        var currentQuote = null;
+        // True while reading the atomic tag's own opening tag. Quote tracking in atomic
+        // mode is limited to that region: quotes in the element's content (text
+        // apostrophes, comments, nested tags) must not affect how the token ends.
+        var inAtomicOpeningTag = false;
         var words = [];
+
+        /**
+         * Consumes a character belonging to a quoted attribute value, updating the quote
+         * state. Quoted values may contain any character including '>', so as long as a
+         * quote is open no tag boundary or atomic tag detection applies.
+         *
+         * @param {string} char The current character.
+         * @param {boolean} canOpenQuote Whether a quote may start at this position.
+         *
+         * @return {boolean} True if the character was consumed and no other handling
+         *    applies to it.
+         */
+        function consumeAttributeQuote(char, canOpenQuote){
+            if (currentQuote){
+                if (char === currentQuote){
+                    currentQuote = null;
+                }
+                return true;
+            }
+            if (canOpenQuote && (char === '"' || char === "'")){
+                currentQuote = char;
+                return true;
+            }
+            return false;
+        }
+
         for (var i = 0; i < html.length; i++){
             var char = html[i];
             switch (mode){
                 case 'tag':
-                    var atomicTag = isStartOfAtomicTag(currentWord);
-                    if (atomicTag){
+                    // Quote handling must come before the atomic tag detection:
+                    // dataHtmlDiffIdRegExp can match right at the opening quote of the
+                    // attribute value, which would enter atomic mode with the quote
+                    // tracking out of sync.
+                    if (consumeAttributeQuote(char, true)){
+                        currentWord += char;
+                        break;
+                    }
+                    // The atomic tag regexps require a delimiter after the tag name, so the
+                    // current character must be included in the check: without it a tag
+                    // ending right at the name (e.g. '<script' + '>') would never match.
+                    var atomicTag = isStartOfAtomicTag(currentWord + char);
+                    if (atomicTag && isEndOfTag(char) &&
+                            (isVoidTagName(atomicTag) || /\/$/.test(currentWord))){
+                        // The atomic tag itself is void or self-closing: it has no content
+                        // that could be swallowed, emit it as a complete token.
+                        currentWord += '>';
+                        words.push(createToken(currentWord));
+                        currentWord = '';
+                        mode = 'char';
+                    } else if (atomicTag){
                         mode = 'atomic_tag';
                         currentAtomicTag = atomicTag;
+                        // we are still inside the atomic tag's own opening tag unless this
+                        // character just ended it
+                        inAtomicOpeningTag = !isEndOfTag(char);
                         // skip standalone tags like <script> and <style>
                         currentAtomicTagDepth = isEndOfTag(char) ? 1 : 0;
                         currentWord += char;
@@ -219,9 +341,17 @@
                     break;
                 case 'atomic_tag':
                     currentWord += char;
-                    // track the same name nested tags depth; 
+                    // Quotes may only open inside the atomic tag's own opening tag, where
+                    // attribute values may contain '>' or '/>' and must not affect the
+                    // depth tracking below. Quote characters in the element's content
+                    // (text apostrophes, comments) are not attribute quotes.
+                    if (consumeAttributeQuote(char, inAtomicOpeningTag)){
+                        break;
+                    }
+                    // track the same name nested tags depth;
                     // end the atomic token only when it returns to 0.
                     if (isEndOfTag(char)){
+                        inAtomicOpeningTag = false;
                         if (isClosingTagOf(currentWord, currentAtomicTag)){
                             currentAtomicTagDepth--;
                             if (currentAtomicTagDepth <= 0){
@@ -231,6 +361,15 @@
                                 currentAtomicTagDepth = 0;
                                 mode = 'char';
                             }
+                        } else if (currentAtomicTagDepth === 0 &&
+                                (isVoidTagName(currentAtomicTag) || /\/>$/.test(currentWord))){
+                            // At depth 0 this '>' can only end the atomic tag's own opening
+                            // tag. When the element is void or self-closing it has no
+                            // content: end the token so trailing content tokenizes normally.
+                            words.push(createToken(currentWord));
+                            currentWord = '';
+                            currentAtomicTag = '';
+                            mode = 'char';
                         } else if (isOpeningTagOf(currentWord, currentAtomicTag)){
                             currentAtomicTagDepth++;
                         }
@@ -314,8 +453,10 @@
         }
         
         // If the token is an a element, grab it's data attribute to include in the key.
+        // Only when <a> is atomic: if it has been excluded from the atomic tags (as done in
+        // recursive inner diffs), the token is just the opening tag.
         var a = /^<a.*href=['"]([^"']*)['"]/.exec(token);
-        if (a) {
+        if (a && atomicTagsRegExp.test('<a ')) {
             return '<a href="' + a[1] + '"></a>';
         }
 
@@ -909,6 +1050,137 @@
         });
     }
 
+    /**
+     * Checks whether a token is an atomic tag that opted into the recursive inner diff via
+     * the data-htmldiff-inner-diff attribute. A bare attribute or any value other than
+     * "false" counts as opted in. Opted-in elements nested inside other opted-in elements
+     * are diffed recursively as well, up to the depth cap; beyond it, opted-in tokens are
+     * rendered verbatim like any other atomic token.
+     *
+     * @param {string} tokenString The token string to check.
+     *
+     * @return {boolean} True if the token should get a recursive inner diff.
+     */
+    function isInnerDiffToken(tokenString){
+        if (innerDiffDepth >= maxInnerDiffDepth || !isStartOfAtomicTag(tokenString)){
+            return false;
+        }
+        var attr = dataHtmlDiffInnerDiffRegExp.exec(tokenString);
+        return !!attr && attr[1] !== 'false';
+    }
+
+    /**
+     * Finds the index of the '>' that ends the opening tag at the start of the given token
+     * string, skipping any '>' inside quoted attribute values (e.g. title="a > b").
+     *
+     * @param {string} tokenString The token string starting with an opening tag.
+     *
+     * @return {number} The index of the closing '>' of the opening tag, or -1 if there is
+     *    none (e.g. an unterminated tag or an unbalanced attribute quote).
+     */
+    function findOpeningTagEnd(tokenString){
+        var quote = null;
+        for (var i = 0; i < tokenString.length; i++){
+            var char = tokenString[i];
+            // quote is closed
+            if (char === quote){
+                quote = null;
+                continue;
+            }
+            // inside quote
+            if (quote){
+                continue;
+            }
+            // quote start
+            if (char === '"' || char === "'"){
+                quote = char;
+                continue;
+            }
+            // not inside quote, check for tag end
+            if (char === '>'){
+                return i;
+            }
+        }
+        return -1;
+    }
+
+    /**
+     * Splits an atomic token string into its opening tag, inner HTML and closing tag.
+     * A token consisting of a single tag (a void or self-closing element) has an empty
+     * inner HTML and no closing tag.
+     *
+     * @param {string} tokenString The atomic token string, e.g. '<div a="b">content</div>'.
+     *
+     * @return {Object|null} An object with openingTag, innerHtml and closingTag properties,
+     *    or null if the token cannot be split (e.g. an unterminated tag).
+     */
+    function splitAtomicTokenString(tokenString){
+        var openingTagEnd = findOpeningTagEnd(tokenString);
+        if (openingTagEnd === -1) {
+            return null;
+        }
+        if (openingTagEnd === tokenString.length - 1) {
+            // The token is a single tag (void or self-closing): the element has no content.
+            return {
+                openingTag: tokenString,
+                innerHtml: '',
+                closingTag: ''
+            };
+        }
+        var closingTagStart = tokenString.lastIndexOf('<');
+        if (closingTagStart <= openingTagEnd || tokenString[closingTagStart + 1] !== '/') {
+            return null;
+        }
+        return {
+            openingTag: tokenString.slice(0, openingTagEnd + 1),
+            innerHtml: tokenString.slice(openingTagEnd + 1, closingTagStart),
+            closingTag: tokenString.slice(closingTagStart)
+        };
+    }
+
+    /**
+     * Renders the recursive inner diff of two matched atomic tokens with equal keys but
+     * different content. The after version's opening and closing tags are emitted with the
+     * diff of the two inner HTML fragments in between. Inside the recursion the default
+     * atomic tags without 'a' are used, so link text is diffed word by word and href-only
+     * changes do not produce any markup. The after version's
+     * data-htmldiff-inner-diff-atomic-tags attribute overrides that list. Nested opted-in
+     * elements are diffed recursively as well, up to a hardcoded depth cap.
+     *
+     * @param {string} beforeString The before version of the atomic token.
+     * @param {string} afterString The after version of the atomic token.
+     * @param {string} dataPrefix (Optional) The prefix to use in data attributes.
+     * @param {string} className (Optional) The class name to include in the wrapper tag.
+     *
+     * @return {string} The rendered element with inner differences wrapped in ins/del tags.
+     */
+    function renderInnerDiff(beforeString, afterString, dataPrefix, className){
+        var before = splitAtomicTokenString(beforeString);
+        var after = splitAtomicTokenString(afterString);
+        if (!before || !after){
+            return afterString;
+        }
+        var atomicTagsOverride = dataHtmlDiffInnerDiffAtomicTagsRegExp.exec(afterString);
+        var outerAtomicTagsRegExp = atomicTagsRegExp;
+        innerDiffDepth++;
+        // we need to clean up the innerDiffDepth update and atomicTagsRegExp restoration in case of an error, hence try/finally
+        try {
+            atomicTagsRegExp = defaultInnerDiffAtomicTagsRegExp;
+
+            if (atomicTagsOverride) {
+                atomicTagsRegExp = atomicTagsOverride[1]
+                    ? buildAtomicTagsRegExp(atomicTagsOverride[1])
+                    : noAtomicTagsRegExp;
+            }
+
+            var innerDiff = diffCore(before.innerHtml, after.innerHtml, className, dataPrefix);
+        } finally {
+            innerDiffDepth--;
+            atomicTagsRegExp = outerAtomicTagsRegExp;
+        }
+        return after.openingTag + innerDiff + after.closingTag;
+    }
+
     /**
      * OPS.equal/insert/delete/replace are functions that render an operation into
      * HTML content.
@@ -931,10 +1203,23 @@
      */
     var OPS = {
         'equal': function(op, beforeTokens, afterTokens, opIndex, dataPrefix, className){
-            var tokens = afterTokens.slice(op.startInAfter, op.endInAfter + 1);
-            return tokens.reduce(function(prev, curr){
-                return prev + curr.string;
-            }, '');
+            // Tokens in an equal operation pair up one to one between before and after. Equal
+            // keys do not guarantee equal strings (e.g. atomic tokens matched by
+            // data-htmldiff-id): elements that opted in via data-htmldiff-inner-diff get a
+            // recursive diff of their content, everything else renders the after version.
+            var result = '';
+            for (var i = 0; op.startInAfter + i <= op.endInAfter; i++){
+                var afterToken = afterTokens[op.startInAfter + i];
+                var beforeToken = beforeTokens[op.startInBefore + i];
+                if (beforeToken && beforeToken.string !== afterToken.string &&
+                        isInnerDiffToken(afterToken.string)){
+                    result += renderInnerDiff(
+                            beforeToken.string, afterToken.string, dataPrefix, className);
+                } else {
+                    result += afterToken.string;
+                }
+            }
+            return result;
         },
         'insert': function(op, beforeTokens, afterTokens, opIndex, dataPrefix, className){
             var tokens = afterTokens.slice(op.startInAfter, op.endInAfter + 1);
@@ -1012,13 +1297,24 @@
      * @return {string} The combined HTML content with differences wrapped in <ins> and <del> tags.
      */
     function diff(before, after, className, dataPrefix, atomicTags){
-        if (before === after) return before;
-
         // Enable user provided atomic tag list.
-        atomicTags ? 
-            (atomicTagsRegExp = new RegExp('^<(' + atomicTags.replace(/\s*/g, '').replace(/,/g, '|') + ')\b'))
+        atomicTags ?
+            (atomicTagsRegExp = buildAtomicTagsRegExp(atomicTags))
             : (atomicTagsRegExp = defaultAtomicTagsRegExp);
 
+        return diffCore(before, after, className, dataPrefix);
+    }
+
+    /**
+     * Runs the diff pipeline with whatever atomic tags regular expression is currently active.
+     * Used by the diff function after resolving the atomicTags parameter and by
+     * renderInnerDiff, which sets the regular expression itself.
+     *
+     * @see function diff for the parameter and return value descriptions.
+     */
+    function diffCore(before, after, className, dataPrefix){
+        if (before === after) return before;
+
         before = htmlToTokens(before);
         after = htmlToTokens(after);
         var ops = calculateOperations(before, after);

package/package.json

@@ -2,7 +2,7 @@
     "name": "@matrixreq/htmldiff",
     "description": "Diff and markup HTML with <ins> and <del> tags",
     "companyname": "Matrix Requirements",
-    "version": "1.4.0",
+    "version": "1.5.0",
     "keywords": [
         "diff",
         "html",

package/test/diff.spec.js

@@ -256,4 +256,63 @@ describe('Diff', function(){
       });
     }); // describe('Adjacent atomic tag combining')
 
+    describe('Atomic tag list override (atomicTags parameter)', function(){
+      // Regression tests for the parameter regexp being built with a literal backspace
+      // ('\b' instead of '\\b'), which made every override match nothing.
+      it('treats a listed tag as atomic', function(){
+        var res = cut('<iframe src="a.html"></iframe>', '<iframe src="b.html"></iframe>',
+          null, null, 'iframe');
+        expect(res).to.equal(
+          '<del data-operation-index="0"><iframe src="a.html"></iframe></del>' +
+          '<ins data-operation-index="0"><iframe src="b.html"></iframe></ins>');
+      });
+
+      it('replaces the default list, so an unlisted default tag loses atomicity', function(){
+        var res = cut('<iframe src="a.html">x</iframe>', '<iframe src="a.html">y</iframe>',
+          null, null, 'p');
+        expect(res).to.equal(
+          '<iframe src="a.html">' +
+          '<del data-operation-index="1">x</del>' +
+          '<ins data-operation-index="1">y</ins></iframe>');
+      });
+
+      it('does not match tags that merely start with a listed name', function(){
+        // 'if' must not make <iframe> atomic: tag names are matched completely.
+        var res = cut('<iframe src="a.html">x</iframe>', '<iframe src="a.html">y</iframe>',
+          null, null, 'if');
+        expect(res).to.equal(
+          '<iframe src="a.html">' +
+          '<del data-operation-index="1">x</del>' +
+          '<ins data-operation-index="1">y</ins></iframe>');
+      });
+    }); // describe('Atomic tag list override (atomicTags parameter)')
+
+    describe('Tags sharing a prefix with atomic tag names', function(){
+      it('should diff <abbr> content although a is an atomic tag', function(){
+        expect(cut('<abbr>old</abbr> t', '<abbr>new</abbr> t')).to.equal(
+          '<abbr><del data-operation-index="1">old</del>' +
+          '<ins data-operation-index="1">new</ins></abbr> t');
+      });
+    }); // describe('Tags sharing a prefix with atomic tag names')
+
+    describe('Quoted attribute values containing ">"', function(){
+      it('should diff the content of a tag with ">" in an attribute value', function(){
+        expect(cut('<p title="a>b">old</p>', '<p title="a>b">new</p>')).to.equal(
+          '<p title="a>b">' +
+          '<del data-operation-index="1">old</del>' +
+          '<ins data-operation-index="1">new</ins></p>');
+      });
+    }); // describe('Quoted attribute values containing ">"')
+
+    describe('Void atomic elements', function(){
+      it('should diff text following a void data-htmldiff-id element', function(){
+        var res = cut('<img data-htmldiff-id="1" src="a.jpg"> old',
+          '<img data-htmldiff-id="1" src="a.jpg"> new');
+        expect(res).to.equal(
+          '<img data-htmldiff-id="1" src="a.jpg"> ' +
+          '<del data-operation-index="1">old</del>' +
+          '<ins data-operation-index="1">new</ins>');
+      });
+    }); // describe('Void atomic elements')
+
   }); // describe('Diff')

package/test/html_to_tokens.spec.js

@@ -161,5 +161,78 @@ describe('htmlToTokens', function(){
                 expect(cut(atomic)).eql(tokenize([atomic]));
             });
         });
+
+        describe('tags sharing a prefix with atomic tag names', function(){
+            it('should not treat <abbr> as the atomic tag a', function(){
+                expect(cut('<abbr>x</abbr> tail')).eql(
+                        tokenize(['<abbr>', 'x', '</abbr>', ' ', 'tail']));
+            });
+
+            it('should not treat <article> as the atomic tag a', function(){
+                expect(cut('<article>hi</article>')).eql(
+                        tokenize(['<article>', 'hi', '</article>']));
+            });
+        });
+
+        describe('self-closing atomic tags', function(){
+            it('should end a self-closing data-htmldiff-id tag without swallowing trailing content', function(){
+                expect(cut('<div data-htmldiff-id="s"/>x old')).eql(
+                        tokenize(['<div data-htmldiff-id="s"/>', 'x', ' ', 'old']));
+            });
+
+            it('should end a self-closing name-based atomic tag without swallowing trailing content', function(){
+                expect(cut('<svg/>tail')).eql(tokenize(['<svg/>', 'tail']));
+            });
+        });
+
+        describe('quoted attribute values containing tag delimiters', function(){
+            it('should not end a tag on ">" inside a double-quoted attribute value', function(){
+                expect(cut('<p class="a>b">text</p>')).eql(
+                        tokenize(['<p class="a>b">', 'text', '</p>']));
+            });
+
+            it('should not end a tag on ">" inside a single-quoted attribute value', function(){
+                expect(cut("<p class='a>b'>x</p>")).eql(
+                        tokenize(["<p class='a>b'>", 'x', '</p>']));
+            });
+
+            it('should not end a void atomic tag on ">" inside an attribute value', function(){
+                expect(cut('<img data-htmldiff-id="1" alt="a > b"> tail')).eql(
+                        tokenize(['<img data-htmldiff-id="1" alt="a > b">', ' ', 'tail']));
+            });
+
+            it('should not treat "/>" inside an attribute value as self-closing', function(){
+                var atomic = '<span data-htmldiff-id="1" data-x="y/>z">c</span>';
+                expect(cut(atomic)).eql(tokenize([atomic]));
+            });
+
+            it('should keep an atomic tag with ">" in an attribute as one token', function(){
+                expect(cut('<div data-htmldiff-id="1" title="a > b">x</div> tail')).eql(
+                        tokenize(['<div data-htmldiff-id="1" title="a > b">x</div>',
+                            ' ', 'tail']));
+            });
+
+            it('should not treat apostrophes in atomic text content as quotes', function(){
+                expect(cut("<div data-htmldiff-id='1'>it's ok</div> tail")).eql(
+                        tokenize(["<div data-htmldiff-id='1'>it's ok</div>", ' ', 'tail']));
+            });
+
+            it('should not treat apostrophes in comments inside atomic tags as quotes', function(){
+                expect(cut("<math><mi>x<!-- don't --></mi></math> tail")).eql(
+                        tokenize(["<math><mi>x<!-- don't --></mi></math>", ' ', 'tail']));
+            });
+        });
+
+        describe('void atomic tags', function(){
+            it('should end a void data-htmldiff-id tag written without a slash', function(){
+                expect(cut('<img data-htmldiff-id="1" src="a.jpg"> tail')).eql(
+                        tokenize(['<img data-htmldiff-id="1" src="a.jpg">', ' ', 'tail']));
+            });
+
+            it('should end a void data-htmldiff-id br tag without swallowing trailing content', function(){
+                expect(cut('<br data-htmldiff-id="x">y')).eql(
+                        tokenize(['<br data-htmldiff-id="x">', 'y']));
+            });
+        });
     });
 });

package/test/inner_diff.spec.js

@@ -0,0 +1,333 @@
+describe('Recursive inner diff (data-htmldiff-inner-diff)', function(){
+    var cut;
+
+    beforeEach(function(){
+        cut = require('../js/htmldiff');
+    });
+
+    function tocEntry(href, name){
+        return '<div class="toc-entry" data-htmldiff-id="123" data-htmldiff-inner-diff="true">' +
+            '<a href="' + href + '">' + name + '</a></div>';
+    }
+
+    describe('when an opted-in element is renamed in place', function(){
+        it('renders inline ins/del inside the single emitted element', function(){
+            var res = cut(tocEntry('#123', '1. Old name'), tocEntry('#123', '1. New name'));
+            expect(res).to.equal(
+                '<div class="toc-entry" data-htmldiff-id="123" data-htmldiff-inner-diff="true">' +
+                '<a href="#123">1. <del data-operation-index="1">Old</del>' +
+                '<ins data-operation-index="1">New</ins> name</a></div>');
+        });
+
+        it('passes className and dataPrefix through to the inner ins/del tags', function(){
+            var res = cut(tocEntry('#123', 'Old'), tocEntry('#123', 'New'), 'diff-cls', 'pre');
+            expect(res).to.equal(
+                '<div class="toc-entry" data-htmldiff-id="123" data-htmldiff-inner-diff="true">' +
+                '<a href="#123"><del data-pre-operation-index="1" class="diff-cls">Old</del>' +
+                '<ins data-pre-operation-index="1" class="diff-cls">New</ins></a></div>');
+        });
+
+        it('keeps diffing the rest of the document with the outer atomic tag list', function(){
+            // The <a> outside the opted-in element must stay atomic after the inner diff ran.
+            var res = cut(
+                tocEntry('#123', 'Old name') + '<a href="x.html">same link</a>',
+                tocEntry('#123', 'New name') + '<a href="y.html">same link</a>');
+            expect(res).to.equal(
+                '<div class="toc-entry" data-htmldiff-id="123" data-htmldiff-inner-diff="true">' +
+                '<a href="#123"><del data-operation-index="1">Old</del>' +
+                '<ins data-operation-index="1">New</ins> name</a></div>' +
+                '<del data-operation-index="1"><a href="x.html">same link</a></del>' +
+                '<ins data-operation-index="1"><a href="y.html">same link</a></ins>');
+        });
+    }); // describe('when an opted-in element is renamed in place')
+
+    describe('anchors inside the recursive diff', function(){
+        it('do not treat <a> as atomic, so href-only changes produce no diff markup', function(){
+            var res = cut(tocEntry('#18036', '1. Same name'), tocEntry('#18037', '1. Same name'));
+            expect(res).to.equal(tocEntry('#18037', '1. Same name'));
+        });
+
+        it('diffs the link text inline when the href changed', function(){
+            var res = cut(tocEntry('#18036', '1. Old name'), tocEntry('#18037', '1. New name'));
+            expect(res).to.equal(
+                '<div class="toc-entry" data-htmldiff-id="123" data-htmldiff-inner-diff="true">' +
+                '<a href="#18037">1. <del data-operation-index="1">Old</del>' +
+                '<ins data-operation-index="1">New</ins> name</a></div>');
+        });
+    }); // describe('anchors inside the recursive diff')
+
+    describe('atomic tags inside the recursive diff', function(){
+        function el(extraAttrs, inner){
+            return '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"' + extraAttrs +
+                '>' + inner + '</div>';
+        }
+
+        it('keeps the default atomic tags (without a) inside the recursion', function(){
+            // Embedded content like svg stays atomic by default: a changed svg is replaced
+            // as a whole, not word-diffed.
+            var res = cut(
+                el('', '<svg viewBox="0 0 1 1"><text>old</text></svg>'),
+                el('', '<svg viewBox="0 0 1 1"><text>new</text></svg>'));
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true">' +
+                '<del data-operation-index="0">' +
+                '<svg viewBox="0 0 1 1"><text>old</text></svg></del>' +
+                '<ins data-operation-index="0">' +
+                '<svg viewBox="0 0 1 1"><text>new</text></svg></ins>' +
+                '</div>');
+        });
+
+        it('replaces the default list with data-htmldiff-inner-diff-atomic-tags', function(){
+            // The override lists only em, so svg is no longer atomic and gets word-diffed.
+            var attrs = ' data-htmldiff-inner-diff-atomic-tags="em"';
+            var res = cut(
+                el(attrs, '<svg viewBox="0 0 1 1"><text>old</text></svg>'),
+                el(attrs, '<svg viewBox="0 0 1 1"><text>new</text></svg>'));
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"' + attrs + '>' +
+                '<svg viewBox="0 0 1 1"><text>' +
+                '<del data-operation-index="1">old</del>' +
+                '<ins data-operation-index="1">new</ins>' +
+                '</text></svg></div>');
+        });
+
+        it('restores atomic anchors (href comparison) when the override lists a', function(){
+            var attrs = ' data-htmldiff-inner-diff-atomic-tags="a"';
+            var res = cut(
+                el(attrs, '<a href="#1">Name</a>'),
+                el(attrs, '<a href="#2">Name</a>'));
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"' + attrs + '>' +
+                '<del data-operation-index="0"><a href="#1">Name</a></del>' +
+                '<ins data-operation-index="0"><a href="#2">Name</a></ins>' +
+                '</div>');
+        });
+
+        it('treats no tag name as atomic when the override value is empty', function(){
+            var attrs = ' data-htmldiff-inner-diff-atomic-tags=""';
+            var res = cut(
+                el(attrs, '<svg viewBox="0 0 1 1"><text>old</text></svg>'),
+                el(attrs, '<svg viewBox="0 0 1 1"><text>new</text></svg>'));
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"' + attrs + '>' +
+                '<svg viewBox="0 0 1 1"><text>' +
+                '<del data-operation-index="1">old</del>' +
+                '<ins data-operation-index="1">new</ins>' +
+                '</text></svg></div>');
+        });
+    }); // describe('atomic tags inside the recursive diff')
+
+    describe('opt-in attribute values', function(){
+        function entry(value, name){
+            return '<div data-htmldiff-id="123" data-htmldiff-inner-diff=' + value + '>' +
+                name + '</div>';
+        }
+
+        it('treats a double-quoted "false" value as opted out', function(){
+            var res = cut(entry('"false"', 'old'), entry('"false"', 'new'));
+            expect(res).to.equal(entry('"false"', 'new'));
+        });
+
+        it('treats a single-quoted \'false\' value as opted out', function(){
+            var res = cut(entry("'false'", 'old'), entry("'false'", 'new'));
+            expect(res).to.equal(entry("'false'", 'new'));
+        });
+
+        it('treats an unquoted false value as opted out', function(){
+            var res = cut(entry('false', 'old'), entry('false', 'new'));
+            expect(res).to.equal(entry('false', 'new'));
+        });
+
+        it('treats a bare attribute without a value as opted in', function(){
+            var res = cut(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff>old</div>',
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff>new</div>');
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff>' +
+                '<del data-operation-index="0">old</del>' +
+                '<ins data-operation-index="0">new</ins></div>');
+        });
+
+        it('only matches exact attribute name', function(){
+            var res = cut(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff-extra="true">old</div>',
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff-extra="true">new</div>');
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff-extra="true">new</div>');
+        });
+
+        it('recognizes the attribute regardless of its position', function(){
+            var res = cut(
+                '<div data-htmldiff-inner-diff="true" data-htmldiff-id="123">old</div>',
+                '<div data-htmldiff-inner-diff="true" data-htmldiff-id="123">new</div>');
+            expect(res).to.equal(
+                '<div data-htmldiff-inner-diff="true" data-htmldiff-id="123">' +
+                '<del data-operation-index="0">old</del>' +
+                '<ins data-operation-index="0">new</ins></div>');
+        });
+    }); // describe('opt-in attribute values')
+
+    describe('edge cases', function(){
+        it('ignores the attribute on non-atomic elements (no data-htmldiff-id)', function(){
+            // Without data-htmldiff-id the div is not atomic, so this is a plain word diff.
+            var res = cut(
+                '<div data-htmldiff-inner-diff="true">old text</div>',
+                '<div data-htmldiff-inner-diff="true">new text</div>');
+            expect(res).to.equal(
+                '<div data-htmldiff-inner-diff="true">' +
+                '<del data-operation-index="1">old</del>' +
+                '<ins data-operation-index="1">new</ins> text</div>');
+        });
+
+        it('diffs text following a self-closing opted-in element normally', function(){
+            var res = cut(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"/>x old',
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"/>x new');
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"/>x ' +
+                '<del data-operation-index="1">old</del>' +
+                '<ins data-operation-index="1">new</ins>');
+        });
+
+        it('marks the content as inserted when the before element was self-closing', function(){
+            // A self-closing element has empty inner content, so the new content is a
+            // pure insertion.
+            var res = cut(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"/>',
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true">new</div>');
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true">' +
+                '<ins data-operation-index="0">new</ins></div>');
+        });
+
+        it('marks the content as deleted when the after element became self-closing', function(){
+            // The after element has no content anymore, so the deleted content is
+            // rendered right after the self-closing tag.
+            var res = cut(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true">old</div>',
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"/>');
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true"/>' +
+                '<del data-operation-index="0">old</del>');
+        });
+
+        it('is not confused by ">" inside attribute values', function(){
+            var res = cut(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true" title="a > b">' +
+                'old</div>',
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true" title="a > c">' +
+                'new</div>');
+            expect(res).to.equal(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true" title="a > c">' +
+                '<del data-operation-index="0">old</del>' +
+                '<ins data-operation-index="0">new</ins></div>');
+        });
+
+        it('is not confused by ">" inside single-quoted attribute values', function(){
+            var res = cut(
+                "<div data-htmldiff-id='123' data-htmldiff-inner-diff title='a > b'>old</div>",
+                "<div data-htmldiff-id='123' data-htmldiff-inner-diff title='a > b'>new</div>");
+            expect(res).to.equal(
+                "<div data-htmldiff-id='123' data-htmldiff-inner-diff title='a > b'>" +
+                '<del data-operation-index="0">old</del>' +
+                '<ins data-operation-index="0">new</ins></div>');
+        });
+
+        it('falls back to the after version when a token cannot be split', function(){
+            // An unterminated atomic tag swallows the rest of the input and has no closing
+            // tag to split on; the inner diff falls back instead of producing broken markup.
+            var after = '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true">new';
+            var res = cut(
+                '<div data-htmldiff-id="123" data-htmldiff-inner-diff="true">old',
+                after);
+            expect(res).to.equal(after);
+        });
+
+        it('renders pure insertions when the before content is empty', function(){
+            var res = cut(tocEntry('#1', ''), tocEntry('#1', 'New name'));
+            expect(res).to.equal(
+                '<div class="toc-entry" data-htmldiff-id="123" data-htmldiff-inner-diff="true">' +
+                '<a href="#1"><ins data-operation-index="1">New name</ins></a></div>');
+        });
+
+        it('do not diffs inner html for moved opted-in elements', function(){
+            function entry(id, name){
+                return '<div data-htmldiff-id="' + id + '" data-htmldiff-inner-diff="true">' +
+                    name + '</div>';
+            }
+            var res = cut(entry('a', 'First') + entry('b', 'Second'),
+                entry('b', 'Second') + entry('a', 'First'));
+            expect(res).to.equal(
+                '<ins data-operation-index="0">' + entry('b', 'Second') + '</ins>' +
+                entry('a', 'First') +
+                '<del data-operation-index="2">' + entry('b', 'Second') + '</del>');
+        });
+    }); // describe('edge cases')
+
+    describe('when the element did not opt in', function(){
+        it('should render the after version as is when keys match but content differs', function(){
+            var before = '<div data-htmldiff-id="chart-1"><svg viewBox="0 0 1 1">' +
+                '<rect width="5"/></svg></div>';
+            var after = '<div data-htmldiff-id="chart-1"><svg viewBox="0 0 2 2">' +
+                '<rect width="9"/></svg></div>';
+            expect(cut(before, after)).to.equal(after);
+        });
+    }); // describe('when the element did not opt in')
+
+    describe('when key and content are both equal', function(){
+        it('should render the element unchanged', function(){
+            var before = 'x ' + tocEntry('#123', '1. Same name') + ' y';
+            var after = 'x ' + tocEntry('#123', '1. Same name') + ' z';
+            expect(cut(before, after)).to.equal(
+                'x ' + tocEntry('#123', '1. Same name') + ' ' +
+                '<del data-operation-index="1">y</del><ins data-operation-index="1">z</ins>');
+        });
+    }); // describe('when key and content are both equal')
+
+    describe('when opted-in elements are nested', function(){
+        function nest(depth, content){
+            var html = content;
+            for (var i = depth; i >= 1; i--){
+                html = '<div data-htmldiff-id="d' + i + '" data-htmldiff-inner-diff="true">' +
+                    html + '</div>';
+            }
+            return html;
+        }
+
+        it('diffs nested elements', function(){
+            var res = cut(
+                '<div data-htmldiff-id="outer" data-htmldiff-inner-diff="true">x ' +
+                '<span data-htmldiff-id="inner" data-htmldiff-inner-diff="true">old</span></div>',
+                '<div data-htmldiff-id="outer" data-htmldiff-inner-diff="true">x ' +
+                '<span data-htmldiff-id="inner" data-htmldiff-inner-diff="true">new</span></div>');
+            expect(res).to.equal(
+                '<div data-htmldiff-id="outer" data-htmldiff-inner-diff="true">x ' +
+                '<span data-htmldiff-id="inner" data-htmldiff-inner-diff="true">' +
+                '<del data-operation-index="0">old</del>' +
+                '<ins data-operation-index="0">new</ins></span></div>');
+        });
+
+        it('do not diffs nested element that do not opt-in', function(){
+            var after = '<div data-htmldiff-id="outer" data-htmldiff-inner-diff="true">t ' +
+                '<span data-htmldiff-id="inner">new</span></div>';
+            var res = cut(
+                '<div data-htmldiff-id="outer" data-htmldiff-inner-diff="true">t ' +
+                '<span data-htmldiff-id="inner">old</span></div>',
+                after);
+            expect(res).to.equal(after);
+        });
+
+        it('diffs all levels up to the depth cap of 10', function(){
+            var res = cut(nest(10, 'old'), nest(10, 'new'));
+            expect(res).to.equal(nest(10,
+                '<del data-operation-index="0">old</del>' +
+                '<ins data-operation-index="0">new</ins>'));
+        });
+
+        it('renders the after version as is beyond the depth cap', function(){
+            var res = cut(nest(11, 'old'), nest(11, 'new'));
+            expect(res).to.equal(nest(11, 'new'));
+        });
+    }); // describe('when opted-in elements are nested')
+
+}); // describe('Recursive inner diff (data-htmldiff-inner-diff)')