Is it possible to use jsdom to take place of $tw.fakeDocument?

Developers
#1

I know $tw.fakeDocument does an excellent job in replacing the document global variable when TiddlyWiki runs in the pure NodeJS environment without any browser, especially when rendering tiddlers to static HTML pages (using --rendertiddlers command). However, I recently ran into some problems with it.

$tw.fakeDocument does not implement everything of real document, such as documentElement for which the MermaidAPI requires. As a result, some extended widgets fail to render results properly during static pages generation.

I find a project called JSDom. It implements the whole DOM and HTML standard and may help a lot for static webpage generation. Is it possible to use jsdom to take place of $tw.fakeDocument, or is there any better solutions? Thanks all.

#2

Hi @Sttot excellent question! I’ve used JSDom in other projects without problems, and I have indeed considered making a plugin that replaces the core fake document implementation with JSDom (which is generally prompted by trying to get a new library integrated, as here). I think it would still be worth exploring, if you’d like to give it a try?

#3

Of course! I’d like to have a try. But I find that the JSDom repo doesn’t provide a finally generated js file like jsdom.min.js, it seems that it can just run in node dev mode. I’m not good at NodeJS, if there are some ways to generate a single static js file, I think it will work well in TiddlyWiki. Thank you.

#4

Hi. I tried to make a rollup project to pack JSDom to a single js file. It may work but there are still some problems. There are some circular dependencies when packing, and some objects can not be imported properly. If you are interested, find my repo here.

#5

Ouch! That’s a bit discouraging. The readme for jsdom used to talk about running in the browser, perhaps it’s no longer a goal for them.

#6

Hi, I think linkedom looks a good alternative. And I give up to pack jsdom or linkedom to a static js file and just run npm add linkedom canvas in node TiddlyWiki lib, then modify the $:/core/modules/utils/fakedom.js tiddler to this:

(function() {

    /*jslint node: true, browser: true */
    /*global $tw: false */
    "use strict";

    var parseHTML = require('linkedom').parseHTML;
    var document = parseHTML('<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></head><body></body></html>').document;

    // Patch
    document.compatMode = 'CSS1Compat';
    document.isTiddlyWikiFakeDom = true;
    document._createElement = document.createElement;
    document.createElement = function(localName, options) {
        var dom = document._createElement(localName, options);
        dom.isTiddlyWikiFakeDom = true;
        return dom;
    };
    document._createElementNS = document.createElementNS;
    document.createElementNS = function(nsp, localName, options) {
        var dom = document._createElementNS(nsp, localName, options);
        dom.isTiddlyWikiFakeDom = true;
        return dom;
    };
    document._createTextNode = document.createTextNode;
    document.createTextNode = function(textContent) {
        var dom = document._createTextNode(textContent);
        dom.isTiddlyWikiFakeDom = true;
        return dom;
    };

    exports.fakeDocument = document;
})();

And it works!

Note that this method can only work in the NodeJS environment (the browser environment doesn’t need it, indeed). I don’t know if JSDom can work in a similar way, but I think it will do.

This solution is quite enough for me, I think I shall remove my JSDom packing repo.

Hope to help you!

#7

A better version that can run both in browser and NodeJS:

/*\
title: $:/core/modules/utils/fakedom.js
type: application/javascript
module-type: global

A barebones implementation of DOM interfaces needed by the rendering mechanism.

\*/
(function() {

    /*jslint node: true, browser: true */
    /*global $tw: false */
    "use strict";

    try {
        var parseHTML = require('linkedom').parseHTML;
        var HTML = parseHTML('<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></head><body></body></html>');
        var document = HTML.document;
        var window = HTML.windw;

        // Patch
        document.compatMode = 'CSS1Compat';
        document.isTiddlyWikiFakeDom = true;
        document._createElement = document.createElement;
        document.createElement = function(localName, options) {
            var dom = document._createElement(localName, options);
            dom.isTiddlyWikiFakeDom = true;
            return dom;
        };
        document._createElementNS = document.createElementNS;
        document.createElementNS = function(nsp, localName, options) {
            var dom = document._createElementNS(nsp, localName, options);
            dom.isTiddlyWikiFakeDom = true;
            return dom;
        };
        document._createTextNode = document.createTextNode;
        document.createTextNode = function(textContent) {
            var dom = document._createTextNode(textContent);
            dom.isTiddlyWikiFakeDom = true;
            return dom;
        };
        exports.fakeDocument = document;
        exports.fakeWindow = window;
        exports.fakeHTML = HTML;
    } catch (e) {
        console.info('Cannot import linkedom as fakeDocument, use native instead.');
        // Sequence number used to enable us to track objects for testing
        var sequenceNumber = null;

        var bumpSequenceNumber = function(object) {
            if (sequenceNumber !== null) {
                object.sequenceNumber = sequenceNumber++;
            }
        };

        var TW_Node = function() {
            throw TypeError("Illegal constructor");
        };

        Object.defineProperty(TW_Node.prototype, 'ELEMENT_NODE', {
            get: function() {
                return 1;
            }
        });

        Object.defineProperty(TW_Node.prototype, 'TEXT_NODE', {
            get: function() {
                return 3;
            }
        });

        var TW_TextNode = function(text) {
            bumpSequenceNumber(this);
            this.textContent = text + "";
        };

        TW_TextNode.prototype = Object.create(TW_Node.prototype);

        Object.defineProperty(TW_TextNode.prototype, "nodeType", {
            get: function() {
                return this.TEXT_NODE;
            }
        });

        Object.defineProperty(TW_TextNode.prototype, "formattedTextContent", {
            get: function() {
                return this.textContent.replace(/(\r?\n)/g, "");
            }
        });

        var TW_Element = function(tag, namespace) {
            bumpSequenceNumber(this);
            this.isTiddlyWikiFakeDom = true;
            this.tag = tag;
            this.attributes = {};
            this.isRaw = false;
            this.children = [];
            this._style = {};
            this.namespaceURI = namespace || "http://www.w3.org/1999/xhtml";
        };

        TW_Element.prototype = Object.create(TW_Node.prototype);

        Object.defineProperty(TW_Element.prototype, "style", {
            get: function() {
                return this._style;
            },
            set: function(str) {
                var self = this;
                str = str || "";
                $tw.utils.each(str.split(";"), function(declaration) {
                    var parts = declaration.split(":"),
                        name = $tw.utils.trim(parts[0]),
                        value = $tw.utils.trim(parts[1]);
                    if (name && value) {
                        self._style[$tw.utils.convertStyleNameToPropertyName(name)] = value;
                    }
                });
            }
        });

        Object.defineProperty(TW_Element.prototype, "nodeType", {
            get: function() {
                return this.ELEMENT_NODE;
            }
        });

        TW_Element.prototype.getAttribute = function(name) {
            if (this.isRaw) {
                throw "Cannot getAttribute on a raw TW_Element";
            }
            return this.attributes[name];
        };

        TW_Element.prototype.setAttribute = function(name, value) {
            if (this.isRaw) {
                throw "Cannot setAttribute on a raw TW_Element";
            }
            this.attributes[name] = value + "";
        };

        TW_Element.prototype.setAttributeNS = function(namespace, name, value) {
            this.setAttribute(name, value);
        };

        TW_Element.prototype.removeAttribute = function(name) {
            if (this.isRaw) {
                throw "Cannot removeAttribute on a raw TW_Element";
            }
            if ($tw.utils.hop(this.attributes, name)) {
                delete this.attributes[name];
            }
        };

        TW_Element.prototype.appendChild = function(node) {
            this.children.push(node);
            node.parentNode = this;
        };

        TW_Element.prototype.insertBefore = function(node, nextSibling) {
            if (nextSibling) {
                var p = this.children.indexOf(nextSibling);
                if (p !== -1) {
                    this.children.splice(p, 0, node);
                    node.parentNode = this;
                } else {
                    this.appendChild(node);
                }
            } else {
                this.appendChild(node);
            }
        };

        TW_Element.prototype.removeChild = function(node) {
            var p = this.children.indexOf(node);
            if (p !== -1) {
                this.children.splice(p, 1);
            }
        };

        TW_Element.prototype.hasChildNodes = function() {
            return !!this.children.length;
        };

        Object.defineProperty(TW_Element.prototype, "childNodes", {
            get: function() {
                return this.children;
            }
        });

        Object.defineProperty(TW_Element.prototype, "firstChild", {
            get: function() {
                return this.children[0];
            }
        });

        TW_Element.prototype.addEventListener = function(type, listener, useCapture) {
            // Do nothing
        };

        Object.defineProperty(TW_Element.prototype, "tagName", {
            get: function() {
                return this.tag || "";
            }
        });

        Object.defineProperty(TW_Element.prototype, "className", {
            get: function() {
                return this.attributes["class"] || "";
            },
            set: function(value) {
                this.attributes["class"] = value + "";
            }
        });

        Object.defineProperty(TW_Element.prototype, "value", {
            get: function() {
                return this.attributes.value || "";
            },
            set: function(value) {
                this.attributes.value = value + "";
            }
        });

        Object.defineProperty(TW_Element.prototype, "outerHTML", {
            get: function() {
                var output = [],
                    attr, a, v;
                output.push("<", this.tag);
                if (this.attributes) {
                    attr = [];
                    for (a in this.attributes) {
                        attr.push(a);
                    }
                    attr.sort();
                    for (a = 0; a < attr.length; a++) {
                        v = this.attributes[attr[a]];
                        if (v !== undefined) {
                            output.push(" ", attr[a], "=\"", $tw.utils.htmlEncode(v), "\"");
                        }
                    }
                }
                if (this._style) {
                    var style = [];
                    for (var s in this._style) {
                        style.push($tw.utils.convertPropertyNameToStyleName(s) + ":" + this._style[s] + ";");
                    }
                    if (style.length > 0) {
                        output.push(" style=\"", style.join(""), "\"");
                    }
                }
                output.push(">");
                if ($tw.config.htmlVoidElements.indexOf(this.tag) === -1) {
                    output.push(this.innerHTML);
                    output.push("</", this.tag, ">");
                }
                return output.join("");
            }
        });

        Object.defineProperty(TW_Element.prototype, "innerHTML", {
            get: function() {
                if (this.isRaw) {
                    return this.rawHTML;
                } else {
                    var b = [];
                    $tw.utils.each(this.children, function(node) {
                        if (node instanceof TW_Element) {
                            b.push(node.outerHTML);
                        } else if (node instanceof TW_TextNode) {
                            b.push($tw.utils.htmlEncode(node.textContent));
                        }
                    });
                    return b.join("");
                }
            },
            set: function(value) {
                this.isRaw = true;
                this.rawHTML = value;
                this.rawTextContent = null;
            }
        });

        Object.defineProperty(TW_Element.prototype, "textInnerHTML", {
            set: function(value) {
                if (this.isRaw) {
                    this.rawTextContent = value;
                } else {
                    throw "Cannot set textInnerHTML of a non-raw TW_Element";
                }
            }
        });

        Object.defineProperty(TW_Element.prototype, "textContent", {
            get: function() {
                if (this.isRaw) {
                    if (this.rawTextContent === null) {
                        return "";
                    } else {
                        return this.rawTextContent;
                    }
                } else {
                    var b = [];
                    $tw.utils.each(this.children, function(node) {
                        b.push(node.textContent);
                    });
                    return b.join("");
                }
            },
            set: function(value) {
                this.children = [new TW_TextNode(value)];
            }
        });

        Object.defineProperty(TW_Element.prototype, "formattedTextContent", {
            get: function() {
                if (this.isRaw) {
                    return "";
                } else {
                    var b = [],
                        isBlock = $tw.config.htmlBlockElements.indexOf(this.tag) !== -1;
                    if (isBlock) {
                        b.push("\n");
                    }
                    if (this.tag === "li") {
                        b.push("* ");
                    }
                    $tw.utils.each(this.children, function(node) {
                        b.push(node.formattedTextContent);
                    });
                    if (isBlock) {
                        b.push("\n");
                    }
                    return b.join("");
                }
            }
        });

        var document = {
            setSequenceNumber: function(value) {
                sequenceNumber = value;
            },
            createElementNS: function(namespace, tag) {
                return new TW_Element(tag, namespace);
            },
            createElement: function(tag) {
                return new TW_Element(tag);
            },
            createTextNode: function(text) {
                return new TW_TextNode(text);
            },
            compatMode: "CSS1Compat", // For KaTeX to know that we're not a browser in quirks mode
            isTiddlyWikiFakeDom: true
        };
        exports.fakeDocument = document;
    }

})();
1 Like
#8

great work! 666 :pinching_hand: :pinching_hand: :pinching_hand:

#9

Great, looks very promising.

I think we also need to import the linkedom module as a tiddler so that we can require('linkedom') (actually, probably should be require('$:/plugins/stott/linkedom/indwx.js'))

#10

The dependency systems of both JSDom and LinkeDom are very complex. I have tried to pack them up without success. A dependency named cssom always causes the circular import, which makes the result js file failed to work properly. So finally I give up.

Is there any better way to solve this problem?

#11

I was working on some mermaid plugins recently, and I did encounter this problem. Mermaid does not support SSR. If the tiddlywiki command line does not support using markdow-export to export tiddler with mermaid, because it is not a browser environment, no document