Saving Node.js Wiki (after converting it from a single-Wiki)

RetRoland · June 2, 2023, 9:39am

I’m doing my first steps with the TiddlyWiki Node.js version . Although it’s rather easy to set up a new wiki I found out that converting a single-file-Wiki could be tricky.

Using the command
tiddlywiki --load ./SingleWiki.html --savewikifolder ./NodeWiki
is doing all conversion very well, but when it comes to save the new wiki with the mechanism Node.js should provide, I got stuck.
Right after the conversion it tries to save to a local file, not to the server.
This is the tiddlywiki.info file:

{
    "languages": [
        "de-AT",
        "de-DE"
    ],
    "plugins": [
        "tiddlywiki/codemirror",
        "tiddlywiki/codemirror-mode-css",
        "tiddlywiki/codemirror-mode-htmlmixed",
        "tiddlywiki/codemirror-mode-javascript",
        "tiddlywiki/codemirror-mode-xml",
        "tiddlywiki/comments",
        "tiddlywiki/filesystem",
        "tiddlywiki/help",
        "tiddlywiki/highlight",
        "tiddlywiki/internals",
        "tiddlywiki/menubar",
        "tiddlywiki/railroad"
    ],
    "themes": [
        "tiddlywiki/snowwhite",
        "tiddlywiki/vanilla"
    ]
}

Compared to the newly created Wiki it seems to miss a plugin,

    "plugins": [
        "tiddlywiki/tiddlyweb",
        "tiddlywiki/filesystem",
        "tiddlywiki/highlight",
    ],

→ "tiddlywiki/tiddlyweb"

But after I added the line to the plugins of the converted Wiki, several error messages were shown:

Get Status Error: XMLHttpRequest Error-Code: 404

Error receiving “skinny” Tiddler list: XMLHttpRequest Error-Code: 404

Sync error while processing save of ‘$:/StoryList’: XMLHttpRequest Error-Code: 404

The last one shows a counter which increases.
[edit]Just noticed that the second message also shows an increasing counter. Not as fast as the third message, but also increasing.[/edit]

I’m pretty sure that I’m only missing some more configuration lines to solve the problem. Maybe it has to do with the build-configuration, which the other (working) Wiki has. But I didn’t catch the point and reading the help didn’t helped me further, unfortunately.

Could someone please point me into the right direction?

saqimtiaz · June 2, 2023, 11:38am

What command are you using to run the wiki on node.js?

linonetwo · June 2, 2023, 11:47am

TidGi app provides a “load HTML wiki” feature in GUI, you can try it out.

tiddlyweb plugin can be omited, and re-added to cli when starting a wiki

github.com/Jermolene/TiddlyWiki5

Built-in externalimages build corrupts original image files

opened 01:41PM - 09 Mar 20 UTC

ejheil

Tiddlywiki version 5.1.21, running on OS X 10.14.6, node version 13.8.0. Summ…ary: running --build externalimages replaces the original image tiddler file in the tiddlers directory with a six-byte garbage file. Demo: ``` ➜ tiddlywikis git:(master) ✗ tiddlywiki test --init server Copied edition 'server' to test ➜ tiddlywikis git:(master) ✗ cd test ➜ test git:(master) ✗ tiddlywiki --listen Serving on http://127.0.0.1:8080 (press ctrl-C to exit) ``` At this point I drag and drop myimage.jpg, a 49K jpeg, into the wiki window and click "import." ``` syncer-server-filesystem: Dispatching 'save' task: $:/StoryList syncer-server-filesystem: Dispatching 'save' task: $:/StoryList syncer-server-filesystem: Dispatching 'save' task: $:/StoryList syncer-server-filesystem: Dispatching 'save' task: myimage.jpg syncer-server-filesystem: Dispatching 'save' task: $:/StoryList ^C ``` The image is now in tiddles/ ``` ➜ test git:(master) ✗ ls -l tiddlers total 120 -rw-r--r-- 1 edheil staff 121 Mar 9 09:33 $__StoryList.tid -rw-r--r-- 1 edheil staff 49303 Mar 9 09:33 myimage.jpg -rw-r--r-- 1 edheil staff 90 Mar 9 09:33 myimage.jpg.meta ``` I build the built-in externalimages target - ``` ➜ test git:(master) ✗ tiddlywiki --build externalimages syncer-server-filesystem: Dispatching 'save' task: myimage.jpg syncer-server-filesystem: Dispatching 'save' task: $:/StoryList ``` And the original myimage.jpg is gone from tiddlers/ replaced with a 6 byte file, though it has been correctly copied to output/images. ``` ➜ test git:(master) ✗ ls -l tiddlers total 24 -rw-r--r-- 1 edheil staff 40 Mar 9 09:33 $__StoryList.tid -rw-r--r-- 1 edheil staff 6 Mar 9 09:33 myimage.jpg -rw-r--r-- 1 edheil staff 127 Mar 9 09:33 myimage.jpg.meta ➜ test git:(master) ✗ ls -l output/images total 104 -rw-r--r-- 1 edheil staff 49303 Mar 9 09:33 myimage.jpg ➜ test git:(master) ✗ ``` At this point if I didn't keep a copy of the contents of that output directory, I've permanently lost the original image! It's unclear exactly what that new 6-byte file is, it appears to be garbage: ``` ➜ test git:(master) ✗ file tiddlers/myimage.jpg tiddlers/myimage.jpg: ISO-8859 text, with no line terminators ➜ test git:(master) ✗ cat tiddlers/myimage.jpg ?w^~)?% ➜ test git:(master) ✗ ```

RetRoland · June 2, 2023, 1:25pm

I’m using the “listen” command:

~/TiddlyWiki $ tiddlywiki ./NodeWiki --listen port=8080 host=192.168.1.1
syncer-server-filesystem: Dispatching 'save' task: $:/StoryList
Serving on http://192.168.1.1:8080
(press ctrl-C to exit)

RetRoland · June 2, 2023, 2:27pm

Thank you very much for information.
You said that “tiddlyweb plugin can be omited, and re-added to cli when starting a wiki” - how do I achieve this?

In general I don’t know if this was the right way to convert a HTML-based single-file-Wiki into a “server-edition”. I expected that after converting it with the savewikifolder param the save mechanism would also be adjusted “automagically” to match the new (Node.js-type) Wiki.
I was wrong.

Mark_S · June 2, 2023, 4:11pm

Yes, the TiddlyWiki.info file that is created is incomplete. That is always surprising, since obviously if you convert it you want it to run on node.js.

Can you show your complete TiddlyWiki.info file after edits?

Scott_Sauyet · June 2, 2023, 4:56pm

A workflow that does well for me:

$ tiddlywiki path/to/MyWiki --init server
$ tiddlywiki path/to/MyWiki --load path/to/my/static.html
$ tiddlywiki path/to/MyWiki --listen port=9876

(Now available at http://localhost:9876)

The first two lines are set-up. After this initial load, that last line alone will start the server whenever I wish.

Usually I run this out of the folder I’m going to use, so path/to/MyWiki is just ..

RetRoland · June 2, 2023, 6:36pm

Yes, you are right - that was surprising.

When I ran the converted Wiki for the first time I got a warning message:

Warning: Plugin(s) required for client-server operation are missing.
"$:/plugins/tiddlywiki/tiddlyweb"

That’s why I added the tiddlyweb-plugin to the plugin-configuration section:

{
    "languages": [
        "de-AT",
        "de-DE"
    ],
    "plugins": [
        "tiddlywiki/tiddlyweb",
        "tiddlywiki/codemirror",
        "tiddlywiki/codemirror-mode-css",
        "tiddlywiki/codemirror-mode-htmlmixed",
        "tiddlywiki/codemirror-mode-javascript",
        "tiddlywiki/codemirror-mode-xml",
        "tiddlywiki/comments",
        "tiddlywiki/filesystem",
        "tiddlywiki/help",
        "tiddlywiki/highlight",
        "tiddlywiki/internals",
        "tiddlywiki/menubar",
        "tiddlywiki/railroad"
    ],
    "themes": [
        "tiddlywiki/snowwhite",
        "tiddlywiki/vanilla"
    ]
}

But this only causes some errors:

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/113.0
Online Status: true

Log:
2023 6 2 20:33:05.356: Getting status
2023 6 2 20:33:05.805: Get Status Error: XMLHttpRequest Error-Code: 404
2023 6 2 20:33:05.805: Retrieving skinny tiddler list
2023 6 2 20:33:05.811: Dispatching 'save' task: $:/StoryList
2023 6 2 20:33:05.935: Fehler beim Empfangen einer "skinny" Tiddler Liste: XMLHttpRequest Error-Code: 404
2023 6 2 20:33:05.940: Sync error while processing save of '$:/StoryList': XMLHttpRequest Error-Code: 404
2023 6 2 20:33:05.940: Checking dirty status
2023 6 2 20:33:10.954: Dispatching 'save' task: $:/StoryList
2023 6 2 20:33:10.986: Sync error while processing save of '$:/StoryList': XMLHttpRequest Error-Code: 404
2023 6 2 20:33:10.986: Checking dirty status

RetRoland · June 2, 2023, 6:45pm

Thank you, this is interesting - running the command inside the Wiki folder shortens the command(s)
Running a single-file-Wiki this way is also an option.

I was curious about the tiddler-files and the format of these. This would help me to generate tiddlers for the Wiki. Now I have a text file with various items in it (call it some sort of an inventory) and I don’t want to re-type all items to move it to TiddlyWiki.
I’m thinking of a simple parser (in Python) which automatically generates corresponding Tiddler-items from the items found in the textfile. This way I could save a lot of time.

Scott_Sauyet · June 2, 2023, 6:59pm

I do this often, though in Node, not in Python. In one example, a wiki that I hope to get back to this weekend after a long hiatus is my Periodic Table Demo, which contains tiddler for elements, element types, and compounds in a data plugin. The tiddlers of that plugin are generated on demand from JSON files with the elements and the compounds, using a Node script that does some minor clean-up, calls the Wikipedia API for its overviews, reformats into .tid format, and saves the results into the Compounds and Elements plugins.

In another example, behind company walls, I use this to generate documentation to match the output of a code-generating rules engine. Every time the code is regenerated, a build step also creates up-to-date docs in Tiddywiki format.

It’s a very useful technique.

RetRoland · June 2, 2023, 7:46pm

That sounds really interesting, thanks for the insight. I’m not a chemist and it was never one of my favourite subjects, but the periodic table is excellent. It would certainly have made learning at school a lot more fun for me.

I think it’s especially cool to keep the documentation up to date in an automated build step. That’s something you don’t like to do and yet it’s required.
It’s a wonderful idea and I’m always surprised at how versatile TiddlyWiki can be used.
Simply great.

digitalap3 · June 2, 2023, 11:48pm

For future reference when I move from single file to node I find the easiest thing to do is spin up a new Node instance and then just drag the single file into it and import what I need. It makes the process so very much simpler.

RetRoland · June 3, 2023, 12:11am

Meanwhile I found a “solution” for my problem:

convert the single-file-wiki using the --load and --savewikifolder command
add the line "tiddlywiki/tiddlyweb", to the start of the plugin configuration section in tiddlywiki.info file
most important: in the tiddlers folder, delete an existing $__config_tiddlyweb_host.tid file

After deleting the file all my errror messages are gone.
Saving the Wiki works fine now

linonetwo · June 3, 2023, 4:31am

Just read the issue link I posted

RetRoland · June 3, 2023, 11:36am

Ah yes, this sheds some light how to use the tiddlyweb-plugin from the command line - thanks.

I read it yesterday and first thought this isn’t what I’ve searched for- mainly because I don’t use external images in my case.
Didn’t understood that you pointed me towards this topic because of the activation of the plugin per commandline, sorry.