Problem with error message as loading tiddlers on node.js crashes

1

I have a brand new tiddlywiki server edition (regenerated at will). I have to import two set of tiddlers. Each set is written is a valid json file. let call them “setup” and “user”

Each set can be imported as the firt one with success.

I can successfully import setup then standard but importing standard then setup will fail.

The error as it occurred is below. not very useful message and factually false (my files were validated with json lint online)

sudo docker run --rm -v /home/jnpr/big/workspace/accessibilite/dockerized/proteva-data:/usr/share/tiddlywiki/proteva-wiki proteva tiddlywiki proteva --load all-system-setup-tiddlers-proteva.json
undefined:1


SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at $tw.Wiki.application/json (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:1614:16)
    at $tw.Wiki.deserializeTiddlers (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:1509:23)
    at Object.$tw.loadTiddlersFromFile (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:1776:23)
    at Object.$tw.loadTiddlersFromPath (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:1828:22)
    at /usr/local/lib/node_modules/tiddlywiki/boot/boot.js:1823:40
    at Object.$tw.utils.each (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:126:12)
    at Object.$tw.loadTiddlersFromPath (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:1821:15)
    at Object.$tw.loadWikiTiddlers (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:2109:21)
    at Object.$tw.loadTiddlersNode (/usr/local/lib/node_modules/tiddlywiki/boot/boot.js:2199:27)
import of all-system-setup-tiddlers-proteva.json done

I did a diff of the directory structure of the wiki in the two scenarios, setup-user for the success case and user-setup for the failed attempt. As expected, I will use that information to try to see what’s gone wrong and report. I think it could be useful.

Here’s the diff. (seulement dans = only in)

Seulement dans setup-user/proteva/tiddlers: $__config_NewJournal_Title.tid
Seulement dans setup-user/proteva/tiddlers: $__config_WikiParserRules_Inline_wikilink.tid
Seulement dans setup-user/proteva/tiddlers: $__DefaultTiddlers.tid
Seulement dans setup-user/proteva/tiddlers: $__SiteSubtitle.tid
Seulement dans setup-user/proteva/tiddlers: $__SiteTitle.tid
diff -r setup-user/proteva/tiddlers/$__StoryList.tid user-setup/proteva/tiddlers/$__StoryList.tid
1c1
< list: [[liste des projets]]
---
> list: GettingStarted
Seulement dans setup-user/proteva/tiddlers: $__themes_tiddlywiki_vanilla_settings_codefontfamily.tid
Seulement dans setup-user/proteva/tiddlers: $__themes_tiddlywiki_vanilla_settings_fontfamily.tid
2

I found out the reason of the crash: it was a json tiddler whose text field was empty. It should have be set to {} to be correct.

I also found out that while on the web wiki, having an empty text field for json tiddler is 100% OK. But stop the server (I’m on node.js) and restart it and hey! you’ve got a crash! This is the same thing with --load command.

Perhaps, testing json content agains empty and providing a default content of {} would be better. Have an error or warning message on the console too.

The interactive support of empty textfield for json tiddlers is the reason for my json file to get one for they were export of my single file wiki for my new node wiki.

3

Thanks @jypre please could you show the source for the tiddler in question?

4

Here the json export of one of these offensive empty json tiddlers (I had several):

[
    {
      "text":"",
      "type":"application/json",
      "title":"lsu-cmp001-web1178-tst",
      "modified":"20210728123821384",
      "created":"20210728123821384"
    }
]

Should I open a bug ticket (where?) to make things documented?

5

Here’s a filter to find any such empty json tiddlers:

[field:type[application/json]field:text[]]

I didn’t remember about the field operator, which is the only way I know to get such a filter. In particular, I had devised [search:type[application/json]] :filter[get[text]match[]] and [search:type[application/json]] :filter[search:text:regexp[^$]]and also [search:type[application/json]] :filter[get[text]is[blank]] that all fail.

I had 7 offensive tiddlers.

6

Apologies @jypre I may not be understanding you correctly. I tried pasting the example JSON tiddler into a file test.json, and then executed the following command without problems:

tiddlywiki editions/empty/ --load ./test.json
7

What I get:

$ sudo docker run --rm -v proteva-data:/usr/share/tiddlywiki/proteva-wiki proteva tiddlywiki proteva --load bug.json
Error: No tiddlers found in file "bug.json"
$ cat bug.json
[
    {
      "text":"",
      "type":"application/json",
      "title":"bug.json",
      "modified":"20210728123821384",
      "created":"20210728123821384"
    }
]
8

Thanks @jypre. Could you try a slight variation of that command:

sudo docker run --rm -v proteva-data:/usr/share/tiddlywiki/proteva-wiki proteva tiddlywiki --load bug.json

(I removed the proteva immediately after the tiddlywiki command, thus avoiding loading your existing wiki folder).

9

No change.

$ sudo docker run --rm -v proteva-data:/usr/share/tiddlywiki/proteva-wiki proteva tiddlywiki --load bug.json[sudo] Mot de passe de jnpr : 
Error: No tiddlers found in file "bug.json"
10

Hi @jypre I’m still unable to duplicate the problem. What version of TiddlyWiki are you using?

Can anyone else help to see what I might be missing - @pmario @xcazin?

11

5.1.23

My Dockerfile:

# Dockerfile pour lancer le tiddlywiki proteva
# copiée et adaptée du modèle elasticdog/tiddlywiki disponible sur le docker hub
from node:16.9.0-alpine3.11

LABEL author="Jean-Pierre RIVIÈRE"
LABEL content="instance de nodejs avec le tiddliwki proteva"

env TIDDLYWIKI_VERSION=5.1.23
env PROTEVA_FEED=proteva-data.json
env WORKDIR=/usr/share/tiddlywiki/proteva-wiki

EXPOSE 8080

WORKDIR $WORKDIR
VOLUME $WORKDIR

run apk update && apk add --no-cache tini
run npm install -g tiddlywiki@$TIDDLYWIKI_VERSION && npm install clean --force

#ENTRYPOINT ["tiddlywiki", "proteva"]
#CMD ["--listen", "host=0.0.0.0"]
ENTRYPOINT ["/sbin/tini", "--"]
CMD ["tiddlywiki", "proteva", "--listen", "host=0.0.0.0"]
12 
Error: No tiddlers found in file "bug.json"

I do get this error message, if the json file couldn’t be found. So make sure, this file gets a path too

13

Thu bug.json content:

[
    {
      "text":"",
      "type":"application/json",
      "title":"bug.json",
      "modified":"20210728123821384",
      "created":"20210728123821384"
    }
]

The error could launch on the --load or on the --listen following it. Just seen that today!

14

I am seeing this issue with 5.1.23 and the prerelease when running with nodejs. In the browser I create a new tiddler with no content and save it as a application/json type. I then kill the nodejs tw processes and rerun the server command and see


SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at $tw.Wiki.application/json (C:\Users\jeffr\Documents\dev\js\TiddlyWiki5\boot\boot.js:1614:16)
    at $tw.Wiki.deserializeTiddlers (C:\Users\jeffr\Documents\dev\js\TiddlyWiki5\boot\boot.js:1509:23)
    at Object.$tw.loadTiddlersFromFile (C:\Users\jeffr\Documents\dev\js\TiddlyWiki5\boot\boot.js:1776:23)
    at Object.$tw.loadTiddlersFromPath (C:\Users\jeffr\Documents\dev\js\TiddlyWiki5\boot\boot.js:1828:22)
    at C:\Users\jeffr\Documents\dev\js\TiddlyWiki5\boot\boot.js:1823:40

This is under window 10 in a git bash shell

note I get a .json.meta file alone with the empty .json tid

15

Thanks @buggyj I can confirm that I am seeing the same results. One ends up with a blank New Tiddler.json file and a New Tiddler.json.meta file containing the created, modified, title, tags, type field.

But I am still confused, because that seems to be a different bug than the one that @jypre is reporting. It works if I try to --load the directory containing New Tiddler.json.

In particular, can anyone else confirm that attempting to --load the following file bug.json gives them an error:

[
    {
      "text":"",
      "type":"application/json",
      "title":"bug.json",
      "modified":"20210728123821384",
      "created":"20210728123821384"
    }
]
16

The problem reported by @buggyj is fixed here:

@jypre does it also solve your problem?

17

I can’t tell yet. In https://github.com/Jermolene/TiddlyWiki5/tree/master/bin, the link labelled “Scripts for building tiddlywiki.com” redirect to https://tiddlywiki.com/static/Scripts%20for%20building%20tiddlywiki.com.html but this does not exist.

Existent documentation on tiddlywiki.com has not helped me building tw5. My problem is I am on Docker to get node.js and then I have a difficulty with the host or the network. If I do a docker run -p 8080:8080 image sh bin/serve.sh , the host is not reachable but I don’'t know how to specify the host. If I create a network and run docker attached to it, it doesn’t work either. I’m still a newcomer to docker, there is something I should have wrong but my docker inspect have not helped me (maybe I don’t know what I should look for in this case).

Here was my Dockerfile:

from node:16.9.0-alpine3.11

LABEL author="Jean-Pierre RIVIÈRE"
LABEL content="instance de nodejs pour construire des éditions de tiddlywiki"

env WORKDIR=/var/opt/tiddlywiki

EXPOSE 8080

WORKDIR $WORKDIR
VOLUME $WORKDIR

run apk update && apk add --no-cache tini

ENTRYPOINT ["/sbin/tini", "--"]
CMD sh

I build the “node” image with

sudo docker build . -t node

in the directory where is Dockerfile.

Here’s a a simple launching of the container:

sudo docker run -it --rm -v $PWD/TiddlyWiki5:/var/opt/tiddlywiki -p 8080:8080 node sh

and once in the container:

# sh bin/serve.sh
Boot log:
  Startup task: load-modules
  Startup task: info after: load-modules before: startup
  Startup task: plugins after: load-modules
  Startup task: startup after: load-modules
  Startup task: story after: startup
  Startup task: commands platforms: node after: story
Executing command: server 8080 $:/core/save/all text/plain text/html   localhost
 syncer-server-filesystem: Dispatching 'save' task: $:/StoryList 
Serving on http://127.0.0.1:8080
(press ctrl-C to exit)

Variation with network:

sudo docker network create --driver bridge --subnet 192.168.0.0/16 twn
sudo docker run -it --rm -v $PWD/TiddlyWiki5:/var/opt/tiddlywiki --network twn node sh

Sorry.

18

Just a bit of a need for a better error message. Type this in your shell:

tiddlywiki test --load nothing

while there is no nothing file you’ll get this error message:

Error: No tiddlers found in file "nothing"

A correct message would be more like

File "nothing" either doesn't exist or can't be read to extract tiddlers from it.
19

Indeed, that’s a bit annoying, but is going to take quite a bit of refactoring to fix.

I’m still unable to duplicate issue with loading a JSON file with a blank text field.

20

@jeremyruston, have you tried with the dockerfile I pasted to have my very same setup? BTW, I’m on debian buster with kernel 4.19.0-17-amd64.

Where could I get a pre-release edition of 5.20 so that I could try with it if I still have this bug? Would it be helpful if I tried with the previous version of tw?