Strange behavior in lazy-images mode: Images and PDFs were deleted

#1

Half a year ago I got to know Tiddlywiki. I was very quickly enthusiastic about the well thought-out, powerful concept and have since been using TW in several Docker containers for private documentation. Especially because of the possibility to easily import screenshots, photos or PDFs, you get a comprehensive appealing documentation very fast and easy.

However, after some time, as the wiki became more comprehensive, I noticed that it took a very long time on my SmartPhone to load the entry page. This was especially inconvenient when I just wanted to write a quick note.

After some research I found the lazy-images mode. This shortened the loading time significantly.

A few days ago I noticed a very strange behavior of Tiddlywiki when exporting tiddlers via the kookma/TW-Commander plugin. The export file did not contain all data. Furthermore, even images and PDFs in the original wiki were deleted.

I raised this problem as an issue under kookma/TW-Commander and described it extensively (including a screenshot story).

See: Images and PDFs are deleted when being exported from Docker-Container using lazy-images mode #29

Unfortunately, kookma can’t recreate the issue because he doesn’t use Docker. He advised me to present it in this forum.

From my point of view, this is a bug in TW-core in conjunction with lazy-images. There, when calling an export function, all tiddlers, that are not yet fully loaded, would have to be reloaded first to be exported. Alternatively, a plugin developer like kookma would have to reload all involved tiddlers with wiki.getTiddlerText() (https://tiddlywiki.com/dev/#LazyLoadingMechanism) before calling the export function.

Under the conditions described, I can understand with the error that the export file is not complete. What I don’t understand, however, is the effect that the images and PDFs not yet loaded are even deleted in the source wiki and the exported tiddlers are marked as modified in the source wiki. (please see my screenshot story in #29). However, this does not happen when doing the export via advanced search.

I wonder why this bug has not been noticed yet. Possibly it is because users are not particularly made aware of it. The lazyload functionality is important. However, it must be completely transparent to the user, i.e. for functions such as export or search the “skinny” tiddlers must be automatically reloaded in the background before the functions are executed.

That’s my opinion. But I’m not the TW-expert. What do you think and how can the problem be solved?

Best Regards
xgadscu

#2

Is this pull request made by @Flibbles related to your issue ? It’s already merged to the TW 5.2.4 pre release I guess.
Also see the below linked issue reported in tiddlymap GitHub repo.

#3

Maybe that’s related. In my screenshot story you can see that not the whole tiddlers are deleted but only the content (image or pdf).

#4

In the meantime, I am convinced that it is related to this.

Without having known the code, I had suspected the following:

From my point of view, this is a bug in TW-core in conjunction with lazy-images. There, when calling an export function, all tiddlers, that are not yet fully loaded, would have to be reloaded first to be exported. Alternatively, a plugin developer like kookma would have to reload all involved tiddlers with wiki.getTiddlerText() (https://tiddlywiki.com/dev/#LazyLoadingMechanism) before calling the export function.

The modified code Fix for #4767: lazy-loading deletes tiddler bodies by flibbles · Pull Request #7014 · Jermolene/TiddlyWiki5 · GitHub is exactly as I assumed:

    if(fields.revision) {
		delete fields.revision;
	}
	// If this is a skinny tiddler, it means the client never got the full
	// version of the tiddler to edit. So we must preserve whatever text
	// already exists on the server, or else we'll inadvertently delete it.
	if(fields._is_skinny !== undefined) {
		var tiddler = state.wiki.getTiddler(title);
		if(tiddler) {
			fields.text = tiddler.fields.text;
		}
		delete fields._is_skinny;
	}
	state.wiki.addTiddler(new $tw.Tiddler(fields,{title: title}));
	var changeCount = state.wiki.getChangeCount(title).toString();
	response.writeHead(204, "OK",{

lazy-images mode should not be condemned as it has been in some comments. The functionality just needs to be implemented properly. For operations like export, search, relink etc. the affected tiddlers or the complete wiki must be reloaded. Then the user has to wait a bit. But in most other cases the performance is better.

The alternative would be to store images and PDFs separately under a web server and link them. But this is much less comfortable.

#5

@xgadscu
I think some of the developers or contributors of TW will respond to your queries. I am just a TW user and doesn’t have any programming knowledge. I just wanted to let you know about that merged GitHub pull request since I had seen it. That’s why I replied to your post.

#6

@arunnbabu81
Thank you very much for your hints. Indeed, i hope that the discussion will continue a bit and the incomplete implementation of lazy-images will be improved.

#7

Take a look at this comment by Jermolene in an old GitHub discussion.