External-Text / _canonical_uri explanation?

[stobot]

Greetings all,

I have a relatively large tiddler/file/database (2MB alone) that I look things up in once in a while that I’ve stored in my TiddlyWiki. Including this essentially doubles the size of the wiki, and I’m wondering if I can externalize it so that it doesn’t get included in load/save operations for performance reasons. I’m currently using the SharePoint/WebDav/ASPX single-file setup for this wiki.

After playing around and referring to the old https://tiddlywiki.com/#Alice%20in%20Wonderland example posted in the home page. I followed this method using a _canonical_uri field and storing it in a relative path (SharePoint https storage). All looks good, it works well in that I can “operate on it” etc. and my hope was that when I saved my wiki, the size wouldn’t have greatly increased. Strangely that doesn’t seem to be the case. It’s going up from the ~2MB to ~4MB with this in it, even though the message says “This tiddler shows content stored outside of the main…”.

I also tried the official “External Attachments” plugin to see if that changed anything (though it’s labelled as primarily for TiddlyDesktop) and that doesn’t see to have changed anything.

Is what I’m intending possible? Not a big deal if not, but it seems like it works for images but not text even though they’re setup in the same way?

[saqimtiaz]

@stobot

After playing around and referring to the old https://tiddlywiki.com/#Alice%20in%20Wonderland example posted in the home page. I followed this method using a _canonical_uri field and storing it in a relative path (SharePoint https storage). All looks good, it works well in that I can “operate on it” etc. and my hope was that when I saved my wiki, the size wouldn’t have greatly increased. Strangely that doesn’t seem to be the case. It’s going up from the ~2MB to ~4MB with this in it, even though the message says “This tiddler shows content stored outside of the main…”.

Right, this surprised me as well. Personally I consider it a bug that the content for _canonical_uri tiddlers gets saved in the wiki, though potentially a tricky one to fix because of how the external content support is implemented. It essentially gets fetched and shoved into the tiddler object, at which point the core treats it like any other tiddler.

Interestingly note that the default behaviour is to save the text field of a tiddler even if it is a _canonical_uri tiddler.

One work around is to have a startup action that creates your tiddler with the _canonical_uri field and exclude it from saving via publishFilter/syncFilter.

[stobot]

@saqimtiaz

One work around is to have a startup action that creates your tiddler with the _canonical_uri field and exclude it from saving via publishFilter/syncFilter.

Interesting - good idea. Thanks

[Mark_S]

You could use your own iframe to display the text.

[stobot]

That’s true @Mark_S - but in this case it’s purpose is to be filtered through for lookups (via <$list>). Good idea for situations where that’s not the case though. Thanks,

[Mark_S]

What do you mean by filtered? Do you mean that you’re filtering on the actual text? Or just the tags? If tags, the tags can live in the enveloping tiddler. If text … I’m amazed that you could filter on external text, which would explain why you’re file size is increasing.

[TW_Tones]

The text could be available in memory, but not when saving!

[stobot]

@TW_Tones

The text could be available in memory, but not when saving!

Exactly. As I mentioned I’m fully including it today. I was just hoping since it’s not an every day need that I could essentially improve speed most of the time (save/refresh) and trade against longer occasional slowdown during occasional use (essentially lazy load the big file on demand). It would also save me having to manually re -import the file when it changes (currently monthly), as I could programmatically push the dB there. At least I can do that part, just not the lazy loading part.