FileSystemPath considered harmful

#1

I’m having some difficulty with $:/config/FileSystemPaths - maybe someone can help. The background:

  • I use the node.js tiddlywiki ... --listen server mechanism with lots (1000+) of individual tiddler files. I have for many years used separate collections of TiddlyWikis; call them Home, Work, Shared, Common, Projects, …

  • an example of how I use these: at home I’ll run a tiddlywiki server with a tiddlywiki.info file in the directory named Home that has

  "includeWikis": [
      {"path": "../Common", "read-only": false},
      {"path": "../Shared", "read-only": false}
    ],

  • I’ll do the same at the office with a tiddler collection in the directory named Work, and so forth: a collection of related tiddlers that I want to keep separate; and a couple of collections I want to integrate into those but in a way that I can make updates to occasionally.

  • The Shared directory hierarchy has tiddlers that I intend to be able to access from the Home, Work, or Project servers. The Common directory hierarchy has tiddlywiki macros, plugins, and such: TW configuration that change rarely but kept in one separate collection so things work the same in other collections that share Common. Any conflicts that I cause in Shared and Common I resolve via git repo conflict resolution. All this to to live by the Don’t Repeat Yourself (DRY) principle.

Here is the issue: when I make changes in Shared tiddlers the tiddler file gets removed from Shared and a new, modified version ends up in Home; or Work, etc… Similarly with configuration tiddlers in Common. I want existing tiddlers in Shared or Common, on the rare occasions when they get modified, to stay rooted in those collections.

I believe this happens because I have a $:/config/FileSystemPaths with contents
[split[ ]join[_]]
The purpose of that is so that I do not end up with tiddler filenames with spaces - which ends up happening much more frequently and is annoying in other ways.

The FileSystemPaths rule that “the first output of the first filter to produce any output is taken as a logical path to be used for the tiddler file. If the logical path has changed a new file is created and the old file is deleted.” accounts for the collection renaming. This happens, I presume, because FileSystemPath’s filter produces output even when tiddlers’ file names have already been modified, though the titles still contain (desirable) blanks.

All of this started back when TW 5.1.14 stopped automatically replacing spaces with ‘_’ characters. Some other growing pains occurred until TW 5.2.1 finally fixed some other issues and now I’m back to this last, changing root, issue.

The rule that the “first filter to produce any output” wins seems overly constraining. However, given that changing the rule is likely to be a non-starter

  1. is there anything I can add to $:/config/FileSystemPaths to restore the collection root?
  2. Is there some smarter FileSystemPaths filter I can use to see if the filename needs adjusting, not the title?
  3. Otherwise something like this old solution from Jeremy, which is the only comparable issue I’ve come across, might be adapted by adding a field to the many Shared and Common tiddlers; though I’m not sure how to make that abide by the first filter rule and yet leave the replace blanks filter to do its job.

Thanks for any other ideas, sorry this got so long.

#2

You’re using the rules just to clean up spaces, but you could also use it for routing. If your work tiddlers were tagged “Work”, “Home”, etc. then you could have rules like:

[tag[Work]split[ ]join[_]addprefix[Work/]]
[tag[Home]split[ ]join[_]addprefix[Home/]]
[split[_]join[ ]addprefix[Common/]]

Apologies if I’ve misunderstood the question.

#3

Have you set the retain-original-tiddler-path option in your tiddlywiki.info file?

If not, I recommend looking at the retain-original-tiddler-path and default-tiddler-location options for tiddlywiki.info files.

https://tiddlywiki.com/#tiddlywiki.info%20Files

#4

Mark,

That might be a work around, but unless I am misunderstanding where in the wiki path addprefix is going to get grafted into, that approach seems like it has the following downsides:

  1. Requires me to change my usage of tiddlywiki.info files’ includeWikis to include the contents of external Shared and Common wikis into separate wikis for Home, Work, Projects, etc.
  2. I’d have to re-structure my git repo and move Home, Work, Project, etc. into a single wiki’s tiddlers, perhaps a combination of Shared and Common.
  3. I’d effectively be running one large tiddlywiki, with the attendant increase in browser and node server memory of handling ~3x the number of tiddlers.
  4. I’d have to bulk add tags to all the Home, Work, Projects tiddlers.

Saq,

Actually I have tried the retain-original-tiddler-path tiddlywiki.info config parameter, which looked like it might have some promise. I already use the default-tiddler-location option that ends up being part of the problem, since the moved Shared / Common tiddlers end up being under that location.

WRT retain-original-tiddler-path, unless I can somehow use the info captured in OriginalTiddlerPaths in some FileSystemPaths filter I’m not seeing how that helps. How would one create a filter that looks up the tiddler title in OriginalTiddlerPaths and returns the original path? Sadly it also seems that $:/config/OriginalTiddlerPaths is an in-memory only tiddler that doesn’t get persisted to disk in tiddlywiki server, so an external solution to tiddlywiki solution, that I’ve considered, ends up being hard too. But the suggestion to look at retain-original-tiddler-path also led me to discover that others have been down similar paths and maybe this is one of those ‘rough edges’ in tiddlywiki server land.

Thanks for the ideas though.

#5

I don’t know if this will work, but the specs only indicate that the path has to be relative. So maybe …

[tag[Work]split[ ]join[_]addprefix[../Work/tiddlers]]
[tag[Home]split[ ]join[_]addprefix[../Home/tiddlers]]
[split[_]join[ ]addprefix[../Common/tiddlers]]
#6

One workaround could be to have the filters in $:/config/FileSystemPaths only match new tiddlers. The assumption being that already existing tiddlers do not have spaces in their titles, and that in the absence of a matching filter they will get saved to where they were loaded from:

[!has[bag]split[ ]join[_]]

1 Like
#7

That little !has[bag] addition seems, from an initial test, to be the perfect solution! Thank you very much for ending that near daily annoyance. I suspected someone smarter than I might be able to. Plus I learned about the bag TiddlerField in the process.

#8

I will file away the possibility of using tags and prefixes containing relative paths for future reference (although allowing a local web app to poke around outside its normal boundaries as I would need to do is generally frowned upon.)