Expanding the is system namespace

[TW_Tones]

Folks,

A quick look at $:/core/modules/wiki.js and $:/core/modules/filters/is/system.js makes me wonder how hard it would be to redefine the function to allow other tiddler prefixes to enter the list of “system tiddlers”. The existing is and all test remain used through out TiddlyWiki but now would include amongst them the new prefix(s), and do not appear when using !is[system]]?

• This would allow the addition of additional prefixes for data tiddlers, index tiddlers and other that designers may appreciate, without needing to modify the core all over the place.
• I am a script kiddy with JavaScript but it looks to me this may be a trivial customisation, but the ability to provide a user mechanism to add prefixes to the definition of system may be very useful.
  • I did try a hack but my regular expressions are also weak.

Why?

Filters are the core functionality of TiddlyWiki and dividing tiddlers between system and non-system is a key feature, basically they are hidden in the standard search and are somewhat hidden from the user. It would be nice to allow a little more flexibility here to the designer.

• As it stands system tiddlers often become quite long, due to the need to prefix with both $:/ and appropriate subtitle/folder structure so as to avoid clashes.
• Existing code already makes this differentiation through the is system and all system operators so it makes sense to leverage this.
• Of course other novel namespaces will appear in the non-system namespaces as needed.

I asked once to also be able to extend the is and all operators so we could add something like is[data], and I would still like this, but it was suggested this would “overload” the operators.

• I doubt more than a few would ever be needed, but regardless
• I do not think is as complex to simply permit more prefixes to match or not match is[system] all[system] !is[system] !all[system]

Your thoughts? and skills for a POC?

[linonetwo]

I want to add is[titleless] for What’s the standard way to not writing Title, I remember the is[system] code is basically something like title.startsWith("$:/", which won’t affect the λ:/, so not much difficult to add a new one.

[TW_Tones]

Related to What's the standard way to not writing Title - #29 by TW_Tones ?

I have done so much work on this from so many directions I am sure we can find ways to solve most questions of this type. This topic has being raised because I see it as an avenue to solve a number of users needs more simply than many of the solutions I and others have raised.

• Most users have a slightly different perspective or requirement, all of which may benefit from the feature I raise here.

[CasperBooWisdom]

This sounds good on the surface, but ONLY if $:/ is the only prefix hard-coded in the core and any new prefixes would then be created on a wiki by wiki basis and stored in a prefixes list tiddler. Otherwise, if you hard-coded additional prefixes in the core, there is no way of knowing what current tiddlers people are using that would contain those new prefixes you want to use, so others currently using those prefixes with non-system tiddlers would suddenly find all of those tiddlers would magically become system tiddlers once they updated, which could instantly break their Tiddlywiki.

So yeah, I agree, this would be a great addition if we could simply have a $:/config/systemPrefixes tiddler that we can add our own prefixes to, thus still being backward compatible.

If not directly in the core, maybe this could be implemented as a plugin.

[TW_Tones]

@CasperBooWisdom

If not directly in the core, maybe this could be implemented as a plugin.

Of course it should start this way. As a plugin, but I am asking for the tiddlywiki designer / user to choose. I want it hackable, I want the capability, not to permanently add anything but the ability to add more. There are plenty of most likely never used prefixes including using Unicode.

@CasperBooWisdom

prefixes with non-system tiddlers would suddenly find all of those tiddlers would magically become system tiddlers once they updated, which could instantly break their Tiddlywiki.

• Actually even if this happens it may be visible, and need correcting, but I think it is unlikely to break much

[Post Script] Presently there are only tiddlers and system tiddlers and all none-system tiddlers are typically plan language titles. The use of other prefixes are made by people to achieve different outcomes, but they are still divided into system and non-system.

• So novel prefixes only exist in the tiddler name space (not the system namespace)
• If I am going to apply a new system namespace to my wiki, it is trivial for me to search all tiddlers, to see if any exist with that prefix before applying a new system prefix.
• If I write a plugin to share with others, perhaps all I need to do is to allow such a namespace prefix to be configured, so that one can be chosen that does not compete with others. This could even be automated. Test if one or more are in use or not, then use one that is not.

[TW_Tones]

@linonetwo I added to my last reply here What's the standard way to not writing Title - #29 by TW_Tones

[pmario]

TLDR;

With the latest development in mind I think there is a chance to get some more convenience operators like [is[json]], [is[dictionary]] and there [all[]] equivalents.

In the meantime

For the other 40+ tiddler types TW can handle the [type[...]] field needs to do the trick.

---

---

@TW_Tones

I asked once to also be able to extend the is and all operators so we could add something like is[data], and I would still like this, but it was suggested this would “overload” the operators.

It would be nice to have a link to “I asked once”, so we can have a closer look to the responses and when that happened.

Until lately data-tiddlers could have been considered unwanted.

But with TW v5.2.5 we got a bunch of new operators jsonget, jsontype, jsonextract, jsonindexes … With v5.3.2 we will get jsonset.

So for me it seems “data tiddlers” will not go anywhere.

There are 2 types of data-tiddlers:

1. Dictionary Tiddlers – type: application/x-tiddler-dictionary used for eg: palettes
2. JSON Tiddlers – type: application/json used for eg: $:/HistoryList

It’s easy to filter for [type[application/json]] which is an equivalent to [is[json]].

The same is true for [type[application/x-tiddler-dictionary]] as an equivalent to [is[dictionary]]

Conclusion

So with the latest development in mind I think there is a chance to get some more convenience operators like [is[json]], [is[dictionary]] and there [all[]] equivalents.

In the meantime

For the other 40+ tiddler types TW can handle the [type[...]] field needs to do the trick.

[TW_Tones]

@pmario If we look at - is tiddler, is system, is missing, orphan, shadow even tag and tiddler to name a few, these have nothing to do with the mime type. I know how and what a mime type means, and must say I have had resistance to being able to add new ones as well (even if a registered mime type). I am referring to a logical differentiation in particular here, through the use of namespaces, for which we presently only have $:/ system and tiddler.

• What I am proposing, is the ability to add additional namespaces (by prefix) to that which we classify as system.
• Do I take from TLDR that you did not read, my thoughtful efforts above?
• I do not need to spell out the difference between system and (non-system) tiddlers for you I am sure. But surely you can see how there may be value not limiting system tiddlers to a single prefix, of $:/ and giving designers the freedom to use other prefixes.
• Think of system tiddlers as a broader category of tiddlers that are “behind the scenes” part of the bones or operation that we do not need to confuse a Mear user (If any exist) with. Many such tiddlers would appropriately “hide behind system”.
• Unlike current non-system tiddlers, where we are free to use any other prefix we wish but all fit in one large category.

PS Thanks for the link to known mime types, this would be useful in the type documentation.

I can easily list categories of tiddlers that may fall into this category but will only do so if you ask in the spirit of brevity.

• We could, even although I don’t believe we must, limit them to the form something:/ like the current system tiddlers.

[pmario]

@TW_Tones

Do I take from TLDR that you did not read, my thoughtful efforts above?

I did read the whole tread - all posts.

It’s an info, that TLDR it is a summary for my post. I did reply to the text that I did quote in my post and I did ask for a link to the thread, where you asked for is[data], so I can read the replies there too.

@TW_Tones

But surely you can see how there may be value not limiting system tiddlers to a single prefix, of $:/ and giving designers the freedom to use other prefixes.

Sorry, but I do not see an advantage of more than 1 system-prefix. It increases technical and mental complexity.

There is more to system tiddlers than hiding them from most lists by default.

We have an additional differentiation within system tiddler namespaces. Mainly for configurations, infos, to avoid saving or syncing them back to the server.

• $:/state/ … are saved – but not synced
• $:/state/popup … not saved with a single file wiki
• $:/temp/ … not saved – not synced
• $:/status/ … not synced
• $:/config/ … used for configuration tiddlers
• $:/info/ … used for system info generated at startup
• … and may be more

All those named prefixes need special treatment all over the places in the UI and some of them also in the core js-code.

Let’s say we add λ:/ to the “system” namespace.

[is[tiddler]]

[is[tiddler]] will return λ:/state/popup as a tiddler and it will be saved with the wiki. But popup states are annoying if saved.
λ:/temp/ will be saved – $:/temp/ will not – that’s inconsistent … and so on.

• Users will request, that their λ:/ namespace also needs special handling.
• Experienced users will want to know, what is the difference between $:/temp/ and λ:/temp/?
• Is $:/config/ more important than λ:/config/?
  • What if both have the same name and only the λ:/ prefix is different?
• What if λ:/ comes from a plugin? It’s a shadow tiddler now.
  • I can not even think about, what side effects that may have.

So my question is: What’s wrong with eg: $:/wl/.. for my wikilabs system tiddlers. Or $:/temp/wl/my-temp-tiddler if I do not want them to be saved.

We do have namespaces. Those namespaced system tiddlers are predictable and do not need any special handling. They are covered by the existing rules.

Every future core change which touches system tiddlers will be valid for my custom $:/wl/ prefixed tiddlers.

[is[system]]

As you requested (at least I did understand it that way)

[is[system]] will return $:/temp/ and λ:/temp … To separate them for different use I will have to write a filter eg: [is[system]prefix[λ:/]]

But [prefix[λ:/]] is much shorter.

@TW_Tones

[. . .] and giving designers the freedom to use other prefixes.

I think we are already free to have as many prefixes as we need if we use namespaces eg: $:/wl/..

They may be marginally more to type, but as my example above shows [prefix[$:/wl/]] is not bad at all.

just my thoughts.
-m

[TW_Tones]

@pmario thanks for your reply. I am still not sure how you use TLDR

I do think you misunderstand my use cases, I left it to your imagination, but perhaps I should not have.

• Why would I use another prefix for $:/temp $:/state $:/config when these are already functional names spaces that do their job?
• Why would I use other than $:/plugin etc… for plugins?
  • however a plugin may very well contain tiddlers of an alternate prefix if that is what I want as a designer.

All I want is the ability to add a prefix(s) to those automatically caught by the existing operator is[system] or all[system]

• This is a simple hackability request.

@pmario

to separate them for different use I will have to write a filter eg: [is[system]prefix[λ:/]]

• Of course I will do that, when I want to make this differentiation.
• Until then if we nominated λ:/ as a new system prefix it will be treaded as $:/ is throughout the wiki. Not in standard search, excluded from !is[system] list in core or user code…
• Alternative names spaces could be designed for all kind of purposes, but I am not asking for that, I am simply asking can I add others to the “system test”.

Not all implementations of TiddlyWiki, hacks applied or plugins produced need to be published to a “broad and innocent audience”. Being able to make use of alternative system prefixes through an optional hack is no different to someone building a bespoke user interface, that will not work unless it is delivered in an Edition.

Perhaps this is more important to me (than to you) because as an experienced Knowledge/Information Manager namespaces are an important tool for me, an area in which I am interested in innovating. What I see here is a simple change that a designer can decide to use to open a new branch of innovation. It is hard to give you insight to into my creative and systems design approach (a lot if which is deep in me head).

• P.S. If I had more JavaScript skills, I would just do it, and you would not know I had. Instead I have to put a full argument to try and get anyone to take my idea seriously.
• I appreciate the cautions and warnings, fine, but I am trying to get additional hackability so we open the design space, even if the designer using it needs to understand the caveats.
• From my perspective it will have no impacts on anyone but my own wikis when I make use of it, and even if something escaped into the wild I am confident the occasional use of additional system prefixes with have little or no impact or side effects on anyone’s wiki.

Some will ask why my posts are so verbose?

• Because until you spell out exactly what you want, justify it, give examples, explain the mechanism, address critisisums, your request is rarely taken seriously.
• Verbose posts are also “rarely taken seriously” so I am often between a rock and a hardplace.
• It is a Loney place being a “creative, systems thinking, tech savvy person”, its like mixing oil and water.

[TW_Tones]

This has gone no where, and I am dependant on some JavaScript Savey person to help me;

In addition to the tiddlers named in the Original Topic post $:/core/modules/filters/is.js is involved, but in $:/core/modules/wiki.js we see

exports.isSystemTiddler = function(title) {
	return title && title.indexOf("$:/") === 0;
};

Now if a clone and edit as follows;

exports.isSystemTiddler = function(title) {
	return title && title.indexOf("#:/") === 0;
};

Then an is system test recognises #:/ as a system tiddler, but no longer $:/

I would like to know how to add one or more prefixes that are considered as is system and all system.

[CodaCoder]

Try

exports.isSystemTiddler = function(title) {
	return title && ((title.indexOf("$:/") === 0) || (title.indexOf("#:/") === 0));
};

[Scott_Sauyet]

I think @CodaCoder’s approach is simple and clear. If you want to expand the system namespace, you might override $:/core/modules/filters/is/system.js with your version, or, uglier, but possible, override $:/core/modules/wiki.js to update isSystemTiddler.

That’s a pure override. It’s worth considering whether TW should offer an extension point. We can easily envision a mechanism, just tag some tiddlers with $:/tags/SystemPrefix, and give them bodies like $:/ (the default one), or #:/ or λ:/. The code to handle this would be simple enough.

If on the other hand, we wanted to just create custom namespaces available to is, that flexibility is already built in. We can just write a module like this:

/*\
title: $:/core/modules/filters/is/lambda.js
type: application/javascript
module-type: isfilteroperator

Filter function for [is[lambda]]

\*/
(function(){

	/*jslint node: true, browser: true */
	/*global $tw: false */
	"use strict";
	
	/*
	Export our filter function
	*/
	exports.lambda = function(source,prefix,options) {
		var results = [];
		if(prefix === "!") {
			source(function(tiddler,title) {
				if(title.indexOf("λ:/") !==0 ) {
					results.push(title);
				}
			});
		} else {
			source(function(tiddler,title) {
				if(title.indexOf("λ:/") === 0) {
					results.push(title);
				}
			});
		}
		return results;
	};
	
})();

and use it like this:

<<list-links filter:"[is[lambda]]">>

You can try this by downloading this and dragging it on a wiki:

lambda.json (1.3 KB)

You will have to save the wiki for this JS module to take effect, but then if you load Lambda Test, you will see it in action.

[TW_Tones]

@Scott_Sauyet

It’s worth considering whether TW should offer an extension point.

I agree, and your example of the is[lamda] is in addition to my request that would allow, making λ:/ test true when asking is[system] or false with !is[system],

• As you suggest this is is[lamda] is already hackable.
• Your [is[lambda]] function is a type not already in the core, so becomes a customisation, and will not interfere, but we can test for its prefix, although its useful to give it another name.

Similarly [all[lambda]] can be achieved with [all[]prefix[λ:/]] and if using the earlier method to including it in the system test, [all[system]prefix[λ:/]]

Much is gained by just being able to include or exclude a prefix from those considered is system.

• This would be the first thing to make hackable, but not allow $:/ to be removed.

I tried to ask for this some time ago and got push back, but its designer beware, and could be hackable, although already is in a sense.

[CodaCoder]

@TW_Tones

I tried to ask for this some time ago and got push back

I think you’re referring to @pmario’s points above – which are wise words pointing out the depth of cognition needed to fully understand the problem domain.

What isn’t clear to me is, why do you need an alternate that merely using prefix[#:/] and !prefix[#:/] doesn’t solve right now? IOW, what does a notional is[system] extension give you that can’t be done with a simple prefix query? I’m obviously missing something…

[TW_Tones]

@CodaCoder thanks for you JS code, from my perspecive that was what I was looking for, for now, a manual hack that provides an oportunity to use another prefix treated as a system prefix. As @Scott_Sauyet points out perhaps it could be a configuration option?.

@CodaCoder

I think you’re referring to @pmario’s points above – which are wise words pointing out the depth of cognition needed to fully understand the problem domain.

Not in this case, @pmario was mistaken on this occasion thinking I wanted to go way beyond the Original Topic;

@pmario

There is more to system tiddlers than hiding them from most lists by default.

We have an additional differentiation within system tiddler namespaces. Mainly for configurations, infos, to avoid saving or syncing them back to the server.

I understand what @pmario was refering to and, originaly and subsequently made clear I was not planing to reproduce these complex behavours of system tiddler prefixes such as $:/temp $:/state etc… because they are fit for purpose. No need to reinvent the wheel.

I am exploring a number of ideas that relate to virtual tiddlers and fields and these are in aid of new innovative and simple ways to get results from tiddlywiki. To do this I did not want to further complicate the system name space or need to use increasingly long prefixes. I just wanted access to one or more additional system prefixes.

• When exploring such ideas we are running with a hunch and the hack you helped me with helps me explore that hunch.

Anyone with a detailed understanding of tiddlywiki, which I have, but always learing, is aware of the code pattern prefix/tiddlername such as is used for button visibility, subtiddlers and more. I have perhaps a dozen areas I am exploring that would use a similar code pattern and it thus became clear other system prefix’s could reduce title lengths, and be clearer to the user.

• I am happy to spell out some of my ideas and hunches if asked, but I do not want to “presume an interest”. They are after all “creative work in progress” and somewhat speculative.
• If I design something of value I will only then put an argument for inclusion in future versions of tiddlywiki.

@CodaCoder

What isn’t clear to me is, why do you need an alternate that merely using prefix[#:/] and !prefix[#:/] doesn’t solve right now? IOW, what does a notional is[system] extension give you that can’t be done with a simple prefix query? I’m obviously missing something…

To answer more directly. I will want to use prefix[#:/] and !prefix[#:/] I just dont want to have to alter every bit of code that currently uses is[system] or !is[system]

[TW_Tones]

If tiddlywiki was new today, I think there could be an argument that tiddlers with the prefix ?/: where ? is any character perhaps even string is how one defines a system tiddler. Its not an unresonable idea, and simply extends the namespaces concidered system but interfears little with the unprefixed namespace often used by designers and users.

More…

A great deal of value can now be found in functions especially when wanting to make an “is test” or “all test”.

• In these example I am not testing the “is system”, which is separate.

\function is.lambda() [prefix[λ:/]]

<$list filter="[is.lambda[]]">

</$list>

• However the above acts as all.lambda as well unless you limit the titles eg;

{{{ [all[current]is.lamda[]then[current is lambda]] }}}

An interesting behaviour change we get by adding a new system prefix, using the above method is it escapes linkification, we could look at fixing this but then that may have advantages in itself.

[TW_Tones]

@TW_Tones

I think there could be an argument that tiddlers with the prefix ?/: where ? is any character perhaps even string is how one defines a system tiddler.

While compiling a solution for wiki interlinking I realised the problem with $/: is its ugly in urls. So an add on namespace that is not an ugly url would be nice for the link use case. One that comes to mind is a character that is not encoded like ~ OR _

• anyone see a problem with a namespace prefix of only one specific character?

I would like to use _templatenames and ~shorttitle

[pmario]

@TW_Tones

I would like to use _templatenames and ~shorttitle

I think both of them cannot be used.

• ~ has a meaining in wiki-link formatting already. It supresses ~WikiLinks from being rendered as links
• _ is a common prefix for “internal” stuff eg: _internal-var, _work-in-progress tiddler and so on.

We cannot take that away from users. There is a reason, why system-tiddlers are prefixed $:/

[TW_Tones]

@pmario thanks for your feedback,

@pmario

I think both of them cannot be used.

• To be sure, they can be used, whether we should or not may be the question.

Please keep in mind,

• At present this is a solution I am building into my own wiki, that I may at most publish for people to use, not to stand as an edition or template.
• No core tiddlers are currently prefixed ~ or _ using these as titles should have no effect elsewhere, using them as variables or in wikitext (except to transclude them) is not intended and would not effect them anyway
• By defining how these tiddlers are to be used I am actually constraining how they will be used and there is unlikely to be any clashes.

@pmario

We cannot take that away from users. There is a reason, why system-tiddlers are prefixed $:/

• I am not taking anything away from anyone, I am giving a lot of opportunities to myself.
• Of course these additional name spaces may become a prerequisite to other features I develop but this is not different to having a plugin as a prerequisite.
• Feel free to tell me the reasons behind $:/ however I am quite comfortable with that and I don’t change it anyway.

Just to be clear, we can currently name tiddlers freely, we can choose naming standards as desired already. ~Hello and _hello even ㎱Hello, 👻Hello and 🗀hello and as designers allowed to treat this as we wish. There are no prohibited titles here.

• The only change I have made is to add the tiddlers with these prefixes to a set, along with $:/ that treats them as system tiddlers.
• Given the functionality I plan for these tiddlers now in the system namespace, they are unlikely to be freely referenced in wikitext (yes in tiddlywiki script) so the connection between the use of ~ to suppress camelcase links (now turned off by default) and with the use of underscore is unlikely and has workarounds.

I can share a wiki with these implemented, if you want to try and break things?