Yes, It would be worth looking to see if there are other cases that exclude missing tiddler titles, even standard search?, perhaps a checkbox option?
I wonder if the ability to filter the ctrl-L to only list according to a selectable filter eg prefix[#] or prefix[@] or clone it for multiple filtered lookup. However if you add search tabs they are visible within ctrl-L, this could be the way to introduce access to missing tiddlers but they will not be found amongst the existing titles.
Yes the more recent design approach of just using titles, could be configured to make other prefixes like @person a trivial addition. The thing is we can draw from this that a #tag is a form of tag typically (but not exclusively) within content, if you open the #tag you see details about where used, similarly this would also be the case for @person tags,
The designer would need to choose if the @person tiddler is also a contact tiddler or if some kind of link or redirect to the actual contact tiddler is needed. See further innovation.
I expect this would be wiki or content related such as wikis with too many tags or content needing in place tagging.
A de facto standard emerging ?
Further innovation
Given the example of an @person or “at tag” makes me ask could such @person (content tags) actually be an alias to another tiddler such as “Firstname surname”. The @person tiddler may exist but what if it looks for the “Firstname surname” tiddler (perhaps with a field or traditional tag of @person).
This idea of tags and or #hashtags or @tags being aliases for other titles is one I would like to explore.
Perhaps we have a content-tags field that may contain #tag and/or @tags (list) to indicate that tiddler pertains directly to the named content tags eg [[Firstname surname]] has content-tags="@person"
Can we find away on use of an @person content tag to “request” assignment to another person tiddler?
Let us not forget using [[Firstname surname]] directly.
If you want to search for a # or @ tag or contact tag (and others a user may use) you have to enter more characters or add two spaces, what if such characters triggered an immediate search rather that wait for the configured minimum of 3 characters to start search?
I have identified the following places we may want to add support for content tags like #tags and @tags.
ViewTemplate on tiddlers that ARE content tags, that allow listing of tiddlers containing that content tag
ViewTemplate on tiddlers that CONTAIN content tags, that allow listing of tiddlers containing that content tag
Sidebar more tab to list all content tags in the wiki, one tab for each content tag
Option to show on sidebar tabs for quicker access
Advanced search tab that lists each content tag meeting search criteria
Advanced Search filter tiddler for searching in Advanced search filter tab
Can you think of any other places we need to provide support for such content tags?
Is it sufficient to package static tiddlers for each content tag that a user installs or would it be worth giving a tool to configure and generate additional content tags (driven by tiddler prefixes)
Eg #hashtag @persontag can also be a place/address tag eg @home
Scott, Yes now days it is good to use functions and also make them available for use as desired.
Here is a working example but only focused on the text field as I am using links and backlinks.
My feeling is if we move into fieldnames as well we may have a separate system for categories, subjects or keywords instead of hash tags which I am defining as “content tags”
\define hashtag-symbol() #
<!-- The prefix symbol we are using, clone and change? -->
\define hashtag-limit-prefix() ##
<!-- eliminate titles with more than one # prefix like '##' leaving such forms for other uses -->
\function hashtag.prefix() [prefix<hashtag-symbol>!prefix<hashtag-limit-prefix>]
<!-- combines hashtag-symbol and hashtag-limit-prefix into the function hashtag.prefix -->
\function all.hashtags() [all[shadows+tiddlers+missing]hashtag.prefix[]sort[]]
<!-- 'all.hashtags' found in wiki and name sorted -->
\function is.hashtag() [hashtag.prefix[]]
<!-- 'is.hashtag' is the current tiddler a hashtag? if not don't return current title-->
\function hashtags.here() [links[]hashtag.prefix[]]
<!-- return the hashtags in the text of the current title -->
\function find.hashtag() [backlinks[]]
<!-- find tiddlers containing the currentTiddler as a hash tag, must start with hashtag.prefix unsorted -->
# hashtag-symbol <<hashtag-symbol>>
# hashtag-limit-prefix <<hashtag-limit-prefix>>
# hashtag.prefix[] {{{ [[#test]hashtag.prefix[]] }}} not {{{ [[@test]hashtag.prefix[]] }}}
# is.hashtag[], title is.hashtag return title {{{ [[#test]is.hashtag[]] }}}
# all.hashtags[] {{{ [all.hashtags[]] }}}
# hashtags.here[] {{{ [all[current]hashtags.here[]] }}}
# find.hashtag[] #test {{{ [[#test]find.hashtag[]] }}} can just use backlinks of the hashtag title
minor edit applied to is.hashtag[]
Edit notes
If I gave up on the hashtag-limit-prefix we could create a contentTag with the prefix defaulting to # but allow any other prefix to be used. what about textTag, bodyTag, userTag where we may use userTag[] defaults to userTag[#] or userTag[@]
Here is an example of a simpler userTag because we provide a default prefix.
This method could permit provision of the fieldname as a second parameter but we would need alternative to links[] and backlinks[] as they default to text.
Noting here this would be a quick way to iterate tags eg [links[tags]] and [backlinks[tags]] and other list fields we expect to contain titles (needs further thought).
\function all.userTags(prefix:"#") [all[shadows+tiddlers+missing]prefix<prefix>sort[]]
<!-- 'all.userTags' found in wiki and name sorted -->
\function is.userTag(prefix:"#") [prefix<prefix>]
<!-- 'is.userTag' is the current tiddler a userTag? defaulting to # if not don't return current title-->
\function userTags.here(prefix:"#") [links[]prefix<prefix>]
<!-- return the userTags in the text of the current title -->
\function find.userTag(prefix:"#") [backlinks[]prefix<prefix>]
<!-- find tiddlers containing the currentTiddler as a userTag, must start with prefix unsorted -->
# is.userTag, title is.userTag return title '{{{ [[#test]is.userTag[]] }}}'
# all.userTags {{{ [all.userTags[]] }}}
# userTags.here {{{ [all[current]userTags.here[]] }}}
# find.userTag #test {{{ [[#test]find.userTag[]] }}} can just use backlinks of the userTag title
;Using @
# all.userTags[@] {{{ [all.userTags[@]] }}}
# userTags.here[@] {{{ [all[current]userTags.here[@]] }}}
# find.userTag[@] {{{ [[@contact name]find.userTag[@]] }}} can just use backlinks of the userTag title
find.userTag for @contact name currently not working
Note;
Each time I write something for tiddlywiki I often see even simpler and more general ways to implement it.
Generalisation 1
In this case the idea would be to generalise user tags, or content tags to any title links in text by default missing or otherwise beginning with a single character found in a list, this would allow viewTemplates etc… to be designed to handle any one of the titles with these prefixes. And perhaps allowing * for all fields, It would leave it to the user which and how they are used.
Generalisation 2
Dump the single character prefixes and let any named prefix to be used
Parameter to be used to name the fieldname for links[] and backlinks[] ?
@pmario perhaps this is another Github request to allow the parameter to name the fieldname for links[] and backlinks[] in which to search for titles, defaulting to text of course. Along with;
One of my issues with this proposal is that it would greatly privilege tags as a data structure. Tags currently have special tools (e.g. a bespoke filter operator, built-in tag manager) to work with them, but for the most part, tags are just another field. Tags would get a meaning within field values, elevating them to the level of wikilinks – which are a more fundamental structure than tags are.
So I think it would muddle things if hashtags were in the core, but I can see this being a popular plugin.
For my part, I would prefer and actually use an interactive mode proposal like this:
The feature to add [[#tags]] entered in the text to become titles, is not unlike what is available through the tags pill. Although I personally would like a form of tag that does not fill the tags field on tiddlers, but can mostly be used in similar ways, like testing if they are in the text, listing them including for selection, and listing the tiddlers they tag.
I am the designer of reimagin tags which extends the tag drop down a lot, So I thought of extending hashtags further, but there is something simple and deterministic about the current title driven process.
We could always create a user interface element to add any hashtag as a tag on the tiddler automagicaly or manually, or via its insertion via an editorToolbar button, but for now at least I don’t think it necessary.
You may also find another organising method flags of interest. In effect no-touch tags.
I don’t think we should be concerned with more options and features for “marking” or organising tiddlers. We will just use what we need when we need it. As I am writing a book this idea of hashtags or content tags appeals because it can be used to mark my content, in tiddler, its more specific.
We do start with plugins, but if it was ever in the core it could require a config item be set, to appear or even just using [[#tagsname]] and it becomes visible.
That’s the point. tags are already “special” in every way. Technically they are not a tiddler field. JS internally, they are an array. The same is true for the list field.
If tags are listed, they use the list-field of a tag-tiddler to provide a custom sort order. There is no other field, that can do that.
Every other field that contains a Title list internally is a string, with special formatting to allow abc [[title with spaces]]. So internally there is a fundamental difference, between tags, list and any other field.
For every tag in the system, the core manages indexes. So if we want to know, which tiddlers are tagged eg: HelloThere, the access is very fast, since the index knows that tiddler list already.
Let’s assume we have 3 tiddlers a, b and c, which contain different “tag-like” titles in my-field
title: a
my-field: a [[a a]]
+
title: b
my-field: b
+
title: c
my-field: a
A) The filter to list tiddlers that contain “a” in my-field would need to look as follows.
List all tiddlers that have the “tag-like” field … has<field>
This functionality needs to read every tiddler in the wiki
This can become a problem if we have 10000++ tiddlers
Make sure we do not list draft tiddlers … !is[draft]
Filter the results :filter[...]
In the filter read my-field … get<field>,
which returns the whole title-list string
Separate the title-list string into a real list we can iterate over … enlist-input[]
Match the title list against the “search string” … match<tagLike>
Return the tiddler title only if it matches
join the found elements with a coma +[join[, ]]
Example B)
Get tiddlers with tag … tag<tag>
This function directly reads form the index, which is automatically updated, whenever a tiddler changes
Make sure we do not list draft tiddlers … !is[draft]
join the found elements with a coma +[join[, ]]
I think it is clear that there is a huge performance benefit. …
Conclusion Tag-likes
So to make “tag-like” titles fast, if they are in fields, we would need a more generalised indexing system.
It would need to allow us to speed up the field:my-field[] operator in a way, that is works like a tag.
May be field:my-field:tag-like<tagLike>
Disadvantage: Every tag-like field would need quite some memory, to store the index.
That’s a completely different part of the core code. links[] and backlinks[] are also very expensive.
To find links and backlinks the whole tiddler text needs to be parsed.
That’s the most expensive functionality we have in TW
So the parse-tree is cached independently from filters
2 indices need to be created one for links, that other for backlinks
They are created “on demand” when they are needed the first time
Once created, they are cached
But these caches is completely destroyed frequently, so it still can be work intensive
Conclusion links, backlinks
IMO these two functionalities have room for improvement.
If you use [[#tag]] this mechanisms are automatically active.
They are not active for tag-like values stored in eg: my-field
I would not use the existing link, backlink mechanisms here
I would use a trie like index, similar to the one, I am using for my “aliases” and “backaliases” handling in my uni-link plugin.
My trie-structure is similar to the Aho-Corasick functionality that is used now for the freelink-plugin. (Wich is optimised to handle “non Latin” character sets)
Those 2 trie-indexes could probably be used to create a more generic mechanism that could also be used for very performant tag-like field link and backlink handling.
It would probably also be able to handle the existing tag indexes.
I am not sure if it could replace the existing links and backlinks indexes. A lot of experimenting would be needed.
I just stumbled upon the listed-operator, which uses a global-cache. So it is cached … But – The main problem here is that the global cache is destroyed with every UI interaction.
I’ll update my post accordingly soon. Need to dig a bit deeper.
Please take note I reopened this thread to implement hashtags which I could also call content tags as they are placed within the text field. My proposal is to simply use titles as links [[#tagname]] and make use of existing mechanisms. I have demonstrated a number of custom functions to make handling the # and other title prefixes easier to use.
This is quite simple yet functional and does not involve turning them into regular or alternative field tags. yet it’s quite easy to locate tiddlers containing them, if the current tiddler has one or more, or test if it contains a particular one. you can open any content tag and see where it is mentioned.
as Mario points out this leverages existing optimisations
So I would appreciate some feedback on this approach specificaly before going too far in another direction and if you are give a reason by comparing with what I proposed.
post script
there is no real need to turn my proposed content tags into regular tags, I am not sure what that would add, just reduce their specificity.
Mario’s comment is a technical performance review, specifically in the context of other fields, that is something we may need to contend with, however out of the box we are not trying to use fields beyond the text field, @pmario as responding to using alternative fields, I am explicitly excluding them in my current solution.
When dealing with [[#hashtags]] I quickly reduce the results of links[] or backlinks[] with prefix[#]
I believe the majority of activity relating to hashtags may only be on the current tiddler, as it is focused on “content tags”.
Something else that is often forgotten is many uses of links and tags for that matter are “on demand” only. My example hashtags are just titles, ie links and leverage the caching and optimisations in the core, and only need to be iterated, when, for example one is looking for all instances of that hashtag. In a similar way when testing for hashtags within the current tiddler you are only referencing the current tiddler.
In many ways this solution benefits from the nature, structure and history of performance improvements in TiddlyWiki.
If someone can demonstrate a performance hit using this approach, we can look at that when it happens. However I would be surprised.
Other approaches I did not follow
I was looking at the parsing, and $link widget, to code features for hashtags, and may do so in the future, however I realised I can satisfy 90% of the requirements without doing so. I could also take the macro route such as <<# hashtag>> but then I need to search for these throughout the wiki.
The macro model could permit setting anchors for navigation, turning off display (for printing) and other dynamic features.
Finally this could also be implemented through template transclusion like {{#hashtag||_userTag}} where user tag provides a link with more button features.
Note: I am looking at adopting the prefix _ for transclusion-template tiddlers
That’s too late. To list all links every tiddler has to be parsed and the parse-tree has to be searched for a link-widget call. Performance here is OKish.
Backlinks are the culprit as we know from many discussions here in the forum. The algorithm to find backlinks needs to search every tag in every tiddler, if it is linked there.
As I wrote, there is room for improvement. The different indexers have been created organically, when we experienced performance problems. So they are implemented according to the distinct issue that needed to be solved. So they are all slightly different.
IMO we should create a more generic functionality now, that we know what actually causes performance issues. – But that’s a lot of work.
But it is true that this performance requirement is only when trying to find all occurrences, and this typically needs a particular tiddler to be visible. ie there is no background performance issue unless say you leave a sidebar tab open that iterated a hashtag.
As @pmario said, this is true of links[] but not of backlinks[], which you were proposing to use…
I’m not sure how you could avoid this issue unless you sacrifice this functionality… and IMO, hashtags don’t offer much benefit if you can’t find all occurrences of a given hashtag in a given set of tiddlers. which (in your proposed solution) would require backlinks, search:text, or similar operators. So the performance issue seems inescapable without separate improvements to backlinks indexing. Thus…
I did understand your intent. What I was implying (and should perhaps have stated more directly) in my previous post was that others are proposing field-based alternatives because they see the performance issues currently inherent to backlinks[] as a substantial barrier to adopting your approach.
Edit: Unless you’re saying that functions like your find.userTag would only incur performance costs when viewing a tiddler (or template) where they were used? That’s technically true, but strikes me as a strong incentive not to use them…
This is a common use pattern, keeping the need to do something in the context in which is is needed and not globalising it.
Why does it?
strike YOU as a strong incentive not to use them
I don’t follow
Perhaps I misread this, @pmario can help us here. I believe his reply was in the context of links and backlinks within additional fields, as I suggested giving links and backlinks a fieldname as a parameter. In turn I was responding to other users raising that. I am not wedded or interested in that now.
No I was not saying that, find user tag is when you are searching for the occurrence of the use of that tag and all.userTags when trying to find all of them. This is when you are trying to collate them, I don’t envision this to be a common need. But I would put it on the hashtag tiddler, like on a tag tiddler, automatically if needed, if title has prefix.
userTags.here is similar to links, and the tags[] operator but only returns those links[] within the currentTiddler tiddler body that have the nominated prefix eg #. We could use this in a viewTemplate to find, and perhaps test for the presence of a particular content tag, then if it exists do something. I am not suggesting we need to, only that we can, just as we can with tags.
The key value I see is allowing a form of annotation within a tiddler body with view and edit modes, something you care about mostly when viewing/or editing that specific tiddler. For example an author I may want to use [[#expand]] to indicate I need to add more here, or [[#reword]]. I expect once done I delete them.
Additional tips,
if these hashtags are in content to be posted to X or something you can use ~ or hide links within the output template.
You can hide the visibility using prettylinks such as [[some in place text|#tag]] but still find the tiddler and list the #tag. [[some text to reword|#reword]] ctrl-L is really good for applying this.
Mario can clarify, but I understood his comments re: links and backlinks to be reflective of their current status (and potential performance impact) — i.e., backlinks[] used to retrieve links from text fields (only).
If nothing else, his technical overview aligns very closely with my personal experiences using backlinks in large wikis (actually for a very similar purpose to the hashtag approach you’ve outlined here). I concluded that, while link syntax is certainly easy to type, collating backlinks from even a subset of a large, highly linked wiki was so slow as to be virtually unusable. I ultimately switched to a system much like this one…
… where all text-field internal [[links]] that matched a certain filter pattern would be automatically converted to list-items in a nominated field whenever I saved the tiddler.
I realize this is exactly what you didn’t want to hear, but it does work and it’s reasonably performant for all the filtering I’d want to do (equivalents of userTags.here, find.userTag, and so on) whereas the links-based filtering really was not.
If it’s mostly maintaining a separate field that you object to, I suppose you could, if you liked, also hack the $link widget to include an inline x button — something like [[#hashtag]] → #hashtag [x], which you could click in view or preview mode to automatically remove the tag from the field.
To a few of your other points…
Really? This is pretty much the primary (only?) function of hashtags as I’ve seen them used: to make it easy to collate and browse posts on a particular topic.
This isn’t a use that would ever have occurred to me, but I can see the utility of having your annotations pre-styled in an eye-catching format. But surely you can do this already, without any supporting functions or ViewTemplates, just by typing hashtags as missing links?
I suppose I’d gathered from the example functions you gave earlier that you were mostly interested in using in-text hashtags to collate content to be presented elsewhere — either on the same tiddler or on a separate hashtag tiddler, backlinks-style. If this isn’t your end-goal, then you’re right: you wouldn’t have to worry about backlinks performance.
On the other hand, if hashtag collation is part of the intended package (and IMO, most people would expect this sort of feature)…
Because, as discussed above, it will be very slow.
This is tapping into the cached list of titles not backlinks.
In the first wiki in which I want to use it, there are not many tags on tiddlers, other than those that add it to the contents. So If I used the above approach to add them to the tags field would be fine.
The solution I was building had another objective, to avoid overuse of tags by introducing another method, as my flags (no-touch tags) solution also does. In my case I expect a few hashtags to be used, they will be added to content needing work, then when the work is done removed, So I dont think the number will accrue too much.
Mario details the issue with non-tags fields but suggests the expensive list and backlinks is already optimised against the text field, although has its limitations. He mentions;
So we have room to innovate.
[Edited]
Perhaps there is value using the very limitations we fact to define different functions, such as local #hashtags (just links) for listing and consumption in a tiddlers text.
List these #thashtags found using links[]prefix[#] on the current tiddler in the story. Whilst we would need backlinks to find these, we don’t. Instead allow hashtags to be globalised with a click, by adding it to the tags. It thus becomes visible as tags[]prefix[#] ie we list only hash tags that are also tags.
: 
