Retrieve all associated (double-chained) paths that exist between two arbitrary tiddlers by double-linking

#21

Then the code I shared above is a very good start. It would have to be updated for links versus tabs, and both inputs would be the full list of all (non-system?) tiddlers.

We can do this for a small list (update: fixed this link!) very easily. But that list has 11 entries with an average number of links of 1.18. That yields 44 paths. If we were to try this on tiddlywiki.com with its 1621 non-system/non-shadow tiddlers with an average number of links of 2.12, I’m guessing it would yield, at a minimum, tens of millions of paths.

If you wanted to select a start tiddler, an end tiddler, or both, this seems feasible. But listing all paths for any medium to large wiki seems unlikely. And it would get tremendously worse if we tried to deal with soft links as well.

2 Likes
#22

A good example already built into tiddlywiki is the Table of Contents, see tiddlywiki.com. This starts are the tiddler TableOfContents and follows all tiddlers that use the current tiddler as a tag.

  • This is a hierarchical structure based on tags.

If you are going to automate this as in the chain of links you followed from HelloThere > TiddlyWiki > WikiText > Markdown > Macros are only a very small subset of links and paths available.

  • Note these links may or may not be in the table of contents, they come from the content of each tiddler you visit.

What I think you want is an equivalent to the TOC based on tags, but for links displayed in each tiddler?

  • Is this correct ?

For example;

Hello there

TiddlyWiki

WikiText

1 Like
#23

Nice, this is one step closer to the ultimate goal, it can list all the double-linked links under the current note, maybe it is not practical to traverse all the paths of the knowledge base in pure wiki syntax, but we can export an index list of the outbound links of each tiddlers, and maybe use python to do a deep walkthrough

#24

I created a tiddler that uses the current wiki to generate JS code you can paste into a JS environment (Node.js, the browser console, some online REPL [such as the one I use], or wherever.)

Download this link: Build List.json (1.3 KB), import it into your wiki, copy the JS code it renders, and run it in some JS environment. Note that almost certainly the number of tiddlers on tiddlywiki.com, especially with the number of links per tiddler, will almost certainly be too large to run in any reasonable amount of time.

For example, I chose, mostly randomly, the wiki https://maths.tiddlyhost.com/, which has 148 regular tiddlers with around 0.37 links each. I dragged that tiddler onto the wiki, imported it, copied the resulting code onto a REPL, and in the output received 464 paths:

"Bit-square theorem → Nonstandard Analysis";
"Bit-square theorem → Nonstandard Analysis → First fundamental theorem";
"Bit-square theorem → Nonstandard Analysis → Fundamental theorem of set theory";
"Bit-square theorem → Nonstandard Analysis → Linear Programming";
"Bit-square theorem → Nonstandard Analysis → Linear Programming → Set Theory";
"Bit-square theorem → Nonstandard Analysis → Linear Programming → Topology";
"Bit-square theorem → Nonstandard Analysis → Linear Programming → Topology → Set Theory";

...

"Theoretical Informatics → Set Theory → Nonstandard Analysis → Topology";
"Topology → Set Theory";
"Topology → Set Theory → Nonstandard Analysis";
"Topology → Set Theory → Nonstandard Analysis → First fundamental theorem";
"Topology → Set Theory → Nonstandard Analysis → Fundamental theorem of set theory";
"Topology → Set Theory → Nonstandard Analysis → Linear Programming";
"Topology → Set Theory → Nonstandard Analysis → Number Theory";
"Topology → Set Theory → Nonstandard Analysis → Second fundamental theorem";

(Note that I added spaces around the arrows, because it looks better to me. You can remove them in the last line of that tiddler’s source code.)

If this looks useful, we can investigate how to turn it into a widget/macro/procedure you can use to show these directly on your wiki.

1 Like
#25

The code is cracking. It’s great!Here I drag and drop this code to a Chinese tiddlywiki forum, and it looks like this, the code works fine, but there is no “->” thing

image
image1920×992 145 KB

#26

That result is just JavaScript source code. You would have to run it in a JavaScript engine. See the directions in my last post.

This is not an overall solution to your request, only a step along the way. If it does what you would like, we could look into turning it into some sort of TiddlyWiki module.

There are three concerns:

  • Is the output—after you run that JS module—what you’re looking for?

  • Do you need more control over the input? Right now it finds all paths between any two tiddlers, ignoring loops. But we might want to find only those paths starting from a particular tiddler, all those ending at a particular tiddler, or both. We might also want to include only a subset of the tiddlers by supplying it a filter.

  • There is a terrible inefficiency in the code. We can probably fix that without too much difficulty, but it’s not clear if that is worth doing. As mentioned above, a relatively small wiki with little interconnectivity ended up with 464 paths. I didn’t think this would usefully scale up to something like tiddlywiki.com, with 1600+ tiddlers and many more links per tiddler. So I think you would need to investigate whether it is going to be useful for the size wiki you want to work with.

1 Like
#27

This is what I need

I want to handle it in python

The expression is similar to that of the knowledge graph in obsidian in the double link path search plug-in

image
image1920×1026 26.9 KB

#28

If all you want is the raw data that you can use in Python, this will generate it in JSON format:

\define links() <$text text={{{ [{!!title}links[]!is[system]] :map[search-replace:string["],[\"]addprefix["]addsuffix["]] +[join[, ]] }}}/>
\define row() {"title": "<$text text={{{ [{!!title}search-replace:string["],[\"]] }}}/>", "links": [<<links>>]}
<pre><code>[
  <$list filter="[all[tiddlers]!is[system]] -[[Build List]]" counter="counter"><<row>><$text text={{{ [<counter-last>match[yes]then[]else[,
  ]] }}} /></$list>
]</code></pre>

List Links.json (626 Bytes)

(Yes, I know that I could replace the counter-last stuff with join in the <$list...>, but I was testing this against some 5.3.1 wikis.)

Let us know how it goes!

1 Like
#29

I just want to point out there are a range of methods to “walk” hierarchies and networks including the table of contents, the TOCP plugin, The Kin operator and the recent taggingtree and tagstree operators.

Although you can build anything similar using a recursive macro or procedure call without javascript. Search here for “recursion”

1 Like
#31

I woke up this morning bothered by the design of the code I shared. I don’t think anyone is looking for it anymore, but I had to clean it up. The earlier version was an attempt to alter other code I’d written for a different purpose and it shows. This code is dedicated to the problem.

Here’s the latest version of the tiddler which creates code from your wiki to find all paths between tiddlers via their links. As before, it simply generates JavaScript code which you could run in another environment:

Create Paths.json (1.2 KB)

Don’t try this on tiddlywiki.com. It will only work for much smaller wikis.

I actually wanted to try it and see if I could run it for main site. I failed miserably. My first attempt ran out of memory after printing something over 500,000 lines; I changed code to output the results as it went, and I ran it with extra memory. It went longer but still crashed after writing out a 48GB file! I have no idea how many paths it contained.

I don’t think anything trying to work with all such paths will work for a larger wiki. But it still might be interesting to find all the paths between two nodes, which would only take a small tweak of my earlier version.

1 Like