In #5700 I introduced the first pre-alpha version of my Nearby plugin. The discussion wandered in many directions – very usefully, I might add.
A new thread was suggested in order to keep things clean. Here it is, started because I want to announce – after a few days away from this – the next minor update, 0.4.0 is released, still pre-alpha. (All versions, including this one are linked in the versions page.)
I’m finally ready to start extending this to a shortest path calculation, and I think that’s up next.
Hi, I am reviewing this on a copy of tiddlywiki.com to explore the value and application. A Few points that may be of interest
On tiddlywiki.com with large tiddlers I have set on $:/plugins/ScottSauyet/Nearby/ViewTemplate list-after to $:/core/ui/ViewTemplate/subtitle so its visible at the top.
I see it remembers if open on all tiddlers, which helps but may want an option for a per tiddler basis like Hidden Setting: Show Edit Preview per Tiddler.
Since the link is provided for each resulting tiddler
perhaps it could retrieve the description as a tooltip if present?
Although even better perhaps a rating or indicator how it came to be on the list.
I tested ctrl-click on the title and it works as expected “opened below without navigating to it”.
I notice on HelloThere that reason for the relationships sometimes appear because HelloThere was used as an example. This raises some thoughts. What guidelines could we give authors to
encourage the tiddler being listed as nearby on a named tiddler
Reference a tiddler but do not include in nearby, or exclude this tiddler from nearby calculations.
[Edited] System tiddlers are not included in Nearby
I will continue to investigate but love your work
Post Script:
My tools for building a network of tiddlers (rather than heirachies) so as to see how they are evaluated by the “Nearby Neighbors Plugin” is still in progress and in part dependant on Exploring default tiddler links hackability in V5.3.0 - #20 by TW_Tones which will allow a rapid composition of networks.
Post Post Script:
I just realised “Nearby Neighbors” is a redundant name as Neighbors are those nearby. Given Neighbors has ugly and differing spelling, perhaps this could just be called the “nearby plugin”.
Please remember that I think of the current UI as disposable, as I move to a more graphical view. It might be worth offering multiple views for different uses. We’ll see.
If we were to offer this as a configuration option, would you suggest anything more than simply allowing users to do what you’ve done, or dragging around the entries in the $:/tags/ViewTemplate pill dropdown? At the moment, I can’t come up with any clean way to do so. But that would not prevent users from doing such things themselves.
I’ve done nothing in particular to make this happen; it’s just how the quick implementation worked. And I’m not quite clear what you’re suggesting for the hidden setting.
That sounds like it might be a good idea, if I even stick with this UI. I haven’t learned how to do that, so if nothing else, it might be a good learning exercise. But again, I’m thinking of this UI as mostly a temporary solution until I can figure out a graphical view.
If I stick with this UI, then I will investigate how to do that. I asked in an earlier topic about ways of combining data (here the rating used to sort and some sort of provenance for the rating) and I never really figured it out. Perhaps I’ve improved enough since that I could figure it out, but it’s not quite clear to me yet.
And of course, again, this UI is not my end-goal.
I guess I never documented this, but for now any tiddler tagged $:/tags/SkipNearby will not show the footer (possibly header, I guess!). This does not prevent it from being included as near to other tiddlers.
As to encouraging authors to do something in particular, remember that the goal is to use this for implicit understanding. Authors already have many ways to make relationships explicit. Think of this as early days of Google’s PageRank, before the millions of SEO attempts to hack around it for higher ratings. It was a simple enough idea. PageRank uses the structure of the Web to determine the relative importance of web pages. Nearby uses the structure of the wiki to determine the relative proximity of tiddlers.
… or will, if I ever get around to the hard parts. (I did start on the shortest path work last night. We’ll see how it goes.)
Yes, and the name everywhere except in my own discussions is Nearby. I think they’re both in my mouth all the time because I vacillated at first over which one was the better name. So now, I seem to have gone for the belt and suspenders approach. I’ll try to stop using “Neighbors”. It is redundantly duplicative.
I am following this with interest although I don’t think I will be able to use it.
Regards the name - is the concept of nearest in a sense an outcome of the algorithmic approach? In everyday terms is “related” a more accurate view for the end user? Specialists work in many different abstract ‘spaces’ of course.
The reason I don’t think I will be able to use it is the run time sensitivity to the number of tiddlers and also the most fertile ground for correlation in my case would be the main text - possibly guided by a dictionary of keywords so that matching is/can be discipline specific and the ‘glue’ words of language don’t skew the result. To cut down on time I would probably given the choice opt for a system that cached and also only calculated for the first tiddler at the top of the story river when I pressed a button - the idea being to select one tiddler as potential “hub” and then find out which tiddlers lack links to it that really should have been linked to it.
By the time my tiddlers are accurately tagged I don’t have much difficulty in identifying closely related tiddlers when I need to - the problem is the tiddlers that were written before a tag was introduced or where I have been lazy or just forgotten one of my tags existed - I have close to 300 topic related tags at the current time so I am conservative creating new ones a deliberate resistance until I have convinced myself I really do need it. I reckon I create around ten to twenty links a week between existing tiddlers that really should have been connected earlier - it’s satisfying and enjoyable and often leads to new observations but I would love a tool that gave reasonably reliable suggestions and hints ( say a 20% success rate in terms of a new link being created vs the recommendation being ignored ).
I will continue to follow and am considering installing in a copy of my TW - it’s a very interesting area you have chosen to explore.
But notice that this might be difficult to convey. The whole point of this concept is to consolidate various kinds of information into a single number. But even that number is by itself meaningless. Except for the cases of 0 and Infinity, their only use is as relative numbers. Adding, even for hover, and indicator that
Tiddler A includes 3 links to Tiddler B
Tiddler B includes 1 link to Tiddler A
Tiddler B is tagged “Tiddler A”
Tiddler A and Tiddler B are both tagged “Foo” and “Bar”
Tiddler B is transcluded by Tiddler A
Tiddler A has field “xyz” with value of “Tiddler B”
Tiddler B has field “abc” whose list of values includes “Tiddler A”
Distance score: 17
doesn’t really offer a great deal in my opinion.
In the end the important thing here is that Tiddler A and Tiddler B are quite close to each other, much closer than, say, are Tiddler A and Tiddler C. That Tiddler B is second-closest to Tiddler A is useful, but most all of that information that I might include – in hover or however – is information already provided by the tag list, or info tab or some other means. The new thing I see here is precisely the consolidation.
I expect the portion of TW users who would find it useful will be relatively small. I described earlier my intended audience. It’s possible it will grow beyond that, but I doubt it will grow to something useful to the majority of TW users who seem to me to be both producers and consumers of their wikis, and so already have the sort of intuitive knowledge this is meant to provide.
Yes, nearness is definitely implying some degree of conceptual similarity, but it is algorithmic, based on weighted versions of counts of tags, links, field values, and (one day) transclusions. I think “Related” is an equally good name. Nothing is perfect :“nearness” for a geospatial wiki will admit ambiguities, as will “related” on a geneaology wiki. That’s yet one more piece of configuration I should add.
Yes, and my long-term plans will only make this worse. Although it’s possible that I will keep something like a cleaned-up version of the current code, which might or might not have issues. I have one report of problems on a 5000-tiddler wiki, but haven’t seen the code to be able to verify for myself. It seems plenty responsive on the 1600-tiddler tiddlywiki.com. But I don’t know what the thresholds are or if there are tricks I can do to speed things up if I’m going to hit them.
That would probably be for another tool. I think full-text searches, possibly of wikified content would be prohibitively expensive on anything remotely large.
I’m thinking hard about how I might cache results and offload their calculation to a worker. I think this will be essential if I try to capture the relations in the whole wiki. But I have no decisions yet.
