Large Knowledge Bases - Ideas About Ranking

[jonnie45]

Reading your post made me think it would be nice to have a touch-button on every tiddler - then touching a tiddler that has not been touched before creates the two new tiddler fields with sensible defaults which can be fine tuned at leisure ( say an initial review-interval of 30 days ) - tiddlers that have the two fields are considered part of the “pool” and ones that don’t are not.

Remove Tiddler From Scheme?: I would probably opt initially for the rather unsightly manual option of edit tiddler and deletion of the two fields to remove a tiddler from the review scheme - yes untidy and not polished but I would want to avoid removing a tiddler from the pool accidentally whilst scrolling up and down the story river. ( by means of example I do occasionally accidentally remove tiddlers from my favourites list - one click solutions are convenient but also convenient for mistakes - perennial usability problem ) - otherwise I end up writing “Are you sure?” conditionals - so my crude method would be unsightly YES inconvenient NO.

I guess one reason I am not so attracted to the more sophisticated and engineered solutions available is that my tiddlers vary from single paragraphs to articles, essays and whole chapters from text books - so not really a binary Q and A situation or exam preparation situation. I like simple solutions where possible and so the idea of being able to populate the story river with the ‘N’ tiddlers currently most requiring a re-read appeals to me most.

[pmario]

Reading your OP reminded me about a proof of concept, that I did create about 6 years ago.

A user wanted to keep track of external links s/he wanted to visit. Everytime a link was clicked the date and time should have been recorded.

While not 100% the same imo the mechanisms that can be used are very similar.

If I do understand your request right

1. You want a reminder, to re-visit existing tiddlers in your wiki.
2. A new “next review” date should be picked automatically, when the tiddler is viewed
   a) Different defaults should be possible eg: +30 days or +100 days from now

------- Switching to brainstorming mode -------

With my POC I did add a “comment”, because I think it’s important, to add new thoughts when “revisiting” a topic. Those notes should be “tracked” too. … At least for me, that’s important, to improve the quality of the wiki if possible.

I personally do not want to “touch” the “content” tiddler, just if I did read the content again. I think for me the tracking-timeline should be separated. Only, if I do change the content tiddler it should get a new modified date.

IMO reminders 1) can be activated using startup-actions

Add 2) Review Workflow

My idea is a bit different to what you describe in the OP.

• Creating a review-overview, which is visible at wiki startup
• re-viewing tiddlers in new window, while
• allowing the main window to be used to track new info
• So 2 windows will be visible at the same time.

could be done by extending the POC, mentioned above, a little bit. … Just some thoughts.

Add 2 a) creating a new re-view timestamp. IMO the default “review priority” should be part of the overview, so it can be changed quickly. May be a dropdown with current defaults. May be a date-picker.

---

The concept developed with the POC also influenced my Script Manager Edition, which is still my main private link and idea collection tool.

The process how the script manager was created is fully documented with video HowTo’s. See Google Groups: INTRO: A script and todo manager workflow / UI experiment + video howto’s

I’ll probably extend my script manager with the new ideas triggered by this thread. … But … No promises

[jonnie45]

After lots of help from more experienced folks (extra special thanks to EricShulman and TW_Tones) I am getting somewhere. Here is a shot of my story river - the tiddler toolbar bell icon is part of the review tool, the tiddlers are self documented in the image shown.

For this case I populated the story river in a way that was designed to show those 3 specific tiddlers.

So I get a view of the ‘review’ situation for each tiddler displayed whatever method I use to populate the story river.

However, if I am particularly interested in catching up on reviewing tiddlers that are overdue ( signified by a red bell ) then there is a new button in my sidebar tab which will show only overdue tiddlers on the story river with the most overdue ones shown first. Like almost all of my custom methods of populating the pool the overdue button filter incorporates a limit on the number of tiddlers displayed which is controlled by a single universal ‘limit’ slider - I have to protect against too many tiddlers.

Areas not fully covered yet…(both with fairly easy manual workarounds).

Remove a tiddler from the review pool - current method simply delete two tiddler fields manually.

Change the review frequency for a particular tiddler from the default of 30 days - current method manually edit tiddler field. The value chosen is likely to depend on a variety of factors including how easily the tiddler content is remembered-understood, the perceived value of the tiddler content and so on, it will be a judgement call and might be changed several times, perhaps if the tool works very well I will find that I am continually increasing the interval on account of increased retention-awareness on my part.

Remove all tiddlers from the review pool - current method not practical it would be a manual deletion of the extra two fields in all tiddlers, if someone starts using this tool they should have a way of removing all changes if they lose interest.

I would add for context that many of my tiddlers are pages or chapters from books which influenced the design decisions - I particularly wanted review information to appear in the story river rather than within a plugin.

Apart from these two items it’s exactly what I wanted - at least in theory - now to see how well it actually works in practice - I have over 2000 tiddlers some quite lengthy so evaluating it will take time, I think it already guarantees that no highly valued tiddler will settle into the mud on the bottom of the pool - at least once I have had the chance to consider all tiddlers for inclusion which will be an ongoing task.

If anyone is interested I will post the code segments - I may even see if it I can put it together as a plugin not sure if anyone else would find it useful?

Thanks again for all the help

[arunnbabu81]

@jonnie45 I have been following your today’s post. This looks like an interesting way to review previously created tiddlers. Once you are finished with the code, please do share it here.

[linonetwo]

No, you can. Just modify the “due” field. You can add a ViewTemplate with pikaday plugin to edit it.

[linonetwo]

You can even use Calendar and Agenda plugin tw-calendar with Mobile frendly agenda page layout plugin to edit the due field with proper configuration (see the doc for example with due field)

[jonnie45]

Hi, I am having a go at packaging it as a plugin initially although totally inexperienced with github so the first share might be a little less organised than that - I don’t think it will be used by many people.

Sidebar Tab

1. Side Bar Tab Red bell button - List 20 overdue ordered by least recently reviewed - this could be improved by ordering so that it shows the most overdue tiddlers first but for now I am taking the simpler coding option. If all tiddlers have the same interval for review then there is no difference - the number of days overdue is dependent on the interval for review as well as the date last reviewed.

2. Side Bar TabGreen bell button - List 20 that are not overdue, ordered by most recently reviewed first.

3. Side Bar Tab Red Bin/Trash Button - feature to remove the two fields that get added to tiddlers that are included in the review pool - this would be a housekeeping act before plugin removal. It features a double nested “Are you sure you want to delete” / “Are you really sure” feature to guard against accidents - I would hate to lose months of review data accidentally.

4. Changing the review frequency interval for a particular tiddler is still done by manual edit of the tiddler - this is only necessary if the default of 30 days is not suitable. I have reasons for not wanting a more sophisticated method if it can be accidentally triggered ( on a smaller touch screen for instance ). For now the manual method suits me.

Story River

The review information is available at all times in the storyriver since each tiddler has either a grey bell, a green bell or a red bell.

• Tiddler Tool Bar Grey bell - tiddler is not part of the review pool and does not have the two extra fields.

• Tiddler Tool Bar Green bell - tiddler is part of the review pool (does have the two extra fields) but it is not overdue. If you click on the green bell then you indicate you reviewed it today even though it was not overdue.

• Tiddler Tool Bar Red bell - tiddler is part of the review pool (does have the two extra fields) and is overdue calculated on the basis of it’s own private review frequency interval - different tiddlers can have different review frequencies (measured in days) - the default is 30. If you click on the red bell you indicate you reviewed it today, it will then turn green.

I am very close to being ready to share so please indicate a method I can do so - I havent shared a plugin before - probably not yet at the stage of wanting to put it on Github etc.

---

It will not work on Tiddlywiki v5.2.0 - but does work on the current version of Tiddlywiki - it relies on something changed ( I think ) at v5.2.3

[TW_Tones]

You can start by exporting as a JSON and adding here. People can try it, then perhaps set up a tiddlyhost site and post in the plugins topic. later you may add it to a library.

[jonnie45]

Here is my first version of the plugin now named ‘ReviewTiddlers’
It works in TW 5.2.7 it does not work in TW 5.2.0 so please only use in TW5.2.7 or greater.

This is my first real project so please try it out in a fresh empty Tiddlywiki - please do not yet use it in a wiki in which you have invested time and effort - in this context I am a “learner driver” you wouldn’t let me loose in your brand new Tesla would you?

Experienced folks should probably inspect the two files containing code first.

The plugin includes a feature to remove the two fields that it creates in a tiddler when that tiddler is added to the ‘review-pool’ as a safety feature for now I have set a limit on 10 tiddlers. That means if you have 30 tiddlers and you have introduced them all into the ‘review-pool’ you will have to perform the delete action three times to remove all tiddler fields created by this plugin. This feature does not remove tiddlers, only the fields created by this plugin. This safety limitation will be removed at a later time.

Background: This was designed with large knowledge bases in mind, in my case some of my tiddlers are of a significant length even entire book chapters and so I usually use a reveal widget to optionally hide the bulk of the content, in addition I have over 2000 tiddlers. I mention this because some folks with tiddlers that contain only a few lines of text each may wonder if it’s not easier if a tiddler is already on the story line just to read the content than look at icons on the tiddler toolbar that tell you whether you need to review the content. Like many tools it’s orientated very much to a specific type of wiki use.

Hope it’s of interest please report bugs or problems.

Jon

$__plugins_jonnie45_ReviewTiddlers.json (16.7 KB)

[jonnie45]

I forgot to say… to test out the situation where a tiddler is overdue a review you could wait 30 days or you could do the following…

• Create a new tiddler and save it.
• Click on the grey bell icon in the tiddler tool bar to include it in the ‘review-pool’ the bell icon should now be green - the ‘last reviewed3’ field in the tiddler will now be set to today.
• Edit the tiddler

Inspect the UTC encoded date field named ‘last-reviewed3’

This tiddler joined the ‘review-pool’ on the 2nd June 2023 so the UTC coded date YYYYMMDD… starts

20230602…

To in this case you might change this to say the 1st May 2023

20230501…

And when you save the tiddler should now display a red bell icon since it now registers a last-reviewed date over 30 days prior to todays date which in this example is 2nd June 2023

Of course in regular use the ‘last reviewed3’ date would not be hand edited like this, it is automatically set by the plugin, this is for the purposes of experimentation only.

The review-interval is always expressed as a negative integer, in the above example it is -30, as we are dealing with the past not the future. This convention originates in the way the tiddlywiki operator ‘days’ works which this plugin relies on.

[Scott_Sauyet]

@jonnie45

Here is my first version of the plugin now named ‘ReviewTiddlers’
It works in TW 5.2.7 it does not work in TW 5.2.0 so please only use in TW5.2.7 or greater.

This is great stuff! I have a wiki at work where I think this might be very useful. It’s a large-ish documentation wiki, and for most users is simply a read-only view, but for authors/editors, this could be a very good way to help keep things fresh. (I would have to hide the new bell icon in read-only mode, but I already hide others, so that should be easy.)

A possible extension of this, or a version 2, might offer a selection of periods: when you click the bell, it shows some simple UI, probably floating right below the bell:

                +----+----+----+====+----+----+----+-------+       +--------+
Review again in |  1 |  2 |  7 ‖ 30	‖ 60 | 90 | Custom ___ | days  | Cancel |
                +----+----+----+====+----+----+----+-------+       +--------+

with each option a button, and the default of 30 days highlighted. (Obviously the values could be configurable.)

Thanks for sharing!

[jonnie45]

Thanks,

I will be thinking about something more elegant than what is offered at the moment for interval adjustment and I like your idea of a cancel - turn it off again for this tiddler as well. I am looking forward to an interesting couple of months finding out how this works in practice, one question I have in mind is will I suffer again from what I call “ranking inflation” - the thing currently being looked at always seems to be the most important and so 1st position ranking tends to be awarded too often. The equivalent here would be setting ever decreasing review-intervals in an attempt to make ‘this tiddler’ stand out against the others I thought were so important when reviewing them two weeks ago etc…

I am reminded of a amateur dance show popular on TV - when the judges award 10’s you always think they have left themselves no “head room” when the next act is even better - human beings, psychology etc…

[markkerrigan]

My solution for this is a bit simple, but I am in the process of doing an “Annual Review” where I go through every tiddler alphabetically and if there’s something I want to return to later I tag it as “Flagged”. Then I keep a list of the reasons why a tiddler is flagged. But the concept is simple, just using tags.

[jonnie45]

With help from Eric Shulman I have added the following to my local prototype…

The Green ( tiddler not yet due for review ) or Red ( tiddler is overdue a review ) icon now displays the review-interval - in the case shown 36 days between reviews.

The faint small up and down arrowheads after the bell icon can be used to raise or lower the review-interval +1 day or -1 -day - it’s quite quick, easy enough to shift the review-interval by 20 days without feeling you are forever tapping away.

Ideally I would like to have the increment and decrement arrows vertically arranged so as to resemble a split button with two actions but that is a later wish.

I will be replacing the icons with more visible ones. I am keeping them small to not take up too much tiddler toolbar space. I don’t expect to be able to get all of this perfectly scaled with good usability for a phone screen in portrait mode - it works fine but adjusting review-intervals would be tricky on a phone.
In context though the tool is aimed at very large wikis where the user will probably use a larger device for more than casual browsing.

The backup method of changing the review-interval is just to edit the tiddler and edit the field directly.

The bell icon can only display a two digit number so the up/down arrow head buttons are limited to a review-interval of between 1 and 99 days. It is possible to set a larger interval by editing the tiddler field by hand but it will not display neatly in the bell icon.

I have improved the sidebar tab to additionally state the total number of tiddlers in the review-pool.

There was a possibility of course to display a different number in the bell-icon for instance perhaps the number of days until the next review. However this tool was written with very large wikis in mind, so in this context the waiting list of tiddlers to review might be quite long. Somehow I think I will be occupied enough with the tiddlers are that overdue a review and less interested in which ones look like they will on the review list next week etc.

I view the review period as the most important number to display in the icon, I already have over 300 tiddlers in my review-pool, my tiddlywiki has over two thousand tiddlers and I am tending to add around 50 to 100 a day to the review-pool.

The first judgement call is what the review-interval should be - I can stick with the default of 30 but usually I choose a starting value based on perceived tiddler value, how well I seem to remember the contents today and so on but I anticipate raising or lowering the review-interval upon next review, I see this as a medium to longer term settling in process.

I will start packaging it up and put out another plugin json file soon for anyone interested.

[jonnie45]

Latest version

$__plugins_jonnie45_ReviewTiddlers1.json (21.8 KB)

As before please don’t risk it with you only copy of a precious Tiddlywiki !

Issues Remaining: Red and green bell Icons will appear together on the day that a tiddler is due review - issue with days operator, yet to resolve - only one should show.

Please let me know of any other issues

Limitations: The clear up code to remove both tiddler fields that this plugin creates is still limited to ten tiddlers per click. Just being careful, limitation will be removed. The clean up appears in the sidebar tab for this plugin, It would be called prior to plugin removal.

If anyone knows how a split button for incrementing and decrementing the interval period could be coded I would be very grateful to hear more, something like this with an up arrow and down arrow all part of the same visual block but with the ability to call different macros, in this case one to increment a value and one to decrement it. In a regular HTML situation components within an SVG image can each be assigned different links but I cannot see how that could be exploited in Tiddlywiki where a button click needs to invoke a macro.

I have tried using CSS to use position relative to align the arrow heads vertically but as expected other elements do not flow into the gap created with this option so other tiddler toolbar items would not shuffle up leaving an unsightly gap.

[Scott_Sauyet]

I think the main problem is that you’re using position wrong. The easiest way I can see is to wrap these two buttons in a container, fix its size and margins, set its position to relative and then for the two buttons, set their positions to absolute (which will be relative to the new container) and set the upper one’s top and the lower one’s bottom to 0

\define reviewIntervalInc()

<$button 
tooltip="increment review interval"
class=<<tv-config-toolbar-class>> style="position: absolute; top: 0px; font-size: 40%" >
<!--                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  --> 
<!-- ... -->
\end

\define reviewIntervalDec()

<$button 
tooltip="decrement review interval"
class=<<tv-config-toolbar-class>> style="position: absolute; bottom: 0px; font-size: 40%" >
<!--                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  --> 
<!-- ... -->
\end

<!--                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  --> 

<$list filter='[all[current]!is[system]!tag[z hidden]has:field[last-reviewed3]]'>
<span style="display: inline-block; width: 1em; position: relative; height: 30px; margin: -10px 0 -5px -5px;"> <!--
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -->
<!-- Add increment button -->
 <<reviewIntervalInc>>
<!-- Add decrement button -->
 <<reviewIntervalDec>>
</span><!--
^^^^^^^ -->
</$list>

This is rough initial formatting, but you can play with the height and the margins for the container to get what you want. I just played with them until it looked reasonable.

As to positioning, I mostly use relative just to establish a new positioning context. All absolutely placed elements will be based on the nearest positioned ancestor. The display: inline-block means that it will take up space in the button row but not start a new row itself.

$__plugins_jonnie45_ReviewTiddlers_TiddlerButton.json (4.6 KB)

[TW_Tones]

Interesting work @jonnie45 I am watching, I like your icon etc…

To be thorough I did mention an opposite approach, for example for weekly review set a weekly-date field, when you “do it”, :stamp it with now", then have a weekly review tiddler (or part thereof) that lists tiddlers with a weekly-date older than 7 days. In this model you may have fields for daily/weekly/fortnight/monthly/day of week etc… but you can also have more than one review period per tiddler.

• You could have a review-field with a date stamp and choose the review period in your tiddler that lists items to be reviewed. So you do not need to set it on the tiddler.

[jonnie45]

Hi thanks TW_Tones, interesting approach using a field for weekly review I will think it through although it sounds as if it would result in quite a number of fields on each tiddler, I would anticipate ten fields to match my current soft limit of 99 days assuming an acceptable granularity of ten days.

Keeping the housekeeping as much as possible on the tiddler itself is a design decision as I anticipate setting and then re-setting each review period quite a few times, it’s a reaction to what I see when I look at the content of the tiddler, I may not be actively thinking about the review aspect of a tiddler initially - I may have landed on it following a group of intertwined tiddlers that represent a sub-topic or area of uncertainty or ongoing research - but when I see it and start reading, musing I start to realise that my recollection of the content is not as fresh as it might ideally be. I might have several things going on, I see relationships that need establishing with links with other tiddlers which I might need to locate via search and so on - there will already be a fair amount of sidebar activity - it is freshingly convenient not to have to do yet another sidebar task to alter the review period, that at least can be done straight away “in situ” whilst I have “eyes on” the tiddler.

I find similar convenience with the star ratings plugin where to change the rating the user only has to click on the golden ( or greyed out ) stars that appear top right on every tiddler. For me that is the nature of working with very large learning/research wikis - ratings, sieves or other methods are used to constantly churn the tiddlers and get back to the valuable but dusty ones to promote, link, cross correlate and spot relationships are a core aspect of the whole workflow and so need to stay with the tiddler itself so that they can be considered every time the tiddler is seen. I appreciate the feature where clicking on a tag pill on a tiddler shows a drop down of every other tiddler sharing that tag, I often use this more than the sidebar options because it says effectively “show me other tiddlers like you - with this quality or aspect” - similar emphasis on links in tiddlers to other tiddlers - back linking and so on so my workflow really revolves around the tiddler not the sidebar, the sidebar just initiates a fruitful avenue of research.

I think there are parallels here with object orientated programming vs structured programming, it makes me wonder if in a future where interfaces might be 3D touch, future crime detectives using software in an investigation ( not a bad metaphor for research in a large knowledge base) might role-play 3D objects representing people, places and objects and access functionality to relate those 3D representations through the 3D representations themselves, perhaps current software where interfaces are structured more around functionality rather than objects reflects on the limitations of mice, keyboards and two dimensional screen space challenges - if these restrictions were limited then certain functional activities might become much more object orientated in terms of their interfaces.

Day dreaming I know but actually writing this post is helping me solidify what direction I think I want to go in with my knowledge base, a preference to initially populate the story river with the principle tiddlers using sidebar functionality but then the next phase tends towards working with the tiddlers themselves in the story river and not the sidebar where tiddlers are usually represented only by titles which in the case of a tiddler that might be a bit rusty only invites viewing on the story river to get the fuller picture anyway.

[jonnie45]

Thanks Scott I will give it a try!

[TW_Tones]

Reading your response, you seem to make some assumptions about the method I proposed, but no problems.

Except for using a separate tiddler or side bar to review tiddlers within a given review period, I propose nothing more or less than you do. Except that I use a fieldname representing a periodical review cycle and time stamp it now, whenever I actually review it.

• The review process sets the period of the review not the tiddler, using a different field name eg weekly, or monthly if wanted.
• Sure your method suits a case where you want an arbitrary number of days for review to be set on each and every tiddler.
• My method is best when you want to do something regularly in a named period or review list of tiddlers not reviewed for an arbitrary period eg: provide a number of days since last touched.

best wishes