I often need to make bulk changes, sometimes to hundreds or even thousands of tiddlers. When these tiddlers are being modified, I’d like to know if there’s a progress bar or something similar that can be displayed. Or is the only option to print logs to the console? Maybe it’s possible to set up a separate tiddler to display this log. Also, bulk changes cause TiddlyWiki to lag. What’s the upper limit for this? Hundreds are manageable, but thousands are noticeably slow. Maybe five thousand or ten thousand is close to the limit.
With TiddlyWiki all actions are synchronous. So there is no asynchronous progress, that can be shown. Bulk actions are done in one run. – Except, if you use a client / server configuration.
But bulk action syncing with 1000th of tiddler may take hours, because file access is slow, compared to actions in PC memory.
If you’re asking strictly about the visual aspect, as in a widget, the Full Text Search plugin had a tiny companion for that: $:/plugins/hoelzro/progress-bar
while true, when developing a plugin to do bulk operations you could:
- first count the number of entries,
- group them into smaller batches and
- update a progress bar after each batch is complete.
so for example, for a 10000 tiddler operation it could break up the work into 10 x 1000 tiddler operations, with a call to update the progress bar to the next 10% after each.
i don’t know if there is a substantial performance hit from doing it this way, but it might at least make it feel faster to have an updating bar…
1 Like
My thoughts;
Any progress bar is likely to slow the process it is representing.
As tiddlywiki is primarily used interactively it is less suitable for long “batch” processes. When doing a lot of processing from the begining to end, out of the box, we see nothing while this is happening.
If you expect this it just requires patience. if you did not know it would take a lot of time and the user should be at least advised. It is best that you are informed before hand that it will take some time. Another trick is to use a GIF so it looks as if it is still running as the batch process takes place. From expierence 99% of the time such long batches are “one off”, during a conversion process.
- In the past it was common to increase the timeout in browsers so the browser did not warn it was running too long, and force the need for multiple “continue” prompts.
Following @Scribs divide and batch suggestion you could do this but rather than return control to the user Interface UI you could send a message to the browser console eg every 1000, 2000 etc… so you know its still running see LogWidget and ActionLogWidget.
- There are a few algorithiums that could programaticaly choose the divided batches.
- Perhaps save the wiki before running incase you are forced to cancel
- Make such batches “restartable”, so if you broke out you can restart without repeating work.
- Be aware that a progress bar assumes what 100% is, you need to calculate it before hand, perhaps just an indicator the process is continuing.
- There is a possibility the browser will timeout, that may need to be addressed
- Consider advising the user with an ActionConfirmWidget
Or perhaps tm-notify
Note: The nature of browsers suggests to me there are numerous ways to notify the user. Multiple processes are running in a browser including timeout functions. Perhaps we need to design some kind of on screen element we can “send” messages to, then have an action that changes it, in the chosen increments. Perhaps a html element then using CSS and its calc functions or something similar to update the count.
- The idea is to minimise the cost of such a progress indicator in both processing and UI impact.