Proposal: WikiShell / TiddlyCLI - A Command Console for TiddlyWiki

Proposal: WikiShell /TiddlyShell /TiddlyCLI – A Command Console for TiddlyWiki

Objective: Introduce a built-in command console to TiddlyWiki that allows users to perform actions, process tiddlers, and manage data efficiently via simple commands. This feature aims to empower users who are not developers and complement advanced user workflows.

Key Features (Initial Scope):

  1. Basic Command Set:
    • Import/export tiddlers.
    • Modify tiddler fields or tags directly.
    • Apply filters for bulk operations.
  2. Syntax Simplicity:
    • Commands use simple, intuitive syntax akin to PowerShell (e.g., export-tiddler "MyTiddler" -path DefauldDirectory or set-tag "MyTiddler" [-important [-urgent).
    • Command auto-suggestions or error handling for improved usability.
  3. Integration with Widgets:
    • Leverage existing <$action> widgets to execute tiddler-defined tasks.
    • Examples:
      • run-action "ActionTiddler-4876487" to execute actions encoded in a tiddler.
  4. Extensibility:
    • Modular design to add new commands easily.
    • Updates via plugin or core updates.

Long-Term Vision:

  • Incorporate as a core TiddlyWiki feature or make it a powerful plugin.
  • Possible integration into the Advanced Filter tab as a dedicated sub-tab.

Benefits:

  1. Simplifies workflows for non-developers, reducing reliance on JavaScript.
  2. Speeds up repetitive tasks and bulk operations for advanced users.
  3. Centralizes power-user tools in one accessible interface.

Example Commands:

  1. Export a tiddler:

    export-tiddler "MyTiddler" --json
    
  2. Add a tag to a tiddler:

    set-tag "MyTiddler" important
    
  3. Filter for tiddlers with a specific field value and change tags:

    filter-tiddlers "[field:author[John]]" | set-tag reviewed
    
  4. Run an action tiddler:

    run-action "DailySetup"
    

Example Proof of Concept (Basic)

Here’s an idea for implementing a small prototype of this idea as a custom plugin.
Note: Since I can’t code directly, I’ll guide you with some basic examples.

Concept:

  • Use the $tw object in TiddlyWiki’s JavaScript environment to parse and run commands.
  • Define a special tiddler (e.g., $:/WikiShell) to serve as the interface or console window.
  • Commands can be processed using existing filter and action widget syntax.

A Basic Approach:

  1. Create a Tiddler for Commands:

    Example content for $:/WikiShell:

    Enter a command below:
    <$edit-text tiddler="$:/temp/WikiShell/Command" tag="input"/>
    <$button>Run Command</$button>
    
  2. Write a Script for Processing:

    Example JavaScript (to be included in a plugin):

    $tw.wikishell = {
        runCommand: function(command) {
            // Split the command into parts
            const parts = command.split(" ");
            const cmd = parts[0];
            const args = parts.slice(1);
    
            if(cmd === "export-tiddler") {
                const tiddlerTitle = args[0];
                // Simulate export logic
                console.log(`Exporting tiddler: ${tiddlerTitle}`);
            }
            else if(cmd === "set-tag") {
                const tiddlerTitle = args[0];
                const tag = args[1];
                // Simulate tag addition logic
                console.log(`Setting tag "${tag}" on tiddler "${tiddlerTitle}"`);
            }
            else {
                console.log("Command not recognized!");
            }
        }
    };
    
  3. Set Up a Button to Trigger the Command:
    Update the $:/WikiShell tiddler:

    <$button>
        Run Command
        <$action-jsrun code="return $tw.wikishell.runCommand($tw.wiki.getTiddlerText('$:/temp/WikiShell/Command'))"/>
    </$button>
    

Disclaimer:
I am not a JavaScript coder nor a core developer of TiddlyWiki. The coding aspect of this proposal was conceptualized with the assistance of Copilot AI. Although, I am a real person and have edited this carefully. I really do hope this idea resonates with the community.

Comparison with Node.js Command Line:
The proposed WikiShell would operate entirely within the TiddlyWiki environment, unlike the Node.js command line, which requires external setup and execution. This internal console would:

  • Be accessible directly from the TiddlyWiki interface, eliminating the need for external tools or environments.
  • Use a unique, user-friendly syntax inspired by PowerShell, DOS, and Bash, making it more approachable for non-developers.
  • Focus on TiddlyWiki-specific operations, such as managing tiddlers and executing actions, without requiring JavaScript knowledge.

While Node.js is powerful for advanced workflows, WikiShell would democratize access to command-line-like functionality, empowering users of all skill levels.

To all of you: Thank you for creating such inspiration through TiddlyWiki. Thank you to the community of users, developers and contributors. I would be honored to work further with you on this idea, should you decide to explore it further. I very much welcome your feedback.

4 Likes

I use Linontwo’s Command Palette, and would love to see even more commands available - so I would love to use something like this, accessible via a hotkey with a button option in the side bar, similar to the ‘new tiddler’ icon etc.
To me this feels more like a plugin rather than core functionality. But if someone were to build it, I would absolutely use it.

File-size-cost-performance-wise, it would be nice to put it in the core and have it as a feature that can be turned on through the control panel settings or offered as a hidden feature; i.e. a booleen system tiddler "yes". At least for basic commands that are unlikely to change. Plugins and modular command updates could carry out more specific tasks in specialized areas of use.

If one is thinking about a Python-type console or a Sublime/VSCode console for file management, etc… that’s something else. The idea here is quite similar to the existing Advanced Search bar except integrating the filter syntax by prefixing them with action commands. It might be necessary to add an aditional tab the Advanced Search Bar to use this feature.

Here’s a simple example: in the Advanced Search Bar, we write our filter: [!is[system]sort[title]] (All tiddlers except system tiddlers). Let’s say we’re just getting rid of all the tiddlers and starting anew. Next to the search bar, we have the Delete Tiddlers icon. In a “command” syntax scenerio, we might write something like: delete -noprompt [!is[system]sort[title]]. In this case, the noprompt would of course get rid of the “Are you sure…” step. Likewise, if one wanted to save those items to a local JSON file prior to deleting them, the syntax might be: delete -save > local/deleted-tiddlers.json [!is[system]sort[title]].

1 Like

Oh good points!

I love those command examples. This is exactly what I kind of fantasize about having in TW :slight_smile:

A little update: making some progress! I’ve got the Command Console bar up with an $action setfield widget tiddler transclusion. It’s not doing anything, yet, but almost.
Imagine a Command Console which can create 1 or several tiddlers with fields and tags set in one line of script! Or delete tiddlers or export or import. Or shampoo your hair while you’re busy scrubbing your feet. All while staying inside of TiddlyWiki.

That’s a very interesting request. You are probably the first one, which describes DOS, bash or PowerShell syntax as “approachable for non-developers”.

There is one important thing. Browsers are not allowed to directly access files.
→ So our CLI would use temporary or named tiddlers instead of files.

Just Brainstorming

No commitment in any way!

Usually CLI commands exist of 3 components as your first example shows:
delete -noprompt [!is[system]sort[title]]

  • command
  • options … prefixed with -
  • parameters … I personally prefer named parameters

In an abstract it is <command> [<options>] <parameters>. [..] means “options are optional”. That’s nice and simple.


The second example looks easy for humans, because our brain is king in “pattern matching” and “interpretation”.

The technical structure of your command
delete -save > local/deleted-tiddlers.json [!is[system]sort[title]] is:

  • command
  • options
  • "send result of command to a file named local/deleted-tiddlers.json
  • parameters

So the command has to use the “result of the parameters” and send it to a new file.

The problem here is that the software has to use 2 passes to execute the command successfully, since the “parameters” are not known, when the file should be created. So the code from the first command can not be reused. That’s a problem.

There are 2 ways a bash command like this would look like:

  1. delete -save [!is[system]sort[title]] > local/deleted-tiddlers.json
  2. delete -save local/deleted-tiddlers.json < [!is[system]sort[title]]

There is an other problem. delete and --save technically are 2 commands. eg:

list --save [!is[system]sort[title]] local/deleted-tiddlers.json
list --delete --noprompt [!is[system]sort[title]]
  • Both commands now have the same pattern: command → options → parameters.
  • Much simpler to parse and probably simpler to implement.
  • Also simpler to debug.
    • Especially since we can write list --delete --simulate --noprompt, wich only creates the “backup”, without actually deleting the tiddlers.

BUT BUT BUT

  • It does duplicate the filter parameter → that’s error prone.
  • There are 2 parameters now, the filter and the filename

OK let’s introduce the PIPE mechanism using the pipe character |. It takes the output of a list and feeds it into the next command as an input.

list [!is[system]sort[title]] |
list --save local/deleted-tiddlers.json |
list --delete --noprompt --nobackup 

It think, that’s as generic as it can be.

  • It is verbose, so every line should be easy to understand
  • The structure of every line is the same
  • No duplicated filter input

The following command will write a list of tiddler titles into the standard output area. Nothing else will happen.

list [!is[system]sort[title]]

BUT BUT BUT the OP did suggest 1 liners.

OK lets introduce shortcuts, which are a new level of abstraction. → Abstractions, reduce verbosity and increase mental complexity


oneliner

list --delete [!is[system]sort[title]] > local/deleted-tiddlers.json

Is a shortcut for

list --delete [!is[system]sort[title]] |
list --save local/deleted-tiddlers.json

This workflow assumes, that list --delete will send all tiddlers it deletes to an output buffer in JSON format, so list --save can handle it.


oneliner

list [!is[system]sort[title]] >> local/deleted-tiddlers.json

Could be a shortcut for append to file/tiddler (browsers can not access files directly)

list [!is[system]sort[title]] |
list --append local/deleted-tiddlers.json

More examples:

The following command would modify every tiddler tagged HelloThere, without any user interaction. If possible.

\rules noprompt yes

list [tag[HelloTere]] | field --set tags="abcd [[tag with spaces]]"

or

let x="aaa" y={{config/tiddler-tags}}

list [tag[HelloTere]] | field --set(overwrite is default) tags=<<y>>

list [tag[HelloTere]] | field --add tags="abcd" |
    field --remove tags=<<x>> |
    field --append test="append this text" |
    field --delete abcd-field

Conclusion and Possible Implementation

I am not 100% sure if the syntax is consistent, but I think it could be implemented in a way, that it can create code, that reuses existing TW wikitext.

So the “executable” code of the whole thing should be valid wikitext.

  • let would obviously be translated to $let-widget
  • list would be translated to $list-widget with temporary input and output buffers/tiddlers
  • field is action-setfield widget or action-**** widgets
  • | writes internal memory to temporary input / output buffer tiddlers
  • # would be interpreted as comment

The whole thing can be covered by a hypothetical - $workflow-widget, which could be very similar to the testcase-widget.

The testcase-widget already has an interactive environment, that is very similar to a CLI

This would allow us to separate the CLI environment from the existing wiki store.

IMO testcase would be preferable, but may be we would need the innerwiki-plugin. … Not sure about this one.

The advantage of a separated store is obvious. It won’t kill a wiki if filters are wrong and it should make debugging much easier. eg:

\rules simulate yes

# The following command would kill all your tiddlers, if simulate would be "no"

list --delete --noprompt "[all[]] :filter[is[test]]" 

Just some thoughts

have fun!
Mario

1 Like

Nice work, Mario! I was never expecting a very clear explanation of the subject, but I can tell you i really appreciate that. If you’re not a teacher you very much should be! More than this, I am certain others reading this post are more enlightened.
I have considered the “|” pipe, but I thought it may be a little off-putting as it’s a PowerShell thing. I come from a DOS and later a little bit of PowerShell background. My Bash knowledge doesn’t go beyond sudo install
I don’t know exactly what you meant by “named” parameters, but I am guessing you are refering to the tendency of “aliases”; for example “ls” instead of “list”, which is quite cryptic and something I personally would like to avoid here.
I’ll have to read your post a few more times. Coming into TW has been such a learning experience and I want more and more people to have the same experience. Making an accessible command-line prompt could be a game-changer.
Have fun? I am!
A

1 Like

A named parameter is eg: list filter="[tag[HelloThere]]", where filter is the parameter name. This has an advantage if there are many parameters. Named parameters are not bound to any order.

Unnamed parameters do need to be in a specific order. eg:

  • copy from-file to-file is “unnamed”
  • copy -Destination to-file -Path from-file is named so the order can be in reverse if the user wants it.

There are some pros and cons.

The | pipe symbol works in any operating system in (almost) the same way. In my case I did suggest it to allow us to combine several lines into 1 command. Otherwise we would need to add \ (bash) or ` (PowerShell) something to the end of the line, wich I do not want.

In my examples

  • If | is at the end of the line it “pipes” the list to the command in the next line.
  • If there is a line-break the command is terminated

hihi, Sorry about that one. – I just saw, that I actually missed some elements that need description.

\rules xxx

For our TW parsers we allow “tiddler global rules” so eg:

  • For testing the CLI commands a new simulate yes rule may disable all store manipulation.
  • A new noporompt yes rule may activate the --noprompt option for every command that understands it.

and so on.

Ahh… got it. I agree.
While the advanced filter syntax is compact, I often wish we could sneak in a command or two:-) Using a special symbol to denote a command, for example: create-list --tags "alphabetical lists" [!is[system]prefix[a]] > "[[Tiddlers Starting With A]]".
The points/problems you are bringing up makes it obvious this project’s a lot more complicated than it would on the surface and that we can’t just rush into this if we want it to be long-lasting.
Some of the fundamental groundwork of the syntax has to be laid down and it has to be robust. I am enjoying the thinking around your framework such as with the order of the commands to be executed.
Like I said, I’m going to have to read your posts a few times because it’s pretty deep.

3 Likes

Thanks @AlfieA it is an interesting proposal, and it is useful to work through it.

I think that this is essentially a new syntax for performing actions that is optimised for typing by humans.

The trouble is that we already have two different syntaxes/languages for performing actions:

  • Commands that are used under Node.js. These commands are also designed for humans to type. Currently we only offer primitive commands that frequently need to be composed together to achieve a desired outcome (eg via a batch script)
  • Action strings using action widgets <$action-...>. Again, the available actions are low level primitives and thus also will often need to be composed together, this time via procedures

I think it is unfortunate that we’ve ended up with two different languages with overlapping functionality. I’ve always hoped we can resolve it by introducing a single, unified action language that can be used everywhere: in action strings, on the Node.js command line, and in this new integrated command line within TiddlyWiki itself

One key wikitext ingredient has been in place for some time: the ability to specify actions as action strings rather than embedded action widgets frees us from having to use a syntax that is compatible with wikitext. In other words, the new syntax could resemble your example, and not need any <$action-...> widgets.

1 Like

Good points.

I know there are sites out there that simulate a programming console, such as W3 Schools or sites that teach how to utilize their IDEs, etc… But for TiddlyWiki, what might tentatively be called “WikiShell” or “TiddlyCLI”, the purpose would be to have full-blown functionality; the creation, deletion, modification and filtering of tiddlers as well as importing, exporting, parsing of tiddlers as well as batch processing. All modelled after PowerShell, Bash and DOS, syntax-wise.

I was thinking that a command console would be a natural fit since TiddlyWiki already has powerful advanced filtering. Two of the challenges specifically the conflicts, I have thought about. The feature would float above existing architecture would require no JavaScript reload, and could act on existing wikitext: by calling on tiddlers which contain those instructions.

In essence, the command line user would be entering proxy commands through the advanced search toolbar and upon the press of the return key, listeners (listener tiddlers) would be set up to determine whether the syntax is a search query or a command, the latter case of which would be handled by the respective tiddlers containing the widgets and wikitext. The command line user need not be familiar with the widgets’ syntax nor JavaScript. Naturally, this would only be the case for simple functions. However, modules could be installed later for more complex operations.

Here is the mock-up I have tried so far (it doesn’t work, though):

I created a tentative search query based on the kind suggestions of pmario (see above)
of the order of operations: modify-field "text"<"yes" "tiddler"<"test Tiddler A" Although a more conventional approach might be: modify-field --field="answer" --value="yes" --tiddler="test Tiddler A"

What I need to do is map each part of that syntax to its respective action.

I went to this tiddler:

Control Panel >>Plugins >>Core >>contents >>$:/core/ui/AdvancedSearch/Filter

In that, I’ve managed to create a console box by using a <$transclude> which directs to this tiddler: $:/custom/console (must be created). And in that “...console” tiddler, I have:

<$edit-text tiddler="$:/temp/command" field="text" placeholder="Enter command" type="text"/>
<$list filter="[[$:/temp/command]is[modified]]">
  <$macrocall $name="parseCommand" command={{$:/temp/command!!text}}/>
</$list>

I’m going to keep working on it and report to you guys. Safety goggles and protective mitts. Fire extinguisher is on the floor next to me.

I was thinking how to implement this for a while and decided to mockup an example of what I think parts would look like… especially to illustrate what i think are key concepts of the standard input, standard error and output for a wiki-command… Then the history of commands entered would look like this :

(abeit with some improved styling - less padding especilly)

The pipe | would then connect the output of one command (most likely always be a list ) to the standard input of the next. And each command could then provide some user friendly info on the execution status and/or errors. Possibly even provide user interactions in the info - like “Are you sure you want to delete 5 tiddlers - yes /no” with links/buttons there re-running that command and modifying the info box accordingly.

Each command i think would then be a procedure or similar like the following:

\procedure deleteTiddlers( stdin-list, stdout-tiddler)
<$set name="missingTiddlers" $filter="[enlist<stdin-list>is[missing]]">
    <%if [<missingTiddlers>count[]compare:gt[0]] %>
          <$action-setfield $tiddler=<<stdout-tiddler>> stderr="Tiddler(s) <missingTiddlers> missing" />
    <%endif%>
<$set>
<action-deleteTiddler $filter="[enlist<stdin-list>]" />
\end

Providing for other command line arguments would probably mean some functions that would substitute a typed value of say --json as an option to exporting a tiddler to a name value parameter like exportFilter="$:/core/templates/exporters/CsvFile" or similar.

I’ve not included the terminal prompt in my mockup - but I think it would be very valuable to have it autocomplete as you type commands (especially open and closing brackets such as <> and [] and select filter operations, commands and maybe variables from those available in the wiki).

:smiley:

Great input, @Christian_Byron !
Visually, very appealing and straightforward.

With a new line after the pipe… there is so much you could do with that. For example, you could “undo” steps, as it were, before executing the process. That’s just a thought, though.

You might want to see what those “5 tiddlers selected” actually are in a list, and that wouldn’t be a problem.

You’re right, the subject of autocomplete and error messages/suggestions comes up a lot. I haven’t thought how that would be implemented; perhaps in the “action” tiddlers themselves.

Hopefully, we can make this so intuitive that error messages would be very rare :slight_smile: Remember in the old days they didn’t tell you much, only: Syntax Error.

Do you have any good name ideas for the command console?

Somewhat different, I note from building Bookmarklets using javascript, many functions can also be called from the browser console. I wonder if one provides aliases, or documented functions, with the ability to reference or give parameters one could make a feature rich command line of sorts from the browser console? The language may thus be a meta language, Javascript or a subset of javascript.

So I started experimenting with the javascript parser peggy.js to see if this could provide a way to interpret the user input as being a valid command etc and so later on provide autocomplete options suitable to that context.

I’ve put the experiement up on glitch at : https://ceebee-peggyjs-poc.glitch.me/

At the moment it only parses the syntax for filter runs, but it validates the operators within that syntax based on the values provided at runtime. So if you use a invalid operator the error message provides the list of valid values ( useful for auto complete to filter on )

I’m pretty sure it would be easy to extend the grammar to a command syntax (commandname then arguements etc ) so that these could be auto completed as well … and once parsed the resulting tree could be used to correctly invoke a procedure…

Lots of possibilities
:smiley:

1 Like

I think a great way to start is using actions inside a button widget. If you find that insufficient, the next step would be to ask in help channels (here, discord, or even ChatGPT) if there is a way to accomplish what you’re trying to do. As developers (generically speaking), it is hard to predict exactly what a user wants to accomplish, but it’s much easier to translate from one existing way of doing things to another, or implement once we understand the desired use case.

Perhaps what we need isn’t command line arguments, but rather Tash (a name I just made up), which executes actions in a wikitext file the same way clicking a button does in the browser.

Also, the only thing keeping PowerShell from being truly great is the lack of intellisense. It would not be hard to build a working interface between TiddlyWiki and PowerShell.

Very interesting. That is something I’d use.