Announcing the MultiWikiServer Plugin

#43

Hi @JanJo I’m hoping MWS will be released before TiddlyWiki’s anniversary in September.

4 Likes
#44

Just to confirm, to install the prototype we need to run this command in the folder of our choice, right ?

git clone -b multi-wiki-support https://github.com/Jermolene/TiddlyWiki5.git
#45

This did it for me:

git clone https://github.com/Jermolene/TiddlyWiki5.git --branch multi-wiki-support

cd TiddlyWiki5

npm install

node ./tiddlywiki.js ./editions/multiwikiserver --listen
3 Likes
#46

Thank you @telumire and @LoryMoney. I’ve updated the plugin docs to improve the installation instructions, and also updated the readme for the branch (see GitHub - Jermolene/TiddlyWiki5 at multi-wiki-support).

1 Like
#47

@jeremyruston

Server-side, are there plans to fire events for updates to tiddlers in bags - such as updating a ‘$:/event/MultiWikiServer’ in $tw.wiki when a user updates a tiddler in a bag - thus would get a ‘change’ event to signal the database has changed?

Or is it already implemented and I am Missing it?

#48

Not exactly. The way that MWS works is that the “tiddlers” table has an ID field that is automatically incremented on each change. (See the SQLite docs for details).

This allows us to easily query the database to obtain a list of all tiddlers that have changed since a given previous change.

So, the equivalent of a change event listener is a periodic process that queries the database for the ID of the last change, and compares it with the last change that it knows about.

Not everything I have described is implemented yet – notably, deletions are not yet represented in the “tiddlers” table.

This will be the basis of a new, instantaneous synchronisation mechanism based on Server Sent Events.

As an aside, over the last few weeks I’ve been working on multipart form upload. This allows a browser form to upload huge files efficiently. On the server side, the incoming files are streamed to the file system and then turned into tiddler attachments, without ever being entirely stored in memory.

As things stand, bags and recipes are mirrored to $tw.wiki and so it would be possible to listen to change events on that wiki to learn of the creation of bags and recipes. But there are no plans to update the admin wiki for every updated tiddler, so change stamps would have to be used to track tiddler changes.

1 Like
#49

Hello, thanks for this interesting evolution of Tiddlywiki. I am not a developer but I started using Tiddlywiki more then 6 years ago, now, even for some specific task concerning my job.
I make use of some well known plugin, such as TiddlyMap and GSD5.
I succeeded to install and run the MultiWikiServer version and the system write perfectly in the SQLite file. I am planning to use this version for prototyping in my projets needing some data classification and visual analysis.
When I try to use those plugins with the recipes and bags, I am not able to upload them. Installing them by the plugin directory in the wiki datafolder or/and by adding lines in the tiddlywiki.info, does not work.
Could you suggest me what I am missing and how I can integrate plugins in the bags, please?

#50

Be aware that this plugin has to be considered beta- or even alpha-status. So I personally would in no way use it for production at this time in the cycle. “beta”-concepts can be still changed in completely incompatible ways, which can cause data loss on your side. As developers we need the freedom to change concepts at will if software is beta.

IMO experimenting is OK. Production may be not safe at the moment.

#51

Hi @jeremyruston

Have written a $tw.mws.store.getBagRecentTiddlers(bag_name, greater_than_tiddler_id, limit) function which returns recent new/changed/deleted tiddlers that can be used to poll for bag changes. If like the direction it is going will do one for recipes too.

Still have some final touches but wanted to see if is going in the right direction.

The response looks like :

{
  bag_name: 'bag-alpha',
  count: 1,
  bag_max_tiddler_id: 18947,
  greater_than_tiddler_id: 18944,
  tiddlers: [
    {
      title: 'In the bag',
      is_deleted: 0,
      tiddler_id: 18947,
      fields: { title: 'In the bag', text: 'This tiddler is still in the bag-alpha' }
    }
  ]
}

The value of bag_max_tiddler_id is passed to the next request to get the next tiddler change if there is one. And empty response looks like :

{
  bag_name: 'bag-alpha',
  count: 0,
  bag_max_tiddler_id: 18944,
  greater_than_tiddler_id: 18944,
  tiddlers: []
}

More details and the code added to SqlTiddlerStore and SqlTiddlerDatabase are in this gist. Plus a REPL output of it working.

If design looks good - will make some changes to harden it up and build the tests. Then work on same design for recipes.

Not happy with the field names. greater_than_tiddler_id should be last_known_tiddler_id ?

1 Like
#52

Thank you @poc2go, that’s great, and very helpful. I’ve just finished a bunch of preparatory refactoring so that I can start on the new synchronisation mechanism. I’m hoping in a few days to have Server Sent Events up and running so that we have near-instantaneous updates.

I agree that “last_known_tiddler_id” is clearer.

Would you be able to prepare a PR? Many thanks.

#53

Yes, indeed, as I said, I’am using the flexibility offered by tiddlywiki for prototyping (that is not production IMO), including operational procedures for knowledge management.

Thanks in advance for your suggestion in order to make TiddlyMap work in bags/recipes architecture deployed by multiwikiserver plugin (i.e. SQLite-based storage)!

#54

If one is running an instance that contains multiple bags, is there a way to tell from within the TiddlyWiki interface which bag a tiddler is associated with, and furthermore change it? I could not see where this is the case. Thank you!

#55

Hi @jeremyruston ,
I am really craving for the new possibilities. Did you create a test instance where we could try the multi-user capabilities.

2 Likes
#56

The bag to which a tiddler belongs is shown at the top of the tiddler info panel.

There will be a way to directly move tiddlers between bags, but at the moment that can only be accomplished via drag and drop.

#57

Thank you @JanJo I am not currently running a public instance of MWS. I am not personally interested in undertaking the obligation of running a freely editable instance, and having to worry about spam or pornbots.

More of a focus for me is making MWS easy enough to install with a cloud provider that anyone can do it; for many providers it ought to be possible to set things up so that installation is a single click.

6 Likes
#58

As a super user not a developer and only at an appropriate time in your development cycle.

Please publish information about how the concepts, interface and options that regular (not developers) will use before this goes to general release. Ideally with a demo we can play with.

We do not currently have a “user acceptance testing phase” in tiddlywiki development, except perhaps for developers, and regular users may have valuable feedback that can both improve the solution, help build the documentation, detect bugs and improve the quality of the first pubished version.

  • Doing this for public release has many benefits;
    • Help reduce the need for multiple release fixing bugs
    • Reduce the Negative responses to change or complaints about limitations
    • Help with adoption thus get real and useful feedback
    • Avoiding too much jargon
    • Including features to meet real world cases, not just that in the LAB

Yours Sincerely
Tony

1 Like
#59

This thread includes an extensive explanation of the bags and recipes model, notably in the original post. I understand the general desire for more documentation, but the most practical way forwards would be to get your feedback on the material that has already been published.

1 Like
#60

At the moment I am more than happy with my single-file Tiddlywiki, but I have to admit that I have one eye on the multiwiki server version.

I’m not keen on the final version only running via a cloud provider. Of course, I realise that cloud applications will become more and more important. But I’m rather old-fashioned and prefer to have the data “within reach” on my own server.

A server version that runs on a Raspberry in my home network, for example, is therefore ideal for my application. I use a VPN for the necessary access while travelling.
It is therefore not important to me whether the MWS would be protected against bot attacks or not and I would welcome it if all the necessary core functions run satisfactorily beforehand.
Bot security is then interesting for the cloud version and can wait a little longer in my opinion.

I’m in no great hurry with the MWS, so please don’t feel pressurised. As I said, I am also very happy with the single-file version.
For me, the motto is: it’s ready when it’s ready.

At this point a big thank you to everyone who has brought TiddlyWiki to where it is today and how it works today - you are fantastic, thank you for your magnificent work! :+1: :sunglasses: :+1:

1 Like
#61

I will, my point is a larger audience, who is not me and less knowledgeable. Even for me some of the Jargon is hard work.

#62

Hi! :slight_smile: It might be obvious, but after cloning the git repository, what command should one use to initiate an empty wiki with MSW?