Edit documentation in TW and use Github API to create PRs directly?

[TW_Tones]

@saqimtiaz this is a very good idea. I am sure we can get a sufficient number of developers and enthusiasts to utilise this method. This will itself help build the documentation.

However I am not sure a novice or even regular users/designers will adopt this. For this reason I think a/the discourse Documentation category could be a place for contributors to post updated tiddlers that those with this ability may load into their Git activated wiki and submit on behalf of the contributor.

• Thus Being able to annotate such PR’s with the contributor or even a talk.tiddlywiki.org link makes sense.
• To make it easy share instructions how to export a modified tiddler or tiddlers for application to the documentation.
• Consider the possibility of being able to submit changes to other wiki repositories/trees.
  ** Eg a links repository, extended documentation or references, self submitted references etc…

Perhaps in the future a publicly accessible GuitHub could allow “anon” contributions into a repository from a website. Still demanding a review before actioned.

Good work @saqimtiaz, looking out for community needs.

[saqimtiaz]

@pmario @jeremyruston @boris and @everyone

This is what a PR generated via TiddlyWiki looks like:
(edit: replaced with a better example)

Docs: remove erroneous tag from enlist-input operator.

What I did:

• Edited the tiddler “enlist-input Operator” in TiddlyWiki
• In the interface shown below:
  • Added a PR title and PR message
  • Added the tiddler title
  • clicked on “Make a PR”
  • smiled like crazy and realized I am up way too late.

I think even I would find this an easier way to make docs PRs than my current node.js workflow.

Good night!

Edit: if the user does not already have a fork of the TiddlyWiki repo it is automatically created.

[saqimtiaz]

@pmario asks on Github:

Is it possible to add changes (new commits) to those tiddlers later on, or is that a new PR?

With the current code it would be a new PR. However, it is possible to implement the ability to add new commits to the same PR as well.

We are using 2 dependencies, a low level library (octokit) that covers the entire Github API and a plugin for it that makes it very easy to create PRs. So either we use the low level library to update the PR or we find a plugin for it.

The real catch is going to be the user side of things and the copy of tw.com that they use.

• we need to make sure they always start with a fresh copy of the site
• if we want to allow updating a PR, then they would need to have saved their changes in TW somewhere
  • unless we use Github actions to generate a TW file for every PR, and then that file can be used to further update the same PR as long as it hasn’t fallen far behind the tw.com branch in terms of changes.
• Or we have a one click option to clone tw.com to say TiddlyHost (with their permission and consideration of cost) as starting point for each PR. The wiki persists at TH while the PR is open.

[pmario]

@saqimtiaz

The real catch is going to be the user side of things and the copy of tw.com that they use.

OK. I haven’t had a look at the dependences yet. …

For the start I would say “keep it simple” and iterate often. So we can deal with several commits to the same PR later.

[saqimtiaz]

@pmario I’ll post a demo soon that everyone can play with. It will create PRs but it will target my fork so as to not spam the main repo while testing

[saqimtiaz]

I have posted a demo:

Demo: updating docs and creating a PR from within TiddlyWiki

A lot of room for refinement as well as constraints in terms of what kinds of PRs are allowed etc, but the basic workflow works. You will need to create a Github Personal Access Token Demo: TiddlyWiki — a non-linear personal web notebook Background: Edit documentation in TW and use Github API to create PRs directly?

[pmario]

@saqimtiaz

@pmario I’ll post a demo soon that everyone can play with. It will create PRs but it will target my fork so as to not spam the main repo while testing

I don’t want to spam your account either. … What do you think, should we create a “throw away” organisation, that we can “spam” with test PRs without collecting too much garbage in our own repos?

[pmario]

@pmario

OK. I haven’t had a look at the dependences yet. …

I did have a closer look at octokit.js … It seems once a PR is created, it will automatically pick up new commits to files in the same “feature branch”. …

So instead of creating a new PR we just need to update the file. … The only thing which seems to be important is the “SHA of the parent commit” … So it may be important what the “create PR workflow” returns. … So a bit more testing is needed.

[saqimtiaz]

@pmario

I don’t want to spam your account either. … What do you think, should we create a “throw away” organisation, that we can “spam” with test PRs without collecting too much garbage in our own repos?

We can, though it really isn’t a problem to spam my fork. I can simply delete my fork and recreate it to get rid of the noise. When I don’t have open PRs I consider forks of public repos to be disposable.

I do have a throwaway account I created for testing CI for twlinks, I can look up the details for that later if need be.

@pmario

I did have a closer look at octokit.js … It seems once a PR is created, it will automatically pick up new commits to files in the same “feature branch”. …

So instead of creating a new PR we just need to update the file. … The only thing which seems to be important is the “SHA of the parent commit” … So it may be important what the “create PR workflow” returns. … So a bit more testing is needed.

Yep if we use the low level octokit it is all doable. The create PR plugin makes creating a PR easier but only returns the PR number. So either we need to use that and the branch name to fetch the SHA for the parent so we can update the same branch, or just handle PR creation ourselves with octokit and not use the plugin. Both would work.

Anyway, I do think we shouldn’t get too caught up on that at the moment and consider that as Step 2, with Step 1 being getting a decent workflow in place for creating a PR in one go (without facilities for updates).

(An important bit with being able to update a PR is that you will need to have the TiddlyWiki you previously used to create it.)

[saqimtiaz]

Leaving this as a note to myself (easy fix): files created for new files don’t currently get a .tid extension.

[pmario]

@saqimtiaz

Anyway, I do think we shouldn’t get too caught up on that at the moment and consider that as Step 2, with Step 1 being getting a decent workflow in place for creating a PR in one go (without facilities for updates).

That’s right. I think the PR-plugin abstracts a lot of “octokit.core” complexity away and is a very good start. …

As a user of our PR workflow I would like to have a bit more feedback. The “button that links to the PR” at the end isn’t enough for me.

Some messages like like:

• Creating a pull-request
• Work in progress … (may be animated dots)
• Success - Link to PR

would already work for the beginning. … Just some thoughts.

[saqimtiaz]

@pmario agreed, as I mentioned both the user workflow and the UI need work. Definitely needs more error handling too.

What is there now is what I considered the absolute minimum for a demo.

Do feel free to tweak and modify if you feel inspired.

[Mark_S]

Various thoughts

I feel like this is usable as is, solving the problem of multiple tiddlers, viewing the tiddler as edited, and eliminating the need for a local repository. For experienced users like Pmario it may be limiting, but for everyone else it’s a door-opener. Among other things, it means you can be at any computer, maybe even a tablet or phone, when you submit changes.

I’m not sure what interface improvements would make it qualitatively better. Dragging and dropping your tiddlers from the “Recent Tiddlers” tab is pretty easy for anyone who’s at the level of someone who is interested in doing this.

The main thing missing from the UI might be a way to add paths for new tiddlers. I’m thinking they could be inserted into “$:/config/OriginalTiddlerPaths” ?

Maybe also instructions to the user to refresh their fork if it’s been awhile since submitting updates. Though I’m not sure if that’s even necessary if the source tiddlers are coming from tiddlywiki. com.

Apparently every PR has to be submitted on a unique branch, which is unfortunate. But the form could allow you to submit a branch name, or branch prefix, for every PR. Which would make it easier for someone who wants to do final touch-ups by hand at GH.

The plugin only allows the creation of PR’s. One way around this limitation would be to have a convention where fix-it PR’s could be submitted with prefix “Supercedes PR#xxxx” . Then whoever is doing the pull could know to skip the prior PR request. This is operating on the theory that most of the time the first PR is good enough.

If the request form and code were included in TiddlyWiki .com , then just by refreshing the page a user would have the latest set of tiddlers. (minus any that haven’t been merged yet).

[saqimtiaz]

Hi @Mark_S thank you for the considered thoughts, very helpful.

Note that I have updated the demo this morning with a few changes:

Demo: updating docs and creating a PR from within TiddlyWiki

I have pushed through some quick changes, hopefully I haven’t broken anything. I have been spending time on this that I really do not have to spare at present, so it is my hope that others will chip in to get this to a state where it is usable. The code itself is very concise and not complex. Changes: Disable “Make a PR” button and show an error message if the tiddler titles entered do not correspond to real tiddlers. Primary concern here is someone forgetting brackets around a title with s…

@Mark_S

I feel like this is usable as is, solving the problem of multiple tiddlers, viewing the tiddler as edited, and eliminating the need for a local repository. For experienced users like Pmario it may be limiting, but for everyone else it’s a door-opener. Among other things, it means you can be at any computer, maybe even a tablet or phone, when you submit changes.

I think the primary obstacle is going to be when someone submits a PR and gets asked to correct something minor before it can be merged. Without delving into the Github UI there will be no way to easily update the PR that they have created. But otherwise I agree, this should already be a step forwards.

@Mark_S

I’m not sure what interface improvements would make it qualitatively better. Dragging and dropping your tiddlers from the “Recent Tiddlers” tab is pretty easy for anyone who’s at the level of someone who is interested in doing this.

The main concern I had was users forgetting brackets around tiddler titles with spaces and thus creating broken PRs. I have introduced some wikitext handling to prevent this.

@Mark_S

The main thing missing from the UI might be a way to add paths for new tiddlers. I’m thinking they could be inserted into “$:/config/OriginalTiddlerPaths” ?

@pmario and I discussed this in either this thread or the one with the demo. Even on node.js there is nothing to guide where new tiddlers go in the directory structure. What would be helpful here both for this work and for updating docs via node.js is a $:/config/FileSystemPaths tiddler for the tw-com edition.

Adding new tiddler paths to “$:/config/OriginalTiddlerPaths” doesn’t help much if all new tiddlers are just being added to the root of the tiddlers directory. I would rather users don’t have to decide where the tiddlers go and it can be calculated from the $:/config/FileSystemPaths tiddler.

@Mark_S

Maybe also instructions to the user to refresh their fork if it’s been awhile since submitting updates. Though I’m not sure if that’s even necessary if the source tiddlers are coming from tiddlywiki. com.

Right. I think the real issue is figuring out the user story. Is this installed on TiddlyWiki.com? or at a different URL that we link to from tw-com and which mirrors it but also adds the plugin? What if the user is working on larger changes and wants to save changes during their work (download saver?). Figuring out a recommended process and documenting would probably help those less familiar with working with TiddlyWiki.

@Mark_S

The plugin only allows the creation of PR’s. One way around this limitation would be to have a convention where fix-it PR’s could be submitted with prefix “Supercedes PR#xxxx” . Then whoever is doing the pull could know to skip the prior PR request. This is operating on the theory that most of the time the first PR is good enough.

Yes that kind of thing can work. What would be needed is a way for them to continue where they left off with their changes. Do we assume that the user has saved the file in which they made changes? If we had the ability to generate TiddlyWiki files from PRs that could help (and is possible via Github Actions). They could take the file from their previous PR and use that as a starting point for revising.

[saqimtiaz]

I don’t expect to have the opportunity to devote much if any time to this in the near future. So I am hoping the community can brainstorm around this and figure out next steps towards getting this usable by those less TW or tech savvy, and then I can chip where really needed when I have the chance.

[saqimtiaz]

I have thrown up the code here: GitHub - saqimtiaz/tw5-docs-pr-maker: Create Docs PRs from TiddlyWiki

And set up a demo that is regenerated every 5 minutes with the latest tw-com content:
https://saqimtiaz.github.io/tw5-docs-pr-maker/

[Mark_S]

@saqimtiaz

I think the primary obstacle is going to be when someone submits a PR and gets asked to correct something minor before it can be merged.

If it’s truly minor they can fix it from GH. If it’s not truly minor, they can submit a new PR and cancel the old one.

@saqimtiaz

The main concern I had was users forgetting brackets around tiddler titles with spaces and thus creating broken PRs. I have introduced some wikitext handling to prevent this.

Dragging and dropping titles automatically inserts the brackets.

My assumption is that anyone submitting documentation knows TiddlyWiki really well – they know about TW titles. They know about saving. They just don’t know or want to know about the arcane, arbitrary, byzantine world of GitHub.

@saqimtiaz

What would be helpful here both for this work and for updating docs via node.js is a $:/config/FileSystemPaths tiddler for the tw-com edition.

I think most users are smart enough to put new tiddlers into directories that match similar existing tiddlers. I can’t imagine that there’s any FSP setting that can substitute for human pattern matching. We could list out the existing directories and have the user choose from one of them.

@saqimtiaz

Right. I think the real issue is figuring out the user story. Is this installed on TiddlyWiki.com?

That would be ideal. But it obviously depends on if Jeremy approves.

Otherwise, if there was a way for a mirror site that is identical to TiddlyWiki .com except that it has your documentation tiddlers, that would be great since it would allow people to make changes from anywhere even without access to a hard drive.

Or, people could just dnd your files into their own copy of TW, save, and reload.

@saqimtiaz

What if the user is working on larger changes and wants to save changes during their work (download saver?).

I think we have different starting places in mind for the user. Any user submitting documentation is likely pretty well versed in the TiddlyWiki world. They know about saving. They know about the emergency saver. Or we can remind them of it if it’s a concern. They can save their work and come back if they need to submit a 2nd PR. Or they can wait for the original PR to get merged, and then work on that.

Because of the one-shot PR mechanism, the ideal use would be for someone who is making a set of changes in one session. And then come back after the changes have been merged if further enhancements are required. Someone who is doing an entire rewrite of the TW core documentation should probably put on their GH hat and do it from a local repository.

I just realized another issue. Does the current code handle tiddler renaming and/or deletion?

Thanks!

[pmario]

@saqimtiaz

And set up a demo that is regenerated every 5 minutes with the latest tw-com content:

hmm. I think 5 minutes is a bit overkill for a prove of concept.

[saqimtiaz]

It makes no difference, GH Action cron jobs run according to availability so it averages out to a few times per hour at most.

[saqimtiaz]

@Mark_S

I think most users are smart enough to put new tiddlers into directories that match similar existing tiddlers. I can’t imagine that there’s any FSP setting that can substitute for human pattern matching. We could list out the existing directories and have the user choose from one of them.

FileSystemPaths was created for exactly this and the fact that they aren’t configured for tw-com is a constant pain point on node.js as well. Most documentation tiddlers follows very obvious patterns for which directories they go in to.

That said, if someone wants to explore a UI for specifying directories that would be welcome.

@Mark_S

Otherwise, if there was a way for a mirror site that is identical to TiddlyWiki .com except that it has your documentation tiddlers, that would be great since it would allow people to make changes from anywhere even without access to a hard drive.

That is what the demo currently does, though ideally we would have the main TW repository trigger an update of the mirror (via a Github Action) every time there are new changes to the tw-com branch, rather than have it on a timer.

@Mark_S

I just realized another issue. Does the current code handle tiddler renaming and/or deletion?

To TiddlyWiki, renaming a tiddler is deleting the tiddler and creating one with a new name. The plugin we are using handles deleting files but we would need to add an affordance for that in the UI, or have a way to track and remember renames and pass that to the code making the PR.