Edit documentation in TW and use Github API to create PRs directly?

[saqimtiaz]

@pmario

I don’t want to spam your account either. … What do you think, should we create a “throw away” organisation, that we can “spam” with test PRs without collecting too much garbage in our own repos?

We can, though it really isn’t a problem to spam my fork. I can simply delete my fork and recreate it to get rid of the noise. When I don’t have open PRs I consider forks of public repos to be disposable.

I do have a throwaway account I created for testing CI for twlinks, I can look up the details for that later if need be.

@pmario

I did have a closer look at octokit.js … It seems once a PR is created, it will automatically pick up new commits to files in the same “feature branch”. …

So instead of creating a new PR we just need to update the file. … The only thing which seems to be important is the “SHA of the parent commit” … So it may be important what the “create PR workflow” returns. … So a bit more testing is needed.

Yep if we use the low level octokit it is all doable. The create PR plugin makes creating a PR easier but only returns the PR number. So either we need to use that and the branch name to fetch the SHA for the parent so we can update the same branch, or just handle PR creation ourselves with octokit and not use the plugin. Both would work.

Anyway, I do think we shouldn’t get too caught up on that at the moment and consider that as Step 2, with Step 1 being getting a decent workflow in place for creating a PR in one go (without facilities for updates).

(An important bit with being able to update a PR is that you will need to have the TiddlyWiki you previously used to create it.)

[saqimtiaz]

Leaving this as a note to myself (easy fix): files created for new files don’t currently get a .tid extension.

[pmario]

@saqimtiaz

Anyway, I do think we shouldn’t get too caught up on that at the moment and consider that as Step 2, with Step 1 being getting a decent workflow in place for creating a PR in one go (without facilities for updates).

That’s right. I think the PR-plugin abstracts a lot of “octokit.core” complexity away and is a very good start. …

As a user of our PR workflow I would like to have a bit more feedback. The “button that links to the PR” at the end isn’t enough for me.

Some messages like like:

• Creating a pull-request
• Work in progress … (may be animated dots)
• Success - Link to PR

would already work for the beginning. … Just some thoughts.

[saqimtiaz]

@pmario agreed, as I mentioned both the user workflow and the UI need work. Definitely needs more error handling too.

What is there now is what I considered the absolute minimum for a demo.

Do feel free to tweak and modify if you feel inspired.

[Mark_S]

Various thoughts

I feel like this is usable as is, solving the problem of multiple tiddlers, viewing the tiddler as edited, and eliminating the need for a local repository. For experienced users like Pmario it may be limiting, but for everyone else it’s a door-opener. Among other things, it means you can be at any computer, maybe even a tablet or phone, when you submit changes.

I’m not sure what interface improvements would make it qualitatively better. Dragging and dropping your tiddlers from the “Recent Tiddlers” tab is pretty easy for anyone who’s at the level of someone who is interested in doing this.

The main thing missing from the UI might be a way to add paths for new tiddlers. I’m thinking they could be inserted into “$:/config/OriginalTiddlerPaths” ?

Maybe also instructions to the user to refresh their fork if it’s been awhile since submitting updates. Though I’m not sure if that’s even necessary if the source tiddlers are coming from tiddlywiki. com.

Apparently every PR has to be submitted on a unique branch, which is unfortunate. But the form could allow you to submit a branch name, or branch prefix, for every PR. Which would make it easier for someone who wants to do final touch-ups by hand at GH.

The plugin only allows the creation of PR’s. One way around this limitation would be to have a convention where fix-it PR’s could be submitted with prefix “Supercedes PR#xxxx” . Then whoever is doing the pull could know to skip the prior PR request. This is operating on the theory that most of the time the first PR is good enough.

If the request form and code were included in TiddlyWiki .com , then just by refreshing the page a user would have the latest set of tiddlers. (minus any that haven’t been merged yet).

[saqimtiaz]

Hi @Mark_S thank you for the considered thoughts, very helpful.

Note that I have updated the demo this morning with a few changes:

Demo: updating docs and creating a PR from within TiddlyWiki

I have pushed through some quick changes, hopefully I haven’t broken anything. I have been spending time on this that I really do not have to spare at present, so it is my hope that others will chip in to get this to a state where it is usable. The code itself is very concise and not complex. Changes: Disable “Make a PR” button and show an error message if the tiddler titles entered do not correspond to real tiddlers. Primary concern here is someone forgetting brackets around a title with s…

@Mark_S

I feel like this is usable as is, solving the problem of multiple tiddlers, viewing the tiddler as edited, and eliminating the need for a local repository. For experienced users like Pmario it may be limiting, but for everyone else it’s a door-opener. Among other things, it means you can be at any computer, maybe even a tablet or phone, when you submit changes.

I think the primary obstacle is going to be when someone submits a PR and gets asked to correct something minor before it can be merged. Without delving into the Github UI there will be no way to easily update the PR that they have created. But otherwise I agree, this should already be a step forwards.

@Mark_S

I’m not sure what interface improvements would make it qualitatively better. Dragging and dropping your tiddlers from the “Recent Tiddlers” tab is pretty easy for anyone who’s at the level of someone who is interested in doing this.

The main concern I had was users forgetting brackets around tiddler titles with spaces and thus creating broken PRs. I have introduced some wikitext handling to prevent this.

@Mark_S

The main thing missing from the UI might be a way to add paths for new tiddlers. I’m thinking they could be inserted into “$:/config/OriginalTiddlerPaths” ?

@pmario and I discussed this in either this thread or the one with the demo. Even on node.js there is nothing to guide where new tiddlers go in the directory structure. What would be helpful here both for this work and for updating docs via node.js is a $:/config/FileSystemPaths tiddler for the tw-com edition.

Adding new tiddler paths to “$:/config/OriginalTiddlerPaths” doesn’t help much if all new tiddlers are just being added to the root of the tiddlers directory. I would rather users don’t have to decide where the tiddlers go and it can be calculated from the $:/config/FileSystemPaths tiddler.

@Mark_S

Maybe also instructions to the user to refresh their fork if it’s been awhile since submitting updates. Though I’m not sure if that’s even necessary if the source tiddlers are coming from tiddlywiki. com.

Right. I think the real issue is figuring out the user story. Is this installed on TiddlyWiki.com? or at a different URL that we link to from tw-com and which mirrors it but also adds the plugin? What if the user is working on larger changes and wants to save changes during their work (download saver?). Figuring out a recommended process and documenting would probably help those less familiar with working with TiddlyWiki.

@Mark_S

The plugin only allows the creation of PR’s. One way around this limitation would be to have a convention where fix-it PR’s could be submitted with prefix “Supercedes PR#xxxx” . Then whoever is doing the pull could know to skip the prior PR request. This is operating on the theory that most of the time the first PR is good enough.

Yes that kind of thing can work. What would be needed is a way for them to continue where they left off with their changes. Do we assume that the user has saved the file in which they made changes? If we had the ability to generate TiddlyWiki files from PRs that could help (and is possible via Github Actions). They could take the file from their previous PR and use that as a starting point for revising.

[saqimtiaz]

I don’t expect to have the opportunity to devote much if any time to this in the near future. So I am hoping the community can brainstorm around this and figure out next steps towards getting this usable by those less TW or tech savvy, and then I can chip where really needed when I have the chance.

[saqimtiaz]

I have thrown up the code here: GitHub - saqimtiaz/tw5-docs-pr-maker: Create Docs PRs from TiddlyWiki

And set up a demo that is regenerated every 5 minutes with the latest tw-com content:
https://saqimtiaz.github.io/tw5-docs-pr-maker/

[Mark_S]

@saqimtiaz

I think the primary obstacle is going to be when someone submits a PR and gets asked to correct something minor before it can be merged.

If it’s truly minor they can fix it from GH. If it’s not truly minor, they can submit a new PR and cancel the old one.

@saqimtiaz

The main concern I had was users forgetting brackets around tiddler titles with spaces and thus creating broken PRs. I have introduced some wikitext handling to prevent this.

Dragging and dropping titles automatically inserts the brackets.

My assumption is that anyone submitting documentation knows TiddlyWiki really well – they know about TW titles. They know about saving. They just don’t know or want to know about the arcane, arbitrary, byzantine world of GitHub.

@saqimtiaz

What would be helpful here both for this work and for updating docs via node.js is a $:/config/FileSystemPaths tiddler for the tw-com edition.

I think most users are smart enough to put new tiddlers into directories that match similar existing tiddlers. I can’t imagine that there’s any FSP setting that can substitute for human pattern matching. We could list out the existing directories and have the user choose from one of them.

@saqimtiaz

Right. I think the real issue is figuring out the user story. Is this installed on TiddlyWiki.com?

That would be ideal. But it obviously depends on if Jeremy approves.

Otherwise, if there was a way for a mirror site that is identical to TiddlyWiki .com except that it has your documentation tiddlers, that would be great since it would allow people to make changes from anywhere even without access to a hard drive.

Or, people could just dnd your files into their own copy of TW, save, and reload.

@saqimtiaz

What if the user is working on larger changes and wants to save changes during their work (download saver?).

I think we have different starting places in mind for the user. Any user submitting documentation is likely pretty well versed in the TiddlyWiki world. They know about saving. They know about the emergency saver. Or we can remind them of it if it’s a concern. They can save their work and come back if they need to submit a 2nd PR. Or they can wait for the original PR to get merged, and then work on that.

Because of the one-shot PR mechanism, the ideal use would be for someone who is making a set of changes in one session. And then come back after the changes have been merged if further enhancements are required. Someone who is doing an entire rewrite of the TW core documentation should probably put on their GH hat and do it from a local repository.

I just realized another issue. Does the current code handle tiddler renaming and/or deletion?

Thanks!

[pmario]

@saqimtiaz

And set up a demo that is regenerated every 5 minutes with the latest tw-com content:

hmm. I think 5 minutes is a bit overkill for a prove of concept.

[saqimtiaz]

It makes no difference, GH Action cron jobs run according to availability so it averages out to a few times per hour at most.

[saqimtiaz]

@Mark_S

I think most users are smart enough to put new tiddlers into directories that match similar existing tiddlers. I can’t imagine that there’s any FSP setting that can substitute for human pattern matching. We could list out the existing directories and have the user choose from one of them.

FileSystemPaths was created for exactly this and the fact that they aren’t configured for tw-com is a constant pain point on node.js as well. Most documentation tiddlers follows very obvious patterns for which directories they go in to.

That said, if someone wants to explore a UI for specifying directories that would be welcome.

@Mark_S

Otherwise, if there was a way for a mirror site that is identical to TiddlyWiki .com except that it has your documentation tiddlers, that would be great since it would allow people to make changes from anywhere even without access to a hard drive.

That is what the demo currently does, though ideally we would have the main TW repository trigger an update of the mirror (via a Github Action) every time there are new changes to the tw-com branch, rather than have it on a timer.

@Mark_S

I just realized another issue. Does the current code handle tiddler renaming and/or deletion?

To TiddlyWiki, renaming a tiddler is deleting the tiddler and creating one with a new name. The plugin we are using handles deleting files but we would need to add an affordance for that in the UI, or have a way to track and remember renames and pass that to the code making the PR.

[Mark_S]

@saqimtiaz

$:/config/FileSystemPaths tiddler for the tw-com edition.

If you had this, how would you use it? And the meta question, why is it not already populated?

My question in the other thread about performance actually relates to this. I was analyzing the official paths. There are 1383 unique tiddlers, and 65 unique paths.
There’s no guarantee, of course, that those tiddlers can all be sorted into the paths by particular rules. It would take awhile to analyze them, and it may be that some existing tiddlers would either need to be moved (physical location), tagged, or be renamed to fit the pattern. So a substantial sub-project. Is it really worth pursuing that approach?

[saqimtiaz]

@Mark_S

If you had this, how would you use it? And the meta question, why is it not already populated?

If the file system paths filters were defined, they could be used to generate file paths for every new tiddler. We already do this in the core for node.js and it is relatively straightforward.

As for why it isn’t populated I can only speculate the facility was introduced after the edition was already established and no one has done the leg work.

@Mark_S

It would take awhile to analyze them, and it may be that some existing tiddlers would either need to be moved (physical location), tagged, or be renamed to fit the pattern. So a substantial sub-project. Is it really worth pursuing that approach?

It is a constant pain point when writing documentation for new features in the core, so I do feel it is worth doing. It need not be a part of this work however.

If someone wants to take on creating an intuitive UI affordance for users to be able to specify the path for a new tiddler I am very much open to that. The UI should be all wikitext work and if that is in place I can try to find time to wire that up to the JavaScript.

My only concern is that the user experience should be as intuitive as possible to allow less experienced users to be able to make smaller documentation contributions and corrections as well if they wish to do so.

[Mark_S]

I may be working towards something like

[pmario]

@saqimtiaz

My only concern is that the user experience should be as intuitive as possible to allow less experienced users to be able to make smaller documentation contributions and corrections as well if they wish to do so.

That’s why I would like a “tags” approach. So if newbie users tag a new tiddler it will be automatically sorted into the right path.

If there is no tag, they land in “tiddlers” … and we will need to tag them later … or they stay there

[pmario]

I also think it would be an advantage, if someone would the TW TOC. There are a lot of tiddlers that are shown in several places. From my point of view that’s a real pain point, even for experienced users.

At TW-com I use search only. The TOC is completely useless for me. … It would be useful if it would “auto-expand” and show open and viewed tiddlers … BUT that’s a completely different topic.

[jeremyruston]

@pmario

I also think it would be an advantage, if someone would the TW TOC. There are a lot of tiddlers that are shown in several places. From my point of view that’s a real pain point, even for experienced users.

Just to say that that is intentional, and is intended to help with the cases that topics belong under multiple headings. For me, that ability is a great advantage of TWs TOC compared to a regular outline or directory structure.

[Mark_S]

Well, talk about winning the battle but losing the war. I’ve got the interface done (donish, there’s always more ) but I’m getting this message when creating the PR:

There was an error in creating the PR. HttpError: tree.path contains a malformed path component

I’m wondering it something in the PR kit doesn’t like new tiddlers with a specified path? The only thing my code does is update OriginalTiddlerPaths . So when the tiddler gets pushed, and the API finds it doesn’t have a match (tree.path ?) … it complains? Maybe?

Here’s the two tiddlers I’ve worked with –

2021-10-27-Push_PR.json (4.0 KB)