RFI: GitHub Actions Static Publishing Script

[saqimtiaz]

@oeyoews

Some configurations can be placed in the tiddlywiki.info file, such as building multiple html at the same time, custom directories, and tiddlywiki commands

I use this approach myself, but it is very unfriendly for non-technical users.

Here is my idea of how to create a script that is user friendly for non-technical users:

• In TiddlyWiki, have a tag that identifies tiddlers that contain static file rendering configurations, for example one configuration for HTML files, one for RSS files and one for JSON files. Each configuration tiddler would include:
  • template to use
  • tiddlers to render
• Have a script that is executed on push via Github Actions workflow that:
  • extracts the information from the configuration tiddlers
  • for each configuration tiddler, it invokes the tiddlywiki command to create the static files
• All static files are published to GitHub pages.

This way, a non-technical user can make all changes in their wiki, and push or save to Github and the static site will be automatically created. The user never has to configure anything in the script or actions.

[oeyoews]

For non-technical users, They often use tiddlywiki (nodejs or html) with other apps, maybe they want a button to publish

[saqimtiaz]

Absolutely. However, implementing this on the client side is considerable work and will require some rather large core improvements to do it properly. However, if we create a script or action as I described above, users can use it today. And in the future the core can provide a UI based mechanism that replaces the need for the script/action, but uses the same configuration tiddlers and templates. So users can easily transition from the script solution to a UI based one.

So the script is a potentially easy way to get non-technical users started with publishing static sites, and creating better templates for them, until the core has better browser based support for static publishing.

[oeyoews]

Combined with the template, is there any specific example, is it similar to the theme of hugo?

[saqimtiaz]

I think the default would be to use the same template as the static version of TiddlyWiki.com. See:

• https://tiddlywiki.com/#Generating%20Static%20Sites%20with%20TiddlyWiki
• https://tiddlywiki.com/static.html

(Note that the above documentation uses the deprecated rendertiddlers command and instead we should user the render command.)

Users could specify a different template in the static site configuration and the script would use that instead.

[boris]

I am closer to having this working. I’m using the defaults from “empty” edition.

@saqimtiaz I couldn’t find examples with render, do you have some?

Progress

Here’s where I’m at:

Couldn’t get it working at all

From single file wiki to output. Just fails silently? Verbose mode does nothing?

Solution: you must have a tiddlywiki.info file

You can grab the default one from empty edition.

Looking at that file was helpful. eg. I changed the name of static to index.html instead, and deleted the commands around alltiddlers.html.

Load from a single file wiki and build a static site

tiddlywiki --load twgroceries.html --build static

Open Questions

This outputs to /output/

• how to change this output folder? Specifically, I don’t want a static.html AND a subfolder of /static/, I’d like them all in one folder
• also I’d actually like to do the current folder I’m operating in, but --output ./ or similar don’t work

The “front page” gets rendered differently (without the custom Notebook theme), but all the individual tiddlers do get rendered with it. I guess I need to include the Notebook theme in the tiddlywiki.info? I don’t know what the path is for that theme.

Helpful links

• Easy method of setting up a static site? - #6 by Mohammad
• GitHub - saqimtiaz/TW5-github-actions-example: Example of how to use Github Actions to build and publish a TiddlyWiki

---

I am working on this here https://github.com/bmann/twgroceries

I’ve also enabled git saving, which is really nice. I can just hit save on my desktop and it pushes it up (still have to pull down the code to get an updated local copy, but decent for a single working session).

I think I can use the LogSeq Publish plugin and the “publish to gh-pages” branch after I get things building correctly.

Right now, if I were to check in the output folder, I could set that as the subfolder to build from, and I’d be done. But that’s completely manual and what we’re trying to fix

[saqimtiaz]

@boris

I couldn’t find examples with render, do you have some?

See tw5-feeds/plugins/feeds at main · saqimtiaz/tw5-feeds · GitHub

@boris

also I’d actually like to do the current folder I’m operating in, but --output ./ or similar don’t work

Place the output command before the build command, for example:
tiddlywiki --load twgroceries.html --output . --build static

@boris

how to change this output folder? Specifically, I don’t want a static.html AND a subfolder of /static/, I’d like them all in one folder

Check the last two lines for the static build target and the arguments passed therein which refence the path static. Not sure if you can set the path to the current folder but you could set the index.html for the wiki to also be in the same folder.

[boris]

Thanks @saqimtiaz!

I see your feed examples for render – and will want feeds! – I will attempt to replace these rendertiddler ones with render using your examples.

@saqimtiaz

Place the output command before the build command, for example:
tiddlywiki --load twgroceries.html --output . --build static

GROAN order dependent command line switches…

(and also silent failing with no errors…)

OK this worked – index.html in root, all other files in /static/

tiddlywiki --load twgroceries.html --output ./ --build static

And, editing the scripts:

"--rendertiddlers","[!is[system]]","$:/core/templates/static.tiddler.html","./","text/plain",

This works … but it deletes all the contents of the current root folder you run it from . Since I was operating in git and had just checked everything in, I just checked it out again so nothing lost. So, does need to go into some sub folder.

Whenever you run the main static tiddler flow, it deletes / recreates the folder, so it has to go first, and then you can generate an index file and it will end up in that same static folder:

"--rendertiddlers","[!is[system]]", "$:/core/templates/static.tiddler.html","static","text/plain",
"--rendertiddler", "$:/core/templates/static.template.html","static/index.html","text/plain",
"--rendertiddler", "$:/core/templates/static.template.css","static/static.css","text/plain"]

But! the index.html assumes a static subfolder, so all the relative paths there are wrong (appends an extra “static”). I guess that means spelunking in these core templates. I don’t want this to be example.com/static/sometiddler.html/, which is why I’m doing this – we should be able to have nice clean URLs like example.com/sometiddler.html.

So, feels like I’m making progress. Now just need to auto-run these commands via GH Actions.

[saqimtiaz]

See PR with build script for static rendering: Added static build script by saqimtiaz · Pull Request #1 · bmann/twgroceries · GitHub

• tiddlywiki.info file is not needed.
• TWCrossLinks plugin removed due to issues with static rendering (this is where browser rendering would be superior)
• renamed TWGroceries tiddler (and adjusted default tiddlers to suit) so that if you copy the static files to the same folder as the wiki, the tiddler file does not clobber the wiki file
• edited static template to remove /static prefix. Future improvement, don’t overwrite the template and override the link generation by providing a variable on the command line.
• uncomment last two lines in the build script if you want the static files in the same directory as the wiki, or you can copy the wiki to the same directory as the static files and publish that directory to GH Pages/Fission.

#!/bin/sh

#install tiddlywiki
npm install tiddlywiki

#clean up any remnants from last build
rm -fr wiki
rm -fr static

#unpack single file wiki into wiki folder
node node_modules/tiddlywiki/tiddlywiki.js --load twgroceries.html --savewikifolder ./wiki

#render each non-system tiddler as a static HTML file
node node_modules/tiddlywiki/tiddlywiki.js ./wiki --output ./ --render '[!is[system]]' '[encodeuricomponent[]addprefix[static/]addsuffix[.html]]'  'text/plain' '$:/core/templates/static.tiddler.html'

#render the css as an external file
node node_modules/tiddlywiki/tiddlywiki.js ./wiki --output ./ --render '$:/core/templates/static.template.css' 'static/static.css'  'text/plain'

#render the index page
node node_modules/tiddlywiki/tiddlywiki.js ./wiki --output ./ --render '$:/core/templates/static.template.html' 'static/index.html' 'text/plain'

#remove the wiki folder
rm -fr wiki

#uncomment below lines to copy static files to root directory of repo, warning if a tiddler has the same name as the wiki file it will be overwritten
#cp -a static/. .
#rm -fr static

[boris]

Thanks. I need to translate this into a GH Actions. Appreciate the examples!

Next is pushing the actions folder out into a separate repo so it can be re-used by anyone.

I’ll get to that and post as I continue.

[boris]

OK, @saqimtiaz I set this up in a separate repo and added you to it https://github.com/bmann/tiddlywiki-static-publish – no need for you to do anything, but let’s use that for issues and what not for anyone else in this thread that it’s interested.

It “works for me” – but barely accomplishes the minimum . I’ll keep tinkering in my repo to get RSS feeds and other things going in there.

[oeyoews]

Maybe it’s more friendly to give users an interactive script command that they can build locally

[Mohammad]

@oeyoews

Combined with the template, is there any specific example, is it similar to the theme of hugo?

This would be great! Having some minimal yet modern templates as optional along with the simple default one!

[Mohammad]

@oeyoews

Maybe it’s more friendly to give users an interactive script command that they can build locally

This is great! While having this also on the GitHub makes things simpler, and lets setup a website.

[Mark_S]

I’m late to the party here, and it looks like the clean-up crew is coming in. I have a small YAML script I use for converting plugins into a plugin library. Looking through code examples, I think on tiddly links, I was inspired by the commit “I hate YAML”.

[boris]

Hey @Mark_S – this is specifically to enable static publishing from a single file wiki or individual tiddler collection.

Here is the github repo to collaborate on / re-use: GitHub - bmann/tiddlywiki-static-publish: A set of scripts / Github Actions for Static Site Publishing for #TiddlyWiki

Here is my personal foodwiki repo GitHub - bmann/twgroceries: A non-linear personal grocery list powered by #TiddlyWiki → and the static version that is auto-published by Github Actions whenever I check in an update → https://foodwiki.bmann.ca

This can be accomplished in a number of different ways – e.g. TiddlyDesktop + git, single file wiki with git plugin, etc – that I’m going to screenshare / document. I’ve got issues in the shared repo on what needs doing Issues · bmann/tiddlywiki-static-publish · GitHub

[oeyoews]

Is it appropriate to refer to hugo, they use docker

GitHub - klakegg/docker-hugo: Truly minimal Docker images for Hugo open-source static site generator.