Has anyone generated an RSS feed from TiddlyWiki?

boris · October 1, 2021, 3:50pm

I am wondering if anyone has a template for static publishing of a TiddlyWiki into an RSS feed?

I haven’t worked much with static publishing but as far as I can tell it should be possible.

For now, I’m thinking of a simplistic version where I publish the single file TW, but also a feed.xml that just points to the #hash-links.

An advanced version would generate the static HTML of all tiddlers, and then re-use the paths of the tiddlers as the permalinks in the feed. Using the same format for generating the path/to/file/slugs.html and running it twice would work, don’t know if WikiText is sophisticated enough to do this in one pass?

Anyone have some existing snippets or guides around this?

Any other documentation / tips around static publishing would also be appreciated.

TW_Tones · October 1, 2021, 10:13pm

Boris,
RSS was provisioned in TiddlyWiki classic but has being neglected since in part because of the file save issues since “the Firefox apocalypse” when browsers got much more restrictive.

Some guidance

The Node version can automatically publish “static or HTML” tiddlers in addition to the interactive wiki. This is done via a template, a version of which could be crafted to publish an RSS feed.
For the single file wikis;
- I expect there is a way to allow “secondary files” generated from a template to be saved to the host. Yet to be developed but long sought after by myself.
- As long as you create a template that can generate the RSS feed, ie extract the required information from tiddlers such as filtered on the modified date you could export and save this along side the single tiddlywiki on the host. Although this is a manual process maybe it need only be done as content changes demands it.
- If you need to generate multiple files for RSS (I can’t recall the format) you could generate a multi-file zip to download and extract on the host.

If you come across a good RSS feed file standard/documentation I could try to build an appropriate template for you.

boris · October 1, 2021, 10:27pm

I’ve manually built RSS multiple times for static site generators, I was hoping that there was an existing template.

Yes my plan would be to do a GitHub Actions build process using NodeJS to generate this automatically. Saq documented this so I should be able to extend it.

Here’s an example done in the Liquid templating language for an Atom feed:

github.com

bmann/bmcgarden/blob/master/feed/blog.xml

---
layout: null
permalink: /feed/blog.xml 
---
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
>
	<channel>
		<title>{{ site.name | xml_escape }} - Blog Feed</title>
		<description>{% if site.description %}{{ site.description | xml_escape }}{% endif %}</description>		
		<sy:updatePeriod>{{ site.feed_update_period | default: "daily" | xml_escape }}</sy:updatePeriod>
		<sy:updateFrequency>{{ site.feed_update_frequency | default: 1 | xml_escape }}</sy:updateFrequency>
		<link>{{ site.url }}</link>
		<atom:link href="{{ site.url }}/{{ page.path }}" rel="self" type="application/rss+xml" />
		<lastBuildDate>{{ site.time | date_to_rfc822 }}</lastBuildDate>
		{% assign feed_items = site.feed_items | default: 10 %}
		{% assign sortedblog = site.posts | sort: 'date' | reverse %}

This file has been truncated. show original

It’s just a single file, the core loop is fairly self explanatory:

<item>
<title>Tiddler Title</title>
<dc:creator>Author</dc:creator>
<description>The full content of the tiddler</description>
<pubDate>RFC822 Publish Date</pubDate>
<link>Link to Tiddler</link>
<guid isPermaLink="true">Link to Tiddler</guid>
</item>

TW_Tones · October 1, 2021, 11:20pm

So just to show how simple this is here is an incomplete macro for generating the RSS file. It would only include tiddlers with the published date and could filter the last N tiddlers.

\define permalink() code for permalink
\define RSSItems()
<item>
<title>$(currentTiddler)$</title>
<dc:creator>{{!!creator}}</dc:creator>
<description><$transclude/></description>
<pubDate><$view field=published format=date template="YYYY0MM0DD RFC822"/></pubDate>
<link><<permalink>></link>
<guid isPermaLink="true"><<permalink>>??</guid>
</item>
\end
<$list filter="[all[]!is[system]has[published]]">
<$text text=<<RSSItems>>/>
</$list>

The resulting rendered template would then be provided to a button/trigger with ## WidgetMessage: tm-download-file

The above example needs

Wrap the items with the rss header info and any variables therein
The macros for the permalink may need wikification, see info tiddlers
Tiddlers with content and the published date, limited by a most recent count?
Format for the RFC822 date
Check if the use of the permalink macro is appropriate in both places.

I hope this helps. I can’t put the time in to complete this weekend but this would allow a download from an actual single file wiki, without Github involvement.

boris · October 1, 2021, 11:26pm

Appreciated @TW_Tones — I should be able to use this as a building block.

TW_Tones · October 1, 2021, 11:43pm

If you know the format of values in the header and the RFC822 date I can do this in time.

saqimtiaz · October 2, 2021, 12:16pm

I have a Feeds plugin that I wrote a while back specifically for use with Github actions that amongst other things generates RSS and JSON feeds. It is currently in the process of a complete rewrite to make it easier to use from within TiddlyWiki as the first incarnation was dependent on nodejs.

I will see if I can quickly extract the salient bits for an RSS feed from dependencies and share it.

saqimtiaz · October 2, 2021, 1:58pm

OK I found the time to have a quick go at trying to extract the RSS bits and resolving non-core dependencies, hopefully I got everything: feeds.json

The main tiddler is $:/plugins/sq/feeds/templates/rss, there are config tiddlers for customizing the title, what tiddlers to include etc. I think everything is configurable now as opposed to the hardcoded first incarnation. I don’t recall how far I got in the refactoring process last time around that I worked on this, so I recommend validating the output to make sure all the fields are there and correctly formatted.

For a single file wiki at the path my-wiki.html:

tiddlywiki.js --load my-wiki.html --render "[[$:/plugins/sq/feeds/templates/rss]]" "feed.xml" "text/plain" "$:/core/templates/wikified-tiddler"

For node.js folder wiki at the path mywiki/:

tiddlywiki.js mywiki/ --render "[[$:/plugins/sq/feeds/templates/rss]]" "feed.xml" "text/plain" "$:/core/templates/wikified-tiddler"

Hope this helps. If you run into errors, or find things that need correcting, let me know.

boris · October 2, 2021, 3:42pm

Thanks! Will run through it and document!

saqimtiaz · October 2, 2021, 5:12pm

Here is a fun little extra (very minimally tested):

Using the excerpts from the Feeds plugin that I shared, you can also save an RSS feed along with your wiki using the Fission or Github uploader with the FileUploads plugin.

Example RSS feed saved with Fission uploader.

In order to get this working, you need a custom save wiki button to save the RSS feed along with the wiki (assumes auto save is turned off) and a custom filter for the FileUploads plugin as seen here: Saq's wiki — a non-linear personal web notebook

This is the sort of thing that hopefully will eventually become a core capability with the intended savers and syncers work as discussed on the community call.

boris · October 2, 2021, 5:53pm

Awesome. I’ll test this on my TW Groceries, too.

I’m currently messing about with filters and documenting my walk through of set up.

e.g. The default filter at $:/config/feeds/filter is basically everything that isn’t a system tiddler:

[!is[system]limit{$:/config/feeds/maxItems}]

For me, this is my filter of non system tiddlers tagged with Notes or Journal in reverse chronological order:

[!is[system] [tag[Notes]] [tag[Journal]] +[limit[25]] +[!sort[created]]

I have this filter pulled out into a tiddler for experimenting, the limit is at $:/config/feeds/maxItems

As always, I’m basically a TW filter noob, so don’t look too closely at my filters

twMat · October 3, 2021, 9:22am

I believe it has been done. I can’t recall the name of the individual who made this but I believe Tobias Beer picked up on it and you can see one use of it in his (incredible) tb5 site - note the RSS symbol next to the wiki subtitle.

Googling “TiddlyWiki RSS” might give a few clues…

boris · October 3, 2021, 4:13pm

Assume that I searched and found nothing

Thanks for pointing out the RSS link on Tobias’ site.

Saq’s template above works well, I’m just documenting and implementing it for my use case and will share a walk through once done.

telumire · October 3, 2021, 4:58pm

There is also the option to use the commits messages (if applicable) as a RSS feed, now that tiddlywiki allows to wikify the commit.

For github hosted wiki, the feed url looks like this :

https://github.com/Telumire/TiddlyTweaks/commits/main.atom

The downside is that we cant put links or images inside, only plain text (AFAIK)

twMat · October 3, 2021, 8:32pm

Here it is:

https://tiddlywiki.narkive.com/7Imfd94p/tw-tw5-generating-an-atom-feed-for-a-static-site

Atom feed by “dullroar” / Jim Lehmer

saqimtiaz · October 5, 2021, 7:15am

Uploading an RSS feed with the FileUploads plugin involves three steps:

Generating an RSS feed from the template and saving it in a tiddler with an appropriate title: e.g rss.xml
Having the FileUploads filter configured to upload that tiddler.
Having a separate button to generate the RSS feed in step 1, triggering its upload, or a custom save button that saves the wiki and generates the RSS feed.

Now with the latest version of the FileUploads plugin you can avoid customizing the filter in step 2 and just use the same button that generates the RSS feed to also upload it directly using the tm-upload-tiddlers message, example:

<$action-sendmessage $message="tm-upload-tiddlers" $param="rss.xml"/>

rmorrow · October 5, 2021, 9:41am

On the off chance that you haven’t seen this already, there is a ‘howto’ of sorts on techlifeweb dot com (have to do this as I’m only allowed one link in a post, apparently :-)). I’ve written up the way I’ve done it based on the information from there on my site, including the relevant links to the pages on techlifeweb.

https://radar231.com/RSS%2520Feed%2520for%2520Tiddlywiki%2520SSG%2520Website.html

telumire · October 5, 2021, 5:19pm

@saqimtiaz This plugin is so great, thank you so much !

@boris I’ve managed to make a RSS feed generator using the file upload plugin thanks to the very useful tips from this thread, here it is :

RSS.tid (4.3 KB)

No commands needed
Enjoy !

PS : You dont need the base64 plugin from tobibeer with the github uploader, it’s only needed for the PUT uploader (it wouldnt upload text file otherwise).

EDIT: @rmorrow wow this is much cleaner than what I’ve come up with :o

saqimtiaz · October 5, 2021, 6:08pm

That is a bug, you shouldn’t need anything extra. I’ll push an update when I get the chance to test properly.

boris · October 5, 2021, 8:20pm

Thank you! The “discoverfeed” is an excellent add on so that various tools can auto find feeds in the header.