Has anyone generated an RSS feed from TiddlyWiki?

boris · October 1, 2021, 10:27pm

I’ve manually built RSS multiple times for static site generators, I was hoping that there was an existing template.

Yes my plan would be to do a GitHub Actions build process using NodeJS to generate this automatically. Saq documented this so I should be able to extend it.

Here’s an example done in the Liquid templating language for an Atom feed:

github.com

bmann/bmcgarden/blob/master/feed/blog.xml

---
layout: null
permalink: /feed/blog.xml 
---
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
>
	<channel>
		<title>{{ site.name | xml_escape }} - Blog Feed</title>
		<description>{% if site.description %}{{ site.description | xml_escape }}{% endif %}</description>		
		<sy:updatePeriod>{{ site.feed_update_period | default: "daily" | xml_escape }}</sy:updatePeriod>
		<sy:updateFrequency>{{ site.feed_update_frequency | default: 1 | xml_escape }}</sy:updateFrequency>
		<link>{{ site.url }}</link>
		<atom:link href="{{ site.url }}/{{ page.path }}" rel="self" type="application/rss+xml" />
		<lastBuildDate>{{ site.time | date_to_rfc822 }}</lastBuildDate>
		{% assign feed_items = site.feed_items | default: 10 %}
		{% assign sortedblog = site.posts | sort: 'date' | reverse %}

This file has been truncated. show original

It’s just a single file, the core loop is fairly self explanatory:

<item>
<title>Tiddler Title</title>
<dc:creator>Author</dc:creator>
<description>The full content of the tiddler</description>
<pubDate>RFC822 Publish Date</pubDate>
<link>Link to Tiddler</link>
<guid isPermaLink="true">Link to Tiddler</guid>
</item>

TW_Tones · October 1, 2021, 11:20pm

So just to show how simple this is here is an incomplete macro for generating the RSS file. It would only include tiddlers with the published date and could filter the last N tiddlers.

\define permalink() code for permalink
\define RSSItems()
<item>
<title>$(currentTiddler)$</title>
<dc:creator>{{!!creator}}</dc:creator>
<description><$transclude/></description>
<pubDate><$view field=published format=date template="YYYY0MM0DD RFC822"/></pubDate>
<link><<permalink>></link>
<guid isPermaLink="true"><<permalink>>??</guid>
</item>
\end
<$list filter="[all[]!is[system]has[published]]">
<$text text=<<RSSItems>>/>
</$list>

The resulting rendered template would then be provided to a button/trigger with ## WidgetMessage: tm-download-file

The above example needs

Wrap the items with the rss header info and any variables therein
The macros for the permalink may need wikification, see info tiddlers
Tiddlers with content and the published date, limited by a most recent count?
Format for the RFC822 date
Check if the use of the permalink macro is appropriate in both places.

I hope this helps. I can’t put the time in to complete this weekend but this would allow a download from an actual single file wiki, without Github involvement.

boris · October 1, 2021, 11:26pm

Appreciated @TW_Tones — I should be able to use this as a building block.

TW_Tones · October 1, 2021, 11:43pm

If you know the format of values in the header and the RFC822 date I can do this in time.

saqimtiaz · October 2, 2021, 12:16pm

I have a Feeds plugin that I wrote a while back specifically for use with Github actions that amongst other things generates RSS and JSON feeds. It is currently in the process of a complete rewrite to make it easier to use from within TiddlyWiki as the first incarnation was dependent on nodejs.

I will see if I can quickly extract the salient bits for an RSS feed from dependencies and share it.

saqimtiaz · October 2, 2021, 1:58pm

OK I found the time to have a quick go at trying to extract the RSS bits and resolving non-core dependencies, hopefully I got everything: feeds.json

The main tiddler is $:/plugins/sq/feeds/templates/rss, there are config tiddlers for customizing the title, what tiddlers to include etc. I think everything is configurable now as opposed to the hardcoded first incarnation. I don’t recall how far I got in the refactoring process last time around that I worked on this, so I recommend validating the output to make sure all the fields are there and correctly formatted.

For a single file wiki at the path my-wiki.html:

tiddlywiki.js --load my-wiki.html --render "[[$:/plugins/sq/feeds/templates/rss]]" "feed.xml" "text/plain" "$:/core/templates/wikified-tiddler"

For node.js folder wiki at the path mywiki/:

tiddlywiki.js mywiki/ --render "[[$:/plugins/sq/feeds/templates/rss]]" "feed.xml" "text/plain" "$:/core/templates/wikified-tiddler"

Hope this helps. If you run into errors, or find things that need correcting, let me know.

boris · October 2, 2021, 3:42pm

Thanks! Will run through it and document!

saqimtiaz · October 2, 2021, 5:12pm

Here is a fun little extra (very minimally tested):

Using the excerpts from the Feeds plugin that I shared, you can also save an RSS feed along with your wiki using the Fission or Github uploader with the FileUploads plugin.

Example RSS feed saved with Fission uploader.

In order to get this working, you need a custom save wiki button to save the RSS feed along with the wiki (assumes auto save is turned off) and a custom filter for the FileUploads plugin as seen here: Saq's wiki — a non-linear personal web notebook

This is the sort of thing that hopefully will eventually become a core capability with the intended savers and syncers work as discussed on the community call.

boris · October 2, 2021, 5:53pm

Awesome. I’ll test this on my TW Groceries, too.

I’m currently messing about with filters and documenting my walk through of set up.

e.g. The default filter at $:/config/feeds/filter is basically everything that isn’t a system tiddler:

[!is[system]limit{$:/config/feeds/maxItems}]

For me, this is my filter of non system tiddlers tagged with Notes or Journal in reverse chronological order:

[!is[system] [tag[Notes]] [tag[Journal]] +[limit[25]] +[!sort[created]]

I have this filter pulled out into a tiddler for experimenting, the limit is at $:/config/feeds/maxItems

As always, I’m basically a TW filter noob, so don’t look too closely at my filters

twMat · October 3, 2021, 9:22am

I believe it has been done. I can’t recall the name of the individual who made this but I believe Tobias Beer picked up on it and you can see one use of it in his (incredible) tb5 site - note the RSS symbol next to the wiki subtitle.

Googling “TiddlyWiki RSS” might give a few clues…

boris · October 3, 2021, 4:13pm

Assume that I searched and found nothing

Thanks for pointing out the RSS link on Tobias’ site.

Saq’s template above works well, I’m just documenting and implementing it for my use case and will share a walk through once done.

telumire · October 3, 2021, 4:58pm

There is also the option to use the commits messages (if applicable) as a RSS feed, now that tiddlywiki allows to wikify the commit.

For github hosted wiki, the feed url looks like this :

https://github.com/Telumire/TiddlyTweaks/commits/main.atom

The downside is that we cant put links or images inside, only plain text (AFAIK)

twMat · October 3, 2021, 8:32pm

Here it is:

https://tiddlywiki.narkive.com/7Imfd94p/tw-tw5-generating-an-atom-feed-for-a-static-site

Atom feed by “dullroar” / Jim Lehmer

saqimtiaz · October 5, 2021, 7:15am

Uploading an RSS feed with the FileUploads plugin involves three steps:

Generating an RSS feed from the template and saving it in a tiddler with an appropriate title: e.g rss.xml
Having the FileUploads filter configured to upload that tiddler.
Having a separate button to generate the RSS feed in step 1, triggering its upload, or a custom save button that saves the wiki and generates the RSS feed.

Now with the latest version of the FileUploads plugin you can avoid customizing the filter in step 2 and just use the same button that generates the RSS feed to also upload it directly using the tm-upload-tiddlers message, example:

<$action-sendmessage $message="tm-upload-tiddlers" $param="rss.xml"/>

rmorrow · October 5, 2021, 9:41am

On the off chance that you haven’t seen this already, there is a ‘howto’ of sorts on techlifeweb dot com (have to do this as I’m only allowed one link in a post, apparently :-)). I’ve written up the way I’ve done it based on the information from there on my site, including the relevant links to the pages on techlifeweb.

https://radar231.com/RSS%2520Feed%2520for%2520Tiddlywiki%2520SSG%2520Website.html

telumire · October 5, 2021, 5:19pm

@saqimtiaz This plugin is so great, thank you so much !

@boris I’ve managed to make a RSS feed generator using the file upload plugin thanks to the very useful tips from this thread, here it is :

RSS.tid (4.3 KB)

No commands needed
Enjoy !

PS : You dont need the base64 plugin from tobibeer with the github uploader, it’s only needed for the PUT uploader (it wouldnt upload text file otherwise).

EDIT: @rmorrow wow this is much cleaner than what I’ve come up with :o

saqimtiaz · October 5, 2021, 6:08pm

That is a bug, you shouldn’t need anything extra. I’ll push an update when I get the chance to test properly.

boris · October 5, 2021, 8:20pm

Thank you! The “discoverfeed” is an excellent add on so that various tools can auto find feeds in the header.

boris · October 5, 2021, 8:30pm

A post was merged into an existing topic: File Uploads plugin: updates

telumire · October 5, 2021, 9:17pm

I’ve read your article, this was very informative, thanks you !
There is another way to make a feed reachable for feed reader : simply add a tag $:/tags/RawMarkup

With this the content of the tiddler is embeded in the head of the document and become reachable

Here’s a demo