Fun with Feeds (a proof of concept RSS reader)

,
#1

A short while back while working on the WebDAV wiki farm stuff, I realized I had most of the ingredients for a simple RSS reader in TiddlyWiki.

Just to demonstrate what might be possible once we have wikitext affordances for HTTP requests in the core, I spent an hour cobbling this together:

https://saqimtiaz.github.io/sq-tw/temp/feedreader.html

Please note that:

  • this is not meant to be usable, nor do I intend to work on it further
  • it relies on the same wikitext affordances for HTTP requests as the WebDAV work
  • it takes a shortcut and uses XSLT (think CSS for XML) to display the feeds as HTML. A more interesting approach might be to convert each feed item into HTML as a temporary tiddler, allowing to easily save individual feed items of interest.
  • the primary constraint is lack of CORS (cross domain communication) support for many of the servers hosting the RSS feeds, which necessitates use of a proxy server.
1 Like
Another Tiddlywiki RSS Plugin
#2

Nice. A shame about the CORS restriction.

Are there any references for what you’re talking about with “wikitext affordances for HTTP requests”?

#3

The basic idea is to introduce a widget message that initiates an HTTP request, and run actions in the callback for the request. Jeremy and I have both worked in this area and came up with our own implementations, though I have since tried to standardize upon version. There is still some work to be done in terms of error handling and providing more affordances for end users to have an overview over and debug requests.

Example implementation of the widget message handling (subject to change).

Example of invoking a request from wikitext.

Fetching an RSS feed and saving it in a tiddler looks like this:

\define callback()
	\define failureHandler()
	<$action-log status="error fetching the feed"/>
	<$action-setfield $tiddler={{{ [[$:/temp/feedreader/error/]addsuffix<feed-uri>] }}} text={{{ [[There was an error fetching the feed: ]addsuffix<feed-uri>addsuffix[. ]addsuffix<error>] }}} component="Feedreader" tags="$:/tags/Alert"/>
	\end failureHandler
	<$list filter="[<status>match[200]] :filter[<error>is[blank]]" variable="null" emptyMessage=<<failureHandler>> >
		<$action-setfield $tiddler={{{ [[$:/temp/feed/]addsuffix<feed-uri>] }}} text=<<data>> />
	</$list>
\end callback

<$action-sendmessage
	$message="sqtm-http-request"
	method="GET"
	bind-status={{{ [[$:/temp/feedreader/http-status/]addsuffix<feed-uri>] }}}
	oncompletion=<<callback>>
	url=<<feed-uri>>
	var-feed-uri=<<feed-uri>>
>
#4

Thank you @saqimtiaz. I can see many uses for this.

I’m happy that after a few months of doing TW work, I understood almost everything in your sample here. (I’ve only glanced at the JS in the GitHub repos, but since that’s my day-to-day work, I expect to be able to follow that.)

The one thing I’m missing is in the emptyMessage to your <$list> in the callback. I’m probably just misled by working with eager languages like JS, but to me that reads as through we would run the failureHandler regardless of whether the list is empty. I assume this is not true. Does that mean that emptyMessage just has the string "<<failureHandler>>" to be evaluated later if the list turns out to be empty?

Do I have that right? I’ve worked a bit with lazy languages like Haskell, and that’s a familiar concept, but I didn’t realize that TW worked that way. This might help we clear up a lot of vague misperceptions about TW that I knew were wrong but didn’t know why?

Or am I on the wrong track altogether?

#5

I use a plugin for my TiddlyWiki to provide an RSS feed. And this is how I add it to the HTML head.

Been using this for years and it works great!

Oh a reader. sorry miss read the title.

1 Like
#6

Almost correct. In the code given above, since the emptyMessage is assigned without quotes around the value, it is the value of the macro/variable that is assigned to emptyMessage as a string. That is a subtle distinction but matters in some contexts.

By default, assigning a variable or some wikitext to the attribute of a widget does not invoke/execute that wikitext, the value of the attribute is provided to the widget as a string and the widget determines how it is uses the attribute value.

The $list widget renders the emptyMessage attribute value when there is no output from the filter provided to its filter attribute. The emptyMessage is the else clause if we think of the $list widget as an if else formulation.

#7

That clears it up. I was thinking in JS terms, believing that <$myWidget prop=<<myRef>> /> was equivalent to JS’s myWidget (prop = myRef ()). That’s not the case, and it’s actually more like myWidget (prop = myRef) (and I know function calls are not a precise analogy either.) It will probably take a few tries for this to sink in, but it makes sense.

Thank you for taking the time here and elsewhere to instruct those of us still struggling. Your help is much appreciated!

#8

This is true for both literal and variable widget attributes. For filtered transclusions such as prop={{{ [some filter here] }}}, the filter is run when the widget is rendered and the output assigned as a string as the value of the widget attribute.

#9

Just to clarify since I have had multiple questions about this now, the link above is a TiddlyWiki. Use ctrl + shift + L to change Layouts.

1 Like
#10

@saqimtiaz, I use miniflux and would be so happy to have this in Tiddlywiki. Is this something that you can establish?

#11

I just hoped to use rsshub to write an rss plug-in, but cors first stopped me from this idea. Maybe this is why many rss readers charge a fee. rsshub supports hosting, which should be able to solve the cors problem, but it is not planned to do so for the time being.

#12

vue-rss

Here is an example of vue implementing rss loading on tiddlywiki

image
image1825Ă—1115 177 KB

2 Likes
#13

In addition, the formats between different RSS are not strictly standardized and unified, and it is necessary to manually adjust the acquisition of some tags.

#14

image
image1770Ă—1071 86 KB

for tiddlywiki newsletter rss

1 Like
#15

I noticed that some of the examples use https://corsproxy.io/. Note that iOS does not support proxy forwarding.

requestUrl - Developer Documentation I saw someone made a similar RSS plug-in for Obsidian, and they used this API to solve this CORS problem, which seems really convenient. I don’t quite understand the details. Is this a similar approach to forwarding through a proxy? @saqimtiaz

PS：Obsidian’s plugin development documentation looks very good. It would be great if our dev manual could reach this level.

#16

My guess is that this is a custom API implemented via electron and exposed to the client and therefore not subject to the same CORS restrictions as browsers. Working in normal browsers, there is no equivalent option for bypassing CORS.

1 Like
#17

Thanks for sharing your script @sukima … I am trying to render the atom.xml via the --build feed method. It does generate the .xml file but the cli hangs and for some reason starts the syncer-server-filesystem.

I am getting this error on --build feed

If you require SSR(server side render), you need to define $height and $width with format like '300px'
ReferenceError: echarts is not defined
    at Object.onUpdate (addon/sevenday.js:10:512)
    at EChartsWidget.renderAddon ($:/plugins/Gk0Wk/echarts/widget.js:1:9651)
    at EChartsWidget.render ($:/plugins/Gk0Wk/echarts/widget.js:1:3877)
    at Widget.renderChildren ($:/core/modules/widgets/widget.js:613:15)
    at ElementWidget.render ($:/core/modules/widgets/element.js:81:7)
    at Widget.renderChildren ($:/core/modules/widgets/widget.js:613:15)
    at ElementWidget.render ($:/core/modules/widgets/element.js:81:7)
    at Widget.renderChildren ($:/core/modules/widgets/widget.js:613:15)
    at ElementWidget.render ($:/core/modules/widgets/element.js:81:7)
    at Widget.renderChildren ($:/core/modules/widgets/widget.js:613:15)
If you require SSR(server side render), you need to define $height and $width with format like '300px'
syncer-server-filesystem: Dispatching 'save' task: $:/StoryList

I wonder if there is a way to suppress the syncer-server-filesystem from starting with a flag since xml file does get generated successfully.

1 Like
#18

Tiddler containing echars seems to cause build issues

#19

@oeyoews I have in the atom feed included tiddlers filter run:
[!is[system]!untagged[]!tag[static]!tag[Private]!prefix[$:/plugins/]!title[Table of Contents]!sort[modifed]]

Shouldn’t build skip over the echarts plugin? :thinking:

#20

No, the plugin itself has no direct impact, you need to exclude the tiddler that uses this echarts widget