How "split" a tiddler with JSON into individual tiddlers?

#1

(Cont from I have a JSON url, how do I make the tm-http-request?)

I have a tiddler containing JSON and I want to “split” (import?) that into individual real tiddlers. I want to do this silently, i.e the UX is to do the JSON import from the external source, and the thereafter the “splitting” is done automatically/silently.

@saqimtiaz kindly gave some pointers:

I’m guessing that this is what the example WidgetMessage: tm-http-request Example - Zotero does, but that is a very complex tiddler and I can’t decipher what are the crucial parts - and there’s the uncertainty that just maybe it lacks or contains some component that is a show stopper for my use case.

Reflection: To “interface” TW with a JSON page seems like a very fundamental use case, so I’m wondering why it is not part of the core? Or is this not an issue for everyone wanting their TW to communicate with other services (…I’m assuming several services feature a JSON interface, right?) If not built into core, we should at least document how to do it. Maybe I’ll be able to make at least a draft after this.

…

Related: As some know, fellow @Mark_S and I worked on the TiddlyGoo project. It was all fun until big Broogle changed the API (or whatever it was they changed). Here’s the introductory illustration from that site that summarizes why having a “link” between Google Sheets and TW would be super useful, and it has become even more useful since the AI revolution:

image
image700×323 65.6 KB

#2

Once you have the JSON, you have to loop through the tiddlers represented within one at a time, creating an actual tiddler for each of them. Hint: use the JSON filter operators, the exact filters will depend on the structure of your JSON.

Look at the procedures zotero-save-items and zotero-save-item.

Another example of doing something similar can be found in this post, in particular the importTiddlers macro though you can skip the step using the deserialize operator.

1 Like
#3

@twMat I hope reading through this step by step helps:

\procedure completion()
\procedure errorHandler()
<$action-setfield $tiddler="$:/temp/fetcherror" tags="$:/tags/Alert" text=`There was an error fetching the URL: $(error)$, statusText: $(statusText)$, status: $(status)$`/>
\end errorHandler
\procedure createTiddlers()
		<!-- iterate over each position in the JSON array -->
		<$list filter="[<data>jsonindexes[]]" variable="index">
			<!-- get the tiddler at that position in the array from the JSON -->
			<$let
				tiddler={{{ [<data>jsonextract<index>] }}} 
				title={{{ [<tiddler>jsonget[title]] }}}
			>
				<!-- make sure we have a title for the tiddler and exclude system tiddlers -->
				<$list filter="[<title>!is[blank]!is[system]]">
					<$action-setmultiplefields
						$fields="[<tiddler>jsonindexes[]]"
						$values="[<tiddler>jsonindexes[]] :map[<tiddler>jsonget<currentTiddler>]"
						$timestamp="no"
						/>
				</$list>
			</$let>
		</$list>
\end createTiddlers
<!-- check if the HTTP request completed successfully -->
<$list filter="[<status>match[200]]" emptyMessage=<<errorHandler>>>
	<!-- request was successfull, create tiddlers from data variable -->
	<<createTiddlers>>
</$list>
\end completion

\procedure fetch-url()
	<$action-sendmessage
		$message="tm-http-request"
		url="https://opensheet.elk.sh/14ps1hhvhK0ctpH-9k8jOseipdRdTGwF-TnLKoipx7Mk/1"
		method="GET"
		oncompletion=<<completion>>
		onprogress=<<progress>>
	/>
\end

<$button actions=<<fetch-url>>>get json data from url</$button>
1 Like
#4

Your solution is, unsurprisingly, superior to my own concoction. Thank you!

#5

I think these two need to be flipped.

#6

@twMat Can you share the code you are using.I tried the code given by Saq in the previous post and its not working for me. Here is the tiddler for fetching google sheet data into json tiddler. Here is the tiddler for splitting json data into individual tiddlers.