How to add, extract, modify "tiddlers" from command line? To do IPFS media transfer

, ,
1

Hi,

I am a new discoverer of Tiddlywiki, and love it immediately!

With the use of this plugin, I would like to use IPFS as storage, sharing system.

For example, detect embedded binary data", copy attachment to IPFS and update tiddlers to load resource from… This will reduce the index.html size, and accelerate loading.

I start reading https://tiddlywiki.com/static/Commands.html
https://tiddlywiki.com/static/fields%2520Operator%2520(Examples).html

With node tiddlywiki, I can use tiddlywiki $USER --verbose --load ~/TW/$USER/index.html --listen port=8282 ( where index.html is created with an ipfs cat ... of my running tiddlywiki)

Meanwhile I continue self exploration. I wonder if some advanced “tiddlywiki commands” user could indicate where to find some code examples concerning “tiddlers” manipulation?

Thanks

2

Hello papiche,

With node you can do things directly from the command line without having to run the server component. You can extract your tiddlers or portion thereof:

https://tiddlywiki.com/#SaveWikiFolderCommand

There’s instructions for how to extract images in this tiddler:

https://tiddlywiki.com/#ExternalImages

As a general tip, it’s best to search tiddlywiki.com directly, rather than depending on the static image version, which are there (I think) mostly to accommodate search engines.

IPFS is still a vague concept in my head. Are there any tutorials you would recommend ?

3

Hello @Mark_S
Thank you for your links, they are pretty useful to comprehend better.

For my first try, I made json tiddlers format file, and drop them into “import sequence” in the web browser.

I wonder how to use tiddlywiki command to make the same ?

IPFS is a peer to peer “cloud” storage, using “hash of files” as reference, it uses libp2p transfer layer. https://ipfs.io is the portal of the project. A perfect collaborative home made storage “CDN” depending on your swarm peers

4

La_Corporation-Documentaire.dragdrop.json (698 Bytes)

For playing IPFS video in tiddler, I made “type”: “text/vnd.tiddlywiki”, json files using <video> html5 inclusion.
What would be the tiddlywiki command to import this file?

5

Per the docs (https://tiddlywiki.com/#ImportCommand), the --import command should do it.

6

I tried –import command, and it works! Thanks :wink:

  1. Get the latest “ipfs online version”
    ipfs cat /ipns/k51qzi5uqu5dkavscgxvllbwh00eo74hqplaltrl4fk347602oq0y5hdc1y2fw > index.html

  2. Add tiddler
    tiddlywiki --verbose --load index.html --import ~/Bureau/Journal/youtube/Original_Atari_PONG-1972-arcade_machine_gameplay_video.dragdrop.json json

Boot log:
  Startup task: load-modules
  Startup task: info after: load-modules before: startup
  Startup task: plugins after: load-modules
  Startup task: startup after: load-modules
  Startup task: story after: startup
  Startup task: commands platforms: node after: story
Executing command: load index.html
Executing command: import /home/fred/Bureau/Journal/youtube/Original_Atari_PONG-1972-arcade_machine_gameplay_video.dragdrop.json json
1 tiddler(s) imported

But it doesn’t change index.html … re-publishing it
NEWIPFS=$(ipfs add -q index.html) && ipfs name publish -k ipfswikikey /ipfs/$NEWIPFS

Now I would like to “export” tiddlers containing external links (“canonical_uri”) to add into IPFS, replace URI republish modified tiddler.

7

If it is possible to save to IPFS via HTTP requests, you could use the FileUploads plugin. It uploads any binary tiddlers to the specified backend and creates canonical URI tiddlers for them.

The choice of backends is extensible via uploader modules, which are relatively easy to write for HTTP APIs. There is one for Fission’s Webnative File System (which also uses IPFS under the covers) but I believe that needs updating for the current WNFS version.

See:

8

OK, thanks @saqimtiaz, the way you explain seems great to add binary files, but my goal is to process already recorded tiddlers.
Some tiddlers could contain binary data already, or canonical_uri field, I want to copy to IPFS and update treated tiddlers.

I still don’t understand how to create new index.html after import step 2

9

Try adding these options:

--output . --render "$:/core/save/all" "newindex.html" "text/plain"

10

The FileUploads plugin uses a filter to determine what to tiddlers to upload, so is completely flexible in that regard.

11

Great !! Your help unlock my first wish :wink:
“Adding tiddler to an online IPFS Tiddlywiki”

My second wish then is “Copy internal/external media binaries from a tiddlywiki to IPFS to reduce index.html size and optimize my hosting”

Do you have an example of a command extracting some tiddlers.json from a specific field or tag values ?

After that, If I want to replace tiddler:

  • Do I need to remove old one, then add the new tiddler.
  • Or “update specific tiddler” command exists?

@saqimtiaz, maybe after I can understand how to use your method, before, I must learn how to manipulate tiddlers from command line

Continuing exploring commands
https://tiddlywiki.com/#FetchCommand
https://tiddlywiki.com/#SaveCommand

And found
http://tobibeer.github.io/tw/filters/#Filter%20Examples

12

Extending the command
tiddlywiki --verbose --load index.html --import ~/Bureau/Journal/youtube/Original_Atari_PONG-1972-arcade_machine_gameplay_video.dragdrop.json json
with

Result in the fact newindex.html and index.html are the same !?

13

I don’t know. Try this.

tiddlywiki --verbose --load index.html --load "~/Bureau/Journal/youtube/Original_Atari_PONG-1972-arcade_machine_gameplay_video.dragdrop.json" --output . --render "$:/core/save/all" "newindex.html" "text/plain"
14

How did you verify this? Are the files the same size? If the added tiddler does not contain a created field, it will not show up in the Recent tab, but it will show up in the All tab.

15

You’re right. My mistake was to put a trailing / in --output /tmp/ as file path

16 
tiddlywiki  --verbose --load index.html \
            --import new_tiddler.json "application/json" \
            --output /tmp --render "$:/core/save/all" "newindex.html" "text/plain"

This is the working “import tiddler” command line.
I was missing "application/json" for import

PS: I get json template from “export tiddler as json” from GUI

17

About SaveCommand I am a bit lost with filters

I can save tiddler with certain tittle.

But some filtering expression examples are missing to me

Like

  • save to json format instead of .tid ?
  • knowing TW last modification date ?
  • “select & save tiddler” containing a tag, a field, with certain values ?
  • save tiddlers created after a certain date ?

Some examples would enhance my learning curve

18

I’m thinking you may actually want the render command. There is an example for saving to JSON, though I feel the example must be wrong (maybe a leftover from a prior command), because the elements are in the wrong order.

19

@Mark_S thanks.

Great, render is the command I was needing.

I could make a script

  • searching for tiddlers “tag=tube”,
  • find URL in “text”,
  • download it with yt-dlp,
  • add mp4 video file to IPFS,
  • create new tiddler removing first one
  • publish new version of TW to IPNS

https://git.p2p.legal/qo-op/Astroport.ONE/src/branch/master/tools/TUBE.copy.sh

I am using TW associated with a crypto key related with IPFS/IPNS key. So anyone can store its tiddlywiki in the “interplanetary file system” only through a “salt/pepper” NaCl key generator converted to ED25519 format

Btw, I still wonder how I could know TW last modification date ?

20

Unfortunately, there aren’t many built-in functions for handling dates. You could filter, say, all tiddlers modified in the last 3 days:

[days[-3]]

Or, if you make up your date like yyyymmdd you can compare it:

[tag[HelloThere]]:filter[get[modified]compare:date:gt[20200101]]