Choosing starting Template, a TW edition (wiki template) list API JSON example

Developers
#1

I’m going to implement wiki template list API this week, I’d like to get some input on this.

My current design will be just setup a wiki like https://tw-cpl.netlify.app/ to collect tiddlers about templates , and use a a render template to aggregate these tiddlers into a JSON like this in Choosing starting Template · Issue #31 · donmor/Tiddloid · GitHub

{
    "locales":{ // all regions and languages available in TW
        "zh-Hans":{ // Take Jane Chinese as an example
            "name": "Chinese - China", // the name in this language
            "alias":[ // alias, corresponding to the locale string obtained under various systems
                "zh-rCN", // Android
                "zh-CN", // standard
                "zh-CN.UTF-8" // Linux/Unix
            ]
        }
    },
    "versions":[ // all versions
        "5.1.21",
        "5.1.22"
    ],
    "templates":[ // list of templates
        {
            "name": "TiddlyWiki", // name
            "locale":[ // all compatible locales, filtered by one
                "*-*" // wildcard, for all languages in all locales
            ],
            "desc": "Official release", // description, with length limit
            "icon": "uri/base64", // icon, recommended size 48x48
            "homepage": "https://tiddlywiki.com/", // homepage
            "uri": "https://tiddlywiki.com/empty.html", // empty.html file URL
            "lastUpdate":1234567890, // Unix timestamp, action pulls the file and gets the timestamp based on the uri field, one of the filtering bases
            "version": "5.1.22", // trial run to pull the HTML file and get the kernel version, one of the filtering bases
            "size":3000000, // file size, one of the filtering bases
            "valid":true, // set to false when no version information is available, client automatically blocks this template
        }
    }
}

Translated with www.DeepL.com/Translator (free version)

Then TidGi and Tiddloid and TiddlyDesktop may fetch this JSON and display available list for new user.

2 Likes
#2

Re-translated:

{
    "locales":{    // All regions and languages available
        "zh-Hans":{    // Take Simplified Chinese as an example
            "name":"中文 - 中国",    // Name in S-Chinese
            "alias":[    // Aliases, locale strings in various OSes goes here
                "zh-rCN",    // Android
                "zh-CN",    // Standard
                "zh-CN.UTF-8" // Linux/Unix
            ]
        }
    },
    "versions":[    // All core versions
        "5.1.21",
        "5.1.22"
    ],
    "templates":[    // List of templates
        {
            "name":"TiddlyWiki",    // Name
            "locale":[    // All locales compatible, one of filtering params
                "*-*"    // Wildcard that matches all regions and languages
            ],
            "desc":"Official release",    // Brief descriptions
            "icon":"uri/base64",    // Icon, 48x48 recommended
            "homepage":"https://tiddlywiki.com/",    // Homepage (based on TW)
            "uri":"https://tiddlywiki.com/empty.html",    // Url to empty.html
            "lastUpdate":1234567890,    // Unix timestamp, generated by actions (by GitHub or something else) which pull the file from "uri" field; one of filtering params
            "version":"5.1.22",    // Version code of core, from actions mentioned above, which tries to boot from the file fetched; one of filtering params
            "size":3000000,    // Size of the file, from the actions mentioned above, one of filtering params
            "valid":true,    // Set to false if failed to boot from empty.html and get version codes; clients will ignore templates that are not "valid"
        }
    ]
}
#3

Hi @linonetwo @donmor3000 we have a similiar but simpler template registry in TiddlyWiki on Fission:

It would be very helpful to devise an extended format that we could use across the community as the definitive way of maintaining references to useful templates.

1 Like
Custom application for TiddlyWiki based on WebDAV
#4

This reminds me of that, we can just use a list even simpler than TiddlyWiki on Fission, just a URL and locale is enough…

We can extract name from the $:/SiteTitle, description from the $:/SiteSubtitle , version from <meta name="tiddlywiki-version" content="5.2.2" />, locale from <html lang="zh-Hans"> (this may need user to choose, because it may have built-in translation)

[
  // List of templates
  {
    // Homepage or documentation, I don't know where we can extract this, so let template author fill this.
    "homepage": "https://tiddlywiki.com/",
    "uri": "https://tiddlywiki.com/empty.html", // Url to empty.html
    "locale": [
      // All locales compatible, one of filtering params
      "*-*" // Wildcard that matches all regions and languages
    ]
  }
]

And generate the complete JSON in Github Actions:

Locale

Let’s just use a simple array from locale-list/locales.json at master · umpirsky/locale-list · GitHub and locale-list/locales.json at master · umpirsky/locale-list · GitHub , based on app’s current language. And I think we should stick to standard locale string, so alias may not be needed, or just trans to standard in app locally.

versions

I think the locale and version fields are a kind of Index, to simplify the use when the app UI reads this data, right? Do we need other index fields?

extracted result:

{
  // I think this can be a simplify to a list
  "locales": [
    // All regions and languages available
    // Take Simplified Chinese as an example
    "zh-Hans" // key from umpirsky/locale-list, you can get value from umpirsky/locale-list 's each translated list
  ],
  "versions": [
    // All versions available in templates
    "5.1.21",
    "5.1.22"
  ],
  "templates": [
    // List of templates
    {
      // number of non-system, non-plugin tiddlers, too many of these pre-packaged tiddlers may make it difficult to upgrade (can't upgrade via clicking upgrade plugins)
      "tiddlers": 15,
      "name": "TiddlyWiki", // Name
      "locale": [
        // All locales compatible, one of filtering params
        "*-*" // Wildcard that matches all regions and languages
      ],
      "description": "Official release", // Brief descriptions
      "icon": "uri/base64", // Icon, 48x48 recommended, from $:/favicon.ico or img tagged with tag
      "homepage": "https://tiddlywiki.com/", // Homepage or documentation (based on TW)
      "uri": "https://tiddlywiki.com/empty.html", // Url to empty.html
      "lastUpdate": 1234567890, // Unix timestamp, generated by actions (by GitHub or something else) which pull the file from "uri" field; one of filtering params
      "version": "5.1.22", // Version code of core, from actions mentioned above, which tries to boot from the file fetched; one of filtering params
      "size": 3000000, // Size of the file, from the actions mentioned above, one of filtering params
      "valid": true, // Set to false if failed to boot from empty.html and get version codes; clients will ignore templates that are not "valid"
      "author": "Jeremy", // extracted based on non-system tiddler's author,
      "plugins": [
        // extracted using prefix[$:/plugins/] , and get additional info from CPL. This makes sure user get a clue about what it will install
        {
          "title": "$:/plugins/felixhayashi/tiddlymap"
        }
      ],
      "themes": [{ "title": "$:/themes/linonetwo/notion" }]
    }
  ]
}
#5

There can be many use cases for a list of templates though. For example, in my WebDAV wikifarm setup, I too have a list of templates that the user can chose from when creating a new wiki. All the logic and code for creating new wikis is client side, and with a local WebDAV server this works offline too. It would be preferable in this scenario to have all the necessary information provided in a single JSON file (such as in TiddlyWiki on Fission), rather than having to fetch every file and extract information from it.

#6

Agreed. Even icons could be embedded in JSON as base64. BTW it’ll be great to add a filter of current locale by default.

#7

Also, the server holding the JSON should be able to provide timestamp of the file, so that clients can cache the JSON and decide to update or not. This is in case the JSON file become way larger than we expected before.

#8

Maybe it is necessary to have an uuid attribute in each JSON item to help clients caching template files.

#9

What do you mean? Filtering can happen in app code side.

Or you mean the default locale of template?

#10

That will be another API? Github pages don’t have this timestamp in OPTION request, but we can add another version.json file for it.

You can still use stale while revalidate strategy to update to JSON on background.

#11

Oh, you mean timestamp of each file, you can just use lastUpdate field for it.

The essential editions of TiddlyWiki
#12

Can also see siyuan’s template list GitHub - siyuan-note/bazaar: 思源笔记社区集市。SiYuan community bazaar.

A preview: preview.png can be also required.
And description can be a link to a readme.md or readme.tid

And it upload the result to an paid OSS to speed the user-side download

#13

We also need a scenario field, so we can mark some editions as template for creating new wiki, and some editions as help (like the tiddlywiki.com and groktiddlywiki.com ).

And we may have more scenarios like social for https://links.tiddlywiki.org , there will be more, these three are what I can come up with now.

scenario field help Apps know where tu put these editions, when creating new wiki, we only list template scenario. When click on “Help”, list the help scenario.

And also a “mirror” list for possible CDN to the HTML, app can retry different mirrors when downloading the edition.

#14

@donmor3000 please review, I’ve written a form for creating a new edition record. In both language.

#15

Apologize for not replying for a long time.

Putting all necessary fields in the template list file reduces network traffic, making it faster to be loaded by clients.

It’ll be more comfortable to see templates in native language of each users. Filters can be manually set by user though. The locale field in the list just indicates what locale it may belongs to.

The timestamp of templates is available in the list file, but not the list file itself. If we can get its timestamp, we can cache this file and thus consume even less network traffic. This is useful especially if the list file become too large.

The template list should be machine-readable and extensible. I think JSON is the best choice. Since using an extensible data structure, clients can make partial implementation to meet their needs.

BTW It’d be easy to generate wiki text or HTML from the JSON, but not vice versa.

#16

Terminating this topic due to complicated reasons. This function will be dropped permanently. Sorry :frowning: