Json in TiddlyWiki by standard

1. Why use Json in TiddlyWiki by standard?

  1. “JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.”

  2. "JSON is a widely used format that allows for semi-structured data, because it does not require a schema. This offers you added flexibility to store and query data that doesn’t always adhere to fixed schemas and data types.

  3. “Json, XML, other markup languages (markdown, wikitext, commonmark, html etc), email and EDI are all forms of semi-structured data.” But… “for embedded Markup” is good if it has Json for standard in TiddlyWiki

  4. “Json is non-linear, tiddlyWiki” too.

  5. Concept

{
  "app": {
    "config": "./settings.json",
    "cwd": "./",
    "src": "example/content/",
    "filePattern": "**/*.md",
    "dist": "example/output.json",
    "name": "markdown-json",
    "version": "0.0.1"
  },
  "data": [
    {
      "section": "Elements",
      "title": "buttons",
      "device": [
        "desktop",
        "mobile"
      ],
      "styles": [
        "https://example.com/styles/structure.min.css",
        "https://example.com/styles/app.min.css"
      ],
      "contents": "<p>Follow some application examples of buttons</p>",
      "id": "buttons",
      "meta": {
        "relativePath": "content/buttons.html",
        "createdAt": "2020-10-08T16:05:30.415Z",
        "lastModified": "2020-10-08T16:05:14.452Z",
        "size": 2095,
        "formattedSize": "2.0 KB"
      }
    },
    {
      "section": "Elements",
      "title": "icons",
      "tags": [
        "icons",
        "base"
      ],
      "contents": "<h1 id=\"icons\">Icons</h1>\n<p>Our icons list still is empty :(</p>\n",
      "excerpt": "<p>Our icons list still is empty :(</p>",
      "id": "icons",
      "meta": {
        "relativePath": "content/globals/js-utils.html",
        "createdAt": "2019-08-27T18:01:33.747Z",
        "lastModified": "2019-08-27T18:01:33.747Z",
        "size": 331,
        "formattedSize": "331 Bytes"
      }
    }
  ]
}
  1. References
  2. https:// www. w3schools .com/whatis/whatis_json.asp
  3. https:// en. wikipedia .org/wiki/JSON
  4. https:// www. json .org/json-en.html
  5. https:// en. wikipedia .org/wiki/Semi-structured_data
  6. https:// en. wikipedia .org/wiki/Unstructured_data
  7. https:// www .integrate .io/glossary/what-is-unstructured-data/
  8. https:// www .xml .com/pub/a/w3j/s3.nelson.html
  9. https:// www .xml .com/pub/a/2004/10/06/deviant.html
  10. http:// essaysfromexodus .scripting.com/tedNelsonWebHypertext
  11. https:// cloud .google .com/blog/products/data-analytics/bigquery-now-natively-supports-semi-structured-data
  12. https:// wikilabs .github .io/
  13. https:// www. vertopal .com/en/convert/markdown-to-json
  14. https:// www. npmjs .com/package/markdown-json
  15. https:// www integrate .io/glossary/what-is-unstructured-data/

Hi all!

1. Question

  1. Would it be possible to have a better abstraction in the custom markup? So that we could separate the logical layer from the application, the interface/interaction layer from the data layer?
    An example of the application logic layer: includes keywords like ‘if’, ‘else’, ‘for’, ‘while’, ‘true’, ‘false’, ‘ListView etc’
    An example of the interface/interaction layer: include keywords like ‘!!! to h3’ or ‘h3’ or ‘#### h3’ etc
    An example each data layer: includes only plain text

2. An example for each usage here

  1. application logic layer:
{
    "time" : 1659600652950,
    "blocks" : [
        {
            "id" : "QG7d-eoSuo",
            "type" : "header",
            "data" : {
                "text" : "Editor.js",
                "level" : 2
            }
        },
        {
            "id" : "kVwdXj215H",
            "type" : "paragraph",
            "data" : {
                "text" : "Hey. Meet the new Editor. On this page you can see it in action — try to edit this text."
            }
        },
        {
            "id" : "H-e1YiUKlN",
            "type" : "header",
            "data" : {
                "text" : "Key features",
                "level" : 3
            }
        },
        {
            "id" : "T3ln9Hapf5",
            "type" : "list",
            "data" : {
                "style" : "unordered",
                "items" : [
                    "It is a block-styled editor",
                    "It returns clean data output in JSON",
                    "Designed to be extendable and pluggable with a simple API"
                ]
            }
        },
        {
            "id" : "KREAmAR5pB",
            "type" : "header",
            "data" : {
                "text" : "What does it mean «block-styled editor»",
                "level" : 3
            }
        },
        {
            "id" : "YOC9ONSAyk",
            "type" : "paragraph",
            "data" : {
                "text" : "Workspace in classic editors is made of a ..."
            }
        },
        {
            "id" : "1vu_B7xnOz",
            "type" : "paragraph",
            "data" : {
                "text" : "There are dozens of ..."
            }
        },
        {
            "id" : "QnnsDw4lDQ",
            "type" : "header",
            "data" : {
                "text" : "What does it mean clean data output",
                "level" : 3
            }
        },
        {
            "id" : "Bz-zR4AT9U",
            "type" : "paragraph",
            "data" : {
                "text" : "Classic WYSIWYG-editors produce raw HTML-markup with both content data and content appearance. ..."
            }
        },
        {
            "id" : "rnI2GILJ4v",
            "type" : "paragraph",
            "data" : {
                "text" : "Given data can be used as you want: render with HTML ..."
            }
        },
        {
            "id" : "VXEb5d5CLm",
            "type" : "paragraph",
            "data" : {
                "text" : "Clean data is useful to sanitize, validate and process on the backend."
            }
        },
        {
            "id" : "u6VJxBO9HW",
            "type" : "delimiter",
            "data" : {}
        },
        {
            "id" : "MpYqDoF6XM",
            "type" : "paragraph",
            "data" : {
                "text" : "We have been working on this project more than three years. ..."
            }
        },
        {
            "id" : "ygqt5IYr9S",
            "type" : "image",
            "data" : {
                "file" : {
                    "url" : "https://example.com/public/app/img/external/codex2x.png"
                },
                "caption" : "",
                "withBorder" : false,
                "stretched" : false,
                "withBackground" : false
            }
        }
    ],
    "version" : "2.24.3"
}

or

  1. interface/interaction layer:
{ // UISchema
  "firstName": {
    "ui:autofocus": true,
    "ui:emptyValue": "",
    "ui:autocomplete": "family-name"
  },
  "lastName": {
    "ui:emptyValue": "",
    "ui:autocomplete": "given-name"
  },
  "age": {
    "ui:widget": "updown",
    "ui:title": "Age of person",
    "ui:description": "(earthian year)"
  },
  "bio": {
    "ui:widget": "textarea"
  },
  "password": {
    "ui:widget": "password",
    "ui:help": "Hint: Make it strong!"
  },
  "date": {
    "ui:widget": "alt-datetime"
  },
  "telephone": {
    "ui:options": {
      "inputType": "tel"
    }
  }
}

or

  1. data layer:
{ // JSONSchema, data layer
  "title": "A registration form",
  "description": "A simple form example.",
  "type": "object",
  "required": [
    "firstName",
    "lastName"
  ],
  "properties": {
    "firstName": {
      "type": "string",
      "title": "First name",
      "default": "Chuck"
    },
    "lastName": {
      "type": "string",
      "title": "Last name"
    },
    "telephone": {
      "type": "string",
      "title": "Telephone",
      "minLength": 10
    }
  }
}

3. References

  1. https:// github .com/rjsf-team/react-jsonschema-form
  2. https:// github .com/Jermolene/TiddlyWiki5/discussions
  3. https:// wikilabs .github .io/
  4. https:// editorjs .io/getting-started
  5. https:// groups .google .com/g/tiddlywikidev/c/Os_LCSCP_l8
  6. https:// json-schema .org/
  7. https:// groups .google .com/g/tiddlywikidev/c/O_jqS1jwKII - “[side topic] The Scratch programming language and UI” - json is semi-structured data, most editors use json for configurations, I think of json as a struct-block

1. idea

  1. logical layer from the application: “editor.js”- “JSONSchema”
    and/or
  2. interface/interaction layer: “UISchema”
    and/or
  3. data layer: “JSONSchema”

2. concept

2.1 concept 1

image

2.2 concept 2

Hi Eduardo
Thanks for your summary. But I’m not really sure what you try to achieve. We already use JSON as internal data store. Even if it is kind of human readable it is way to complicated for standard users.

TiddlyWiki is designed to be used by users that have no technical background. They should still be able to change components that usually only can be changed by programmers.

I’m on mobile at the moment, so I’ll reply late in more detail.

1 Like

Right. I’m one of those. And want to stay one :slight_smile: .

I have got as far as understanding JSON “flat-file”. That I can understand.

I do think the internal storage in TW via JSON clever and very manipulable, if needed.

@anon5541130 Why would we need anything more complex for data interchange?

A comment, TT

1 Like

@TiddlyTweeter @pmario thank you for feedback!

1. Why is it important?

  1. This reduce tiddlywiki file size - why?
    Response: This separate specific data types like JSONSchema, UISchema, JSONSchemaEdit-js, plaintext - you separate configuration parts from data and application logic parts which follows my line of thought, concept I propose and result

  2. My idea is the following we have a template template and the text file. We can send the copy of the template template to some backup service like: google drive/dropbox etc - I’m thinking of a universal plugin for this

    • Getting back to what I was saying, in the case of separating the template template from the text template
    • you get it with a good chance of having more security - since you can sync the template template to a remote server and sync the text file on another remote server. The idea here would be to split the file in two, in such a way that no part of that file is lost. It’s even easier to manipulate the template part and the text part.
    • A good example of what I’m talking about is the case of EDL, you can split part of the text file and synchronize with the json configuration part.
    • This guarantees, for example, that if the model is modified, it does not make the text part manipulable, you guarantee greater reliability in the parts of the file without them suffering interference. A good example of this specific use is Xanadu.
  3. Template file: model-layout(database: JSONSchema, UISchema, JSONSchemaEdit-js), plain text file " edl", “opendocument”, “object data linking”.

  4. It is highly modular

  5. Easy to use, manage

  6. Makes life easier for users

  7. It is aesthetically beautiful

  8. Great for multiple sync

  9. What I’m proposing here is the tiddlywiki file format which is made through these layers: ‘logical layer from the application’, ‘interface/interaction layer’, data layer.

2. Sample “library-js: hope.js”
0-43:h3
45-139:p
116-126:a href="http://en.wikipedia.org/wiki/Ted_Nelson"
141-368:p
205-213:a href="http://en.wikipedia.org/wiki/Project_Xanadu#Original_17_rules"
371-683:p
441-464:a href="http://blog.codinghorror.com/what-you-cant-see-you-cant-get/"
685-939:p
941-1223:p
1185-1191:em
1205-1211:em
1225-1356:p
1358-1368:h3
1370-2351:ul
1370-1468:li
1470-1622:li
1624-1698:li
1700-1759:li
1761-1925:li
1927-2072:li
2074-2350:li
2352-2453:p
2403-2428:a href="https://github.com/poef/hope"
2.1 sample: “editor.js”- “JSONSchema”+ “UISchema”+ “JSONSchema”+“hope-js” = json in tiddlywiki by standard
{
  "app": {
    "config": "./settings.json",
    "cwd": "./",
    "src": "example/content/",
    "filePattern": "**/*.md",
    "dist": "example/output.json",
    "name": "markdown-json",
    "version": "0.0.1"
  },
  "data": [
    {
      "section": "Elements",
      "title": "buttons",
      "device": [
        "desktop",
        "mobile"
      ],
      "styles": [
        "https://example.com/styles/structure.min.css",
        "https://example.com/styles/app.min.css"
      ],
      "contents": "116-126:a","reference:href", "link:"http://en.wikipedia.org/wiki/Ted_Nelson",  
      "id": "buttons",
      "meta": {
        "relativePath": "content/buttons.html",
        "createdAt": "2020-10-08T16:05:30.415Z",
        "lastModified": "2020-10-08T16:05:14.452Z",
        "size": 2095,
        "formattedSize": "2.0 KB"
      }
    },
  ]
}
4. References
1 Like
  1. This is responsible for customizing the text content in a specific part
  2. 116-126(start characters and end characters)
  3. Maybe in the future we can create an RFC in IEFT about the .tiddlywiki file format

TLDR;

There was a discussion about interchangeable field names with the help of Ontology definitions

IMO the linked topic above and your thread here share some common interests. … They both can help to increase interoperability. … They are not the same but imo close enough to create the connection.

But both topics are for developers only.


Not really. JSON keys and values can’t be “minified”. If we remove linebreaks and indentation we will loose human readability. So I don’t see any size related win here.

One of the biggest advantages of TW is, that it is schema-less. Our users will need to be able to store any data they want. 95% have no idea what a schema is and that’s OK. They shouldn’t need to care about it.

So is wikitext and the tiddler format, which internally is an Object store, which can easily be serialised into JSON if we need to.

100% of the TW UI is built using wikitext, that is parsed and internally translated into widgets. The advantage is, users don’t need to know about it.

Not true for non tech users. Creating a JSON structure that parses without errors in a consistent way the first time you save the tiddler, is hard. Even for experienced programmers. The most common error is missing commas and braces.

May be if you are a programmer. No if you read my last response.

If we want to make the file size smaller and minify the JSON data, then no. It’s unreadable in that form, without external help.

Internally we already us JSON as our main data format, but it doesn’t need to be exposed to our end users.

2 Likes

@pmario

1. Idea\concept

This can be done using the edl concept - here is a brief example:

"contents": "116-126:a", // 116-126(start characters and end characters) to anchor 'a'
"reference":"href", // reference('a') with href
"link":"http://en.wikipedia.org/wiki/Ted_Nelson",   // reference('a') with link
// <a href="http://en.wikipedia.org/wiki/Ted_Nelson"> words words .... </a>

2. other concept

<script src="https://cdnjs.cloudflare.com/.../json2html.min.js"></script>
 
<script>
    
    let data = [
        {"title":"Heat","year":"1995"},
        {"title":"Snatch","year":"2000"},
        {"title":"Up","year":"2009"},
        {"title":"Unforgiven","year":"1992"},
        {"title":"Amadeus","year":"1984"}
    ];
    
    let template = {'<>':'div','html':'${title} ${year}'};
    
    //render
    document.write( json2html.render(data,template) );
    
</script>

3. References

  • https:// json2html. com/
1 Like

@anon5541130 … That’s a very interesting thread. I do enjoy it.

That’s interesting, but the first paragraph catches my eye:

Note: this project is superceded by cobalt

cobalt is interesting too, but the last release was 2015 … So for me personally it may be interesting from an archaeological point of view, to study the concepts involved. But a version number 0.1.0 imo is far away from production.

I don’t know the library, but it seems to be a templating engine that can convert a JSON structure to HTML output. … That’s nice for static HTML output.

But what, if we have a templating engine that allows ordinary users to create a fully dynamic, basic task manager with 13 lines of code, that works in the “story river”, in the sidebar-tab and is part of the TOC (table of contents)

We create 2 tiddlers

  • 1st TaskDraggableTemplate with the content below
  • 2nd simple-task-manager with the content below
title: TaskDraggableTemplate

<$checkbox tag="done"> <$link to={{!!title}}><$view field="title"/></$link></$checkbox>

Second tiddler

title: simple-task-manager
caption: Tasks
tags:  $:/tags/SideBar TableOfContents 

!! Outstanding tasks

<$button>
<$action-sendmessage $message="tm-new-tiddler" title="New Task" tags="task" />New Task
</$button>

//Drag the tasks to re-order them//

<<list-tagged-draggable tag:"task" subFilter:"!has[draft.of]!tag[done]" itemTemplate:"TaskDraggableTemplate" emptyMessage:"You don't have any active tasks">>

!! Completed tasks

<$list filter="[!has[draft.of]tag[task]tag[done]sort[created]]">
<div>
<$checkbox tag="done"> ~~<$link to={{!!title}}><$view field="title"/></$link>~~</$checkbox>
</div>
</$list>

TW wikitext is very very powerful.

I need to think about this one. … But I do like the idea of the Xanadu project

I also like the idea of Hypercore which is a decentralised peer to peer protocol with security built in.

If you want to capture edits, then atJSON might be interesting to look at https://github.com/CondeNast/atjson

1 Like

This is not seen by the user but internally by the application.

example in markdown

my concept

reference

1. intro

  1. My concept is that the tiddlywiki file is readable by the browser - but it is a binary file like the PDF - however unlike the PDF the tiddlywiki file can be viewed, changed to create new modified tiddlywiki files - the difference from the approach with the PDF is that here tiddlywiki looks like opendocument in a structural sense - another difference is that how it opens the information in the browser - that same information can be taken elsewhere
  2. Think about this concept here: PDF+OpenDocument+Json+Browser+Web Bundle = tiddlywiki ;D

more note

  1. by technical definition any binary file is unstructured data, which by definition of the word unstructured data makes me wonder if tiddlywiki follows the idea of unstructured data or if tiddlywiki is just semi-structured data.
  2. See, Ted Nelson in your articles and ideas - I could be wrong, but I don’t know personally just that they speak in different it groups etc - I’ll cut to the chase… your ideas are for unstructured data, including the idea of inline markup was exactly that, an embedded markup that was binary, that is, not readable to humans and only machines
  3. a good example of this use case would be wysiwyg editor - you just need to mark up the text you want just by looking at the markup you’d like to do - things like Microsoft Office/Libre Office do the same thing

proof

references

benefits

  1. read only by machines “json in tiddlywiki by standard” is binary file
  2. it is highly shareable: “plugin odbc for tiddlywiki-json-by-standard”
  3. it is highly modular: “atjson”/“plugins tiddlywiki”
  4. it is highly small: “is as bson”

with your tiddlywikis you can generate other files like pdf, markdown etc

please, see this: github-com/clarkduvall/JSOL -

my idea

  1. my idea would be to autogenerate plugins like create-react-apps
    https://tiddlywiki.com/dev/static/How%20to%20create%20plugins%20in%20the%20browser.html
  2. https://tiddlywiki.com/dev/static/Developing%20plugins%20using%20Node.js%20and%20GitHub.html
  3. https://tiddlywiki.com/static/PluginMechanism.html
  4. Creating Plugins: Grok TiddlyWiki — Build a deep, lasting understanding of TiddlyWiki

This is still a short draft, and I’m open to new ideas, proposals, improvement or fixes

Standards(formats\apps\technologies) I’m Watching To Make This Possible

  1. “package.json”
  2. JSONSchema: “UISchema”
  3. JSONSchema: “DataSchema”
  4. JSONSchema: “Editor-js”
  5. EDL
  6. BSON
  7. OpenDocument
  8. JSON
  9. Wikitext
  10. Json2HTML
  11. Created-react-apps for “created-tiddywiki-plugins”
  12. atjson
  13. PDF
  14. Web Bundle
  15. JSOL

concepts

  1. tiddlywiki → “package.json”+“EDL”, JSONSchema: “UISchema”+JSONSchema: “DataSchema”+JSONSchema: “Editor-js” → struct tiddlywiki “file format as web bundle/OpenDocument/PDF/BSON”
  2. Created-react-apps “created-tiddywiki-plugins” → generate plugins tiddlywiki
  3. tiddlywiki-app-engine: “Json2HTML”, “atjson”, “JSOL”, “Wikitext”