Integrating AI Communication Into TW is Great - AI Created Wikitext not so Much

[etardiff]

Originally in response to AI-Integration in Tiddlywiki

Is it safe to assume that the model you’re using hasn’t been trained on TiddlyWiki syntax in particular? I asked it a TW question just to test your interface, and the results were not at all encouraging.

EDIT: I had posted the results of asking the LLM model showcased in @stelzi’s PerplexityWiki how to use conditional syntax in TiddlyWiki; @Mario removed them as per Talk TW’s AI content policy. But I do think it’s important to illustrate just how wrong the AI answer was, and I think the issues were more obvious in Discourse, which didn’t struggle with the codeblocks to the extent that the TW parser did. I hope that including a screenshot rather than raw code will be an acceptable compromise.

A screenshot sample of the redacted code

---

I should say that, despite the utterly fictitious answer, your UI does work smoothly… though the TW parser did seem to struggle with rendering the output. I assume the misused transclusions and/or the unclosed “HTML” element is tripping it up… interesting that Discourse doesn’t have the same issue.

Screenshot of the above code as rendered in the wiki

It’s an interesting experiment, and if a LLM (the same model you’re using in the wiki?) generated the bulk of the code… well, I’m impressed that you got anything so functional out of it.

[stelzi]

For creating the code in Tiddlywiki I use Perplexity AI pro and selected there the model “Anthropic Claude 3.5 sonnet”. It is actually a very new model, which creates good code, also for Tiddlywiki, but it’s not free.

I tried your question in Claude 3.5 like this:

How do I use conditional syntax in TiddlyWiki? Respond in Tiddlywiki syntax

The result was:

!! Conditional Syntax in TiddlyWiki
In TiddlyWiki, you can use conditional expressions to display or hide content based on certain conditions. Here are some examples:
!!! Simple Condition
text

// Moderated since code was wrong

!!! Condition with else branch
text

// Moderated since code was wrong

!!! Comparisons
text

<$list filter="[<currentTiddler>get[priority]compare:number:gt[3]]">
This text will only be displayed if the priority of the current tiddler is greater than 3.
</$list>

!!! Multiple Conditions
text

<$list filter="[<currentTiddler>tag[important]tag[urgent]]">
This text will only be displayed if the current tiddler has both the "important" and "urgent" tags.
</$list>

!!! Variables in Conditions
text

<$set name="myTag" value="important">
<$list filter="[tag<myTag>]">
<<currentTiddler>>
</$list>
</$set>

These examples demonstrate various ways to use conditional syntax in TiddlyWiki. You can combine and adapt these techniques to create more complex conditions and dynamically shape your TiddlyWiki content.

You see, there are different AI models which produce different output. The actually integrated model is
llama-3-sonar-large-32k-chat.

[Mario]

@stelzi
Your example wiki is impressive. Well done.

I’ll create a second post to discuss some possible code improvements.

@stelzi

Please have a look at: How to Show Backticks in Discourse Threads, which shows how we need to use “backticks” to mark code here in the forum.

Otherwise the forum parser will hide code eg: <some text> will be completely hidden. Or <<myMacro>> will only show up as <> when rendered here.

[stelzi]

Uuuhh sorry! I forgot this!
Thank you

[etardiff]

@Mario I understand the reasoning for removing the nonfunctional code from my post, but I’d shared it specifically to illustrate how very wrong the results were—as a cautionary note for anyone who might hope to use this for TW code—and I thought I’d signposted it as wrong (“utterly fictitious”). Is there a better way to demonstrate AI problems like this? Should I have stuck to screenshots only?

I did initially intend just to post the screenshot, but I thought the code errors were actually more obvious in Discourse, since the codeblock formatting didn’t break in the same way.

I was also going to point out that while the results from @stelzi’s paid model were better—they certainly looked a lot more like plausible TW code—they still included some nonexistent widgets ($if tiddler=). Your edits to his post are certainly helpful if the end goal is to demonstrate functional TW code, but they make it difficult to illustrate the ways in which the AI is unreliable.

[Mario]

@etardiff

I was also going to point out that while the results from @stelzi’s paid model were better—they certainly looked a lot more like plausible TW code—they still included some nonexistent widgets ($if tiddler=). Your edits to his post are certainly helpful if the end goal is to demonstrate functional TW code, but they make it difficult to illustrate the ways in which the AI is unreliable.

You are right, it’s hard to show, what’s wrong if it will be moderated and I do not have an answer right now.

I think @stelzi’s showcase it impressive and I’ll move the discussion about AI created content into it’s own thread, because the discussion about the showcase should be about the wiki and how the plugin was created and not about the shortcomings within AI Q&A’s

[etardiff]

@Mario

the discussion about the showcase should be about the wiki and how the plugin was created and not about the shortcomings within AI Q&A’s

Fair enough. I don’t want to detract from @stelzi’s showcase, which is impressive! At the same time, I think that, as he describes it as “fully created by AI”, people may reasonably expect that the AI he’s incorporating can reliably produce correct TW syntax… and I wanted to emphasize that this is not a safe assumption to make.