You may be surprised, but I’m impressed with Custom Instructions. Great idea.
(@CodaCoder thanks for reminding me to add it to the list)
And here I thought the best touch was pasting to upload
Mostly I just got tired of opening the JS in order to add new instructions 
2 Likes
Minor Fix - 0.9.1:
$__plugins_NoteStreams_Expanded-Chat-GPT (0.9.1).json (95.3 KB)
Realized Transclusion pattern for images was interfering with text, minor fix which distinguishes images from text and, since I was in there, also pushes a stream-list for text tiddlers, if it exists
I know I’m going against what I said only a few days ago, but I’m releasing a not-insignificant upgrade,
-
because of this plugin’s mention in the Tiddlywiki Newsletter likely to bring it more traffic, and
-
because I finally had the agent misunderstand and perform a task that was frustrating enough to undo that I felt it amounted to a “fix” that I could not ignore
$__plugins_NoteStreams_Expanded-Chat-GPT (0.9.3).json (116.9 KB)
The “fix” in question is technically a new feature, but probably deserves to have been included earlier in the process – an undo button will now undo most any action the agent can perform. Just a reminder that this includes Creation and Modifying of tiddlers.
As you can see here, one of the “experimental” features that’s packaged with this update is the ability for the agent to open and close Tiddlers in the story river. Another, which can also be affected by the undo button, is a function for Renaming Tiddlers
Behind the scenes, there’s another improvement that I believe might qualify as a “fix” – the agent must now perform a process verifying its actions before it reports that it has performed them. This eliminates a problem in which the model would either
-
hallucinate that it had performed an action without even attempting to do so, or
-
attempt to perform an action, following the methods for doing so that it has been provided, and reporting that the action had been completed based on the assumption that calling the function was always sufficient for an action to be completed.
This latter point may be a bit confusing, as it’s not unreasonable to assume that functions would consistently perform if they are constructed well – to elaborate, I will use another improvement as an example:
There are occasions in which the agent might be responding to “partial hits” – let’s take the case of “Please add the tag Yolo to all tiddlers with their Generation field value set to Millennial”
-
This type of complex task requires multiple steps which the agent first breaks down into a list performs one at a time.
-
It is very possible in a situation like this that only some of the appropriate changes are performed, even if the model has correctly identified all the changes that need to be made.
-
In such a case, it would be reasonable for the agent to affirm that modification of the tiddlers has taken place, because it is accurately reporting partial successes.
A change to the modifyTiddler function has reinforced the agents ability to perform this kind of multi-step task – Additionally, the verification step makes sure that the agent confirms that the action has actually been performed, attempts a maximum of 3 times if it has not, and then informs the user of the results, rather than arbitrarily reporting.
This is a feature that still very much qualifies as experimental. Validation can become complex quickly and balancing that against wait times and timeouts and retries is all a bit complicated… While it’s definitely stable and safe to use at this point, there’s definitely still refinement I’ll be making to this process before it’s completely seamless.
While I have a few other “fixes” along these lines that I am working on, in addition to UI features and prompt refinements that I would like to make, I feel that it is timely and valuable enough to release this improved version for people to experiment with.
Without going too deeply into those fixes/features, it might be worth mentioning that I have begun with some experimentation that would allow the agent to perform the undo operation however, for the time being, it is incapable of that.
I think the button is a more valuable feature, frankly, but imagine a situation where the agent has performed an incredibly complex operation involving more than a dozen unique tasks in one query. The ability to tell an agent to undo “the last action,” could, effectively, undo dozens of actions at once, rather than requiring the user to sit and click the button dozens of times – a possibility I consider relevant.
2 Likes
Could this Plugin use handwriting recognition in you have a ChatGPT Account?
@JanJo, one will need to have an API key to use this at all, but other than the account associated with that, there is no need for a ChatGPT subscription or anything like that.
If one has that, it will perform handwriting recognition, yes, you can see in Expanded ChatGPT Interface - #12 by well-noted that it has taken an image of my (pretty poor) handwriting and done a solid job of interpreting it and setting the text of the tiddler accordingly.
2 Likes
On a separate note, the validation and verification steps that I am trying to to incorporate for more complex tasks are not quite there yet - - I intend to do further work to refine that, but any suggestions from the group (either specific code or abstract ideas about how the operations would go) are greatly appreciated.
In the meantime, the plugin continues to perform very well and I’ve incorporated it into many of my daily tasks – one should just verify that tasks have been performed correctly, which is much easier since you can prompt the agent to open the tiddler for you to validate, and the undo button works very consistently.
1 Like
Could the API-Key be hidden to make the function accessible without revealing the Key?
I’m not totally sure what you’re asking, @JanJo, so I will just give some context and you can ask for further clarity if I don’t address your question:
The API key that you get from platform.openai.com is a secret key that you only get once when you sign up for it. From your platform you can create more, but if you lose that key you cannot go back and get it.
If you have this plugin installed in your wiki there is a place to input the API key – that gets saved to the wiki as the text field of a tiddler.
The key is accessible only to those who have access to that tiddler. If you had a publicly available wiki, you would need to find some way to obfuscate and encrypt that tiddler… Although you would probably not want to have this plugin be available for people to use in a way that they could access your key, as each query sent by a user would charge your account (usually less than a penny for a simply query, but still)
It could pretty easily be modified for public use so that a user would have to input their own API key in order to make queries, which would not carry over through saves.
If you have some specific use case in mind, I’d be interested to help work through it with you.
I am a teacher and I would like to build an interface that gives pupils feedback on a handwritten input - so it would have to be my API-Key.
In the first step it should recognize the text and give hints on orthography.
In the second it should evaluate the content.
That’s an interesting usecase (and the first time I’m coming across the word orthography, thank you for that! 
)
I believe that it would be capable of recognizing misspellings and making suggestions - - I haven’t tested on elementary handwriting, but I imagine it would do a fairly decent job, though further tests would need to be done.
As far as your API key being used, possible misuse could be minimized pretty easily: The OpenAI platform allows you to associate any API key with a specific project title and then set limits on how much can be charged on that project per month. I have not had to reload my credits since throwing in $20 at the beginning of the year, so I think it could be fairly affordable and if you set the limit at say $5, it would be safe to allow students to use the interface with your API key without running the risk of going destitute.
This would protect you even if you had the API key publicly available and were just acting on the basis of trust – if you ever had reason to suspect the API key had been compromised, the damage would be minimal and you could just swap it out.
Another simple process would be creating a new API key for each assignment, that would again have guardrails against severe misuse.
As far as hiding the API key, though, I’m afraid I don’t have too much experience with sharing my wikis with others, and from what I’ve seen around here, the ability to hide or encrypt a tiddler in a sufficient way to prevent malicious intent seems challenging… though perhaps will become less challenging with the new release? (no citation on that, it’s just the general sense I have)
There may be someone who knows more about encryption and hiding tiddler content, however, that could give you more info.
1 Like
An easy way to make the API key less visible would be to hardcode it within the widget javascript – I don’t know the skill level of the average student, but it’s possible that the key would be sufficiently hidden from most people within the context of a very large codeblock.
But the API key would need to be accessible by the widget somehow… Would be very interested to hear if anyone has a method for extracting information from a source outside the wiki to use within it.
1 Like
There is an encrypttiddler-plugin by danielo that does a decent job on one or a list of tiddlers.
1 Like
What about entering a hashed key by a qr-code ?
1 Like
Great! Then all you would need to do is to encrypt the single api tiddler ( $:/plugins/NoteStreams/expanded-chat-gpt/openai-api-key) and you would be good to go.
Although you should still set up the guardrails in the openAI platform, since a user could still query the agent far more than necessary (Unless maybe you were to also modify the system message to forbid this kind of behavior… that would be interesting to attempt).
Would this be a key the student would input at time of use? If so, the key would have to decryptable for the widget but not decryptable for the student.
My cryptography skills are extremely minimal.
I guess the student should have to authentificate.
I also would love to have an LDAP or OAuth plugin for TW
Keep me up to date if you decide to attempt something like this, or would like to have further brainstorming on it. I often find that my usecases are niche, and find a lot of excitement and intrigue in hearing about and discussing those of others!
1 Like
Thanks a lot.
I guess this week I won’t have the time. Maybe in two Weeks.