Sorry, my answer below doesn’t work. I’m elaborating another answer.
I would advise to put triple double-quotes around the widget, like this (untested):
<$button>
<$action-setfield $tiddler=demolower lowered="""<$transclude $variable=makelower input={{!!input}}/>""" />
Set field "lowered" to lowercase version of entered input
</$button>
Please tell me how it goes
Fred
Hi Fred,
This is very interesting, your code seems to work, but doesn’t have the truly desired outcome. It sets the value of the “lowered” field to <$transclude $variable=makelower input={{!!input}}/> so it doesn’t have the desired outcome of setting the lowered field permanently
to the simple text string of “hello world”.
If you display {{demolower!!lowered}} what is displayed will change according to what is entered into the input box.
But I have now figured out the solution!
It is to use <$wikify>
As follows:
<$wikify name=processed text='<$transclude $variable="makelower" input={{demolower!!input}}/>' output="text">
<$button>
<$action-setfield lowered=<<processed>> />
Set field "lowered" to lowercase version of entered input
</$button>
</$wikify>
You are right, wikify is the solution in this case.
I suppose your makelower code is just an example (there’s already a core lowercase filter operator which does exactly this), but I think makelower should be a function, not a macro: it takes an input, transforms it and returns a result, which is better served by a function than a macro. Macros are supposed to substitute text with few transformations, not return a result. Functions are evaluated like filter operators, which adds more flexibility.
If makelower was a function, you could write this:
<$button>
<$action-setfield $tiddler=demolower lowered={{{ [function[makelower],{!!input}] }}} />
Set field "lowered" to lowercase version of entered input
</$button>
Links:
Fred
Thank you, Fred. Yes it was just an example, and what I have in mind is to create a more complex text processing operation, with lowercase just being the first step. AI created some elegant and short javascript code for me that does exactly what I want in terms of input and output.
I had already figured out a way to do this complex processing with filter operators, but it involves a huge number of search-replace:gi[x],[y] and other operations which feels clunky. I also like the idea of being able to call a macro in any tiddler, whereas I believe the function needs to be defined inside every tiddler where it is called?
But thank you for the insight, the truth is I have avoided learning how to use functions, and there are probably many places where using them would be the best option!
Maybe your existing javascript code can be turned into a javascript filter operator definition?
Wikitext function definitions, defined with the \function pragma, can be placed in a tiddler tagged $:/tags/Global in order to make them available in any tiddler.
Javascript filter operators are also available in any filter in any tiddler.
I tend to use filters and functions very frequently, so I may be biased, but I can’t recommend using them highly enough!
Fred
This use of AI (LLM) is possibly why you are coming at this from the wrong direction, writing javascript macros to solve problems or develop solutions in TiddlyWiki is not the preferred approach, in TiddlyWiki, it should start with wikitext, filters, widgets and functions, what I call “TiddlyWiki Script”. These are the tools TiddlyWiki provides to manipulate TiddlyWiki. 96% of requirements can be satisfied this way, and most of the rest by using plugins already developed.
In a very small number of cases, writing a javascript solution may be needed, but these should also flag a need to update the core of TiddlyWiki to reduce the need in future. If we needed to do this we would more likely modify existing code to enhance feature or build on top of existing JavaScript definitions of filter operator, widgets etc…
Thank you @TW_Tones and @tw-FRed for your nudging me away from using macros as a first approach. I understand from what you have written that it is for the best, and follows the TiddlyWiki philosophy.
Here is my attempt at just describing my problem and desired outcome:
This was all started with a frustration that there wasn’t already a built-in feature in the search operator.
I wish there were an accentinsensitive flag. Or I wish that, by default, searches were not sensitive to accent/diacritics.
In most blingual dictionary software, when you look up a word and are not sure of the accents in the spelling, or are too lazy to type them, it doesn’t matter. Typing “ele” into the input field will match both “element” and “élément”
My wikis are multilingual and I want to, for example, do a search such as [search:notefield:accentinsensitive[element]] and it would match tiddlers that contain any of these spellings in the notefield: element, Element,élement,Élement ,élément, Élément (the middle two are actually incorrect French spellings)
Currently, when using the search operator, “ele” will only match the first two, and “élé” will only match the last two.
My solution is that any time a notefield is created or updated in a tiddler, another unaccentednotefield is created (now with a javascript macro, but in my previous solution with transcluded filter operations) and when I search the wiki, I have things set up to search both of these fields when looking for a word [search:notefield,unaccentednotefield:[element]]
But maybe there is a simpler, built-in way of doing “accent-insensitive” searches of fields, without having to pre-generate “unaccented” duplicates of these fields, and include them in the search operation? If so, I am all ears!
Being French myself, I had to use a similar technique in a wiki I designed which heavily relies on search. By the way, it’s also useful for sorting.
FWIW, I think it’s a typical problem for which the solution would be better suited inside the core search/sort functions. But the subject is complex (different alphabets, syllabic languages, ideograms, …) and a core solution would probably need to be generic enough to cover most languages peculiarities, probably with a plugin for each language category.
Does any javascript fluent community member know existing solutions for this?
Fred
Yes I use it for sorting as well! And yes I agree, it would be wonderful to have accent-insensitive searching and sorting, or a way to toggle it on or off, included in Tiddly Wiki’s core features.
I understand your requirement here and think it is a reasonable to suggest its something the core should handle. How we achieve this needs to be determined, perhaps rather than a filter operator suffix we could have a global search setting in a config tiddler? Perhaps it could even turn on with the installation of any language that uses any “accent/diacritics”. There may need to be cases where only the literal is found for example if one was looking for words that did not use the accents.
Perhaps we should start a new topic with this specific need? As a discussion first, and if no solutions appear raise one for the developers topic.
If you are in desperate need of a workaround perhaps we can hack $:/core/modules/filters/search.js with an LLM to include diacritical’s. It will also demonstrate how we may change it in the core.
I and I have tried to kick it off, if none of the core team chime in, ask and I will create the hack with the LLM.
For a work around you can possible go a long way using regular expressions such as;
Note: I have not tested this.
Thank you for looking into this @TW_Tones
I already am happy enough with my current inelegant workaround of generating duplicate “unnaccented” fields of any data I want to be able to search or sort. It works for me in a reliable way, and as I don’t deal with many accented languages (only French amd German) this explicit definition of character replacements is fine for my use case. So I am not in urgent need of a solution.
But thank you for following up on this, as I do think it would be a great feature to have for the whole TiddlyWiki community, to have “diacritic insensitive” searching and sorting for all latin-script languages that use diacritics.
I have no technical skills to contribute to development here, so I can only express my encouragement and gratitude as an end-user, and also reassure you that I personally am not in urgent need of a solution (as I already have a usable workaround).
Just a small note: Tiddlywiki prerelease 5.4.0 was published a few days ago. It adds a new prefix to the sort filter operator in order to make it locale aware:
https://tiddlywiki.com/prerelease/#%24%3A%2Fchangenotes%2F5.4.0%2F%239400
Kudos to the dev team!
Fred
This is great news! Congratulations and thanks to the developers!
Just for clarification, is the actual approach to locale-specific sorting rules being designed and implemented with this update, or is this change simply opening up the opportunity to integrate this feature in future?
Are there plans to do this for search operator as well?
I am very keen to see how sort (and search!) can be made more friendly to international users. I hope that multiple locales can apply in the same operation, so that fr-FR and de-DE sorting rules, for example can both apply to what is being sorted.
Or I hope that there is a generic “ignore diacritics in latin-script characters” for both searching and sorting (this could cover almost all of the accented characters one finds in latin script languages, with only a few edge cases needing to be explicitly coded like ß->ss in German or œ-> oe in French, æ->ae in English and some Scandinavian languages.
This is a broad diacritic-removing function (doesn’t work for ß and œ and æ).
function removeDiacritics(inputStr) {
return inputStr
.normalize('NFD') // Decompose characters
.replace(/[\u0300-\u036f]/g, ''); // Remove diacritical marks
}
// Example usage
const text = "Café façade naïve résumé";
const result = removeDiacritics(text);
console.log(result); // Output: "Cafe facade naive resume"
AFAICT sort operator already sorts -at least French- accented characters correctly in current version (v5.3.8):
y ÿ ŷ à ç é è e ö ô y ï ê +[sort[]join[ ]] returns “à ç e é è ê ï ô ö y ŷ ÿ”
The new parameter allows the operator to sort according to a specific locale when different locales lead to different results.
Look at the documentation and examples for the sort operator in the preview here.
These enhancements will be available as soon as v5.4.0 is official.
I didn’t see anything about search in the release notes for v5.4.0 preview, but there might be open bugs or pending PRs for next releases.
Fred
This looks way more effective than my pure wikitext implementation… 
Fred
Here it is for the record:
<!-- Text substitution patterns -->
\define Any-a() [ÀÁÂÃÄÅàáâãäå]
\define Any-e() [ÈÉÊËèéêë]
\define Any-i() [ÌÍÎÏìíîï]
\define Any-o() [ÒÓÔÕÖØòóôõöø]
\define Any-u() [ÙÚÛÜùúûü]
\define Any-y() [ÝŶŸýŷÿ]
\define Any-c() [Çç]
\define Any-n() [Ññ]
\define Any-ae() [æÆ]
\define Any-oe() [Œœ]
<!-- Text sanitizer for sorting/searching -->
\function .toascii(src)
[<src>]
:map[<currentTiddler>search-replace:gi:regexp<Any-a>,[a]]
:map[<currentTiddler>search-replace:gi:regexp<Any-e>,[e]]
:map[<currentTiddler>search-replace:gi:regexp<Any-i>,[i]]
:map[<currentTiddler>search-replace:gi:regexp<Any-o>,[o]]
:map[<currentTiddler>search-replace:gi:regexp<Any-u>,[u]]
:map[<currentTiddler>search-replace:gi:regexp<Any-y>,[y]]
:map[<currentTiddler>search-replace:gi:regexp<Any-c>,[c]]
:map[<currentTiddler>search-replace:gi:regexp<Any-n>,[n]]
:map[<currentTiddler>search-replace:gi:regexp<Any-ae>,[ae]]
:map[<currentTiddler>search-replace:gi:regexp<Any-oe>,[oe]]
\end
Fred
Yes it is very efficient (and provided by AI 
). It’s not actually what I am currently using, though.
This is what I am currently using. Converting to lowercase avoids the need to specify É and é, for example. And even though some letters are redundantly repeated in the German section, it is visually clear what substitutions are being applied and easy to check you have everything you want (there won’t actually be duplicate entries in the accentmap dictionary, but its helpful for human readability and sanity checking).
/*\
title: mymacros/deaccent.js
type: application/javascript
module-type: macro
\*/
/*
Macro to deaccent a string
*/
"use strict";
exports.name = "deaccent";
exports.params = [{name: "input"}];
exports.run = function(input) {
// Define the mapping of accented characters to their simplest equivalents
const accentMap = {
// French
'à': 'a',
'â': 'a',
'ä': 'a',
'ç': 'c',
'é': 'e',
'è': 'e',
'ê': 'e',
'ë': 'e',
'î': 'i',
'ï': 'i',
'ô': 'o',
'ö': 'o',
'ù': 'u',
'û': 'u',
'œ': 'oe',
// German
'ä': 'a',
'ö': 'o',
'ü': 'u',
'ß': 'ss',
};
// Convert input string to lowercase and split into individual characters
const characters = input.toLowerCase().split('');
// Replace each character with its simplest equivalent if it's in the accentMap
const replacedCharacters = characters.map(char => accentMap[char] || char);
// Join the characters back into a single string
return replacedCharacters.join('');
};
If I used the newer script in my previous post, however, I could probably just add two lines to get the same effect:
.replace('œ' , 'oe');
.replace('ß' , 'ss');
@TW_Tones has mentioned the possible technical approaches for diacritic insensitive searching here, in the thread I linked a few posts up (title: diacritic-insensitive-searching-of-fields-for-non-english-languages)
Sadly I am not technically knowledgable enough to understand how to make diacritic insensitivity a core TW sort/search feature, but I do hope the developers add it to their new feature plans!
I haven’t been following closely and don’t have the time for a proper investigation, but is String.prototype.toLocaleLowerCase an appropriate solution? Clearly it would have to be folded into search, since, I assume, it’s not already there. But I think it would likely beat the best custom implementations we’re likely to develop.