Using the lingo macro for my own plugin translations

[Scott_Sauyet]

I’m wondering if there are established best practices around using the lingo macro and the $:/language/* internationalization technique for user wikis, in my case for a plugin/extension mechanism.

I have an external way to internationalize the main content, but I’m not happy with my current approach for internationalizing templates, procedures, functions, and the like. I was thinking of adopting something similar to how the core is internationalized, adding tiddlers like

title: $:/language/plugins/MyName/MyPlugin/Chapter

Chapter

for English

and

title: $:/language/plugins/MyName/MyPlugin/Chapter

Capítulo

for Spanish

I don’t have to worry about the language loading mechanism that the core uses. My version would simply have the language tiddlers overlay the default ones.

But I am worried about name-spacing. I haven’t seen any description of conventions here. The core uses 16 basic tiddlers, plus 39 namespaces, to hold slightly over 1000 language tiddlers.¹ Would it make sense that I simply claim the $:/language/plugins namespace for this? That would mean it’s not available for the core. I think the core would likely use $:/language/Plugin if it ever needed something like this. But I don’t want to step on toes.

Would that be reasonable? Is there any objection to my starting a convention that $:/language/plugin/MyName/MyPlugin/* is to be used for plugin-specific language translation tiddlers?

Pinging @jeremyruston, @pmario, @saqimtiaz, @EricShulman

---

¹ These:

Language Tiddlers

$:/language/ConfirmAction	$:/language/DropMessage
$:/language/ConfirmCancelTiddler	$:/language/LazyLoadingWarning
$:/language/ConfirmDeleteTiddler	$:/language/LoginToTiddlySpace
$:/language/ConfirmDeleteTiddlers	$:/language/No
$:/language/ConfirmEditShadowTiddler	$:/language/OfficialPluginLibrary
$:/language/ConfirmOverwriteTiddler	$:/language/PluginReloadWarning
$:/language/Count	$:/language/UnsavedChangesWarning
$:/language/DefaultNewTiddlerTitle	$:/language/Yes

Language Tiddler Namespaces

$:/language//*	$:/language/Error/*	$:/language/RecentChanges/*
$:/language/AboveStory/*	$:/language/Exporters/*	$:/language/RelativeDate/*
$:/language/BinaryWarning/*	$:/language/Filters/*	$:/language/Search/*
$:/language/Buttons/*	$:/language/Help/*	$:/language/Shortcuts/*
$:/language/ClassicWarning/*	$:/language/Import/*	$:/language/SideBar/*
$:/language/CloseAll/*	$:/language/InternalJavaScriptError/*	$:/language/Snippets/*
$:/language/ColourPicker/*	$:/language/LayoutSwitcher/*	$:/language/Switcher/*
$:/language/ControlPanel/*	$:/language/Manager/*	$:/language/SystemTiddler/*
$:/language/Date/*	$:/language/MissingTiddler/*	$:/language/SystemTiddlers/*
$:/language/Diffs/*	$:/language/Modals/*	$:/language/TagManager/*
$:/language/Docs/*	$:/language/Notifications/*	$:/language/ThemeTweaks/*
$:/language/EditTemplate/*	$:/language/OfficialPluginLibrary/*	$:/language/Tiddler/*
$:/language/Encryption/*	$:/language/PageTemplate/*	$:/language/TiddlerInfo/*

[pmario]

The main problem here is, that the core does not have any functionality to switch / extract your texts.

There has been a PR, which is now closed because @jeremyruston is developing an alternative approach: Introducing bundled sub plugins #8957 at the moment.

The structure in the “now closed” PR was as shown in the screenshot from OP

As you can see, the

• active texts follow $:/plugins/<author>/<plugin-name>/language/...
• available texts follow $:/plugins/<author>/<plugin-name>/languages/<language-code>/...

I do not know, how @jeremyruston’s approach is with the new plugin, but I think it needs to be similar, to avoid naming problems.

[Scott_Sauyet]

@pmario

The main problem here is, that the core does not have any functionality to switch / extract your texts.

In my case, I don’t think that will be a problem. This is designed to allow the same codebase to be used across a number of wikis, with just a few language-specific tweaks. All the important content will be in just one language. There is no dynamic switching of language.

This is related to things I’ve discussed in three previous posts regarding Bible wikis.

Currently I handle the small bit of internationalization needed with lines like this:

  {
    "title": "${title}",
    "tags": "${TableOfContents}",
    "text": "<div class=\"tc-table-of-contents\" style=\"column-width:12em;\">\n\n<<toc-selective-expandable \"${Book}\" \"nsort[seq]\" >>\n\n</div>\n",
  },

(note the use of ${title}, ${TableOfContents}, and ${Book})

which uses lines like these from a translation file:

  "Chapter": "Capítulo",
  "Book": "Libro",
  "Books": "Libros",
  "Verse": "Versículo",
  "Contents": "Contenido",
  "TableOfContents": "Contenido",

and this bit of code:

const update = ({title, language: {Book, Books, Chapter, Verse, Contents, TableOfContents, books: {Psalms}}}) => (tiddler) => 
  Object.fromEntries(Object.entries(tiddler).map(([k, v]) => [
    k, 
    v.replaceAll('${title}', title)
      .replaceAll('${Book}', Book)
      .replaceAll('${Books}', Books)
      .replaceAll('${Chapter}', Chapter)
   // ...

to create a tiddler like

title: "La Biblia Reina Valera Gómez"
tags:"Contenido"

<div class="tc-table-of-contents" style="column-width:12em;">

  <<toc-selective-expandable "Libro" "nsort[seq]" >>

</div>

That works fine for creating the various Bible versions. But the idea of extensions for these wikis, created not just by me but by others means that I have to revisit this to handle that. Here’s where I thought of using <<lingo>>. I can add language tiddlers for each of those terms above, and the extension can use them with <<lingo>>. If the extension has other internationalization needs, it can define its own language file that extension translations can also implement.

Of course I can define my own macro in place of <<lingo>>, but using the $:/language namespace seems to make sense. I’m just hoping that it’s reasonable to use the namespace the way I suggested.

[pmario]

@Scott_Sauyet

Of course I can define my own macro in place of <<lingo>>, but using the $:/language namespace seems to make sense. I’m just hoping that it’s reasonable to use the namespace the way I suggested.

I think even if Jeremy will change something, we still need to have the possibilities to switch between languages. So IMO the linked namespaces still stand. So for active texts I would use: $:/plugins/<author>/<plugin-name>/language/... which should be compatible, if you set the lingo-base variable right.

@pmario

As you can see, the

• active texts follow $:/plugins/<author>/<plugin-name>/language/...
• available texts follow $:/plugins/<author>/<plugin-name>/languages/<language-code>/...

I do not know, how @jeremyruston’s approach is with the new plugin, but I think it needs to be similar, to avoid naming problems.

[Scott_Sauyet]

Well, if I’m not going to use the $:/language namespace, there’s little reason to use the <<lingo>> macro. I can write my own more specific to the problem. But I thought it would be a good idea to keep all internationalization text in one place. Clearly you disagree, but I’m afraid I don’t quite understand your reason.

While one could certainly change the interface language in one of the wikis, it would seem an odd experience, with the content still all in the initial language:

My language tiddlers, under the $:/language/plugins/<author>/<plugin>/ namespace would not currently be replaced when the language changes; the core includes no tiddlers there. I was hoping that we might agree to keep it this way.

[Scott_Sauyet]

/bump

I’m simply hoping this might get more attention. (Thank you @pmario for your responses. @jeremyruston, any thoughts?) The basic question is,

@Scott_Sauyet

Is there any objection to my starting a convention that $:/language/plugin/MyName/MyPlugin/* is to be used for plugin-specific language translation tiddlers?

[linonetwo]

Anyway, those are going to be replaced by bundled sub-plugin. We could do anything now, but later just don’t forget to migrate to bundled sub-plugin and follow its standard.

I’m currently still using what I wrote in feat: i18n framework based on extracting translations from plugin's /languages/ to /language/ by linonetwo · Pull Request #8435 · TiddlyWiki/TiddlyWiki5 · GitHub , a lingo macro, in my many plugins. But I will migrate to bundled sub-plugin once it is done.

[jeremyruston]

Hi @Scott_Sauyet I prefer $:/plugins/MyName/MyPlugin/language/Chapter over $:/language/plugins/MyName/MyPlugin/Chapter, primarily because I think of the $:/language/ namespace as being for the core. The secondary reason is that I prefer the primary grouping of shadow tiddlers to be their source, rather than their function, because a tiddler can have only one source, but might have multiple functions.

The upcoming addition of bundled sub-plugins that @linonetwo mentions will not enforce any particular naming convention, it is primarily concerned with allowing language plugins to be packaged within the parent plugin. It will allow a plugin author to publish a plugin with integrated translations for multiple languages.

[Scott_Sauyet]

@jeremyruston

Hi @Scott_Sauyet I prefer $:/plugins/MyName/MyPlugin/language/Chapter over $:/language/plugins/MyName/MyPlugin/Chapter, primarily because I think of the $:/language/ namespace as being for the core.

That doesn’t bother me particularly. I haven’t used lingo-base for anything, and in fact have done little with internationalization in TW altogether. I was thinking that the order I suggested would simply allow for the use of the lingo macro directly in these plugins. But either writing my own version or figuring out how to use lingo-base should be easy.

I’m not sure how much I like the idea of that namespace belonging to the core, partly because the whole design of TW seems to allow a continuum of uses from boot through core through plugins into userland. But I have no objection to this decision, Moreover the following is much more compelling to me:

@jeremyruston

The secondary reason is that I prefer the primary grouping of shadow tiddlers to be their source, rather than their function, because a tiddler can have only one source, but might have multiple functions.

My personal use-case is quite different from this one:

@jeremyruston

It will allow a plugin author to publish a plugin with integrated translations for multiple languages.

My idea is that a plugin-author could use a lingo-like mechanism that allows independent authors to add their own language extensions.

This is for what might be an unusual usage of TW: A collections of wikis with identical structures but content that varies by language/translation. Each wiki is a collection of bible verses collected into chapters inside books, where field names and tag values such as Verse, Chapter, and Book will vary by language. What this wouldn’t allow is dynamic changing of the interface. I can see little need for that in this case. If the bible text is in Spanish, I would expect the interface to also use Spanish.

But I also want to support shareable user-created extensions. These would modify or extend the functionality of the base wiki for various needs. If the initial author uses the supplied lingo-like mechanism to handle those parts of the text that might need internationalization, then it would work for their own case, but would be easily extensible to other languages with simple translation tiddlers. The current example extension I’m playing with would be one that adds New/Old Testament to the otherwise flat list of books, and adds sections like Law, Minor Prophets, and Gospels to those two new groupings, listing the books in those sections.

A more in-depth discussion of the overall mechanism, and some implementation options is at Design question: how to configure tool to alter wikis with a specific structure?. I’m hoping to start working on this again soon.

[Xrizzy]

@pmario

As you can see, the

• active texts follow $:/plugins/<author>/<plugin-name>/language/...
• available texts follow $:/plugins/<author>/<plugin-name>/languages/<language-code>/...

@pmario If use this appoach with Projetify to create an UI that can switch between languages, will it switch automatically together with the TW language or do I have to adapt Projetify further?

[pmario]

@Xrizzy

@pmario If use this appoach with Projetify to create an UI that can switch between languages, will it switch automatically together with the TW language or do I have to adapt Projetify further?

Sadly that info is obsolete. As far as I remember Projectify did their own version. So you have to analyse that one. I do not know, how they did it.

There will be a new mechanism Introducing bundled sub plugins by Jermolene · Pull Request #8957 · TiddlyWiki/TiddlyWiki5 · GitHub . But that one is not finished yet and I do not know how it works atm.

[Xrizzy]

@pmario It looked to me like the english/german menubar worked, but I could not find where to get it.

Projetify does not switch. You change the language by importing the set of language tiddlers for one of the other 4 languages.

[Scott_Sauyet]

@Xrizzy

Projetify does not switch. You change the language by importing the set of language tiddlers for one of the other 4 languages.

I’m going to have to look at that implementation. That’s certainly similar to what I want to do when I get back to this project. lingo will be an easy macro to replicate, but I really would prefer to use the $:/langauge namespace for these! (When I get back to that work, that is!)

[TW_Tones]

Whatever the solution going forward, I think there should be a simple way to externalise text from TiddlyWiki script when writing solutions and/or plugins, but with no expectation that the designer will provide the translations, but make it possible for other parties to translate and store the alt language tiddlers. They could always forward a set of translations back to the author to include.

• But the key issue is the documentation for designers.

Further tools such as an easy way top select some text and exercise/macroise it to a language tiddler under a configured language namespace, or clone one or more language tiddlers into a new language allowing a systematic translation of all language tiddlers, from the source to new language.

• The easier you make it, the more people will do it
• Consider the idea of a place where someone posts their “first language” tiddlers and people who speak it and at least another language can voluntarily step through and send the result to the author.
  • All this with the translator not needing to know how tiddlywiki or a plugin works, just translate.

[Scott_Sauyet]

@TW_Tones

Whatever the solution going forward I think there should be a simple way to externalise text from TiddlyWiki script when writing solutions and/or plugins, but with no expectation that the designer will provide the translations,

Absolutely. And, while I applaud the existence of the Translator’s Edition, that wouldn’t be the best model for what I would expect to be much less extensive sets of strings needing translations.

For the case that inspired this, I could easily imagine asking translators to complete a single data tiddler:

title:  $:/plugins/MyName/MyPlugin/languages/es-ES
language: Español

main.Chapter: Capítulo
main.Book: Libro
main.Books: Libros
main.Psalm: Salmo
main.Verse: Versículo
main.Contents: Contenido
main.TableOfContents: Contenido

book.Genesis: Génesis
book.Exodus: Éxodo
book.Leviticus: Levítico
book.Numbers: Números
book.Deuteronomy: Deuteronomio
book.Joshua: Josué
...
book.1_John: 1 Juan
book.2_John: 2 Juan
book.3_John: 3 Juan
book.Jude: Judas
book.Revelation: Revelación

[pmario]

@Scott_Sauyet

Would that be reasonable? Is there any objection to my starting a convention that $:/language/plugin/MyName/MyPlugin/* is to be used for plugin-specific language translation tiddlers?

The actual structure for plugin translation is PR 8975

@jeremyruston
As far as I know, it is not finished yet. I did not test the PR yet. I did review it once in Feb. but I do not know how it works in detail.

[Scott_Sauyet]

@pmario

@Scott_Sauyet

Is there any objection to my starting a convention that $:/language/plugin/MyName/MyPlugin/* is to be used for plugin-specific language translation tiddlers?

The actual structure for plugin translation is PR 8975

I did read and briefly participate in that thread. IIRC, it’s all about bundling, about the mechanism we might use to package subPlugins. For me, at the moment, that’s much less of a concern. I suppose it might end up being useful, but my main concern here was about where in the tiddler namespace to put the translations.