Translating TW documentation - workflow proposal

Hi,

I intend to update and improve French translation of TW documentation, ie the fr-FR edition of TW.

One problem with this task is keeping translations up to date with subsequent modifications of the tiddlers in the original edition.

Thanks to the help of the numerous plugins, tutorials and resources provided by the community, I came up with a workflow that I think could be useful to others, and maybe improved through your comments.

General idea

The general idea is to collect and extract metadata from the original documentation at tiddlywiki.com, and then import it into the French edition, with some utility tiddlers and plugins, in order to:

  • Easily see if a tiddler has been modified since its last translation
  • Easily compare French and English contents of a tiddler

First step: collecting English metadata

For this I use a tiddler containing this code:

\define tTitles() $:/C.Titles
\define tCreated() $:/C.Created
\define tModified() $:/C.Modified
\define tText() $:/C.Text
\define tTags() $:/C.Tags

<$button>
<$list filter="[all[tiddlers]!prefix[$]!is[draft]!tag[NoExport]sort[]]" counter="cpt">
<$action-setfield $tiddler=<<tTitles>> $index=<<cpt>> $value=<<currentTiddler>>/>
<$action-setfield $tiddler=<<tCreated>> $index=<<cpt>> $value={{!!created}}/>
<$action-setfield $tiddler=<<tModified>> $index=<<cpt>> $value={{!!modified}}/>
<$action-setfield $tiddler=<<tText>> $index=<<cpt>> $value={{!!text}}/>
<$action-setfield $tiddler=<<tTags>> $index=<<cpt>> $value={{!!tags}}/>
</$list>
Go !
</$button>

The tiddler must be imported in tiddlywiki.com (we want the most recent English data available).
Clicking on the Go! button creates 5 data-tiddlers in which the title, created, modified, text and tags fields of every tiddler is stored. They can be exported to JSON using the following filter in Advanced Search: [prefix[$:/C.]]

Second step: preparing French edition

Metadata

Here again we want the most recent data available, so we go to https://tiddlywiki.com/languages/fr-FR/index.html and import the JSON metadata file.

The following code injects the metadata into new fields in each French tiddler:

\define tTitles() $:/C.Titles
\define tCreated() $:/C.Created
\define tModified() $:/C.Modified
\define tText() $:/C.Text
\define tTags() $:/C.Tags

!! Insert metadata into tiddlers

<$let
  tTitlesOK={{{ [<tTitles>has[text]then[✅]else[❌]] }}}
  tCreatedOK={{{ [<tCreated>has[text]then[✅]else[❌]] }}}
  tModifiedOK={{{ [<tModified>has[text]then[✅]else[❌]] }}}
  tTextOK={{{ [<tText>has[text]then[✅]else[❌]] }}}
  tTagsOK={{{ [<tTags>has[text]then[✅]else[❌]] }}}
>

;Checks:
:Titles imported: <<tTitlesOK>>
:Creation dates imported: <<tCreatedOK>>
:Modification dates imported: <<tModifiedOK>>
:Text imported: <<tTextOK>>
:Tags imported: <<tTagsOK>>

</$let>

If each line above is ✅ then <$button>
<$list filter="[<tTitles>indexes[]sort[]]" variable="cpt">
<$let
  myTitle={{{[<tTitles>getindex<cpt>]}}}
  myCDate={{{[<tCreated>getindex<cpt>]}}}
  myMDate={{{[<tModified>getindex<cpt>]}}}
  myText={{{[<tText>getindex<cpt>]}}}
  myTags={{{[<tTags>getindex<cpt>]}}}
>
<$action-setfield $tiddler=<<myTitle>> $field="C.created" $value=<<myCDate>> $timestamp="no"/>
<$action-setfield $tiddler=<<myTitle>> $field="C.modified" $value=<<myMDate>> $timestamp="no"/>
<$action-setfield $tiddler=<<myTitle>> $field="C.text" $value=<<myText>> $timestamp="no"/>
<$action-setfield $tiddler=<<myTitle>> $field="C.tags" $value=<<myTags>> $timestamp="no"/>
</$let>
</$list>
Go !
</$button>

The new fields are C.created, C.modified, C.tags and C.text. The action-setfield widgets use the $timestamp="no" parameter to preserve modification date of destination tiddlers.

The C.created field is useless though, because new English tiddlers are automatically copied to the French edition, so they share the same creation date.

Styles for outdated translations

The new cascades mechanism provides an easy way to highlight outdated translations with 3 tiddlers:

The first one named $:/title-when-outdated:

\whitespace trim
<h2 class="tc-title">
🆘⏰🇫🇷 &nbsp;<$view field="fr-title"><$view field="title"/></$view> &nbsp;🇫🇷⏰🆘
</h2>

Then $:/title-when-original, for untranslated tiddlers:

\whitespace trim
<h2 class="tc-title">
🆘🇬🇧 &nbsp;<$view field="title"/> &nbsp;🇬🇧🆘
</h2>

And $:/title-filter-for-translation for the cascade, tagged $:/tags/ViewTemplateTitleFilter:

[has[modified]has[C.modified]get[modified]compare:date:lt{!!C.modified}then[$:/title-when-outdated]]
[has[modified]has[C.modified]get[modified]compare:date:eq{!!C.modified}then[$:/title-when-original]]

The French translation uses a specially crafted ViewTemplate to display a translated title for tiddlers, stored in the fr-title field. This ViewTemplate isn’t compatible yet with cascades, so the above tiddlers won’t work until the shadow tiddler $:/core/ui/ViewTemplate is deleted.

Plugins

@pmario wrote two very useful plugins for translation: Toggle Field Visibility and Multiline Field Editor, both available here:
https://wikilabs.github.io/editions/field-editor/#%24%3A%2Fplugins%2Fwikilabs%2Ffield-visibility:%24%3A%2Fplugins%2Fwikilabs%2Ffield-visibility%20%24%3A%2Fplugins%2Fwikilabs%2Ffield-editor

After importing them into the French edition TW, configure the Multiline Field Editor for the C.text field – or create this tiddler (presented in TID format for convenience):

title: $:/config/wikilabs/field/C.text
description: English text
icon: {{$:/languages/en-GB/icon}}
tags: $:/tags/wikilabs/fieldswitcher
type: text/vnd.tiddlywiki

C.text

After configuration, the Multiline Field Editor plugin features an EditMode split view displaying two fields of the same tiddler at the same time, thus easing comparison of English and translated texts.

Cleanup

After translations are done, and before preparing a PR, English metadata must be removed from tiddlers:

\define tTitles() $:/C.Titles
\define tCreated() $:/C.Created
\define tModified() $:/C.Modified
\define tText() $:/C.Text
\define tTags() $:/C.Tags

!! Remove metadata from tiddlers

<$let
  tTitlesOK={{{ [<tTitles>has[text]then[✅]else[❌]] }}}
  tCreatedOK={{{ [<tCreated>has[text]then[✅]else[❌]] }}}
  tModifiedOK={{{ [<tModified>has[text]then[✅]else[❌]] }}}
  tTextOK={{{ [<tText>has[text]then[✅]else[❌]] }}}
  tTagsOK={{{ [<tTags>has[text]then[✅]else[❌]] }}}
>

;Checks:
:Titles imported: <<tTitlesOK>>
:Creation dates imported: <<tCreatedOK>>
:Modification dates imported: <<tModifiedOK>>
:Text imported: <<tTextOK>>
:Tags imported: <<tTagsOK>>

</$let>

If each line above is ✅ then <$button>
<$list filter="[<tTitles>indexes[]sort[]]" variable="cpt">
<$let
  myTitle={{{[<tTitles>getindex<cpt>]}}}
>
<$action-setfield $tiddler=<<myTitle>> $field="C.created" $timestamp="no"/>
<$action-setfield $tiddler=<<myTitle>> $field="C.modified" $timestamp="no"/>
<$action-setfield $tiddler=<<myTitle>> $field="C.text" $timestamp="no"/>
<$action-setfield $tiddler=<<myTitle>> $field="C.tags" $timestamp="no"/>
</$let>
</$list>
Go !
</$button>

Exporting translations

Translated tiddlers are listed with the Filter tab of the Advanced Search, using this filter:
[all[tiddlers]!is[draft]!prefix[$:/]] :map[get[modified]compare:date:gt{$:/C.Titles!!created}then<currentTiddler>]

Each tiddler should be exported to a separate TID file, it’s easier to add them to a PR in Github.

Workflow improvements (AKA: todo :sweat_smile:)

  • Remove created metadata from tooling (useless).
  • Improve and update ViewTemplate of the French edition: use cascades.
  • Add a filter showing outdated tiddlers to tooling.
  • Update @pmario Multiline Field Editor plugin for cascades mechanism: in its current state it breaks TW’s Import function. Many thanks @pmario anyway :slightly_smiling_face:
  • Use @Mohammad 's Gatha plugin to package utility tiddlers with @pmario plugins, for an easy import into translated edition.

Hi, I’m interested – But I need time to test your workflow with the German translation

That’s why those plugins are labelled “experimental” at: https://wikilabs.github.io

I prefer the bundler plugin to bundle plugins. I don’t know, if Gatha can put several plugins into a package and install them on import. bundler does handle that well

Gatha has two publishFilter and allows you to export plugins (themes, languages, …) to the reproduced Tiddlywiki.

But when creating a plugin and export it, it does not make sense to append another plugin to it!

Goodbye workflow, welcome Toolbox !

Well, not quite… The workflow is still the same, but using it was error prone, so I needed more tooling.

Enter frTT!

frTT_v0.3.1.json (15.2 KB)

frTT stands for French Translation Toolbox, but of course it can be used for other languages.

Here’s what frTT looks like:

SideBar1

The first step takes place on https://tiddlywiki.com. After importing ‘‘frTT’’, open the Metadata Export tiddler:

Export-metadata1

Click Go! to extract English metadata. The UI changes:

Export-metadata2

After exporting the JSON file, close https://tiddlywiki.com

You can now open your language TiddlyWiki edition and import frTT. The translation process still relies on @pmario plugins, see frTT readme for instructions on their installation.

The main frTT UI is a SideBar tab, with some tools buttons, and a list of translated tiddlers below.

SideBar2-before

From left to right, the buttons are:

  • Help button: links to frTT readme, which details all the workflow
  • Import button: just the plain old import button
  • Store button: inserts English metadata into user’s language TiddlyWiki
  • British flag button: opens a list of not-yet-translated tiddlers
  • Clock button: opens a list of outdated translations
  • Clean button: cleans English metadata from tiddlers, before exporting translated tiddlers

The warning message reminds you to clean up metadata before export.

Translated tiddlers bullet icons are :construction: WIP icons as long as they contain English metadata fields.
After using the Clean button, the list is modified:

SideBar3-after

Each translated tiddler can now be exported thanks to its export bullet button.

For more information, drop frTT JSON file on https://tiddlywiki.com and click the help button of frTT sidebar tab.

frTT_v0.3.1.json (15.2 KB)

(edited: update frTT package to v0.3.1)

3 Likes

Thanks @tw-FRed it’s great to see work in this area. Although translations are conceptually simple to understand, there is a lot of friction in keeping them up to date.

I’d welcome any of these enhancements to be added to the core “translators” edition, to make it the officially recommended workflow.

There was a bug with export bullet button and tiddlers whose title contains spaces.
Fixed using code inspired by @pmario’s bundler plugin.

frTT_v0.3.1.json (15.2 KB)

1 Like

Good! For those of us needing multiple language support sometimes @tw-FRed’s things definitely look like good additions.

Best, TT

frTT new release

This is frTT v0.4, with some enhancements:

  • Added a translation todo-list, and a ViewMode button to add tiddlers to the list. The use-case is this: you are translating a tiddler which contains a link to another tiddler of interest. You click the link, the tiddler opens, you click the TODO button, then you come back to your translation.
  • The todo-list can be exported/imported between translation sessions.
  • The frTT sidebar tab now also lists modified tiddlers in the $:/editions/ subtree (useful for documentation macros translation)
  • Better bullet buttons for translated tiddlers: clean then export

You can get it here:
frTT_v0.4.json (19.1 KB)

For more information, drop frTT JSON file on https://tiddlywiki.com and click the help button of frTT sidebar tab.

@tw-FRed … I did just upload the latest version of the field-editor and field-visibility plugin.

To use field-editor both plugins are needed!

  • field-visibility uses its own tab now and version moved to v0.1.0 … so it’s beta now and out of experiemental.
  • field-editor was a complete rewrite. It doesn’t modify the core edit-template anymore. It just uses a custom template now, that can easily be removed. …

Both of them are compatible with TW 5.2.1 and TW 5.2.2 now.

… So I can have a closer look at your workflow now :wink:

New bugfix release of frTT:

[FIX] Tiddlers whose title contains " can’t be added to TODOlist

frTT_v0.4.1.json (19.2 KB)

Enjoy!

Fred