[tw5] How can I prevent TW from trimming leading space in field transclusion?

[Misterel85]

How can I prevent TW from trimming leading space in field transclusion?

When transcludings two fields one after the other, for instance:

<$transclude field="foo" /><$transclude field="bar" />

if the value of field “bar” starts with a space, that space is automatically removed in the transclusion.

In this example the successive transclusions of:

“foo”=> “This is foo”
“bar”=> " and this is bar."

render as:

“This is fooand this is bar.”

Inserting a trailing space in “foo” or a   between the two transclusions is not an option for me.
Placing a   at the beginning of the value in the second transcluded field works, but that’s a bit of a hassle with multiple transclusions.

Why does TW behave so and what is needed to achieve the result I am looking for (a kind of ‘untrim’ operator)?

Sorry if that question seems trivial or was already asked before.

-Stéphane L.

[pmario]

Hi,

Try this: <$transclude field=“foo” /> <$transclude field=“bar” />

See the red space! It should do the trick

-m

[joshuafontany]

Hi Misterel85,

This behaviour is found in TiddlyWiki — a non-linear personal web notebook

/*\
title: $:/core/modules/widgets/transclude.js
type: application/javascript
module-type: widget

Transclude widget \*/

...
var parser = this.wiki.parseTextReference( this.transcludeTitle, this.transcludeField, this.transcludeIndex, { parseAsInline: parseAsInline, subTiddler: this.transcludeSubTiddler })
...

And we can lookup the wiki.parseTextReference method here: TiddlyWiki — a non-linear personal web notebook

and see that it uses the `parseText’ method. This means we actually need to look at the $:/core/modules/parsers/wikiparser/wikiparser.js module, where we find:

/*
Push a text widget onto an array, respecting the configTrimWhiteSpace setting
*/
WikiParser.prototype.pushTextWidget = function(array,text) {
if(this.configTrimWhiteSpace) {
text = $tw.utils.trim(text);
}
if(text) {
array.push({type: "text", text: text});
}
};

Now we are getting somewhere. Searching the system tiddler for configTrimWhiteSpace gives us $:/core/modules/parsers/wikiparser/rules/whitespace.js

Where we see the 2 pragma defined as:

\whitespace trim
\whitespace notrim

AHA!

Now we know that if we put the “pragma” of \whitespace notrim at the very beginning of any Tiddler’s text field, then all parsers in that tiddler will not trim whitespace when parsing text.

Try \whitespace notrim before your other wikitext.

Best,
Joshua Fontany

[TW_Tones]

Misterel

Joshua does a great technical explanation here, and Mario gives the standard solution.

• The spacing you wish is something best done in the display of the data and not in the data, what if they changed order?
• A field containing a list of titles or tiddlers is typically space delimited. So we often need the leading and trailing spaces not to be included.
• Create a new tiddler and give it leading and ending spaces and you will see how these end up trimmed and do not become part of the title.
• You can access fields in a different way if you really need to have leading or training spaces as part of the data
• Transcluding not only brings the fields value, in but what follows is it is rendered, I believe this is when the spaces are trimmed.
  Try

‘<$transclude field=afield/>’ or ‘{{!!afield}}’
‘{{{ [all[current]get[afield]] }}}’

The transcluded filter output demonstrates the field contains a leading space of it has one.

Regards
Tones

[pmario]

I can only second that. If the data needs to be treated special, you will get a maintenance problem and it’s errorprone. Everytime you forget about the special handling. It ends up to be more work. …

-m

[Misterel85]

Thank you for your replies, PMario, Joshua and Tones.

@ Joshua: Thanks a lot for taking the time to write this comprehensive and clear explanation.

I have followed your solution to insert \whitespace notrim at the beginning of a tiddler text, but unfortunately, it doesn’t work.

I also tried to place it into a macro like this:

\define mymacro()
\whitespace notrim
<$transclude field="field1" /><$transclude field="field2" />
\end

but in vain. In the documentation, the default is set to ‘notrim’, so adding such an instruction in a macro is useless.

@ PMario:

I am aware of the solution you provide, but inserting a normal or non-breakable space between the two transclusions is not an option.

A very simplified example of what I would like to do is connecting two parts of something similar to a sentence together, but that sentence may not need a complement in the second part.
For instance:
if field1 contains: ‘The cat is eating’
field2 might contain: ’ a mouse.’ or ’ cat food.’ (note the space), or ‘.’ (no space required before a full stop / period).

Your solution would always insert a space, even before a punctuation sign such as a period or a comma…

@ Tones

Your advice is worth following ‘for standard use cases’. But what I wish to implement is probably not an ordinary one.

You wrote:

• The spacing you wish is something best done in the display of the data and not in the data, what if they changed order?

I don’t understand what you mean by ‘what if they changed order?’.

What would change order? The data? The fields? No, I wouldn’t need to change their order. See my example above. Why would I need the ‘complementary’ part of the data to show before the ‘main’ part?

As for the spacing, it is an integral part of the data here (as a word separator within a sencence, for instance). Therefore putting it anywhere else in this case wouldn’t be a wise choice.

• A field containing a list of titles or tiddlers is typically space delimited. So we often need the leading and trailing spaces not to be included.

Sure, but that’s not my use case. And ‘often’ is not ‘always’!

• You can access fields in a different way if you really need to have leading or training spaces as part of the data

Well, that’s exactly what I’m after!

‘{{{ [all[current]get[afield]] }}}’

The transcluded filter output demonstrates the field contains a leading space of it has one.

That filter output is interesting. But it doesn’t render HTML tags or Wikitext, contrary to $transclude.
If ‘afield’ contains: This is <strong>my</strong> //field//.,

then

‘{{{ [all[current]get[afield]] }}}’
‘<$text text={{{ [all[current]get[afield]] }}} />’
‘<$view field=afield />’
all render raw text as:

’ This is my //field//. ’
whereas

‘<$transclude field=afield/>’

renders as:
‘This is my field.’

Seems that I have to trade in the leading space for formatting and vice-versa. Can’t have my cake and eat it.

Regards,

Stéphane

[TW_Tones]

I think you can have your cake and eat it, as long as you use the tiddlywiki recipe/ingredients.

First lets see if I can explain about order.

If this was my first design (with space between them)
<$transclude field=“field1” /> <$transclude field=“field2” />

And I decided I wanted;
<$transclude field=“field2” /> <$transclude field=“field1” />

• I modify the presentation in the above code, I do not need to modify my data, just its presentation. This may be happening inside a list widget for example.
• In your case you may need to change both the presentation and the data.

Now, If I understand correctly you want to correctly treat the joining of “snipits” found in fields correctly, you would be better served building some standards for your data and tools to do the joining. Since there are already implied standards for handling those values lets adopt them, then if you can identify occasions where appending one snippet to the other by separation with a space, is not working, let us detect and alter that case.

In the original post I think you go down the path of altering the way they are presented by modifying the data.

Could you give us some examples where you are finding it hard to join such snippets?

I would suggest {{!!foo}}{{!!bar}} for your first example (As did Mario)

Tones

[Misterel85]

Thanks for your complementary explanation about order, Tones, and you’re quite right.

But as I wrote in a previous message, in that particular TW instance I am adamant that I won’t change the field order.

Concatenating / Joining snippets of text from two fields in immutable order, while retaining Wikitext or HTML formatting or links is exactly what I want to do.

The basic example I gave earlier is thus:

if field1 contains: ‘The cat is eating’
field2 might contain: ’ a mouse.’ or ’ cat food.’ (note the space), or ‘, while the dog is playing.’ or ‘.’ (no space required before a full stop / period or a comma).

Actually I need that to reference sources for tiddler contents. It is a bit more complex because field1 is in the source tiddler, providing general info, whereas field2 is in the content tiddler, with references specific to that content.

As an example:
field1 in ‘source’ tiddler might contain: ‘Doe, John. //My Great Book.// (2020)’
while field2 in ‘excerpt’ or ‘content’ tiddler might contain details such as: ‘, p. 22.’ or ’ — see chapter 8.’ or ‘. John Doe develops that idea in the first 3 chapters.’
So the leading character in field2 might either be a period, a comma, a space, depending on how I wish to gracefully join the two parts punctuation-wise.

It is pretty similar to connecting words and parts of speech in a sentence.

What I have achieved so far works fine, except for the space trimming in the transclusion of field 2.

I first thought there would have been a more straightforward solution to that issue, but actually I come to think that it should be possible to inspect the first character in the contents of field 2:

If value of field2 starts with space, then insert   between the transclusions, else just proceed with both transclusions.
But I don’t think I have enough TW knowledge to do that, unfortunately.

Best,

Stéphane

[EricShulman]

Try this:
<$transclude field=“field1”/><$text text={{{ [{!!field2}split[]first[]match[ ]] }}}/><$transclude field=“field2”/>

Notes:

• The {{{ […] }}} is a “filtered transclusion”
• The filter starts by getting the value of the desired field, using {!!field2}
• It then splits that value into separate characters, using split[]
• Next, it gets just the first character, using first[]
• and compares it with a space, using match[ ]
• The result of the filter is either nothing, or an actual space character (not an  )
• The <$text> widget then converts the output to plain text (instead of displaying a link to a space!)

enjoy,
-e

[Misterel85]

Yay, it works!!! Thank you very much for your quick reply and your simple and straightforward solution, Eric!
After a bit of trial and error, I could adapt it to my TiddlyWiki.

No need for an   anymore indeed.
Thanks again to all of you for your help, solutions, explanations and suggestions.

Best,
-Stéphane

[TW_Tones]

Misterel,

I believe I understand what you are trying todo. Yet I would persist in suggesting you separate the data from its presentation.

Of course its is fine to use the pragmatic approach as you wish.

What I would do is build the concatenation process to always include a space between elements, no leading or trailing spaces needed, except when it starts with period or any other exceptions you identify.

Basically I would get the logic to handle the language concatenation and leave such fiddling, outside of the data. Ideally I would even leave leading commas out of the data as this is again all about presentation eg if I have a list of three items the first is separated from the second by a comma the third by “and”. and the full stop comes at the very end. ie just store the items because you need not know how you will use them in the future and when using them format according to the presentation. The recent and next version 5.2.0 contains some support for this first, seconds and last treatment such as with the counter variable, even css can handle this.

Experience has shown me such an approach is actually much simpler in the short term and few exceptions are required, where the approach you are taking will ultimately be the opposite, more complex and have more exceptions, while also loosing flexibility.

But yes you are an autonomous being do what you want.

Tones

[joshuafontany]

We’ve found a bug!

The wikitext parser object always tries to read Pragmas from the beginning to parse a text, and this always trims whitespace from the text (moves the parser position past any leading whitespace).

The $:/core/modules/parsers/wikiparser/wikiparser.js module needs to updated.

The Parser’s this.skipWhiteSpace = method needs to check the this.configTrimWhiteSpace value before moving the parser position, or reset the parser position if no pragmas are read, etc.

This type of change should definitely have Jeremy’s eye on it.

I am terribly busy at work, so I will come back and file a Bug/Issue on GitHub if no-one else can.

Best,
Joshua Fontany

[Misterel85]

Tones,

Oh, now I better see what you mean by ‘separating the data from its presentation’.
It kind of reminds me of my attempts at concatenating genealogical data in LaTeX with AppleScript years ago. What you advise is something I more or less tried to do at the time.

I fully agree with what you wrote, but your precious advice stumble over a huge impediment: my poor skills at programming in general, and Wikitext in particular! I have absolutely no programming or STEM background, unfortunately, and I am only a tinkerer.

When I manage to copy-paste and adapt existing code snippets to meet my needs, I am very happy.
What a seasoned programmer would conceive and write in half an hour will take hours --if not days-- for me to think about and assemble, and I am overwhelmed with joy if it works, even if the resulting code would be scoffed at by students, let alone their teachers, after a few months in a 101 Programming class.

Being both a perfectionist and a dabbler, one should never been given such good advice as yours! They will keep what you suggest in mind and want to implement it without ever knowing how to get started! How terrible!
My project is certainly not well-thought-out, and if it was, I would probably need more code examples like the solution Eric Shulman kindly provided me with. And TW v. 5.2.0 being in prerelease, I haven’t looked into it yet.

I am aware that that TW project of mine is personal and is not designed to be published. Unless I get struck with Alzheimer’s disease soon --in which case I won’t need that project anymore–, for the moment I should be able to remember its quirks when I use it. And there’s room for improvement in several other priority aspects of it. I am already spending way too much time trying to achieve a project that probably wouldn’t need all those bells and whistles to work properly, to the detriment of other --as / more-- meaningful projects.

Best,

Stéphane

[Misterel85]

Joshua,

Thanks for letting me know! Glad to have contributed to the bug chase through that seemingly inconspicuous question!

I don’t know much about GitHub, therefore I let you file that bug.

Best,

Stéphane

[TW_Tones]

Misterel,

Perhaps my writing is not so helpful, but my advice is intended to make it simpler for you. I would be happy to write the code to resolve the concatenation of various text snippets into sentence like text. All I would need is samples of the text you whish to join (without presentation related hacks eg leading spaces etc…

The first examples would be foo=“This is foo” bar=“and this is bar.” please join as a sentence.

Feel free to be pragmatic and decline my offer but also feel free to ask.

Here are some examples to look at, paste into a tiddler on tiddlywiki.com, add the foo and bar fields as before.
See how alice and bob values are independent of the order in which they will be presented, used?

\define full() {{!!foo}} {{!!bar}}
\define full2() $(foo)$ $(bar)$
\define full3() $(alice)$ and $(bob)$.

\define full() {{!!foo}} {{!!bar}}
\define full2() $(foo)$ $(bar)$
\define full3() $(alice)$ and $(bob)$.

<$vars
foo=“This is foo”
bar=“and this is bar.”
alice=“this is Alice”
bob=“this is Bob”
cat=“this is cat”

<<foo>> <<bar>> <> <>

{{!!foo}} {{!!bar}} {{!!foo}} {{!!bar}}

{{{ [<foo>] [<bar>] +[join[ ]] }}} {{{ [] [] +[join[ ]] }}}

<$text text={{{ [<foo>] [<bar>] +[join[ ]] }}}/> <$text text={{{ [] [] +[join[ ]] }}}/>

<$text text={{{ [{!!foo}] [{!!bar}] +[join[ ]] }}}/> <$text text={{{ [{!!foo}] [{!!bar}] +[join[ ]] }}}/>

#<<full>> <>
#<<full2>> <>
#<<full3>> <> | {{{ [sentencecase[]] }}}
#{{{ [<alice>] [<bob>] +[join[ and ]addsuffix[.]sentencecase[]] }}} {{{ [] [] +[join[ and ]addsuffix[.]sentencecase[]] }}}
#{{{ [<bob>] [<alice>] +[join[ and ]addsuffix[.]sentencecase[]] }}} {{{ [] [] +[join[ and ]addsuffix[.]sentencecase[]] }}}

</$vars>

Regards
Tones

[Misterel85]

Thank you for your very interesting and useful examples, Tones. I didn’t know about sentencecase! I’ll keep that post as a reference.
And thank you very much for proposing your help.

Sorry for not replying sooner. I’ve spent the last two days thinking those field connexion rules over for my project and I can say they are not that obvious to define. Anyway, here is something that would correspond to my needs, but it needs ‘conditional transclusion’. So I gladly accept your offer.

So, in order to manage my sources, I have a ‘reference’ field for general information and a ‘details’ field for any additional information, that I must transclude in that order: reference + details.

Here’s how I see things:

• the value in ‘details’ may start with p. (for page) or pp. (for pages).
  Then I would connect ‘reference’ and ‘details’ with , (comma + whitespace).
  For instance:
  Doe, John. //My Great Exercise Book// (2019), pp. 28-32.

• the value in ‘details’ may start with a word in lowercase: exercise 2, chapter 4, etc.
  Then I would connect ‘reference’ and ‘details’ with — (whitespace + dash + whitespace).
  For instance:
  Doe, John. //My Great Exercise Book// (2019) — chapter 6 exercise 5 p. 61.

• the value in details may start with the first word having a capital letter (uppercase), to start a short sentence.
  Then I would connect ‘reference’ and ‘details’ with . (period + whitespace).
  For instance:
  August 2018 training session with John Doe. See short video for this exercise on YouTube.

• if the details field is empty, then I would need to end the transclusion of reference with . (period).

I’m not too sure what I should do if the value in details field starts with a character that is not a letter, which may happen in very rare cases. And I can still alter the value accordingly:
For instance:
[[Comprehensive Exercise Website|www.to.the.link.com]] — “Exercises” tab, bottom of page.
might be replaced with:
[[Comprehensive Exercise Website|www.to.the.link.com]]. See “Exercises” tab, bottom of page.

I hope the goals I have set are clearly explained and above all doable.

I suppose that making liberal use of list filters and operators such as ‘split’, ‘join’, ‘append’, ‘match’, it could be possible to build those ‘conditional field transclusions’, but I don’t think I could do that alone!

Regards,

-Stéphane

[joshuafontany]

Stéphane,

Some-one on GitHub in the bug report (link) mentioned that you can “view” the raw text of a field with the View Widget.

So, where as {{foo!!bar}} is a shortcut for

<$tiddler tiddler=“foo”><$translcude tiddler=<> field=“bar”/></$tiddler>

You can create a macro that uses the View Widget like so:

\define viewField(tiddler,field)
<$tiddler tiddler=$tiddler$><$view field=$field format="text"/></$tiddler>
\end

<<viewField "tiddler a" "foo">> <<viewField "tiddler a" "bar">>

You could even use the View Widget directly if the defaults would do fine:

<$tiddler tiddler="tiddler a">
<$view field="foo"/><$view field="bar"/>
</$tiddler>

If you had wikitext in the field that needed to be proccessed, you could use a different “format” value: TiddlyWiki — a non-linear personal web notebook

Best,
Joshua Fontany

[TW_Tones]

Stephanie

It took a little time to wade through the combinations. I also found there are possibly many ways to do this.

The attached json has a view template that displays the detail or references if they exist according to your specs, the view template itself is for all other setting and the testn tiddlers demonstrate each of the formats.

Here is the content of the viewtemplate

\define reference-only() {{!!reference}}.
\define page-reference() {{!!reference}}, {{!!details}}
\define not-page-reference()
<$list filter="[lowercase[]match]" variable=nul>
{{!!reference}} — {{!!details}}
</$list>
<$list filter="[lowercase[]!match]" variable=nul>
{{!!reference}}. {{!!details}}
</$list>
\end

<$list filter="[all[current]has[details]] [all[current]has[reference]] +[limit[1]]" variable=nul>
<$wikify name=details-start text="""{{{ [all[current]get[details]split[ ]first[]] }}}""">
<$list filter="[all[current]get[details]]" variable=nul emptyMessage=<> >
<$list filter="[match[p.]] [match[pp.]]" emptyMessage=<> variable=nul>
<>
</$list>
</$list>
</$wikify>
</$list>

Basically I define some alternative layouts with define, some with logic. These transclude the reference and details fields so any wiki text therein is rendered.

Then from

• first we test if we have anything todo - reference and/or details has content
• Then we get and wikify a filtered transclusion to get the first word
• Then if not details exist we use the reference-only layout
• else if we do have details we test if it contains a page-reference and use that layout if so
• else (emptyMessage) we use the not-page-reference layout
  • Every other case uses this.
    To handle the first character upper or lower case I am simply testing to see if the first word matches all lower or not, it does the same job but could be tricked. eg “tHis” would trip the capital test. This can be refined, however I expect you would then edit it to correct it anyway.

So the main mechanism I use is the list widget with filtered tests and emptyMessage. As I asserted best put the formatting separate from the content/data.

Regards
Tones

(Attachment details-references.json is missing)

[Misterel85]

Hi Tones,

Thank you very much for your solution. Being away from my computer for the last few days, I could try it only this afternoon. I managed to twist it to my needs and it seems to work fine, which is great!

I understand the way you implemented your solution, but I have three specific questions about the code you used, in order to grasp the details that still elude me.

1. You used variable=nul 5 times in your code, for at least two (seemingly) different list filters. Why did you use the same variable name? And why did you choose that name? How useful is it to declare a variable in a list filter? Could we make do without it?

2. You also chose to use $wikify at some point. Why did you choose that?
   In my adaptation of your code, I had to replace:

<$wikify name=details-start text="""{{{ [all[current]get[details]split[ ]first[]] }}}""">

with:

<$set name=details-start value={{{ [all[current]get[details]split[ ]first[]] }}}>

because it wouldn’t work properly otherwise. (Some closing tags such as </$list> would show as text.)

3. How does +[limit[1] work? What does the plus sign mean here? Wouldn’t it be okay to just use [all[current]has[fieldname]]?

Thanks again, Tones.

-Stéphane

[pmario]

@Misterel85

You used variable=nul 5 times in your code, for at least two (seemingly) different list filters. Why did you use the same variable name?

I’m not Tony, but anyway. … Sometimes it’s useful create a variable like this, to indicate for other programmers that the variable isn’t used in the body of the list widget. Instead the <<currentTiddler>> variable is used inside that list widget.

The value “nul” “null” or “ignore” means the same thing. … Not needed or currentTiddler should be used.

By default the list widget will always assign it’s filter results to the <<currentTiddler>> variable. But sometimes that’s not what you need. That’s why the list widget has a variable parameter, that overwrites the default name.

See: TiddlyWiki — a non-linear personal web notebook