I have a JSON file that has 2135 records, each beginning
{"created"
This equates to 1068 actual data objects (objects referring to an actual artwork) and the same number of _canonical_uri objects referring to an external image of the art work.
The problem I have encountered is that after import there appear to be only 835 artwork records
Have I come across a limitation in JSON importing?
bobj
After much more testing, I lean towards an issue with my data. Of all the tests I ran, the records tagged with âLinksâ all get imported OK but those tagged âArtworksâ see to lose some or maybe they are misplaced.
More testingâŚ
bobj
The key requirement for the JSON is that the representation of each tiddler should have a title, and it should be unique.
From your screenshot from your first 6 records 3 have the same title: âPainting: Untitledâ - So they will overwrite each other and only the last one will be imported.
As Saq wrote. Titles have to be unique.
Thank you @pmario. After saqâs reply, I have been wondering about how to get unique titles as there are many times an artwork may have a title of untitled. My thoughts are to prefix each title string with the asset number, ie. YB123. These strings are guaranteed to be unique even if they make the title cumbersome.
Good to know that an import of a json record will overwrite an existing title though. I have been wondering about how to update the data from the FileMaker Pro database which generates the json via a script. Overwriting is perfect in his case.
Bobj
You can make thy YB numbers a suffix eg: Painting: Untitled (YB...). So it does not make the title too cumbersome.
Since you do use a database output, it should be possible to make the titles unique in the export logic.
Just a quick note: I too have lots of experience generating JSON for TiddlyWiki from Filemaker (my original bibliographic database with thousands of records and quite elaborate data structure). So Iâm happy to problem-solve together about the process.
Thanks everyone for their thoughts and offers of assistance with Filemaker. I have been a Filemaker developer for many years and so am easily able to generate unique titles and generate JSON. For your info, the database currently has a script that generates valid HTML files which I can simply upload to my web server to provide access to the artworks. I have created a companion JSON script.
I was curious to see how TW might do the job since my work with the Central Street Archive, which is also, essentially, a list of objects being accessed and displayed.
This exercise has highlighted how I can not just mimic the Filemaker/Central Street Archive structures as TW requires the title, usually a text string, to be unique and as we have seen it cannot be guaranteed. Prefixing by artwork type, painting, print, etc, makes more records unique but again it can not be guaranteed.
The Filemaker database has a unique key, a running number, generated by Filemaker and I use that, prefixed by âYBâ as the asset number of any artwork/record. Each filemaker artwork record includes an image of the work alongside other attributes.
My current TW structure does not accommodate that as I generate a YBxxx record for the _canonical_uri record to point to the external image and then have a field on the actual data tiddler that references the YBxxx number so it can transclude the image when the data tiddler is displayed. This is how the Central Street Archive works.
I am now going to revisit my TW structure and explore merging the two tiddlers for each artwork and specify each artwork tiddler of type Image/jpg with a _canonical_uri field and use the asset number to, hopefully, show the image at tiddler display time. I am trying to avoid having the images inside the wiki as there will be over 1000 of them at present with the number growing.
I understand that by using _canonical_uri to reference image files, these can not be resized on display. Is that still correct for the current TW version?
If so, then I am going to work out how to have a field in the artwork tiddler that allows the viewtempate to display the image at different sizes in different contexts (ie. thumbnail, full size, etc)
So, much more work to be done and I am sure many more requests for assistance.
bobj
I have a wiki that hosts a few thousand image references. I use a number as a title and a subtitle field if I need a textual title.
If you control the JSON, perhaps you could map the stored title to a tiddler subtitle field and generate a sequence number for the actual titles.
Aside: I also use a uuid field which becomes the single source of truth as regards an âidâ. Then it doesnât matter if I change the title or even the subtitle, the stored uuid uniquely identifies the actual image source.
Just something for you to think over.
@CodaCoder that is sort of what the filemalker database does. Anyway, I am not replacing the filemaker database, just providing a web visibe version of selected artworks. So the id number and hence the asset number/id is controlled by filemaker, the title can be anything. I have been playing with each artwork having an âartwork_titleâ field and mapping the database title field to that in generating the JSON.
Does yout wiki allow for resizing of images in different contexts? If so, how did you do that? Or do you store different sized versions of images and link to one or other as needed?
bobj
I just supply a CSS width value. In the host tiddler, it tends to occupy the space available (width of the tiddler). When itâs consumed elsewhere (transcluded into other tiddlers) I just change the width value via a macro.
\procedure bk-image(tiddler-title, width:"250px", style:"")
<<bk-image "CH1-045" "200px" "float:left; margin-right:15px;border:3px solid #fff;">>
Just t tidy up my issue, I have successfully converted my wiki to merge the previous two tiddlers into a single tiddler. I have used the asset number as the title of a tiddler, so that guarantees uniqueness. I have also added two fields, artwork_title and sorting_title. The former holds the actual title (eg: A Snow Gull), whilst the sorting_title holds just the type plus title (Painting: A Snow Gull).
I have also amended my list-link macro calls to sort on the field sorting title but list the artwork_title field.
So, its beginning to look how I envisaged.
Now to tackle two remaining requirements
bobj
@CodaCoder are your images accessed via _canonical_uri mechanism or an [image...] specification (eg. [img[YB100]])?
bobj
Neither.
Referring back to my previous (where the target image tiddler is titled CH1-045), hereâs the algorithm:
In the following, understand âCHâ stands for âChapterâ. So a numbered image, 045 in chapter 1 gives
CH1-045.
path field, set url to it and exit this algorithm.OtherwiseâŚ
Get field img-dir from CH1-000 â some/path/
-000 sets up data common to all images throughout the chapter. One of those is the path to use for all images unless a path is set on the image tiddler (see step 1).
Split target at the hyphen to get the image number CH1-045 â 045
Get extension field from CH1-045: (default to .png if blank) â .jpg perhaps
Combine to form url â some/path/045.jpg â chapters/ch1/045.jpg
Then, basically, I use <img src=<<url>> blah blah />
Sorry, I neglected to mention how width is handled, which is part of your requirement. There are two related parts: width itself and a filter â CSS filter properties.
In the display macro, setup width variable to use <<__width__>>, if supplied, otherwise get the width field from the image tiddler. If blank, use 100%.
Set the ifilter variable from the filter field on the image tiddler. If blank, use none.
Build the style attribute using macro img-style
\define img-style()
width:$(width)$;filter:$(ifilter)$
\end img-style
Then we have:
<img src=<<url>> style=<<img-style>> />
For my setup, I had planned to do the same. I noticed that a list of 1000 (-000 > -999) was terribly slow1. In fact, I stopped at around 800 and split them up, now limiting my âchaptersâ to 200 images per.
[1] And that was using $eventcatcher, too.
@CodaCoder thak you for your descriptions, they have given me more to think about. FYI, I have re-built my wiki using <$image widget, specifying a source rather than using _canonical_uri. My thoughts are to use a special data tidler, $:/TSL/width, to control the width of images as they are being displayed by passing the contents of that tiddler in to the image widget width parameter. So, as my context changes, I can set that tiddler to the required width and have the images displayed at the correct width (assuming I can work out how to do all that of course). For source tiddler display, I use the ViewTemplate to set the image display to 100%
<$image source={{!!object_link}} width="100%" />
I am happy to continue exchanging thoughts here unless the group would prefer we took it off line.
bobj
Just to complete my design, I have managed to display a list of thumbnail images and their corresponding artwork titles with a link on the artwork title to display the source data tiddler with the image at full size.
<table>
<$list filter="[all[tiddlers]tag[Artworks]object_type[Artist Book]]+[sort[sorting_title]]" >
<tr>
<td><$image source={{!!object_link}} width={{$:/TLS/image_width}} /></td>
<td><$link to={{!!title}}>{{!!artwork_title}}</$link></td>
</tr>
This has come about because I converted to use the $image widget instead of _canonical_url and of a very early post I did regarding displaying tables that @TW_Tones responded to showing how to populate each table row and the $link widget to include the clickable link back to the source data tiddler.
Testing shows that there is latency in loading large lists even coming from my local drive (as forwarned by @CodaCoder above)
Overall, this is magic!!
Donât understand why I donât need to terminate the table nor list tags though. If I do, then the text of those two tags is displayed at the base of every image cell.
bobj
I viewed your wiki and added </$list></table> to the end of the code, and everything looks ok. It doesnât display those tags as the end of every image cell.
When internal syntax does unexpectedly âfall outâ and become visible, it generally indicates some kind of tag mismatch or out-of-sequence error occuring before those tags. Perhaps you previously had an error that you ultimately fixed, so that the âfall outâ issue no longer occurs.
In any event, the reason you donât need the </$list></table> is because they are the last syntax occuring in the tiddler, and any unterminated widgets are automatically closed when the end of the tiddler is reached. Given that the problem doesnât seem to appear any longer, you should add the </$list></table> to your code for properly terminated syntax.
Also, I note that the above syntax occurs in several tiddlers, so thereâs quite a bit of repeated code where the only difference is the specific filter you are applying to generate each particular table. To eliminate this repetition (and make any future improvements to your table layout MUCH easier) you could define a global procedure to render your table output.
Try this:
Create a separate tiddler (e.g., ArtworkTable), tagged with $:/tags/Global, containing:
\procedure artwork_table(thisFilter:"[{!!title}]")
<table>
<$list filter="[tag[Artworks]] :filter[subfilter<thisFilter>] +[sort[sorting_title]]">
<tr>
<td><$image source={{!!object_link}} width={{$:/TLS/thumbnail_width}} loading="lazy" /></td>
<td><$link to={{!!title}}>{{!!artwork_title}}</$link></td>
</tr>
</$list>
</table>
\end
Then, in each tiddler where the table occurs, you can write things like this:
<<artwork_table>>
or
<<artwork_table "[object_type[Artist Book]]">>
or
<<artwork_table "[get[year_created]compare:date:lteq[2029]compare:date:gteq[2020]]">>
Note how the first example above doesnât have any âfilterâ parameter. This is because the global macro automatically begins by selecting only those tiddlers tagged with âArtworksâ and also uses [{!!title}] as a default subfilter, which acts as a âpass-thruâ, accepting all input titles if no subfilter is specified.
enjoy,
-e