Importing a large json file containing hundreds of new tiddlers

Beginning User. I’m trying to import a large JSON file containing hundreds of new tiddlers but i don’t want to overwrite the tiddlers that all ready exist. I could go through and uncheck those that exist but was hoping there was a way to mass uncheck the existing ones. Also there is a preview option on the import screen but I could not find anything documenting its function.

Welcome @tjstokes

There is a lot we can do to help here, but depending on the complexity of your input data etc… there may be some quicker ways.

  • Can you return to the JSON source and be more selective there, before importing?
  • Can you import to an intermediate wiki?
  • Backup, Export or copy your wiki for safety

I believe a more complete solution maybe to build your own import mechanism, not as complex as you think.

  • Use this button to import the JSON, not as seperate tiddlers but as a single JSON tiddler <$browse deserializer="text/plain">
  • Then using the new JSON operators write your own method for creating only the missing tiddlers as real tiddlers.

I do think there may be some other even simpler approaches, I will look again later today.

Question

  • Are the tiddlers you are importing regular tiddlers and not replacing any existing shadow tiddlers?
    • If so I may have a solution already.

Thanks for the response. I’m a believer in the simpler approach.

These are just regular tiddlers. I’ve done some editing to the one’s I don’t want to replaced. A simple example would be downloading a company directory on a regular basis that creates a tiddler for each person. After the first load there would be editing and adding information to the person tiddler. I just want to add the ones that don’t exist.

A work around would be to export the existing tiddlers, import the new JSON file and then re-import the edited ones I exported.

2nd: I can’t figure out what the “preview option” on the import screen is for. It would be nice to know but I could not find anything documenting its function.

Click on the arrow on the left of the to-be-imported tiddler title: you get a preview of its content, and the ‘‘Preview’’ list lets you choose what this preview looks like. For example, if you choose ‘‘Diff (Fields)’’, you’ll see the differences between the existing and imported fields of the tiddler.

Fred

That’s right.


I was thinking about the possibility to create a configuration, that would allow users to set the “default selection” for existing tiddlers. …

It should be a relatively straight forward plugin. …

1 Like

6 posts were split to a new topic: How to export / import $:/state tiddlers – Folded states in particular

I knew there was a simple way

As soon as you told me

  • This became obvious.

I have not tested this but you may want to try this;

  • Download a copy of tiddlywikis empty.html
  • Drop your Import JSON on this new wiki ''Don’t Import"
  • Edit the Import tiddler $:/Import and
    • give it a new title eg “Import Plugin Feb 2023”
    • Edit the field plugin-type and change it to plugin
    • Save the tiddler

You now have a plugin containing your import tiddlers, as shadow tiddlers. Make sure your main wiki has a backup, then.

  • Drop your “Import Plugin Feb 2023” tiddler on your main wiki

All done

  • The new tiddlers are now available as shadow tiddlers
    • but any existing ones will show as overridden shadows, or tiddlers ie they remain the current version.
    • You may need to change any lists or filters to use [all[shadows+tiddlers]… to include the shadow ones.
    • You can edit a tiddler and use the preview to see “difference from shadows” at least for the text field.
  • Next time follow the same process to update all the shadows but not the “edited” tiddlers.

This is a little messy because you may already have many existing tiddlers you have not edited, but are already tiddlers (as opposed to shadow tiddlers). Had you followed the above approach from the beginning you would not have these.

  • We could possibly come up with a batch process to clean up your current wiki. ie delete all tiddlers for whom you have an “identical” shadow tiddler, ie no differences. Possibly ignoring the created and modified date in this comparison.
1 Like

Here is an even simpler solution.
Assuming you don’t want to replace any of your existing tiddlers with the same title:

  1. Export all your current tiddlers
  2. Load the new tiddlers, replacing whatever exists. You will get them back with the next step.
  3. Import the JSON file created in step 1. replacing everything.

Always back up your wiki. Good luck!

1 Like

@tw-FRed Thank you - old eyes missed the arrow

Thanks everyone for the quick response. I went with the simple solution this time but will be trying to remember and use some of the others. Thanks