Corrupt TW with JSON and .tid files export and import problems

#1

@Birthe @Mark_S
As my problem seems to have hijacked the post at
Help required with Checkboxes
I have created this new post to continue discussion on my problems with tiddlers including javascript.

Exporting tiddlers as JSON and .tid files causes an error when either are imported to my existing single TWs
@Mark_S highlighted that
“The files seem to be packed with javascript related to geolocation. Do you have anything related to geo in your plugins?”
Suggested solution was to disable all plugins, save TW, reload and try to export problematic tiddlers for another try at file importing.
Sadly, this did not fix the problem.

@Birthe
Kindly suggested removing overridden shadow tiddlers by finding them in advanced search using existing filter option.
Again this did not fix the problem.

I have downloaded a clean/ new v5.2.0 TW and populated it with some new toddlers created afresh, not imported or dragged from another wiki.
These tiddlers can be exported and subsequently imported to fresh TWs as either JSON or .tid files.
This is normal practice and works as intended.

If I cut and paste code into a new tiddler, say from the above post, the resultant tiddler works fine but again has export/ import problems.
I am now at a loss as to how I can rectify this problem.
Can anyone offer a solution / fix.
Kind Regards
Scot

Problems with browser extension disturbing use of tiddlywiki
#2

I tried one of the tid files I downloaded the other day. Opened it in a text editor and removed the strange parts. Saved and tried to import the edited version. That seems to work okay - only I need all the other tiddlers to see the subsumes…and all that.

If you have many tiddlers I guess that will be quite time consuming. Easy enough, you will find the part you need in the bottom of the file.

#3

@Birthe
Thank you again for your suggestion.
I do think that the number of tiddlers would make this impractical, however so is starting afresh.
I am puzzled as to why my clean TW still had a problem with a new tiddler with code copied from a post
I will have to try and replicate this, in case it was an error I introduced.
Your help on this matter is much appreciated. I would be interested to hear if anyone else has come across this problem.

Thanks also to @Mark_S for his time and suggestions.

Best Regards,
Scot

#4

The strange text writes about something being injected in the boot sector of your wiki. I even saw some text about being careful ending the script right.
I think that is the code error we saw earlier on.

That is the geo, fake-geo part.

#5

@Birthe
I think the problem is of my own making. @Mark_S asked about the Geo code and I could only think of a Tiddler I was using for Google map info a while back.
For a while now, I have been using Dropbox in conjunction with TW5 in the cloud, to access my TW’s. In addition, fairly recently I added an app to my phone and tablet that also syncs with Dropbox keeping all 3 locations synced. I am pretty certain now, that this could be the cause of my problem as I sometimes add new TW’s from phone or Tablet rather than straight to Dropbox, which I only upload on my PC.
I will test a clean TW in Dropbox and TW5 in the cloud, which I think will be fine.
I will also try a file added or synced with the app and see what that produces.
I will report back, soon.
Scot

#6

Is the TW private, or just recipes (which is what you were working on) ?

I’m thinking that it might be possible to use a text editor and edit out the parts that are causing the problem.

#7

Hi Mark_S
just recipe stuff.

that’s exactly what @Birthe was suggesting.
I added a TW5.2.0 to my phone and created a single file testphone1
Exported to phone and then uploaded to tiddlywiki.com
Note:
File downloads as JSON
Tid File downloads as .html
Upload to tiddlywiki.com (from phone)
JSON file loads and opens fine - no error

.html (was .tid) imports as a html file (filename testphone1.html)
file has created, modified tags Text content

Snyced file and open from TW5 in the cloud (in the sky)
imported JSON file has error

{
    "tiddlers": {
        "JSON error: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data": {
            "title": "JSON error: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data",
            "text": ""
        }
    }
}

.tid file has file with geo code

testphone1.tid

function waitGetCurrentPosition() { if ((typeof hookedObj.fakeGeo !== 'undefined')) { if (hookedObj.fakeGeo === true) { hookedObj.tmp_successCallback({ coords: ....

It seems that the syncing process adds the geo code to the file in the wiki.
I will stop using phone and tablet apps.

Still to test TW5 in the sky to PC and vice versa.

New TW5.2.0
create file test-pc1
export JSON and .tid file
Upload to tiddlywiki.com

test-pc1.tid

function waitGetCurrentPosition() { if ((typeof hookedObj.fakeGeo !=

test-pc1.JSON

JSON error: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

This pure TW changed in Firefox 92.0.1

test-pc2
Uploaded to TW5 in the cloud
created file test-pc2
Exported file as JSON and .tid
Imported to tiddlywiki.com
JSON file has error as above
.tid file has Geo code as above

#8

No, she was suggesting that you edit each export. I’m suggesting that it might be possible to edit the source TW file and fix it. Are you willing to share the file?

I don’t know what you mean by upload to tiddlywiki.com. Unless your name is Jeremy, you can’t upload to tiddlywiki.com

TW5 in the cloud. Have you been using this all along? I’m wondering if it could insert the code. Do you have an iphone or a smart phone, and what is your standard save mechanism?

#9

I meant imported to tiddlywiki.com sorry.
I’m using a pc, smartphone and tablet (both android)
I edited my previous post to show what I’ve been doing in order to show the errors

Mark I can share the file if you like.

#10

I used dropbox for a long time and edited my TW’s with TW5 in the cloud. No problems ever. But I do not own any smart phone or tablet.

#11

What app? What synch mechanism?

#12

I explained to Birthe that I use an app (Dropsync) on my phone and tablet to sync with dropbox and TW5 in the sky - which is what I thought may have been the problem (i was using these, but didnt think of the GEO code possibly being that, I thought it was with the TW I was using with google map data a while back).

I am going to simplify my data by just keeping master TW on my PC and on Dropbox +TW5 in the cloud

#13

@Scot

I still think so, remembering a long discussion in GG a couple of years ago about a script for getting those data.

#14

Birthe
Are you still using TW5 in the cloud and Dropbox without any problems?
If so then I can dismiss it as a source of my corrupt files.
Scot

#15

Scot,
I use TW5 in the cloud and Dropbox regularly. Haven’t had a problem yet.

#16

Birthe,
maybe I should re-install TW5 in the cloud and try again.
Scot

#17

@Mark_S @ Birthe
I copied some of the Geo code into Google and was directed to stackoverflow.com page.

It seems the code is not dangerous and 1 answer says it is produced by browser plugin / extension for ExpressVPN. I have Express VPN installed on my devices, so this could be the problem.

PS
ExpressVPN code on Github.com

Regards,
Scot

1 Like
#18

@Scot
Thanks for telling us! At least you know now!

Well thought @Mark_S

#19

I will delete the ExpressVPN plugin temporarily to see if it resolves the problem, as suggested. Not sure whether it would have to be deleted permanently to use TW though or a ExpressVPN free browser may work.
Thanks @Birthe and @Mark_S for your help and patience.
Scot

#20

@Birthe @Mark_S

Deleted ExpressVPN plugin from Browser.
Refreshed TW and created new tiddler and exported as JSON and .tid
All seems fine.
Scot

try this tiddler.json (139 Bytes)
try this tiddler.tid (115 Bytes)

1 Like