Help needed about HTML2TW: Macro

#6

I’ve put an updated version of the HTML2TW macro here. This fixes the image link by making the image and link wikitext separately. Keep in mind that in some cases (most actually) that the link around the image and the image itself will be two different things. So we don’t want to throw away the link. In many cases, the link will be more important than the image, which might just be a stock image.

I wanted to test from a copy of your MW43 site. But in edit mode, why, for the love of all that is alphanumeric, does Shift-L invoke a popup? This means I can’t type important words like “Liberty”, “Litany”, “Liturgical”, and most importantly, “Link”.

When running the web-clipper, I got an error “This page can not find removelfmarco”. I searched for “removelfmarco” on MW43, but couldn’t find it. The error didn’t block clipping, so I was able to proceed. Maybe you know how to fix it?
.

Do you have a link to the article?

Thanks!

1 Like
#7

Thank you @Mark_S for fixing it. I am currently not on my desktop. So unable to test it.

I disabled that keyboard shortcut. I hope it won’t get triggered when you press shift+L nowonwards. That was modal search experiments done by me recently.

I haven’t seen this error when I clipped from the demo wiki the last time. Since it’s based on multicolumn layout, I think only one ensemble of this wiki has tiddlyclip enabled. You have to click on the Tiddlyclip link in the topbar to go to this ensemble created for the purpose of Tiddlyclip demo. I will confirm once I am back on my desktop. Not sure why that error occured.

This is the link of that article. Its one of our main Journal. So many of the references will be taken from this site. If this site is covered, it will be a huge relief.

#8

I’ve included a fix for this. It turns out they had their own source attribute which was confusing the regular expression.

#9

Both works fine. Thank you once again. Where in the macro code is the second fix added. I just wanted to learn how to do it in case I find similar issues in other websites.

Any solution for this code blocks issue.

Do you use this macro these days. I found it to be more reliable than a markdown addon i tried recently after these fixes.

#10

Do you have a link to a sample? This is a little off from HTML2TW conversion (since now it’s more like JSON2TW )

The original code looked like this.

   /* IMAGES */
    pattern = new RegExp ("<img.*?src=\"(.*?)\".*?>" ,"gi") ;

The problem was that the image tag had it’s own attribute containing “src” (data-lg-src). Which is the one the regular expression was finding:

<img class="figure__image" src="https:...gif" data-lg-src="/cms/10.11..." title="" >

The fix was to require a white space (\s) in front of the “src” attribute. Perhaps a word boundary would work as well or better. @TiddlyTweeter might be able to comment:

   /* IMAGES */
    pattern = new RegExp ("<img.*?\\ssrc=\"(.*?)\".*?>" ,"gi") ;
1 Like
#11

I have added 8 example tiddlers arranged in 3 columns - each tiddler has 2 parts - first part uses clipping by HTML2TW macro and second part uses html clipping to see the html mark up. All tiddlers are folded for ease to find them. Click on the tiddlyclip link in the topbar to see all relevant tiddlers together. Hope you can identify the tiddlers using the permalink. If you want to see only one tiddler at a time, use the fullscreen button in the viewtoolbar of each tiddler

  1. Column 1 - first tiddler - Missing images

    I tried to clip the OP of this post in Talk Tiddlywiki forum which had multiple images…all images after the first image were missing in the clipped tiddler.

  2. Column 1 - second tiddler - Missing text

    This was clipped from github. Some parts of the text were missing (I saw the disclaimer regarding missing text in the macro tiddler. Still if something can be done it would be nice).

  3. Column 2 - first tiddler - Another example of missing text while clipping from talk tiddlywiki

  4. Column 2 - second tiddler - Example of image with anchors showing bug with link display
    Link is not shown seperately from the image inspite of the new updated macro being used (I found it accidently. New macro was working well for this feature in all other cases).

  5. Column 3 - first tiddler - Example for quotes and single backtick code blocks in Github

  6. Column 3 - second tiddler - Example of triple backtick codeblock in Github

  7. Column 3 - third tiddler - Example of single backtick codeblock in talk tiddlywiki

    Also numered list is seen as bullet list on clipping.

  8. Column 3 - fourth tiddler - Example of triple backtick codeblock in talk tiddlywiki

    Code is not seen at all including the html clip.

Just wanted to point out these issues which I found in addition to the code and quote block feature inclusion. If not fixable, there are workarounds I guess like selective html clipping or clip selectively using html2tw macro instead of clipping an entire post in single session.

#12

Ok. See if current version fixes this.

Edit: How do I shrink/expand columns?

1 Like
#13

wow. It works now. I have updated that tiddler.

Ctrl+Alt+B to reduce the number of columns.
Ctrl+Alt+N to increase the number of columns.

#14

I am keen to incorporate a HTML2TW into a dropbar, if html is dropped on it, it appends the html as TW Markup to the text field.

  • What level of maturity is HTML2TW at?
#15

I think @Mark_S is the better person to answer it.

From this post, it seems like this macro was created way back in 2016. So many must have used it - just an assumption.
I started using it recently only. It’s reasonably reliable except for a few bugs I have reported in this thread. Also it depends on the type of websites we need to clip contents from. I will definitely use it.

In the current state, it’s best used along with tiddlyclip. Tiddlyclip and this macro is a deadly combo if some of these bugs are fixed. In two or three click, you can clip text, images to your wiki with preserved wikitext formatting.

I wonder why people hardly talk about these two tools here in the forum - dont know whether it’s because less people use it or is it because nobody has any doubts regarding their usage. Also many people might not be aware of these tools.

1 Like
#16

I’ve made some tweaks that should fix (sort of) problems 1 thru 4.

It’s six years old. How many mature six year olds do you know?

There’s been almost no interest in the last 6 years. I think part of the problem is that’s it’s kind of hard to figure out the configuration for bj’s web clipper – the documentation is old, scattered in various places, and doesn’t include the latest developments. It’s not an out-of-the-box experience like with Evernote or some other app.

I know for myself, I generally just use zotero or something to save articles. Lazy.

That said, it’s surprising that there isn’t more interest.

I can only assume that most people don’t need web-clipping, or they are content with HTML clipping, regardless of the management implications.

The HTML2TW macro is all based on regular expressions. This is very flexible and easy to understand. But it can go off the rails with poorly formatted text. Probably a more robust method would be to put the text in an HTML object and then parse the tree using standard JS tools. I sort of thought someone would have done that by now.

1 Like
#17

Bingo! The man (buggyj · GitHub) is brilliant! Likely fedup with supporting something with no revenue stream over years??

TT

#18

That’s true. I have asked too many doubts to him regarding tiddlyclip and his other plug-ins in github and he have replied to all of them with answers without any delay. Just see this discussion page of Tiddlyclip plug-in and see the number of topics he have answered may be within the last one year or less. I hope to set up a demo wiki for tiddlyclip for demostarting various clipping methods I have learned from him with examples.

Sadly that also must be true.

1 Like
#19

@Mark_S Thank you. I have updated the demo wiki. First 4 image related issues are fixed now except for the one thing which I had forgot to mention before. In the 4th issue, the numbered list was seen as bulllet list after conversion with some extra gap between each bullet.

Earlier today I found another bug related to images. See this tiddler. The first image is seen missing from this clip (html added for comparison). With the earlier version of the macro, first two images were missing.

I think if I use more, there will be some random issues like this. But that doesn’t limit the usefulness of this macro. It saves so much of our time. Should I report such small issues here or just ignore them.

#20

Maybe you’re using the version before the update? When I try clipping the first 3 images and some text, it seems to be all there. I’m a bit overwhelmed by the size of the images :wink:

redsign-of-landing-page.json (1.3 KB)

Re multi-columns, how do you change the size of the columns (not the number, which I was able to find easily).

Thanks!

#21

I am using the version 0.7

Will check once more and reply.

Ya, thats true :grinning:. These are just for testing purpose.

Double click on the top of the each column to increase its size. Alt+Double click on the top of the each columnto reduce the size of that column.

#22

I once again added the updated macro to my wiki. I tried various combinations of selections, but the first image was not getting clipped unless I clip in seperately. How you were able to do it?

Is the code block and quote stuff doable?

#23

Is there any way to convert such videos to be embeded in our wiki.

#24

Another update. Ordered lists might work now.

When I try a straight HTML capture from twitter it doesn’t seem to give me code containing the appropriate video information. That is, it doesn’t display correctly even as HTML, let alone wikitext. But I can select it here on Discourse and get the working MP4 address. So Discourse must know how to negotiate with the twitter server to get the real video address.

#25

I would first look to see if the video platform offers the embed code for your wiki, if you have any difficulty just ask the community.

  • I know YouTube does.
  • Its usually some html + an include you use a raw tag for.