Pluggable backends for uploading files

Continuing the discussion from Elephant in the room:

I am wondering if you are aware of the FileUploads plugin? It is pluggable and can be extended to support most backends with a RESTful API. If you let me know your preference with regards to where you want to store images, I may be able to provide some guidance.

See:

2 Likes

This is likely a segue … but I have never understood the issue that requires anything special on external images. They just images at an address. What is the problem?

I do wonder what problem I am not having?

Best, TT

I think he’s referring to the process of taking images that are stored internally and converting them to images that are stored at an external location. Currently you have to save each image one by one, and then create the external image tiddler. Saq’s plugin will do all that for you automatically.

Maybe Saq is also suggesting that he can even actually upload to a hosted file location, in which case I should be paying better attention. Currently I have to upload to a hosted location using a separate upload tool.

1 Like

Essentially I’d like to be able to do what I do here in Discourse forums - Ctrl+V an image from clipboard and have it:

  1. Be uploaded into my server and stored as a separate file.
  2. Have an image tiddler created with _canonical_uri set to that file’s URL.
  3. Insert a transclusion of that image-tiddler into the currently edited tiddler.

I don’t want to store the binary image data as tiddlers and have them be needlessly loaded every time I load my TW instance. And since I host my TW instances myself using the built-in Node server, that’s where I’d like those files to be.

From what I gather this part of the documentation already explains how to do it, at least part of it.

As for the plugin I am pretty sure I saw it before but noticed it hasn’t been updated in a while and it still says “Under Development” so I was probably wary of trying it :).

Perhaps this is something simple to solve but I just never had the time to do it and the possible solution in the docs uses stuff I don’t quite understand how it works yet, so I haven’t tried it either.

1 Like

This is the use case that FileUploads plugin tries to address.

The plugin itself is stable with the only real unaddressed issue being that tiddler titles are used as file names without any further processing to ensure valid file names.

The plugin supports different uploader modules, where each uploader module adds support for a different backend, for example Fission, WebDAV, Github or node.js. The uploader module for node.js is highly experimental, however I don’t see it getting any further attention without user feedback or interest. The way it works is by enabling the /files route of the node.js TW server to accept PUT requests.

If you are interested in trying it, you can install the following from the SQPL plugin library:

  • FileUploads
  • the following sub-plugins for FileUploads:
    • FileUploads: PUT
    • Node.js files PUT uploader support

You may also need to create the config tiddler with the title:
$:/config/file-uploads/PUT/servertype
and content:
node.js

1 Like

The concept of the FileUploads plugin is that you can write a uploader module that adds the functionality for uploading to a given backend (hosted location) that provides a RESTful API, for example Fission, Github or WebDAV.

Uploading to storage without an API would require some kind of serverside component. Where do you upload your images/files to?

Got it! Tx.

I had never done, or thought about, doing that.

You might warm me to it!

TT

I don’t see a module for WebDAV or node.js listed. Are there modules for them?

A file hosting service. The most common one would be Google Drive, but they have a complicated authentication setup. So … ignore that one.

So, other, simpler hosts that offer ftp, local mount, or webdav.

If there’s a module for webdav, then that might work. But, as I mentioned, I didn’t see that listed.

Some webhosts create a local mount. So this would require the PUT to put the files somewhere that is not necessarily a relative path (because you can’t control the mount point).

I think part of the problem is that your original roll out is so closely linked to fission. This makes it seem like a very niche application that would not be of great interest to most people. But maybe that’s just me. I certainly didn’t realize originally that it could export files automatically.

And then, there’s the terminology of calling it an “uploader”. Which I guess means that it’s uploading to fission. But if you use it locally, it’s more like a “downloader”.

So maybe some other term like, “Image De-embedder”. Or “Put-to-Bedder”. Ok, maybe not those exact terms, but something that helps people see the relevance.

Thanks!

1 Like

I’m rarely this technical. But I find @Mark_S right-on explicating the relevance of this thing.

Given that @Mark_S is The Prisoner

Screenshot 2022-10-11 171236 he is likely right.

TT

1 Like

The development of this plugin was a test run at finding a sustainable model for community driven and supported development, where community members contribute via discussion, ideas and donations. The discussions around the original concept and the subsequent fund raising happened here in this forum, and after my involvement began I did my best to make sure the discussion was end user friendly. However, from my perspective, there was very limited buy in and involvement from the community.

The reason that the Fission Webnative uploader was centre stage during development is because it was the only one actively supported enough via both discussion and OpenCollective funding.

We also had a community call in August 2021 that demoed the functionality using both Fission and Github uploaders, emphasized that the plugin was not Fission specific, and the presentation on the call was again designed for users without a developer background.

I have explained the status of the node.js uploader earlier in this thread, and the situation with the WebDAV uploader has not changed since you and I last discussed it. Both of these are experimental and something that I have worked on in my own time.

User contributions to move any of them further, whether they take the form of code contributions, help with fundraising via Open Collective, or helping to find developers to work on them would be highly appreciated.

This feedback would have been helpful during the development process, starting from the initial public discussions and including the beta testing phase, during which feedback was explicitly invited after the public demo on the community call. The terminology could still be changed if there was community consensus behind the change and a developer to take on the work.