I’m not sure what you mean by “files”. Image files are notorious for bogging down a TiddlyWiki project if they are actually stored as data within image tiddlers…
But this question about the performance of a <<list-links>>
macro — one whose filter searches every other tiddler’s text field — should not be directly affected by this question of where your “files” are stored.
It does, however, matter just how much text is being searched.
Here’s a good test case: try visiting the TiddlyWiki recently shared by @Scott_Sauyet — a wiki that serves up the whole King James Bible — and make a tiddler called “Joses” with this content:
<<list-links filter:"[search<currentTiddler>]">>
The fact that Joses is only mentioned a handful of times doesn’t matter. This list-links macro still trawls through all 32 thousand tiddlers, looking for “Joses” in the text field. So it takes a couple seconds to render.
Looking for [<currentTiddler>backlinks[]]
(tiddlers with links to the current tiddler) is faster, though, since links are more efficiently tracked.