Playing with Parsers

#21

I think it’s possible, it’s definitely something I’ve been considering, but I wanted to finish cleaning up the videoparser, incorporating some of the things I’d learned from building out the audioparser.

The floating player was actually inspired as a mobile feature, as I sometimes need to pause and skip back and forth when taking a note, and within that context:

  • it works fairly well, but I find it doesn’t consistently play flawlessly well with all screen interactions (opening context menus in streams, having keyboard open, etc).

  • I also think I will not have scrubbing through in the floating window, since (at least on android) those capabilities should be in the notification window

Dragging was an afterthought once I saw how well it seemed to work on desktop, when testing – and, I agree with your assessment that, in that context, scrubbing would be useful as well.

I think you may be right, the state mechanism and the qualify macro are both areas I could use more work in, so perhaps this is the time – it is also a bit more complex, which is why I went with field values initially, but perhaps you could help me work through this conceptually?

  • The state would need to be unique both to the source and the context, i.e. $:/state/src.instance

  • Notes that are made referencing the “current time” would need to capture that from the state tiddler… so we would need (in plain language, to start) a way to describe the relationship between the context in which that create new note button appears and the $:/state - - so conceptually, is there a way to do that within a template that audio tiddlers are passed through, or would this create new note button need to be a part of the audio parser

  • Would you agree that it would be appropriate for timecodes in notes to be field values, since they are not “states” but are static values (or ranges?) In which case, how should the time be called?

  1. in the context of the $:/state/src.instance, which would directly tie the tiddler to the instance (and context) of the material in which it was generated,
  2. the src itself, which would require altering the state for the src before calling it,
  3. within the context of the note itself, i.e, an embedded player in each note (not my favorite solution, but may work if we have multiple template we’re passing through, or
  4. some completely different thing I haven’t considered yet?
#22

Ah, yes. The current setup would only allow a “start time” but it has occurred to me to store notes as a range – the tricky thing is that for me, conceptually, recording the note should be as simple as possible… the ideal being click the button, take the note though something where I could drag-select a range, take the note could also be reasonable (though more complex to execute well).

The course you take would be too complicated for me, manually entering multiple edit boxes… here’s another riff, though, what about push button to record start time AND/IF initiate transcription of currently playing, push button to record end time AND/IF suspend transcription, take note?

#23

https://mediaplayerava.tiddlyhost.com/

My annotation template was inspired by this mediaplayer by @buggyj

This should also be enough.

#24

Other use of the mediaplay is here

https://mediaplayers.tiddlyhost.com/

#25

can you share a screenshot of how your video tiddler looks like so that I can test the code from the PR

#26

It’s certainly possible to do it with a template (in the traditional TW sense). I’m not familiar with the parsers and the ways they intersect with content-types… You could use the ViewTemplate cascade to supply a default template when rendering any tiddler with an audio/... type, but this wouldn’t help you when audio/video tiddlers get transcluded into other contexts, so I suspect you’d need to modify the parser if you wanted an annotation mechanism to be universally available.

Broadly speaking, though, qualify works the same way whether you’re using it to generate a field name or a tiddler title – so if you can retrieve the value from a qualified field name, you can retrieve it from the text field of a qualified $:/state tiddler.

Yes, I’d say time-stamped notes do fall into the category of “content”, not “state”. I’m not sure how likely it is that you’d want different notes/timestamps on the same media in different contexts, but I can contrive some possible circumstances — for instance, I might want one set of notes on the topics discussed, and another on linguistic phenomena displayed. And as I said previously, I’m not keen on the idea of “dirtying” the src tiddler with fields that won’t be meaningful if taken out of context, since TW users do commonly import tiddlers from other wikis.

  • I’d generally prefer to keep any timecode notes on the tiddler in which the player was embedded, since that’s presumably the context in which they’re most relevant.
    • I think my personal ideal would be one note field per embedded player per host tiddler, with each timecode and its associated note appearing on a new line (akin to a dictionary tiddler, but stored in a field).
      • In my own (very simple, manually-edited) template, I’m using the text field this way, but a) I only have one embedded player per tiddler, and b) I’m not using the text fields of those tiddlers for anything else. For “playlist” tiddlers, you’d presumably want a different field for each player.
  • I could also see a case for storing notes in JSON tiddlers associated with the “host” tiddler.
    • This would avoid the potential field pollution issue altogether, and it would let you use just one “notes” tiddler per “host” tiddler, regardless of the number of embedded players.
    • Of course, a secondary “notes” tiddler would be less closely linked to both the media src tiddler and the “host” tiddler displaying it. This could be either a downside or a benefit, depending on your perspective.
      • On the one hand, it’s an extra tiddler to import if I want the same notes in another wiki.
      • On the other hand, perhaps I want your video collection but not your annotations?
    • The other, larger downside to JSON tiddlers is that reading and especially writing to them is still rather difficult, IMO. You’d need a fully automated writing/retrieval system that wouldn’t require any end-user knowledge of what was going on behind the scenes.

I do think we’re getting rather far into the territory of personal preference here, and I’m not convinced that there is one “optimal” solution. Other people may want to store/access/display their notes in ways I haven’t considered, and this may be too opinionated to hard-code into a parser.

I think it’d be more in-line with the general TW ethos to keep the parser/player itself as simple and flexible as possible and use procedures or widgets to provide more complicated systems like annotations. This would let you use {{My Video}} to display just the player and <<annotated-video "My Video">> to display a template that generates/displays your timecode notes.

  • This is basically the approach @buggyj takes in his (very impressive!) media player plugins… though I admit that they’re so complex I found them a little intimidating to use.
#27

I’m sorry I don’t quite understand what you’re asking here. I included a gif above that shows what the transclusion looks like when the parser has been replaced, and the code for the tiddler is just a transclusion of the src tiddler {{billionaires want you to know they could have done physics.mp4}}

#28

Well, let’s just say we have a tag “Audio” or something, for simplicity’s sake – what would the plain-english way of describing the relationship between the note and the context be? For example, "Create a new tiddler which has a field X with the value <<currentTime>>" x being some value that would communicate both the parent tiddler (The one tagged audio, transcluding the src) and also the particular audioplayer instance (in case you are transcluding multiple audio files in the same Audio tiddler)

Agreed, I’d like to find a generic enough solution that will work for a wide range of usecases, hence my asking so many questions (forgive :slight_smile: )

I use Streams for notes, so each note is its own tiddler which is being transcluded in the stream-list of the parent tiddler. This allows me to transclude notes in multiple place – that’s helpful because, in some contexts, notes are being used in a processing/reference context, and others in a reference/citation context, which I am constructing an argument which might be supported by evidence the note points to.

#29

Ah, I suppose that would be a third possibility: each note goes in its own tiddler, with a title derived from both the parent tiddler and the src title — <<parent>>/<<src>>/<<timecode>>, or possibly $:/notes/<<parent>>/<<src>>/<<timecode>> if you want to keep them out of a standard search?

You could also take a cue from Streams, let users configure the naming scheme, and store the parent/src/timecode values in fields for easy use in filters. That would add a total of 3 field names rather than a potentially infinite number.

If you were going this route, it might also be worth adding a button to export all notes associated with a src file (or a particular embedded instance?)

#30

Changing to a $:/state tiddler would almost certainly solve this problem, given me assumption here is correct

I’m inclined toward this solution, so then we have a modified streams node template that not only sets the parent value as it currently does, but also gets the state value… and this is where I run into trouble conceptually because, if currentTime is not a field of the currentTiddler {{!!timestamp}}, what is the generic solution?

But, talking through it, perhaps the solution is that the Audio-tagged tiddler has a template which transcludes the {{!!src}} field, which itself contains the transclusion of the particular source for that video. And then the New-Node button would adopt that value (to allow for your suggestion of referencing all notes associated with a src) as well as the parent, and it can set the field timecode to $:/state/{{!!src}}.{{!!title}}

And then, in the template for the note, we would have a button which says, "Set $:/state/{{!!src}}.{{!!parent}} to {{!!timecode}} and that would result in the media instance jumping to that timecode if it’s already running, else start at that timecode the next time it loads.

#31

@well-noted a few notes on seperating information from one tiddler to another.

If you look at the parameters widget, you will see how the parameters can be stored as an array and iterated back into a set of variable name/value pairs.

There are numerouse ways to move this complexity behind the sceens whilst provisioning the tools to support drag and drop of a set of tiddlers to keep related tiddlers bundled together. One option may include notes, the other not.

  • One approach is two draggable buttons, one each for with or without related content such as notes. These can only appear on video tiddlers in the toolbar or viewTemplate in a particular mode.
  • You can support the use of a filter to be used in advanced search, so you can then choose what to do with the tiddler bundle.

Other notes;

I would make sure you are familular with the way we use $:/state $:/config $:/temp and $:/temp/volatile/ and see also RefreshThrottling.

  • To use these we need to generate compound titles and there a range of ways to do this;

The Qualities of Qualify

Keep in mind that tiddlers are by definition unique, so if you simply prefix the current tiddler with one of the above prefixes it to will be unique. If the parent may be renamed you can install the relink plugin to keep the referential integrity, or provide the user with a rename tool.

Qualify generates a title not just based on the the details given to it but by using context information like where the code is within other code, it can be used to generate multiple serial numbered titles if needed. It is best used when you are not after a simple one tiddler to another relationship. Rename the tiddler and typicaly qualify returns a different title.

  • If the title is to be referenced more than once save the title in a variable to reuse it.
1 Like
#32

To return to this, @etardiff, I have modified to use state tiddlers, however they do not persist through refreshes - - I saw some previous discussions about allowing persistence in state tiddlers, did anything ever come of that? If not, is there an alternate method you would recommend?

Originally I’d thought this might be a problem, which is why I’d gone with the field solution :sweat_smile:

#33

Ah, I assume you’re using Node.js. $:/state tiddlers do persist in single-file wikis (though they’re not included by default when upgrading); it’s $:/temp tiddlers that aren’t saved. I’m not sure why the handling differs in this way, to be honest… it makes most core UI states kind of meaningless.

I tend to use system tiddlers with a different namespace for state-like tiddlers I want to be absolutely sure I won’t lose. In this case, I’d probably use $:/timecode/ or something similar.

#34

I’m a bit reluctant to pile on additional tiddlers, generally, which is why I’d gone with a field value originally… will have to reflect some on how to address. Perhaps I can allow the user a toggle.