Basically if it confuses you it will leave me on the Old Town Road.
Best, TT
We still have the problem of delayed gratification. At this moment I have at least 4 PR’s (non tag) pending. None of them particular controversial. One of them provides examples for one of the many tm- messages. If it is approved, I may go on to write up examples and corrections for the other 20 messages. But as long as it sits there, I’m not doing another one because it might be that I would have to change something on ALL of them. The longer the delay, the more likely I am to lose interest and never come back to the project. Also, the more likely someone else is to make some other change, which will mean that the merge can’t be fast-forwarded.
When you’re working free, the only ‘payment’ is seeing your work get used. You don’t even get a name attribute (except if you want to read the github commit history log).
1 Like
I think what we all need to understand is that everyone gravitates towards what what they are used to. Non-technical new users want things to be similar to what they might be used to from elsewhere, maybe that is markdown or a sprinkling of wiki mark up. We go out of our way to accommodate them and help them feel welcome and get started.
It is important to understand that when someone with a technical skillset, let’s say a JavaScript developer, joins the community they are just as much at a loss as the non-technical new users. None of their prior skillset applies or is relevant in the world of wikitext. So it makes sense that they gravitate towards using what they know, and we should be just as patient with them. There is a lot more that could be said here - and probably should be - but that would probably take us even further off topic than I have already done, so if we want to discuss this further we best do so elsewhere.
To be honest, personally I really don’t care about acknowledgement. I would be more than happy to keep doing my thing in the background and for no one to ever know about it or mention it, as long as we could stop with the uninformed hostility that new development work is sometimes met with, as well as the rather unreasonable statements that crop up far too often implying that developers are not doing enough.
3 Likes
IMO it depends on the size of the tiddler. I personally would go with 1 tiddler per PR if eg: 1/3rd of the content is changed or the internal structure is changed.
If you have more than 1 heavily modified tiddlers in 1 PR the chances are high that 1 tiddler is OK, but the other one causes problems. … So the whole thing can’t be immediately merged.
If it would be 2 PRs 1 could be merged and the other one may need some discussion.
BUT
If tiddlers belong together and changing 1 without the other doesn’t make sense, they should be in 1 PR.
1 Like
Yes. If we see a PR we generally have a look at the diffs. If something looks suspicious you can be sure that you’ll get a comment with our thoughts.
If every thing looks OK there only may be a “thumbs up”-emoji or a “looks good to me” comment. So Jeremy knows that someone else also has looked at it already.
The final decision is made by Jeremy. … and you can be sure that “backwards compatibility” is a thing 
@Mohammad if you look at the list I referenced, I am very confident that writing documentation for the following is well within the abilities of most of the people participating in this thread:
- Filter run prefix documentation tiddlers missing for most prefixes
- even ignoring the newer prefixes, we need documentation for the very old ones like +, ~ and even the no prefix case. There are examples to follow that already exist for the structure of the documentation.
- Examples for insertbefore operator
- Examples missing for haschanged operator
- Provide examples for the ButtonWidget
From my point of view there isn’t much of argument to be made claiming that documentation cannot be written due to a lack of understanding, when the ones that easily could be written are still wanting for attention.
Furthermore, many cases of people claiming that documentation is lacking are not carefully reading what is already there.
Take the EventCatcherWidget for example, the documentation is complete and includes a functional example. It also mentions that it is an advanced widget that requires understanding of HTML and DOM events. It isn’t intended for everyone, we have other widgets that are easier to use and do not require such knowledge, the entire TiddlyWiki user interface has been built without EventCatcher for instance.
Teaching users about HTML and the DOM is far outside the scope of what TiddlyWiki documentation should be expected to explain. However for users willing to spend the time to understand the relevant concepts about the DOM, this opens up a lot of interesting possibilities without ever needing to write or understand JavaScript. This in itself is a significant achievement, it cannot be simplified further - well it can but then we end up with widgets like the ButtonWidget which we already have.
1 Like
The issues you raised are very much about “perceptions”. I think they matter. I’m not sure how much we can address them. Just FYI I worked with “developers” in TW in a naive way to MY benefit. @pmario on Custom Markup and Bundler. @Mark_S on Polly. Moransanue on TW Icons. I think we get along.
I would happily comment in a new thread that champions developers. You are the rock stars.
TT
1 Like
There is a Documentation Styleguide and it should be used.
But as you found out, there is room for improvement. … If I remember right, the “Documentation Macros” date back to some discussions with Tibias Beer about “improving the docs” 2014 or 2015. …
They should give us the possibility to globally “re-define” and “theme” doc-styles if the macros are consistently used in the docs. … (there is still some way to go)
They are mainly used with filter operator docs and scattered all over the other doc tiddlers.
1 Like
If you try to completely change the structure, you should definitely discuss the new structure first. … The best way to go is “start small” …
Create a demo wiki with eg: 10 tiddlers that show the new structure and let’s discuss it – here or at GitHub Discussions
1 Like
Yes. It may look like this, where I thought it would be a quick fix. … The bad thing is, it can be frustrating. … The good thing is: What ends up in the docs is better as the first try.
1 Like
All contributiors are listed with every release at the end of the official release note: https://tiddlywiki.com/#Releases
Dear M.S.
We take you home and feed you on squirrel-and-mash. Yeah I’m feedin’ you smart food–yeah I’m gonna take my goat to the lawn, leanin’ on my Tiddly. I SEE your thing, cross town in your sports pod spending time on us. I’m gonna take my Tiddly out on to the back, think about the work you do for nothing at all. It’s not fair, time given 'as come upon. The Marlboro Man with no fags left. I wanna light that match for a candle at least, at least.
TT
Hmm. Might be country music. Just don’t know which country.
1 Like
OK, First I don’t have the patience to read every detail in this thread.
In fact I am a little annoyed;On the subject of documentation I have commented an number of times and apart from only Mohammad seeming to acknowledge the issues in my last post I feel I or others have being “slighted” a few times here;
- My comments about a minimum amount of documentation needing to come from developers holds, though only for specific features.
- There are a number of recent threads discussing new ways to facilitate, including comments on mine about making it as easy as possible, even my suggestion for a helpful button has being poopooed. But what is now the definitive documentation path? Have we found a "best approach, or a few even?
- It ideally should NOT demand GitHub, though we can facilitate more of this
- I have commended on New users with Javascript skills prone to reinventing the rules with Javascript solutions when such solutions are already built in. Good documentation may reduce this but it is a valid point and people using a Javascript hammer to bang all kinds of inappropriate things. TiddlyWiki get more complex when solutions are developed through Javascript and present themselves as the solution when they are unnecessary.
- I also feel slighted by suggestions people talk about things and Don’t do any thing. I for one have a large repository of tips, tricks, examples documentation in my own repositories, just waiting for movement into documentation. But I barely have time for my own development I shy away from undertaking time wasting tasks, where the cost is greater to me than the value to the community.
You do not motivate contributions by complaining about the lack there of. You encourage and facilitate, because if you do not you push people away.
There has being a number of highly relevant impediments to Documentation raised previously and I don’t want to “regurgitate” them. But mark my words my contributions have being realistic, truthful and based on work and a deep understanding, but if the result of that was almost nothing it hardly motivates me.
Lets start with compassion, understanding and facilitation.
Mario, with all due respect that is incorrect, it only lists people who have submitted GitHub PR as far as I can see.
I have being deeply involved in each version from testing to popularisation and a myriad of requests that get unactioned then appear somewhat reinvented a few versions later, with no acknowledgement.There is a “blinkered view of contributions”.
You see I currently contribute to the conversation, not PR’s, for example I usually identify limitations in widgets and wiki text and ask they be fixed, because its outside my skill set. But this rarely occurs, even if I get convoluted work arounds.
My only acknowledgement in a release was due to a PR, versions ago, and the reason I have not done it again is because I am not sure if I now have a “stale tree” version tree in GitHub. And as I have complained the documentation submission process is complex, and for someone who values integrity I do not want to introduce issues.
The current documentation process and that as stated above by @saqimtiaz is easy for someone who is a developer, but no one else, but there are more problems than just this like poor documentation on the “documentation macros”, so its a bit rich complaining that non-developers are not contributing enough and when they do they rarely get acknowledged in the release.
</new rant>
All, Please Listen or be doomed to repeat these errors.
@TW_Tones, I think you’re onto something here about there being a lot of people who want to contribute but don’t want to because:
- It takes extra time and steps to do so, many of which are difficult or unclear;
- You have to make sure everyone is OK with changes first.
My initial thoughts were about not knowing how to get started, which indeed I don’t know how to do, but @pmario’s responses and your thoughts made me realize there is a more general problem after those things get resolved. The more I think about it, the more skeptical I am that a pull-request-based process is the right way to write documentation, especially when we are trying to encourage contributions from more users. We are gating things and thereby eliminating contributions to make sure the documentation quality is kept high, when the actual problem is that the quality is much worse than it could be because we don’t have enough contributions.
Here’s why the gating is discouraging contributions.
It’s hard to squash general improvements to documents into PRs
I think the focus on familiarity with GitHub that has so far characterized these discussions is misplaced. That’s a factor for lots of users, but I think there’s a larger problem.
For me, it isn’t familiarity at all. I’m a software developer, I work with pull requests all the time, and I am pretty much obsessive with producing a clean Git history that is easy to follow and explains why I made each change in detail. I will happily spend 5 minutes rebasing something if I accidentally added two lines to a commit that don’t belong. I also was the Git, SCM, and development automation expert at my last workplace for 3 years and learned all sorts of esoterica I am not happy to have needed to know. So I am very Git, GitHub, and PR-positive in general. But I’m also a writer, and when I am sitting down and working on improving a large document, I cannot get my prose to fit effectively into neat little changes that align with pull requests. I have tried numerous times going back to when I learned distributed version control in 2008 – because I would love to be able to produce a coherent timeline of my “development” process – and I just cannot make it work.
Here’s, as best I can explain, why it doesn’t work:
When you’re writing software, you can decide you want to add X feature or fix Y bug; exactly what behavior you want is clearly definable and clearly defined. Then you go and make changes until that’s implemented. You know exactly when you’ve finished implementing it, because before it didn’t work and now it did. You finish testing it, and then you commit it and move onto X’ or Y’. If you change a feature that has documentation associated with it, you can also go update that documentation and include it in the commit, if you like. The changes you need here are fairly circumscribed and clearly connect back to the functional changes you just made.
But then we have the situation where there’s a document out there. For our purposes here, we’ll say the TiddlyWiki documentation. This contains, in addition to topics that are tightly connected to sections of the code, numerous explanations of best practices, tutorials, and so on, and we see some part of it that’s not as clear as it could be, or that could use additional information, or could maybe be reorganized, or whatever. I don’t know about you, but I don’t do this by picking out some extremely specific change I want to perform, doing it, committing it, and then moving on. I open the document to the section that looks like it needs work, read it, think about what’s missing, try adding some text, go remove some other text, rewrite some text, realize that this is partially duplicating another section, go over there and swap some stuff between sections for a little bit, notice a broken link in the supporting material for that and fix that, come back, realize that something is tagged wrong and fix that, notice that I can make things clearer by reorganizing this into two sections…eventually 45 minutes have passed and the document is better than it was when I started, in a way generally according with what I set out to improve, but maybe in other ways as well.
TiddlyWiki’s structure often makes this effect even more pronounced, at least for me, because one topic will likely be spread over several tiddlers, and it’s therefore easier to encounter other things that need to be improved on the way. The number of things that need to be reshuffled to end with an effective organization can also be higher. Sure, I could add all those items I come across to a to-do list and come back to them later, but if the problem is, e.g., “these italics over here should be changed to a macro,” or “there’s a missing comma in this tiddler,” that’s kind of ridiculous. Yet if you start allowing these kinds of things to squish into an existing commit, the boundaries start to blur, and in my experience it’s very difficult to avoid making larger and larger additional changes, even unintentionally, and these then become extremely challenging or impossible to separate. Also, in some cases two changes are intimately connected with each other and it’s not really possible to separate them at all.
So then put them into one PR, that’s OK, they’re part of the same change right? But then you end up with a huge PR and people complain that there’s too much in it, and it takes 53 days for all the reviewers to look at it, and by that time you’ve forgotten what you were doing, you’re on vacation, and you’ve given up on editing the TiddlyWiki documentation because it took forever to get anything done. (This isn’t really an exaggeration; it took 30 days to have my 4-character typo fix reviewed in September. Some of this could be fixed by ensuring that new PRs get triaged better, but I think anyone who has worked on open-source can testify that sometimes, they just take a long time to deal with.)
In the end, I find that my changes smoosh together enough that it really isn’t worth trying to separate out the changes that precisely. Obviously it’s possible to broadly group them rather than working for 8 hours before committing anything – which is sometimes important if you’re working collaboratively on something – but the kind of precision naturally expected by version control doesn’t come easily, if it’s possible at all. When I work on something by myself, I often don’t even bother with general groupings and just do end-of-day checkpoints so I have backups, as I don’t find the groupings to be helpful in any way.
Submitting things for review causes excessive friction
I’ve used over a dozen different tools for documentation, and without exception the tools that required extra steps to submit changes for review, merge them, etc., have been the ones that went out of date and the ones where I often left ugly bits and even outright errors because it was too much trouble to go fix them. Sure, you can say “it only takes 45 seconds to submit a pull request,” and that’s true, but only to a point:
- The more steps there are in a process, the harder it seems to get started. Have you ever not gone to get something you wanted from the next room because you had to get out of your comfy chair? I’m a young and reasonably fit guy, and I do that all the time. The inconvenience can be completely trivial, and it’s enough to stop you from doing something that is obviously beneficial.
- Unless you own the project and there are no other reviewers, once you take 45 seconds to submit a pull request, you have to keep thinking about it. It’s not actually public yet. Someone might come back and challenge what you wrote, and then you have to go defend it or update it. Someone else’s PR might get merged first, and then you have to go fix merge conflicts. People might forget about it entirely, and then you have to go poke someone. As mentioned in the previous section and by @Mark_S, this can take a long time.
When we’re talking about functional software, running these hurdles is part of the game, and they’re totally worth it. Because it’s relatively easy to bundle software changes into small, neat groups, even if you leave it for a while and have to come back to it, it’s usually pretty easy to re-grok your request and get it moving again. Meanwhile, introducing bugs into software that a lot of people rely on is pretty bad, even when it’s TiddlyWiki and it’s unlikely to hurt anybody, and mistakes can easily lead to more time and trouble than doing the code reviews over the long run.
Again, general improvements to a large document are a different animal. They’re hard to bundle and delays are much more costly because changes build on each other more often. Waiting for an existing PR takes up a lot of mental space that makes it hard to keep working on other parts of the document, and you usually can’t hop back to the current mainline revision and work on a different feature, then merge them together, like you usually can with software. There are a lot fewer objective problems that need to be caught; while the docs can certainly contain factual errors, the debates about them are usually more about organization, surface errors, etc., which are subjective to a significant degree, and “failures” in them are a matter of making the docs a little harder to use rather than making something outright not work. Realistically the worst thing that can happen is “the documentation is a little harder to understand and we might have to revert it when somebody notices.” Given all the downsides of a gated review process in this case, I think it’s really hard to justify that.
Look at what happened to Wikipedia – the project behind it was originally an encyclopedia farmed out to individual experts, with a review process and so on. The wiki part started as an experiment for small portions of the encyclopedia, and it sounded like an absurd idea…but we all know now that the little upstart experiment worked so dang well that it’s one of the most visited websites in the world and they canceled the initial project entirely. This works by getting more people involved. Maybe 95% of the (non-vandalism) edits are improvements; the others get noticed and fixed because there are a lot of people involved, and overall you trend upwards.
I don’t mean any disrespect (y’all are doing great work!), but I think the folks who are in charge of the current process for handling documentation do the vast majority of their documentation writing in the “writing documentation to go with a change” mode. For this mode, pull requests work pretty well, and there’s no denying that having the documentation’s history tied to the code is useful. So naturally, they’re biased towards this mode. But the reason our documentation is tricky to navigate, in places hard to understand, and in general in need of revisions and additional content of some types is that more or less all of it was written to go with a specific change, and some other types of work are needed from time to time as well. Why is all of it written to go with a change? Because it is a pain in the rear end to do anything else currently.
Have you ever worked in a bureaucracy where you have to ask a committee for permission to do something, and then they go talk to 6 other people, and then they come back and you have to go to two other meetings so you can explain what you want to do multiple times to people who don’t understand what you’re doing? Then once it finally gets approved, you have to go back and do the next step in your 6-step project plan? It can take months or years to get simple things done under these processes. That is what I see from afar when I think about working on editing a document where I don’t have some form of direct write access, and I’m inclined to run away.
How can we fix it?
I’m not suggesting changing platforms. That’s a bother, and it would be kind of embarrassing if the TiddlyWiki docs didn’t use TiddlyWiki! If it were me, I would try getting rid of reviews entirely on documentation files in the tiddlywiki-com branch – have them merged instantly by a bot or a scheduled process and sent live on a schedule. Ask for a commit message, but don’t be too worried about exactly what’s changed.
Since we wouldn’t have the benefit of instant rollbacks by anyone like you would in, say, MediaWiki, I would want a couple of precautions:
- Require contributors to be approved the first time they contribute. (As things are, we probably want them to sign the CLA anyway.) It’s probably too dangerous to let anyone with a GitHub account commit any changes they want to the docs directory, but hopefully we can trust established members of the community. Worst case, we could even require people to apply to get this access if “known, has committed OK things before, and signed their name in the license agreement” still seems too risky, but I don’t like bureaucracy for projects like this.
- Limit the number of tiddlers and/or lines that can change in one PR to a reasonable number – this can prevent accidents and make sure that each PR is small enough to read through if appropriate.
- Tag/otherwise mark any particularly dangerous or high-profile tiddlers (e.g., the front page) as protected and require approval for those.
- Since I saw concerns earlier in another thread about changing things that would affect inbound permalinks, something to detect renames without redirects and block PRs until this is fixed should be implemented. There may be many additional automated checks that would make sense.
If we aren’t willing to do that, then something needs to be done to streamline the review process. Unless I can expect leniency on exactly what (maybe partly unrelated) changes go in a PR and the turnaround time is vastly improved from the status quo, I just don’t think I can write effectively, and I doubt I’m unusual. Further, in most cases I’m not sure blocking changes until things reviewers suggest are improved really makes sense for documentation; unless the changed version is clearly way worse, usually it’s better to just make the changes and then have someone go back in and make additional improvements if others have suggestions.
It might also be helpful to add some kind of subscription or recent-changes feed so people could see what’s changed more easily and see if they disagree with anything. And there should be a clear forum to bring up questions, whether they’re about changes already made or about potential future ones.
The work in progress to make it easier to submit changes directly from TiddlyWiki could help to address some of the remaining problems with the tools being developer-centric.
Maybe none of this will work, but the basic approach seems to work in most places it’s been tried across the web – I do not think it’s pie in the sky by any means.
4 Likes
I largely agreed with what @sobjornstad and @TW_Tones and @Mark_S have stated above!
Lets accept the barrier to contribution is high! All of these people showed their commitment to Tiddlywiki and its community and each of them has invaluable contributions!
At the same time I understand the pressure on developers! and I understand how much energy @saqimtiaz, and few other core developers put in every release!
By the way, as the one who started this thread, the purpose was not to question anyone or official documentation! I just announced request for help to share demos/working examples/docs for filters! These may have potential later to be added to official docs!
1 Like
I aggree almost totaly with your responce. For this to be possible we musy be tackling a universal ly experienced systemic fault. I too have a number of ways to address this but will resist because i dont want to put th effort in if few read it and it goes no where.
Hi @sobjornstad, thank you for the time you are putting into reading and engaging with discussion thoroughly, it is highly appreciated.
A few comments and clarification from my perspective on some points below, though forgive me if I stray into more general discussion around the topic at times as opposed to replying to you directly:
All of these same constraints apply for developers as well when it comes to writing documentation. Sure, we might have greater familiarity with the git part of things, but the rest with regards to where new tiddlers go, tags and macros to use are a struggle. I’ll often have prospective PRs languish for weeks before being submitted just because I need to write documentation. So it might be helpful for all to understand that this isn’t a problem for end users alone - I think you already understand this.
While I agree with most of what you have said about why gating discourages contributions, I will point out that I have seen proposals for documentation updates that have had such poor writing/grammar and lack of proof reading, that were it my project I definitely would not want them to go live even for an instant. Especially since tiddlywiki.com is the official public face of the project. From my perspective some sort of gating and quality control is needed for tw-com.
Which raises the question that perhaps the definitive reference documentation lives on tiddlywiki.com but there could be a community documentation collaborative on tiddlywiki.org with a laxer review process. Jeremy may have suggested something along these lines recently, though there is always the chance I may have misunderstood his intent. Even if we don’t merge PRs automatically, with a separate repository and site, we could have more people reviewing them. One of the issues with the current TiddlyWiki repository is that we are not able to have fine grained permissions and Jeremy is the only one who can merge PRs. Once in a while documentation improvements from the .org site could be cherrypicked and merged into .com.
I would also like to point out that for a project where only one person holds the repo keys (and with good reason in this instance for the code base) I think Jeremy does a great job these days of replying to Issues and PRs in a timely manner. Yes we do have a lull immediately after a release, or one can get unlucky and make a PR while Jeremy is away, but a bump more often than not is enough to get things rolling.
I am not entirely sure who you refer to here, the only one really “in charge” of anything is Jeremy. Everyone else involved in the project is a volunteer, we are just volunteers with different skillsets all of whom have to work within the current limitations of how documentation is written. I doubt that Jeremy is tied to a particular way of writing documentation, its just the best we have. The tricky thing as always is that for there to be a better way to write documentation, someone has to do the leg work. So either we need people to step up and volunteer and do the hard work, or we pool together funds and hire someone. Just talking about things and hoping someone else does it, wont get us anywhere.
Speaking personally for myself, there is no bias or preference for a particular way to write documentation. I have been working within the bounds of what is currently possible - a git based workflow - to make it easier for users to make documentation changes. In response to people asking for gaps in documentation to be filled, and identifying the lack of Git knowledge as the primary road block, I’ve been dedicating time to make that process easier for the layman so that they do not need to understand or use Git directly- and I believe we have managed that, we just need to present it better and make it easier to find.
I would agree with you that making longer and far reaching documentation changes is difficult via individual PRs, and in my personal opinion these are the kinds of changes that would be most helpful in terms of onboarding users. Not the detailed documentation for individual widgets etc but introducing the general concepts, that is where I personally think the weakness lies.
However, these aren’t the things that the community has recently been asking for more documentation on. There have been specific requests for gaps in docs to be filled in regarding primitives like individual filter operators, widgets etc, and these are precisely the type of things that can be written rather easily by most involved in this discussion - as demonstrated by their use of those very primitives and especially given new tooling - and also go through PR review quite quickly and painlessly. Whatever challenges still exist in writing this type of documentation apply just as equally to developers. Which is why the idea of “developers need to write this documentation” expressed by others is completely misplaced in this instance.
I wholeheartedly agree with this and it has been on my mind this very week. If the updated documentation is better than what was there and has no obvious errors, it should be accepted and any further improvements requested as another PR. This requires both a change in mindset for people reviewing PRs (myself included) and for Jeremy to adopt this way of thinking about documentation updates.
I think what is really needed to effect meaningful change is for a group of interested people to take the lead on improving the documentation process and overall work on documentation, help facilitate these conversations and try to translate them into something actionable, and direct that eventual work. If there prove to be resources required, be they developer skills or funding, lets figure what those needs are and then see what we can come up with as a community.
4 Likes
@saqimtiaz we don’t need more from developers (except some really basic info on some key features provisioned by developers), the community can take it further. We don’t need funding and we don’t need more “Developer thinking”, I have argued this at length. We need to empower the community. Given my words in this thread you do not respond to and your statements in your reply I must say I feel as if your are either avoiding what I say all together, or being passive aggressive. Please take on board some of what I have said, perhaps even just acknowledge reading it, and stop speculating in a vacuum.
We must past beyond this barrier, or remain behind it. I respect your views but I am not sure you are understanding the views of others.