Looking for feedback on the Markdown syntax

JoshyPHP

I'm currently looking for feedback on the Markdown flavour used in Flarum, called Litedown. Litedown was created a little more than 4 years ago as an abridged version of the original Markdown. Since then, other flavours with slightly different syntactic rules have appeared or gained popularity, users' expectations have shifted away from the original Markdown and it's time to reevaluate some of the Litedown syntax.

Are you aware of any syntactic gotchas that often trip users? For example, take this text:

Is this a list?
 - yes
 - no

Most flavours are faithful to the original Markdown and do not display it as a list but it would make sense to break from it to better match users' expectations. For instance, Litedown already differs from the original Markdown when it comes to a_b_c.

I'm looking for any real-world examples where users are having troubles with the syntax. You're welcome to comment here or on this GitHub issue but commenting here is probably easier for everybody.

Pollux

JoshyPHP I'm looking for any real-world examples where users are having troubles with the syntax.

One thing, that struck me recently, is the handling of the superscript tag (^).

I understand the arguing, that 1^st and cm^2 should do the trick without the need for a closing tag, because it looks nicer, but it is counter-intuitive i.m.o., especially if 1^st^ and cm^2^ do the same thing most of the times. All other inline tags need to be closed, this is the only exception. More so, having only a white space to end the superscript sequence leeds to ugly consequences when non-superscript characters are supposed to immediately follow the superscript.

Last Friday, February 1^st^, I had an invitation ...
Last Friday, February 1^st, I had an invitation ...
The area of the circle (3.14 cm^2^) was greater than ...
The area of the circle (3.14 cm^2) was greater than ...

renders to:

Last Friday, February 1^{st^,} I had an invitation ...
Last Friday, February 1^st, I had an invitation ...
The area of the circle (3.14 cm^2⁾ was greater than ...
The area of the circle (3.14 cm²⁾ was greater than ...

Neither of them is correct.

Comparison with other markdown implementations

Even a backslash, which helped me out so often in other cases, can't solve this. There is no other way than to wrongly put a white space after the last superscript character:

Last Friday, February 1^st , I had an invitation ...
The area of the circle (3.14 cm^2 ) was greater than ...

Last Friday, February 1^st , I had an invitation ...
The area of the circle (3.14 cm² ) was greater than ...

Does anyone know another way?

clarkwinkelmann

One thing that often triggers me is the fact following lines are not merged.

It makes a lot of sense on forums, because users expect a new line to be rendered as such. But when I copy GitHub READMEs into the forum I often need to merge lines so they render as a single continuous paragraph ?

Recently I've seen non-tech people confused by an empty list turning the line above into a title, like:

Hello
-
-

Renders as:

Hello

-

This was on Trello, which apparently also allows lists without a spacing. I don't have any preference on forcing spacing or not, but using spacing prevents some Markdown issues like that.

Pollux

I wrote about my headaches with superscript here nearly two years ago: Pollux

And to my surprise and pleasure, the two most annoying shortcomings have already been adressed with the latest iterations of the flarum software: Not beeing able to actively close the superscript tag and a missing subscript tag (yes, that's what I intended to say).

If I may present another wish: I don't understand how on earth the underscore (underline) character doesn't produce underlined text, i.e. _text_ doesn't produce text. It's counterintuitive imho., especially as emphasis is also covered by the asterisk.

By the way: Is there a reason for the different behaviour of _ and *? It seems, that single underscores don't work inside of words.

m*m*m >> mmm
m_m_m >> m_m_m
m**m**m >> mmm
m__m__m >> mmm

And one more unexpected behaviour: If you combine subscript and strike-through via triple tildes (~), a double tilde terminates both:

~~~m~~m~ >> _mm~

JoshyPHP

I think that some implementations use optional parentheses around the superscript text. Of course that means that literal parentheses have to be escaped where ambiguous.

The area of the circle (3.14 cm^(2)) was greater than ...

I'm 50% sure that Reddit uses something like that but I think it doesn't work recursively over there. That's a shame because a main application of superscript text is to make it sound farther^{and^{farther^{and^{farther^away.}}}}

Might be worth considering but I need to look at the various implementations out there.

Pollux

JoshyPHP I think that some implementations use optional parentheses around the superscript text.

pandoc 1.19.2 and RDiscount 2.1.8 convert as intended (3.14 cm^2^) and (3.14 cm^2) resp. and fail with the other, MultiMarkdown 5.1.0 and MultiMarkdown 6 0.1.4a convert both.

3.14 cm^2^ seems most intuitive to me. The Reddit variant can be tought to techies, but not to ordinary folks.

What's more: I miss subscripts. How can I survive without water?

What about https://github.com/sgraber/markdown.subscript?

ShawnPConroy

May I ask where we are on this? Will Flarum still use litemark in the next release?

In theory I'm a big fan of the idea behind CommonMark, but I appreciate you wanting to balance what is typically used elsewhere and what would cause common problems for users. I believe this year they plan to release CommonMark 1.0. I don't really know much about it, but some extensions I think are important would like to see used. Not sure how they define or group extensions, however.

clarkwinkelmann

ShawnPConroy that's a good question for @JoshyPHP

I assumed Litemark was the name of the "markdown compatible" component of TextFormatter (the library used by Flarum to parse, store and render content) based on an existing Markdown specification and it just happened to have a few "quirks", or was it meant to be a markdown specification itself ?

Making Litemark CommonMark compliant would probably solve the issues reported above by simply following a specification instead of creating a new "markdown" specification with its own strange quirks...

Not going for an existing Markdown specification would probably end up like this:

JoshyPHP

Litedown is the name of the plugin. It's loosely based on Markdown and is more or less compatible with most implementations. It's not meant as a specification.

clarkwinkelmann Not going for an existing Markdown specification would probably end up like this:

Technically, Litedown is the 14th competing implementation and CommonMark is the 15th.

clarkwinkelmann Making Litemark CommonMark compliant would probably solve the issues reported above by simply following a specification instead of creating a new "markdown" specification with its own strange quirks...

No, it wouldn't and CommonMark has introduced its own quirks that didn't exist in Markdown and didn't exist in any other implementations at the time. I have no intention on spending time replicating those quirks for the sake of being compatible with one specific implementation.

Maybe someone could use a CommonMark implementation to develop a CommonMark plugin if they see value in it. If you want to do that you can glance at the Your own plugin section of the docs and look up the Parser API for add*Tag() methods. You can tag me in relevant discussions whenever you'll need help with it.

ShawnPConroy

JoshyPHP So, I take it from this we are going to continue to use the Litedown plugin since it seems to work well enough? I have had no problem with the Markdown syntax and think it works pretty well. But no changes are expected in the short term?

JoshyPHP

ShawnPConroy That's correct.