site  contact  subhomenews

Limitations of markdown

September 03, 2017 — BarryK

I am beginning to have doubts about markdown, for anything but the most simplest usage. Blogs, github, maybe ok, but for more complex pages, there are serious limitations.

I was porting the "How Easy works" page to markdown. This was originally written in SeaMonkey Composer, WYSIWYG HTML editor.

Due to the limitations of standard markdown syntax, as per the Daring Fireball markdown to HTML converter:

I had to insert actual HTML. OK, markdown allows this, and passes it through unchanged. But not quite...

First, a comment. For a reasonably-complex tutorial, here I am having to insert HTML. So, why am I not writing the whole thing in HTML, and avoid all the markdown-to-HTML conversion gotchas?

And yes, there are many gotchas.

Secondly, I was puzzled, as the markdown converter was mangling some of the HTML code. What I am doing is inserting syntax-highlighted HTML that was exported from Geany text editor.

Geany outputs lines that look like this:

<span class="style_2">###find&nbsp;drives###</span><br />
<span class="style_2">#find&nbsp;the&nbsp;drive&nbsp;we&nbsp;are&nbsp;booting&nbsp;on&nbsp;(has&nbsp;vmlinuz,&nbsp;initrd.q,&nbsp;q.sfs),&nbsp;and&nbsp;working&nbsp;drv...</span><br />
<span class="style_7">.&nbsp;/</span><span class="style_8">BOOT_SPECS&nbsp;</span><span class="style_2">#has&nbsp;BOOT_DISKID,&nbsp;BOOT_PARTNUM,&nbsp;BOOT_FS,&nbsp;BOOT_DIR,&nbsp;WKG_DISKID,&nbsp;WKG_PARTNUM,&nbsp;WKG_FS,&nbsp;WKG_DIR,&nbsp;Q_DISTRO_VERSION</span><br />
<span class="style_8">echo&nbsp;-n&nbsp;-e&nbsp;</span><span class="style_5">"\\033[1;35mFinding&nbsp;drives:\\033[0;39m\n&nbsp;"&nbsp;</span><span class="style_2">#purple</span><br />
<span class="style_8">CNT</span><span class="style_7">=</span><span class="style_3">0</span><span class="style_7">;&nbsp;</span><span class="style_8">Pb</span><span class="style_7">=</span><span class="style_6">''</span><span class="style_7">;&nbsp;</span><span class="style_8">Pw</span><span class="style_7">=</span><span class="style_6">''</span><span class="style_7">;&nbsp;</span><span class="style_8">BOOT_DRV</span><span class="style_7">=</span><span class="style_6">''</span><span class="style_7">;&nbsp;</span><span class="style_8">WKG_DRV</span><span class="style_7">=</span><span class="style_6">''</span><br />
<span class="style_4">while&nbsp;</span><span class="style_7">[&nbsp;</span><span class="style_9">$CNT&nbsp;</span><span class="style_7">-</span><span class="style_8">lt&nbsp;</span><span class="style_3">8&nbsp;</span><span class="style_7">];</span><span class="style_4">do&nbsp;</span><span class="style_2">#drives&nbsp;may&nbsp;take&nbsp;couple&nbsp;seconds&nbsp;to&nbsp;become&nbsp;available.</span><br />
&nbsp;<span class="style_8">sleep&nbsp;</span><span class="style_3">1</span><br />
&nbsp;<span class="style_8">CNT</span><span class="style_7">=$((</span><span class="style_9">$CNT</span><span class="style_7">+</span><span class="style_3">1</span><span class="style_7">))</span><br />
&nbsp;<span class="style_4">for&nbsp;</span><span class="style_8">aDRV&nbsp;</span><span class="style_4">in&nbsp;</span><span class="style_7">/</span><span class="style_8">sys</span><span class="style_7">/</span><span class="style_8">block</span><span class="style_7">/</span><span class="style_8">sd</span><span class="style_7">[</span><span class="style_8">a-z</span><span class="style_7">]&nbsp;/</span><span class="style_8">sys</span><span class="style_7">/</span><span class="style_8">block</span><span class="style_7">/</span><span class="style_8">mmcblk</span><span class="style_7">[</span><span class="style_3">0</span><span class="style_7">-</span><span class="style_3">9</span><span class="style_7">]</span><br />
&nbsp;<span class="style_4">do</span><br />

I put this inside <code></code> tags, and it is supposed to be passed through as-is, but isn't. For example, the text BOOT_DEV something WKG_DEV gets translated to BOOT<em>DEV something WKG</em>DEV.

At first glance, I thought "great", but then I noticed the mangling here and there. Reading the Daring Fireball docs:

Unlike block-level HTML tags, Markdown syntax is processed within span-level tags.

Even though the <span> tags are within a <code block makes no difference. Now the word that came to mind was "cr*p".

The problem is, there doesn't seem to be any way around it, using the Daring Fireball converter. So, googling, found that the Kramdown converter supports this:

By default, kramdown parses kramdown syntax inside span HTML tags. However, this behaviour can be configured with the parsespanhtml option. If this is set to true, then syntax parsing in HTML spans is enabled, if it is set to false, parsing is disabled.

It is also possible to set this on a per-tag basis.

So, I need to investigate alternatives to Daring Fireball. However, the first point I raised is very valid.

HTML is for everything, from the basic markup that markdown emulates, to sophisticated page layout. However, there is no difficulty in entering basic non-page-layout HTML tags when creating a page. Which does raise the fundamental question, why use markdown?
Why drop down to such a primitive syntax, when you find that anything beyond the most basic usage requires insertion of HTML anyway?

This is a valid question. I am thinking of extending shellCMS to enabled writing posts in SeaMonkey Composer. These will be raw non-page-layout HTML, to which shellCMS can add header and footer. Expect another post on this topic!

Tags: shellcms

shellCMS uploaded, online docs

September 01, 2017 — BarryK

I have beavered away, adding features to shellCMS. Got to say, this is fun.

Documentation is starting, and is to be found here:

Yes, is now driven by shellCMS, installed at the root of the site, as explained here:

It is most interesting to consider what more can be done to enhance shellCMS. Inherited from BashBlog, is support for Twitter or Disqus comments plugin. I tried the latter, didn't like it, plan to give Twitter a go.

shellCMS, as with BashBlog, is a single small shell script. Inherited from Bashblog is the ability to edit the html files after they are generated from markdown, or even not use markdown at all and create the posts directly with html.
There may be problems with this though, and I will probably change the script to insist on posts created and edited by markdown. That change would simplify parts of the script.

Another thing that would be great would be a GUI. probably using gtkdialog. The shellcms.tar.gz tarball could have gtkdialog bundled in it, for convenience when using on other Linux distributions.

Also, a search-box would be great, perhaps using google.

Tags: shellcms

BashBlog forked to shellCMS

August 31, 2017 — BarryK

This news blog started life powered by BashBlog, project page here:

It is very simple, too simple I soon realised. I have been hacking on the script. Some hacks are specific to my requirements, some are generic and could be adopted by the BashBlog developers if they wish.

As my script has become very different from BashBlog, I decided to give it a new name, shellCMS. It will be hosted on my site soon, expected to be here:

Here are the main changes from BashBlog:

Local development
shellCMS is designed for local development, whereas BashBlog is designed for development on the remote site.
shellCMS allows use of a nice local markdown editor, such as Geany with markdown plugin, which has real-time preview.
A script remote-sync uses rsync to upload the files to the remote site.

Redesigned page layout
There is now an overall table, of fixed width, to optimise fitting in a mobile device window.
There is a banner image at top, with links just below the banner.

Document mode
shellCMS can now be a document manager, not just a blog. There is a variable, MODE, that can be set to "blog" or "document".
Blog mode is what you are seeing now, on this blog. Here is an example of document mode:

BashBlog stores everything in the same folder. The filesystem will slow down as the number of posts increases, so BashBlog does not scale very well.
shellCMS stores posts in sub-folders.
In document mode, the sub-folders are named by tag (category) (and only one tag is allowed per post).
In blog mode, the sub-folders are named by post-date, in format yyyymm, for example 201708.

Another reason that BashBlog does not scale well, is that a page view of all posts with a particular tag, can be very big, if, for example, there are 1,000 posts. In fact, this is a horrendous situation, as an enormous page is created.
shellCMS will only create a page showing the first 8 of those 1,000, with "Older" (and "Newer") links. Up to 5 older pages can be viewed, past which a history summary page is displayed.

Tags: shellcms