Convert markdown links to html with Pandoc

Question

In my new project I have multiple markdown files which are linked to each other. These links refer to the original .md files.

Example: File README.md

...
1. [Development documentation](Development.md)
1. [User documentation](Usage.md)
...

If I convert these files with Pandoc, e.g. to html files, all links are still pointing to the original .md file. What I'm looking for is a way to convert also the link type, which means that output files should refer to the output file type such as HTML, PDF, TeX etc. Is there a way to convert the internal link type with Pandoc?

I use this to convert the files:

pandoc -f markdown -t html5 input.md -o output.html

Solution 1

You can create a filter that checks every link element and--if the url ends with .md--replaces it with .html.

Example with Python, using the panflute package:

import panflute as pf

def action(elem, doc):
    if isinstance(elem, pf.Link) and elem.url.endswith('.md'):
        elem.url = elem.url[:-3] + '.html'
        return elem

if __name__ == '__main__':
    pf.run_filter(action)

Solution 2

Example with the built-in Lua filters:

# links-to-html.lua
function Link(el)
  el.target = string.gsub(el.target, "%.md", ".html")
  return el
end

Then:

pandoc -f markdown -t html5 input.md -o output.html --lua-filter=links-to-html.lua

Solution 3

I had a similar problem, so I made this: https://github.com/MatrixManAtYrService/md_htmldoc

It finds all of the .md files in a directory and then makes a separate directory where all the markdown has been converted to html.

It fixes hyperlinks (thanks to @Sergio_Correia's answer).

It also gathers up any local file references so that links to images and such still work

Solution 4

Assuming you are going to serve you html pages via webserver it is relatively simple to resolve all *.md urls as *.html ones instead of rewriting them via pandoc, e.g. using NGinx:

location ~ \.md$ {
  if (!-f $request_filename) {
    rewrite ^(.*)\.md$ $1 permanent;
  }
}

location / {
  try_files /$uri /$uri.html;
}

Alternatively, you can replace all md links with html using sed (taken from here):

Change all internal file urls from pointing to *.md links and instead point to the local *.html file

1. recursively run this sed command (programatically replace FILENAME)

   sed -n -i.bak '/href="\./s/\.md/\.html/' FILENAME.html
   

2. alternatively, run the following command instead (programatically replace FILENAME)

   sed -e '/href="\./s/\.md/\.html/' FILENAME.html > FILENAME.html.tmp && mv FILENAME.html.tmp FILENAME.html`
   

View more solutions

Comments

[P.Tail]

Thanks, that's great! I am just wondering if there's no simple way by using Pandoc options?

[Waylan]

@P.Tail no, there is no out-of-the-box way to do that. Markdown assumes that your links already point to the rendered HTML documents and does not alter your URLs. In fact, any altering of URLs would be a bug. Of course, using a custom-built plugin (like the one here) is possible with various Markdown parsers. But it has to be custom because only you know your specific needs and no single solution could possibly meet the needs of most (let alone all) users.

[hoijui]

I would not recommend this approach, because there could be link in code sections which you would not want to convert, and similar other complications you might not think of at the moment, and which you automatically circumvent if you use an other tool for parsing, like pandoc.

[hoijui]

I saw in your code (get_references.py), that you use regex to find links in markdown. I would not recommend this approach, because there might be links in code sections which you would not want to convert, and similar other complications you might not think of at the moment. you could automatically circumvent this, if you always use an other tool for parsing, like pandoc (as you also do per panflute).

[Stefan]

@hoijui: get_references.py isn't used to convert something - it's just used to find references. Every found reference gets checked if it refers to a local file - if yes, then it gets added to doc_relevant, which then is used to (1) either compile a markdown to html or (2) copy that file to HTML_DIR

[hoijui]

ok, thanks :-) still, you might miss references this way, for the reasons mentioned above. Only building an AST can do it properly (which is what pandoc does, for example). using filters and AST(raw?) output with pandoc, it is quite easy to do.

[Piper]

Note that # is not a valid Lua comment. Use -- if you want to keep the first line.