Creating a website generator to generate my website

I like to write. I mostly write code, but sometimes I get so excited about things I discover that I feel the need to share it! That’s why I also write blog posts like this.

But this post is special: it’s the first one that I am writing using a static website generator that I wrote myself.

I called it Magnanimous because that’s the first kind of unique word I could come up with that was associated with something great¹.

D. Pedro II, The Mananimous. Emperor of Brazil from 1831 to 1891

The reason I decided to abandon my previous blog post platform, Google Sites, was the fiasco with code samples in my last blog post on it: Fearless Concurrency….

Even though that blog post was one of my most popular posts ever, the Google Sites editor destroyed some of the source code sample’s indentation for some reason, so many of the samples were quite hard to read! The samples were supposed to show the beauty of concurrent code in languages like Erlang and Pony, where indentation actually matters a lot! I tried all I could think of to fix the problem, but just couldn’t do it (the generted HTML is impossible to edit directly, so a bug in the editor is nearly impossible to circumvent).

I never really liked Google Sites anyway… my old website looked outdated and ugly (and the new look they are rolling out didn’t even support source code highlighting last I tried)… I only coped with that because adding new content was so convenient! I still needed something to push me over the limit, and that definitely did it.

I had tried a few static website generators before (Jekyll, Hugo), but didn’t like them². But I knew that a static website generator was the ideal solution for me because I wanted to make sure I controlled the content and how it was presented, and because of the plethora of options to host such websites for free (Github Pages, Netlify, Digital Ocean etc).

So why not just write a simple solution myself to do only what I needed?! Should take a weekend or two, I thought 😀!

Thus, I spent a few weeks, which turned into a few months, doing it. But I finally got to a place where I think Magnanimous is usable, and I am ready to not only write my own website on it, but write about it so that others may also give it a go.

It even got a logo (thanks to a bored marketing colleague on the last day on the job):

Footnotes:

2 I wrote a couple of blog posts on Jekyll when GitHub started promoting it as the default tool for GitHub Pages… after some time passed, I pushed a tiny update to one of the blog posts, which caused all of my code samples to lose code highlighting… it was some update on the libraries or something, but that cost me several hours to fix, enough to make me avoid Jekyll at all costs. I also used Hugo for my RawHTTP project’s website - and the problem is that they tried to make things too magical… trying to customize themes is painful and most themes do things their own undocumented way, so you have to actually check the source code to see how to change something. Not cool.

The Journey

Choosing a language

I chose the Go Programming Language because I wanted something that can run a short-lived process as quickly as possible. Go compiles to machine code directly and is one of the only languages that can cross-compile to any other platform with only its own toolchain. It creates a single, statically linked binary, so installation is trivial on any OS (just download the file). It’s no surprise it has become extremely popular for developing command-line utilities (such as Magnanimous).

I had already written a few applications in Go, so I was quite productive from the start. Go rarely gets in your way, even if it famously lacks abstraction tools common in most other languages, like generics and exceptions.

Inventing a templating system

The hard part was designing the template’s expression language. There’s a lot of templating engines out there, but the ones I looked at are not designed to perform file cross-referencing the way I envisioned.

For example, I wanted to be able to:

• include a file into another.
• define variables that can be used in included files, so the included content can be customized.
• iterate over files in a certain directory to create a content summary.
• write content in markdown.
• include highlighted code samples in many languages.

It became clear to me that I would need to create my own expression language to do all these things.

At the time, I was learning Racket, so I took some inspiration from Lisp.

As an example of what Magnanimous instructions look like, to define a variable in Magnanimous, you do:

{{ define name "world" }}

To actually include the contents of a variable in the current file:

Hello {{ eval name }}

I intentionally limited the language so that it would not become a Turin-complete language. For example, it is not possible to nest instruction blocks or create functions. Every block has the same basic form:

{{ instruction args }}

The acceptable args are determined by the actual instruction. Even the syntax of the arguments can be determined by the instruction, so it is possible, for example, to refer to another file by its path without using a quoted string:

{{ include path/to/another.file }}

Many instructions accept simple expressions as arguments. For example:

{{ define componentsDir basePath + "/components" }}

Shortcuts

Despite the Lisp inspiration, I decided to use C-like expressions for two reasons: they are easier to understand for most people (compared to Lisp prefix notation), and I could use the Go built-in parser to parse them, saving me the effort to parse general expressions.

I will probably change this at some point, but not having to write an expressions parser from scratch allowed me to save a few weekends’ work, and keep me motivated to get something working within a reasonable time frame.

I still had to evaluate the AST produced by the parser, but that was fairly easy.

Converting markdown to HTML was also almost trivial thanks to Go libraries being available for everything I needed: Blackfriday to convert markdown to HTML, and Chroma for code highlighting (easily integrated with Blackfriday thanks to bfchroma) for virtually any programming language.

func writeAsHtml(c []Content, writer io.Writer, files WebFilesMap, stack ContextStack) error {
	if len(c) == 0 {
		return nil
	}
	mdBytes, err := asBytes(c, files, stack)

	var chromaRenderer = blackfriday.WithRenderer(
		bfchroma.NewRenderer(bfchroma.WithoutAutodetect(), mdStyle))

	html := blackfriday.Run(mdBytes, chromaRenderer)

	_, err = writer.Write(html)
	if err != nil {
		return &MagnanimousError{Code: IOError, message: err.Error()}
	}
	return nil
}

Go sample code showing Chroma code highlighting. The code shown is actually used by Magnanimous to write a Markdown file as HTML.

In a few days I had something that looked useful, so I started writing Magnanimous’ own website using Magnanimous itself.

The big refactoring

Upon actual usage, I found out the way I had designed variable scopes was wrong and, sometimes, variables didn’t get resolved the way a user would expect. My mistake was to try to resolve each file’s scope during parsing. That’s not possible because any file can be included in any other file, so the scope of a file can only be resolved when actually writing the file (I was trying to clean up and then reset the scope on each write, but that was complicated and error-prone). To fix this would require a big refactor of the source code.

But because by now there was already a few thousand lines of code written, refactoring would be a big task… this hiccup de-motivated me enough to set the project aside and switch to more interesting, easier adventures for a few months!

But I did find energy to come back and start the refactoring. Go may not have a powerful type system, but at least it has one! And that made the refactoring a mostly mechanical process: once the code compiled again, it would most likely work - and I had written enough tests to give me confidence of that.

After 6 days (this is a side project, so I was working on the weekend and evenings), I finally created a pull request to go back to the main branch with the main scoping problem completely solved, and a few other nice clean-ups as well.

Documenting

After that, it was a matter of documenting Magnanimous in full - if it’s not documented, it doesn’t exist for all but the most dedicated users (who are willing to look into the source code to figure things out)!

I spent a long time on the Docs, including writing a Tutorial to make the core concepts clear - not only for new users, but for myself, as doing that really helps to make sure things are clean and consistent, which is easy to mess up when you’re the only user and author.

I made a few improvements as a result of writing the basic tutorial and the (thus far, unpublished) internationalization tutorial.

You can check out the results on this website you’re reading, on the Magnanimous Website and on the Basic Tutorial’s Demo Website.

The road ahead

I am very happy with the results of both my website (it may not look incredible, but I can make it look like anything I want) and the Magnanimous project!

I am not sure if anyone else will like and start using Magnanimous to build their websites, and maybe even start contributing to it (I wrote a CONTRIBUTING file just in case). It doesn’t matter, really. The important for me is that I now have the tool I wanted to create pure HTML/CSS websites (not sure yet if I will ever add JavaScript to the mix, but if you like that, you can do it on your own websites, of course - just notice that you might not need it). And, of course, I learned a lot doing it, and it was a lot of fun.

There’s still a few things I wish Magnanimous had, like an easy way to include a ToC (Table of contents) and maybe add a RSS feed option to websites created with it.

But I have a few ideas already to make that possible, and from my experience, adding features to Magnanimous is a breeze!

Maybe you’d like to give that a go? Let me know if you do… create an issue on GitHub and we can discuss it.

For now, though, I will probably focus on making this website look nicer, and perhaps add some non-blog-post content to it.

Thanks

Thanks for reading and hope you come back for my next blog posts.

Big thanks to Netlify for providing free hosting and seamless https support via Let’s Encrypt.

Also thanks to BitBucket for hosting some of my private projects, including the source for this website.