felixfoertsch.de/Content/Publishing System.md

Publish, publish, published!

A dive into Publishing Systems

Ready made knowledge can only be memorized. Knowledge is not truly our own, until we are capable of reproducing the given content in a form of our own making. Memorizing is but a negative condition. True or organic assimilation is impossible without inner transformation of what we learned.

All rules for study are summed up in this one: Learn only in order to create. Only by his divine capacity for production is man truly man. Without it no more than a tolerably well devised machine.

— Friedrich Schelling

Some Requirements

I have so many ideas, but all of them have one thing in common: they have to be documented nicely. Only what's written is permanent. And only what's permanent is real. I need a system that helps me to publish my thoughts. Here are some additional requirements:

• Markdown (or equivalent)
• Needs to help me organize my thoughts
• Needs to be publishable
• Needs to be convertible into book/thesis form (I want to write master thesis with it)
• Needs to integrate into spaced repetition system?? -> Work through something, publish it, make it learnable?
• Write math in Unicode symbols?

One of the most important things to really understand something is to actually apply it. If application is not possible or straight-forward, an alternative is teaching someone else. The important aspect is the transformation. Transforming existing knowledge into your knowledge.

A good way that is completely independent of other people is simply forcing yourself to publish your transformed knowledge. It doesn’t matter if they are being read, applied, or anything else. You are publishing for your own sake.

I have been using different blogging tools all my life. The latest one I tried was WriteFreely. The idea is great: have a blogging system that also speaks ActivityPub. This way, the integration with Mastodon is easy as pie.

However, after using it for a while, I cannot recommend the system. The UI is horribly confusing. And then there is issues like this where a feature is only half-baked, because it is “a design decision”. Sadly, as is often the case with open source software, you reach a point where something does not work, is not well thought out, and completely dismantles your use case.

And, to be honest, by now I am convinced that horrible UI is like a badge of honor for open-source projects.

Researching Alternatives

I have not published in a while, because it is actually a hard thing to do. Learning something properly is difficult. But It is the only real way if you want to be qualified in a topic. And because is hard, encountering issues like the ones mentioned above are a hinderance for an enjoyable workflow. It can stop any motivation dead in its tracks. Therefore it is necessary to find new way to publish my knowledge and thoughts.

I have a few requirements for my new system. It needs to be able to read and convert Markdown files. I can have styling, but the source of the posts have to be files. A file based system is the only one that is really portable.

This means the system has to have one of the following:

• An idempotent way to import a folder.
• Serving directly from a folder on the fly.
• A compile and deploy step that converts a folder to a static website.

Additionally it needs a default design for a post, but it has to be possible completely change the styling of a single page, because I want to explore different designs for different topics.

It needs proper support to display mathematical formulas.

It needs to support a footnote/glossary system where terms like idempotent can be explained once and then being applied to the whole system.

Linking Publishing as book

I have been collecting potential projects that I want to have a look into:

• Quarto
• Pollen
• Scribble

These are the results of my preliminary research: Quarto is a publishing system. TODO

Pollen and Scribble share a common root. They both use Racket, a Lisp dialect, to implement the programmatic aspects of the system. However, Pollen and Scribble differ in their goals. Pollen, similar to Quarto, is a complete publishing system that aims at creating beautiful books, websites, etc. That means it can render different formats as well. It’s highly flexible and customizable. Scribble’s main purpose, however, is to publish technical documentation, which results in a less stylized appearance of the document. It’s single output format is HTML.

Decisionmaking After weighing the pros and cons a little bit, I decided on trying Pollen first. It seems to be closest aligned with my goals in publishing different ideas on different pages. If it turns out to be the wrong choice, I’ll try Quarto next.

The installation of Pollen on macOS is straight-forward. Install Racket with Homebrew then install pollen wir the Racket package manager. That’s it.

After diving deeper into Pollen, I decided against it. For one specific and maybe irrational reason: When Pollen renders files, it puts the output in the same folder. To me, that is horrible. It starts littering files. I want a clean working environment. At least it should clean up after itself, but it doesn’t.

Let’s try Quarto.

Custom System

Based on the Unix philosophy it would be smart ot encapsulate every task in a single program. This way, the system can be easily extended and modified. So we need to have a step-by-step plan to convert the source files into the target files.

• The output files have to be copied to a directory on a web server via ssh and rsync

Content

Content Structure

• Archive: Folder that contains old ff.de-versions, should not be published
• Drafts: Folder that contains articles that are in progress. No stubs, only articles, that actually contain content.
• Ideas: Folder that contains stubs for articles.
• img: Folder that contains images for articles the filenames should be Time-based UUIDs so they can be sorted
• Published: Folder that contains the published articles in the source format. They have to be converted into the target format.
• Tweets: Tiny "articles" without a title.

Content Types

• There are different types of files that have to be accounted for. - Blog posts: Have a time stamp and are published on the web in chronological order - Books: Have one root page and everything else is linked from there - Papers/Thesis: Very similar to books (probably the same thing in our context) - Glossary: Is a list of things that should be linked/underlined everywhere - Footnotes: Can be part of all texts and should be at the bottom of the page. - Bibliography: - Sources should be linked on the page in the footnotes. The Bibliography should auto-generated from the used sources. - Can have style-templates

• Markdown *.md

• Textile

• Asciidoc

Converter

• Pandoc
• Hugo
• Jekyll
• Gatsby
• Eleventy
• VuePress
• Docusaurus
• Gridsome
• Nuxt.js

Targets

• The converter converts the source using a template
• The template has to be specified in the front matter (if there is no template, use a default)
• HTML
  • Templating
• PDF
• Epub