2021-01-06

Your Own Publishing Portal

Since I now dual post a lot of material to both the web and geminispace I've built tools to accomplish this. I thought I'd share some of this, because the issue of how to easiest or best publish ones stuff often comes up. This is *one* solution, and it works very well for me. Hopefully there's something you can take away from it too. Consider any code examples here to be MIT licensed.

Two+ Parts

Any executable file that I publish here with the .cgi suffix will be executed by the server when called. Your mileage may vary here; if you're on a tilde CGI scripts may be disallowed, or more often all located in ~user/public_html/cgi-bin -- check with your friendly sysadmin what applies, and place your file where it needs to be.

My publishing CGI script will serve up a HTML Form when called with an HTTP GET method, and handle input data to publish the way I want it to be published when called with an HTTP POST method. You may see this as two parts: the form and the publishing logic. In reality it's a lot more parts than that, because I decided to break my publishing routine into smaller parts and make a small tool for each. You'll see what I mean.

The Form and Main Script

Here's my publishing CGI script, warts and all:


#!/bin/bash

AUTH_TOKEN_SUM="[[[ HERE BE SECRET STUFF! ]]]"

echo "Content-Type: text/html"
echo
echo
echo '<!DOCTYPE html>
<html>
  <head>
    <style>
      body {
        max-width: 45em;
        margin: auto;
      }
    </style>
    <meta charset="utf-8">
    <title>Post Editor</title>
  </head>
  <body>'

# Serve a form, if nothing was submitted
if [[ $REQUEST_METHOD == "GET" ]]; then
echo '    <form action="editor.cgi" method="post">
      <label for="authtoken">Auth token: </label><input type="password" name="authtoken" id="authtoken"><br/>
      <label for="posttext">Post text:</label><br/>
      <textarea name="posttext" id="posttext" style="width: 100%" rows="20">Type here</textarea><br/>
        <input type="radio" name="publish" id="gemini" value="gemini">
        <label for="gemini">Gemini</label>
        <input type="radio" name="publish" id="html" value="html">
        <label for="html">HTML</label>
        <input type="radio" name="publish" id="both" value="both">
        <label for="other">Both</label><br/>
      <input type="submit" value="Submit">
    </form>'

# When something is submitted, treat it and post
elif [[ $REQUEST_METHOD == "POST" ]]; then

POST_DATA=$(cat -)
POST_VARS=(${POST_DATA//&/ })

for i in "${POST_VARS[@]}"
do 
        STRING=${i//+/ }
        POST_VAR_VALUE=$(echo -e "${STRING//%/\\x}")
        case "${POST_VAR_VALUE}" in
                authtoken=* ) AUTH_TOKEN="${POST_VAR_VALUE/authtoken=/}";;
                posttext=* ) export POST_TEXT="${POST_VAR_VALUE/posttext=}";;
                publish=* ) PUBLISH_TO="${POST_VAR_VALUE/publish=/}";;
        esac
done

if [[ $AUTH_TOKEN_SUM != $(echo "${AUTH_TOKEN}" | sha256sum | awk '{print $1}') ]]; then
        # Unauthorized!
        echo "<html><head><style>body { max-width: 45em; margin: auto;}</style><meta charset="utf-8"><title>Post Editor</title></head><body><h1>Authentication Failed!</h1><p>It appears you provided invalid credentials.</p></body></html>"
else
        # Authorized! :D
        export POST_TITLE=$(echo "${POST_TEXT}" | grep -m 1 -E '^# ' | sed 's;^# ;;')
        export POST_FILENAME=$(echo "${POST_TITLE}" | grep -oiE '[a-z0-9]+' | xargs echo | sed -e 's; ;-;g' -e 's;[A-Z];\L&;g')
        cd ~/Journals

        echo "${POST_TEXT}" > "allposts/${POST_FILENAME}.txt"

        if [[ ${PUBLISH_TO} == "html" || ${PUBLISH_TO} == "both" ]]; then
                for script in $(find html -type f | sort); do
                        $script
                done
        fi

        if [[ ${PUBLISH_TO} == "gemini" || ${PUBLISH_TO} == "both" ]]; then
                for script in $(find gemini -type f | sort); do
                        $script
                done
        fi
fi
fi

echo '  </body>
</html>'

FFS! That's 80 lines of ugly bash! Explain!

Heh. Yeah.

Alright, the important bits:

Obviously there's an AUTH token. If you can block access to this script with HTTP Basic Auth (through .htaccess or similar) then that's better. This isn't by any means perfect, but it will stop random bots from posting stuff on your blog/gemlog. The authtoken field in the HTML Form is of the sort "password" because that way your browser will ask if you want to save it to your keyring and can auto-fill it later. Pick a random string that's not used as a password anywhere else. Save the sha256sum of it in the AUTH_TOKEN_SUM variable.
The data you receive will be percent encoded, but spaces will be +. The STRING=${i//+/ } and POST_VAR_VALUE=$(echo -e "${STRING//%/\\x}") replace the + signs and decodes the data, respectively. As the variables are separated by & we split them up first with POST_VARS=(${POST_DATA//&/ })
If the wrong AUTH Token was provided, nothing is published. If it's correct we'll change our working directory to ~/Journals -- this works on my server because the CGI script is executed as me and has access to my home directory. If the web server you're on does not do setguid/setuuid for CGI scripts you may have to place your tools elsewhere (maybe even in some subdirectory in ~/public_html).
In the ~/Journal directory I have a few subdirectories, the most important of which are "html" and "gemini". These contain the publishgin scripts for each of those, and these will be executed in alphabetical order.

My Publishing Scripts

I realised that I want to do a whole bunch of different things when I publish a post:

Add the raw text to a folder, if I want to rebuild my site differently later.
Convert raw text (written in text/gemini syntax) to HTML for the web side of things, publish the post in both cases.
Update the Archive page (check /posts/index.html or /posts/index.gmi on this site to see what I mean).
Update the Atom feed.
Update the index page with a new Latest Posts listing.

As you see these are pretty isolated things. By executing them one by one and providing the needed data (POST_TEXT and POST_TITLE) as environment variables they can be written and tested in that order without interfering with each other. Each script does one thing, the way *I* want it done. Some are written in bash, some are written in python. I won't show them here, because this is the part you really need to build yourself for your own setup. My gemlog is as spartan as they go, and my blog mimics that; I don't expect anyone else to want such a minimalist web setup.

One thing you should be aware is that a lot of data coming from your browser will contain CRLF (\r\n) line endings, because of legacy web reasons. I have dos2unix calls in a lot of places to correct that.

Let me know if you see any glaring problems with my code, or have any questions that I might be able to answer 🙂️

-- CC0 Björn Wärmedal