Why I love to program as a writer

Paraphrasing Alan Kay: Literate Programming to me means only prose and code in order of human comprehension, with the order of the code required by the compiler being irrelevant. It is possible to do in Emacs Org-mode and in Noweb. There are possibly other systems in which this is possible, but I'm not aware of them.

Lately I have been programming two big systems. One is a project management software and the other is an Eiffel library. And that made me appreciate the gifts of Literate Programming.

The two gifts of literate programming

I went back to my Eiffel library after 5 months of not touching it. Of course my mind was blank when reading the code.

Fortunately for me I have programmed it in literate programming style. So I had 1300 lines of prose and code to explain to myself. What I had decided and why!!

It was a gift from my past self to present me. The gift of understanding. The gift of knowing that when I come back to this software, I can see why it's designed the way it is.

The second gift is the gift of focus, focus on the goals that matter. Most software is programmed to work in layers. We have specialized tools for each layer: Database Manager for the data layer, coding editors for the business logic, and design tools for the user interface. We have specialized professionals for each layer, Database Administrators, Programmers, and User Experience.

However our users don't think in terms of layers. Our users have goals. And our software must help them achieve their goals.

So the first thing we need to change is to think in terms of goals not tools. And the way I have found to do that is to organize my thinking as an article. Where the goal is the first paragraph. Subsequent paragraphs and code blocks are written in service of the goal. This of course means that current tools are not useful in this way of writing. I cannot write SQL in the python interpreter, nor CSS in the command prompt. But I need all of these languages and tools mixed in order to fulfill the goal of my user.

Example of a literate program

Our goal is to show the users stored in a SQLite database. So we would write an article as follows:

#+title: How to Use SQLite with Python

In this tutorial we would learn how to connect SQLite and Python. (goal)

* Requirements
First of all you need to check if you have Python and Sqlite installed.
So write this command in your terminal.
And they should return an output similar to the one shown.

#+name: (pre-conditions)
#+begin_src shell :exports both :results output
,  python --version
,  sqlite3 --version
#+end_src

#+results:
: Python 3.10.12
: 3.37.2 2022-01-06 13:25:41 872ba256cbf61d9290b571c0e6d82a20c224ca3ad82971edc46b29818d5dalt1


* Setup (installation)
#+begin_src shell
,  pip install cs50
,  touch database.db
#+end_src

* Connecting

#+begin_src python :tangle main.py :mkdirp yes
,  from cs50 import SQL

,  db = SQL("sqlite:///database.db")
#+end_src

* CREATE
#+name: (a_code_block)
#+begin_src python :tangle main.py
,  db.execute("""
,      CREATE TABLE IF NOT EXISTS users (
,          name     TEXT,
,          age      NUMBER,
,          fav_food STRING
,      )
,  """)
#+end_src


* Tangle and Run                                                   :noexport:
#+name: (compile)
#+begin_src shell
,  emacs --batch \
,        --eval "(require 'ob-tangle)" \
,        --eval "(org-babel-tangle-file \"python-sqlite-tutorial.org\")"
,  python main.py
#+end_src

Observe that:

goal: First I start with the goal or user story this document will satisfy.
pre-conditions: Every program has some assumptions about its environment. I like to make those assumptions explicit. Because in 6 months when this doesn't compile I need to check if the requirements were fulfilled.
installation: In this day and age any functionality requires some library that the Open Source community has done for us.
a_code_block: This is where the actual work begins, sometimes I start deductive (top down). With a general idea of the code as it could work. Like an outline of the code I'm going to write. Other times it's inductive (bottom-up). I start with a specific example and modify it until it can handle the general case.
compile: Going back to the context, we compile our program and we test against our expectations.

But with a powerful literate programming editor like Emacs, everything is in the same view. Our user goals are prose, environment configuration are blocks, business rules are code. Everything in the same document, so just scrolling up and down is enough to understand how this goal is realized in every layer. For my brain, this has been a lifesaver. This state of flow in the age of distraction, has made me so much more productive.

Discarded alternatives

Obviously Emacs and Noweb aren't the only tools in the game. But I have tried them all It's hyperbole, but I have tried a good chunk. . And all of them fall short in some way.

In general, and the most damaging, is that every single tool in this list treats a paragraph as an individual node in a graph of ideas. But that separation, dilutes the goal in many little files. To the point that each file doesn't have an idea of the single goal you were trying to achieve.

I do not want to criticize any one tool, instead I'm going to mention my favorite tools, that aren't emacs. And why I don't use them to do literate programming.

Leo-Editor and its clone LeoJS

These tools are almost perfect. Their literate programming capabilities are top notch. Their node organization is amazing. But, they separate each node, either code or prose in their own little note. And that is the problem. I start with a mixed set of ideas (prose) and code. And I want the editor to separate them into nodes. And then create the document for each audience:

The SQL files for the database manager.
the code for the compiler and
the prose for my fellow programmers or, more likely, future me.

Tiny Wiki.

On the opposite side we have tiny wiki. Which is the perfect note taking tool. Every note is just a block in a single long document. And it supports transclusion Which is the fancy way of saying it inserts blocks from another document into the current one. . Which makes it perfect to take little ideas that you have accumulated along the years. And assemble a coherent whole. But its literate programming story is a lot less good.

Jupyter Notebooks.

I personally don't like Jupyter notebooks. I think their data format is good. But they have gone too far on the other direction. Since you don't have a tangle operation i.e., assembling the code out of the fragments that you have written. Then Jupyter notebooks end up being the only environment in which your code can be executed. But that destroys the layers and the fact that each layer is perfectly optimized for its function.

How to become a literate programmer.

To learn how to become a literate programmer, I think a good exercise is to take any programming tutorial that you already know or have read in the past It is important that this is a tutorial that you have done in the past. Your goal is to learn literate programming not the contents of the tutorial. . A tutorial whose prose and content you have already enjoyed.

And type the whole tutorial again, no steps omitted. Every time the tutorial said: "and now you execute our program" or where it said: "in the browser click here and there" Now you must find a way to write an executable code block that will be part of your document.

This practice is great to learn to write in a literate programming style, since you would be wearing the shoes of both an experienced writer and programmer.