Learn how Pratt parsers work by writing one. I assume familiarity with grammars, parsers and TypeScript.

The code is public domain. Send feedback by email.

Motivation

This is an intuitive expression grammar.

expr = expr ("+" | "-" | "*" | "/" | "^") expr
        | "-" expr
        | number
        

The problem is it's ambiguous. 1 + 2 * 3 can be parsed as (1 + 2) * 3 or 1 + (2 * 3).

The textbook way to resolve ambiguity is to convolute the grammar with associativity and precedence rules.

expr = expr ("+" | "-") term
        | term
        
        term = term ("*" | "/") factor
        | factor
        
        factor = factor "^" number
        | "-" factor
        | number
        

This grammar is not intuitive. It gets worse if your parser cannot handle left recursion. And worse as you add more operators.

The resulting syntax trees are cumbersome.

Overview

Pratt parsing uses grammars of this form.

expr = head tail*
        
        head = number
        | "-" expr
        
        tail = ("+" | "-" | "*" | "/" | "^") expr
        

And produces syntax trees of this form.

expr = expr ("+" | "-" | "*" | "/" | "^") expr
        | "-" expr
        | number
        

The parsing routine applies precedence and associativity rules to resolve grammar ambiguity.

Pratt parsers are easy to understand, implement, and integrate.

We'll build a series of gradually better parsers to explain how it works.

Precedence and Associativity

Left-Associative Parser

Start simple. Assume all operators are left-associative and without precedence.

Parse 1 + 2 + 3 * 4 * 5 as ((((1 + 2) + 3) * 4) * 5).

The expression grammar supports this with a repeated tail that immediately applies the operator to expand the left expression.

expr = number (tail_op number)*
        |    | |               |
        └head┘ └──────tail─────┘
        

The implementation reflects the grammar.

function expr(ctx) {
        let left_expr = number(next_token(ctx));
        while (has_token(ctx)) {
        const op = tail_op(next_token(ctx));
        const right_expr = number(next_token(ctx));
        left_expr = infix(op, left_expr, right_expr);
        }
        return left_expr;
        }
        

Right-Associative Parser

Now assume all operators are right-associative and without precedence

Parse 1 + 2 + 3 * 4 * 5 as (1 + (2 + (3 * (4 * 5)))).

The expression grammar supports this with a right-recursive tail that expands the expression to the right before applying the operator.

expr = number (tail_op expr)?
        

The implementation reflects the grammar.

function expr(ctx) {
        let left_expr = number(next_token(ctx));
        if (has_token(ctx)) {
        const op = tail_op(next_token(ctx));
        const right_expr = expr(ctx);
        left_expr = infix(op, left_expr, right_expr);
        }
        return left_expr;
        }
        

Mixed-Associative Grammar

Sometimes the parser should apply an operator immediately as in the first parser. Other times it should wait until later operators are applied as in the second parser.

Merge the grammars to create an ambiguous grammar that enables this choice:

expr = number (tail_op number)*   // Parser 1: left-associative
        expr = number (tail_op expr  )?   // Parser 2: right-associative
        expr = number (tail_op expr  )*   // Merged: mixed-associative
        

Take the second parser and rewrite if to while.

function expr(ctx) {
        let left_expr = number(next_token(ctx));
        while (has_token(ctx)) {
        const op = tail_op(next_token(ctx));
        const right_expr = expr(ctx);
        left_expr = infix(op, left_expr, right_expr);
        }
        return left_expr;
        }
        

This parser resolves the grammar's ambiguity by always choosing to recurse, which makes it indistinguishable from the second parser.

Respecting Associativity

From experience we know right-associative operators should recurse.

Left-associative operators should instead return the current expression to become part of the parent expression.

function expr(ctx, parent_op) {
        let left_expr = number(next_token(ctx));
        while (has_token(ctx)) {
        const op = tail_op(peek_token(ctx));
        if (assoc(op) === "left") break;
        next_token(ctx);
        const right_expr = expr(ctx, op);
        left_expr = infix(op, left_expr, right_expr);
        }
        return left_expr;
        }
        

Now the parser respects associativity but not precedence.

Respecting Precedence

Pratt parsing uses nested levels of recursion to represent nested expressions of equal or increasing precedence.

By definition, a level exits when an operator with lower precedence appears.

By definition, associativity applies when operators have equal precedence.

The parent operator determines a level's precedence. The root expression has no parent, which means minimum precedence.

// Compares the precedence of two operators
        function cmp_precedence(op1, op2): "<" | "=" | ">";
        
        function expr(ctx, parent_op) {
        let left_expr = number(next_token(ctx));
        while (has_token(ctx)) {
        const op = tail_op(peek_token(ctx));
        const order = cmp_precedence(op, parent_op);
        if (order === "<") break;
        if (order === "=" && assoc(op) === "left") break;
        next_token(ctx);
        const right_expr = expr(ctx, op);
        left_expr = infix(op, left_expr, right_expr);
        }
        return left_expr;
        }
        
        expr(ctx, Op.Root); // Expected syntax for parsing an expression.
        

This parser respects precedence and associativity.

Adding Operators

Head and Tail Parsing

To simplify future extensions, split the grammar into head and tail rules.

expr = expr_head expr_tail*
        
        expr_head = number
        
        expr_tail = tail_op expr
        

Refactor the parser accordingly.

function expr(ctx, parent_op) {
        const left_expr = expr_head(ctx);
        return expr_tail(ctx, parent_op, left_expr);
        }
        function expr_head(ctx) {
        return number(next_token(ctx));
        }
        function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        const op = tail_op(peek_token(ctx));
        const order = cmp_precedence(op, parent_op);
        if (order === "<") break;
        if (order === "=" && assoc(op) === "left") break;
        next_token(ctx);
        const right_expr = expr(ctx, op);
        left_expr = infix(op, left_expr, right_expr);
        }
        return left_expr;
        }
        

Prefix Operators

Extend expr_head to support prefix operators.

expr_head = number
        | prefix_op expr
        

Operators like - can have overloaded infix and prefix definitions because prefix_op and tail_op don't conflict in the grammar.

function expr_head(ctx) {
        const token = next_token(ctx);
        if (is_number_token(token)) {
        return number(token);
        } else {
        const op = prefix_op(token);
        const right_expr = expr(ctx, op);
        return prefix(op, right_expr);
        }
        }
        

Postfix Operators

Update expr_tail to support postfix operators.

expr_tail = postfix_op
        | infix_op expr
        

If an operator is both postfix and infix, the parser must look ahead to differentiate the two cases. Disallow this to simplify parsing.

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        const op = tail_op(peek_token(ctx));
        const order = cmp_precedence(op, parent_op);
        if (order === "<") break;
        if (order === "=" && assoc(op) === "left") break;
        next_token(ctx);
        if (is_postfix_op(op)) {
        left_expr = postfix(op, left_expr);
        } else {
        const right_expr = expr(ctx, op);
        left_expr = infix(op, left_expr, right_expr);
        }
        }
        return left_expr;
        }
        

Parentheses

Extend expr_head to support parenthesized expressions.

expr_head = "(" expr ")"
        | number
        | prefix_op expr
        

Parentheses prevent other operators from interacting with the enclosed expression, so precedence is reset.

function expr_head(ctx) {
        const token = next_token(ctx);
        if (token === "(") {
        const right_expr = expr(ctx, Op.Root);
        next_token(ctx); // consume ")"
        return right_expr;
        } else if (is_number_token(token)) {
        return number(token);
        } else {
        const op = prefix_op(token);
        const right_expr = expr(ctx, op);
        return prefix(op, right_expr);
        }
        }
        

) tokens appear where the parser otherwise expects tail operators. They mark the end of an expression.

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        const token = peek_token(ctx);
        if (token === ")") break;
        const order = cmp_precedence(op, parent_op);
        if (order === "<") break;
        if (order === "=" && assoc(op) === "left") break;
        next_token(ctx);
        if (is_postfix_op(op)) {
        left_expr = postfix(op, left_expr);
        } else {
        const right_expr = expr(ctx, op);
        left_expr = infix(op, left_expr, right_expr);
        }
        }
        return left_expr;
        }
        

Subscript Operator

The (array) subscript operator [ resembles a postfix operator because there is no right argument.

Brackets prevent other operators from interacting with the enclosed expression.

expr_tail = infix_op expr
        | postfix_op
        | subscript_op
        
        subscript_op = "[" expr "]"
        

Like ) tokens, ] tokens mark the end of an expression.

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        const token = peek_token(ctx);
        if (token === ")" || token === "]") break;
        const op = tail_op(token);
        const order = cmp_precedence(op, parent_op);
        if (order === "<") break;
        if (order === "=" && assoc(op) === "left") break;
        next_token(ctx);
        if (op === Op.Subscript) {
        const middle_expr = expr(ctx, Op.Root);
        next_token(ctx); // consume "]"
        left_expr = subscript(left_expr, middle_expr);
        } else if (is_postfix_op(op)) {
        left_expr = postfix(op, left_expr);
        } else {
        const right_expr = expr(ctx, op);
        left_expr = infix(op, left_expr, right_expr);
        }
        }
        return left_expr;
        }
        

Ternary Operator

The ternary operator left ? middle : right resembles an infix operator because it has an unenclosed right argument.

The middle expression is enclosed so other operators cannot affect how it is parsed.

expr_tail = infix_op expr
        | postfix_op
        | subscript_op
        | ternary_op
        
        ternary_op = "?" expr ":" expr
        

: tokens mark the end of an expression.

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        const token = peek_token(ctx);
        if (token === ")" || token === "]" || token === ":") break;
        const op = tail_op(token);
        const order = cmp_precedence(op, parent_op);
        if (order === "<") break;
        if (order === "=" && assoc(op) === "left") break;
        next_token(ctx);
        if (op === Op.Ternary) {
        const middle_expr = expr(ctx, Op.Root);
        next_token(ctx); // consume ":"
        const right_expr = expr(ctx, op);
        left_expr = ternary(left_expr, middle_expr, right_expr);
        } else if (op === Op.Subscript) {
        const middle_expr = expr(ctx, Op.Root);
        next_token(ctx); // consume "]"
        left_expr = subscript(left_expr, middle_expr);
        } else if (is_postfix_op(op)) {
        left_expr = postfix(op, left_expr);
        } else {
        const right_expr = expr(ctx, op);
        left_expr = infix(op, left_expr, right_expr);
        }
        }
        return left_expr;
        }
        

Ternary Operator with Optional Else

Modify the ternary operator so : right is optional.

Pair : to the closest unpaired ? so that a ? b ? c : d parses as a ? (b ? c : d).

ternary_op = "?" expr (":" expr)?
        

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        // ...
        if (op === Op.Ternary) {
        const middle_expr = expr(ctx, Op.Root);
        let right_expr = null;
        if (peek_token(ctx) === ":") {
        next_token(ctx); // consume ":"
        right_expr = expr(ctx, op);
        }
        left_expr = ternary(left_expr, middle_expr, right_expr);
        }
        // ...
        }
        return left_expr;
        }
        

Non-Associative Operators and Relative Precedence

Non-Associative Operators

Is << left-associative or right-associative?

Since it's not obvious, forgo associativity and instead require disambiguating parentheses.

a << b passes because it has one interpretation but a << b << c fails as ambiguous and requires parentheses to fix.

function error_assoc(op) {
        return new Error(
        `Operator "${op}" is not associative and requires parentheses`,
        );
        }
        
        function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        // ...
        const order = cmp_precedence(op, parent_op);
        if (order === "<") break;
        if (order === "=") {
        if (assoc(op) === "left") break;
        if (assoc(op) === "none") throw error_assoc(op);
        }
        // ...
        }
        return left_expr;
        }
        

Relative Precedence

Is << or ** higher precedence?

Since global precedence is confusing, let's instead define relative precedence between operator pairs and require parentheses for unrelated operators.

For example, if Precedence(**) > Precedence(/) is the only known precedence relationship, then expect the following parse results:

The parser's cmp_precedence() now returns "!" for unrelated operators.

function error_unrelated_ops(op1, op2) {
        return new Error(`"${op1}" is unrelated to "${op2}"`);
        }
        
        function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        // ...
        const order = cmp_precedence(op, parent_op);
        if (order === "!") throw error_unrelated_ops(op, parent_op);
        if (order === "<") break;
        if (order === "=") {
        if (assoc(op) === "left") break;
        if (assoc(op) === "none") throw error_assoc(op);
        }
        // ...
        }
        return left_expr;
        }
        

Precedence Groups

Defining the relative precedence between every pair of operators is unscalable in languages with many operators.

One solution is to organize operators into groups that share the same associativity and relative precedence.

Let's define a variety of operators, group them, and define each group's relative precedence.

Given precedence P, P(Multiplication) > P(Comparison) because precedence is transitive: P(A) > P(B) and P(B) > P(C) imply P(A) > P(C).

Precedence groups form a digraph. Note how BitwiseShift is unrelated to Exponentiation, Multiplication, and Addition.

Implementation

Encode the precedence table in TypeScript.

const enum Op {
        PostfixPlusPlus,
        PostfixMinusMinus,
        Subscript,
        // ... remaining operators
        }
        const enum Group {
        Postfix,
        Prefix,
        BitwiseShift,
        // ... remaining groups
        }
        const groups = [
        {
        id: Group.Postfix,
        ops: [Op.PostfixPlusPlus, Op.PostfixMinusMinus /* , ... */],
        assoc: "none",
        gt: [Group.Prefix],
        },
        {
        id: Group.Prefix,
        ops: [Op.PrefixPlusPlus, Op.PrefixMinusMinus /* , ... */],
        assoc: "none",
        gt: [Group.BitwiseShift, Group.Exponentiation],
        },
        {
        id: Group.BitwiseShift,
        ops: [Op.BitwiseShiftLeft, Op.BitwiseShiftRight],
        assoc: "none",
        gt: [Group.Comparison],
        },
        // ... remaining groups
        ];
        

Precompute mappings from operators to groups and associativities.

const map_op_to_group = new Map<Op, Group>();
        const map_op_to_assoc = new Map<Op, "left" | "right" | "none">();
        
        for (const group of groups) {
        for (const op of group.ops) {
        map_op_to_group.set(op, group.id);
        map_op_to_assoc.set(op, group.assoc);
        }
        }
        

assoc() is a map lookup.

function assoc(op) {
        return map_op_to_assoc.get(op)!;
        }
        

For each precedence group, precompute lower precedence groups using depth-first search.

const precedence_gt = new Map<Group, Set<Group>>();
        
        for (const src of groups) precedence_gt.set(src.id, new Set(src.gt));
        for (const [_, dsts] of precedence_gt) {
        const explore = [...dsts];
        while (explore.length) {
        const dst = explore.pop()!;
        dsts.add(dst);
        const next_dsts = precedence_gt.get(dst)!;
        for (const new_dst of next_dsts) {
        if (!dsts.has(new_dst)) {
        explore.push(new_dst);
        }
        }
        }
        }
        

cmp_precedence() refers to the map.

function cmp_precedence(op1, op2) {
        const group1 = map_op_to_group.get(op1)!;
        const group2 = map_op_to_group.get(op2)!;
        if (group1 === group2) return "=";
        if (precedence_gt.get(group1)!.has(group2)) return ">";
        if (precedence_gt.get(group2)!.has(group1)) return "<";
        return "!";
        }
        

Wrap Up

Error Handling

Handle errors by checking for unexpected tokens.

Start with the head parser.

function error_bad_token(token) {
        return new Error(`Bad token "${token}"`);
        }
        
        function expr_head(ctx) {
        const token = next_token(ctx);
        if (token === "(") {
        const right_expr = expr(ctx, Op.Root);
        const rparen = next_token(ctx);
        if (rparen !== ")") throw error_bad_token(rparen, ")");
        return right_expr;
        } else if (is_number_token(token)) {
        return number(token);
        } else if (is_prefix_op_token(token)) {
        const op = prefix_op(token);
        const right_expr = expr(ctx, op);
        return prefix(op, right_expr);
        } else {
        throw error_bad_token(token ?? "EOF");
        }
        }
        

And now the tail parser.

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        const token = peek_token(ctx);
        if (token === ")" || token === "]" || token === ":") break;
        const op = tail_op(token);
        if (op === undefined) throw error_bad_token(token);
        const order = cmp_precedence(op, parent_op);
        if (order === "!") throw error_unrelated_ops(op, parent_op);
        if (order === "<") break;
        if (order === "=") {
        if (assoc(op) === "left") break;
        if (assoc(op) === "none") throw error_assoc(op);
        }
        next_token(ctx);
        if (op === Op.Ternary) {
        const middle_expr = expr(ctx, Op.Root);
        let right_expr = undefined;
        if (peek_token(ctx) === ":") {
        next_token(ctx);
        right_expr = expr(ctx, op);
        }
        left_expr = ternary(left_expr, middle_expr, right_expr);
        } else if (op === Op.Subscript) {
        const middle_expr = expr(ctx, Op.Root);
        const rbracket = next_token(ctx);
        if (rbracket !== "]") throw error_bad_token(rbracket);
        left_expr = subscript(left_expr, middle_expr);
        } else if (is_postfix_op(op)) {
        left_expr = postfix(op, left_expr);
        } else if (is_infix_op(op)) {
        const right_expr = expr(ctx, op);
        left_expr = infix(op, left_expr, right_expr);
        }
        }
        return left_expr;
        }
        

Grammar

This is the final grammar.

expr = expr_head expr_tail*
        
        expr_head = number
        | prefix_op expr
        
        expr_tail = infix_op expr
        | postfix_op
        | "[" expr "]"
        | "?" expr (":" expr)?
        

The parser produces syntax trees of this form.

expr = number
        | prefix
        | postfix
        | infix
        | ternary
        | subscript
        
        prefix = prefix_op expr
        postfix = expr postfix_op
        infix = expr infix_op expr
        ternary = expr "?" expr (":" expr)?
        subscript = expr "[" expr "]"
        

Parser

Code for the final parser is here It includes a scanner and makes greater use of types and enums.

Alternative code that uses an explicit stack (instead of the call stack) is here.

Run tests with deno test parser.ts.

Binding Power

Other posts explain Pratt parsing using "binding power."

Each operator is assigned a left and right binding power based on its precedence and associativity:

The condition for exiting a level is simplified to a binding power comparison.

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        // ...
        if (binding_power_right(op) < binding_power_left(parent_op)) break;
        // ...
        }
        return left_expr;
        }
        

I think binding power is less intuitive than the equivalent precedence and associativity checks.

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        // ...
        const order = cmp_precedence(op, parent_op);
        if (order === "<") break;
        if (order === "=" && assoc(op) === "left") break;
        // ...
        }
        return left_expr;
        }
        

I think requiring parentheses on expressions that are ambiguous to humans is good language design.

function expr_tail(ctx, parent_op, left_expr) {
        while (has_token(ctx)) {
        // ...
        const order = cmp_precedence(op, parent_op);
        if (order === "!") throw error_unrelated_ops(op, parent_op);
        if (order === "<") break;
        if (order === "=") {
        if (assoc(op) === "left") break;
        if (assoc(op) === "none") throw error_assoc(op);
        }
        // ...
        }
        return left_expr;
        }
        

But binding power requires operators to have global precedence and associativity.

• Canned chickpeas, 1 can/15oz/425g
• Tahini, 1/3 cup
• Olive oil, 2 tablespoons
• Lemon, 1
• Garlic, 2 cloves
• Cumin, 1/2 teaspoon
• Salt, 3/4 teaspoon
• Ice or cold water
• Sumac

Steps

1. Drain and rinse chickpeas
2. Add tahini, olive oil, lemon juice, crushed garlic, cumin, and salt to food processor
3. Blend until smooth
4. Add half cup chicpeas and blend until smooth
5. Add remaining chickpeas and blend until smooth
6. Add ice or cold water until desired consistency
7. Serve in wide container, drizzle olive oil, and sprinkle sumac

• Always use timestamptz.
• Always set the timezone to UTC using timezone = 'UTC' in postgresq.conf.
• Always specify the timezone when updating tables or working with timestamp literals.

Definitions

timestamptz represents a specific instance in universal time. It can be used to represent when an account is created, when a user posts a comment, or when a meeting will start. timestamptz is not the combination of a timestamp and a time zone. It is impossible to recover the time zone used to create a timestamptz value.

timestamp represents a conceptual time that might vary based on the frame of reference. For example, New Year's Day starts on Janaury 1 at midnight, but midnight occurs at different times across planet Earth. timestamp is not a timestamp relative to the UTC time zone; although the underlying storage format and existing misuse may imply as such.

reality is chaos. If you work on a system with different interpretations of timestamp and timestamptz, embracing the chaos may help preserve sanity.

Literals

Timestamps can be created using the ISO 8601 text format.

The optional time zone suffix at the end of a literal indicates the offset relative to UTC, e.g. UTC is UTC+00, PST is UTC-07, and IST is UTC+05:30

If a timestamp is created from a literal with a time zone suffix, the suffix will be silently ignored.

select timestamp '2004-10-19 06:23:54+02';
        select timestamp '2004-10-19 06:23:54-07';
        -- Both queries return 2004-10-19 06:23:54
        

If a timestamptz is created from a literal without a time zone suffix, the time zone is assumed to be the session time zone.

set timezone = 'America/Los_Angeles';
        select timestamptz '2004-10-19 06:23:54';
        -- Returns 2004-10-19 06:23:54-07
        
        set timezone = 'America/New_York';
        select timestamptz '2004-10-19 06:23:54';
        -- Returns 2004-10-19 06:23:54-04
        

Storage

Both timestamp and timestamptz are stored as 64-bit integers representing the offset in microseconds since 2000-01-01 00:00:00 UTC.

select extract(epoch from timestamp '2004-10-19 10:23:54');
        select extract(epoch from timestamptz '2004-10-19 06:23:54-04');
        select extract(epoch from timestamptz '2004-10-19 03:23:54-07');
        -- All queries return 1098181434
        
        select extract(epoch from timestamp '2004-10-19 10:23:54')
        - extract(epoch from timestamp '2000-01-01 00:00:00');
        -- Returns 151496634
        
        -- 151496634 seconds = 151496634000000 microseconds = 0x000089C90F0DE280 microseconds
        -- If you go digging, you'll find 64-bit integer 0x000089C90F0DE280 in memory.
        

Even though timestamp values are stored as offsets relative to the dawn of the millennium in UTC time, they should not be interpreted as timestamps in the UTC time zone. Instead, refer to the prior definition of timestamp.

Note that timestamptz does not store the time zone. The time zone used to create a timestamptz value cannot be recovered.

Serialization

timestamptz values are serialized using the session time zone.

set timezone = 'America/Los_Angeles';
        select timestamptz '2004-10-19 06:23:54-03';
        -- Returns 2004-10-19 02:23:54-07
        
        set timezone = 'America/New_York';
        select timestamptz '2004-10-19 06:23:54-03';
        -- Returns 2004-10-19 05:23:54-04
        

Serialization of timestamp values is time zone independent. The returned value mirrors the source literal.

set timezone = 'America/Los_Angeles';
        select timestamp '2004-10-19 06:23:54';
        -- Returns 2004-10-19 06:23:54
        
        set timezone = 'America/New_York';
        select timestamp '2004-10-19 06:23:54';
        -- Returns 2004-10-19 06:23:54
        

Ingredients

• Fresh kumquats, ~500 grams
• Fresh lemon, 1
• Cane Sugar, 3/4 weight of kumquats
• Ginger (optional), 1 teaspoon
• Sanitized Jars

Steps

1. Wash kumquats and remove blemishes
2. Slice kumquats into ~3-5mm circular cross sections into dish
3. Zest lemon into dish
4. Extract lemon pulp into chunks and remove rind, then add to dish
5. Weigh mixture, then add 3/4 as much sugar by weight
6. Mix sugar into mixture
7. Macerate mixture by letting it sit at room temperature for a couple hours or overnight in the fridge
8. Place mixture into pot
9. (Optionally) grate 1 teaspoon ginger into mixture
10. Boil mixture on medium heat
11. Periodically mix with silicon spatula
12. Take it out when it's ready (watch a video)
13. Pour hot mixture into sanitized jars and seal
14. Allow jars to sit until cooled to room temperature
15. Refrigerate and eat

Notes

• All quantities are guesstimates
• 1-to-1 is the standard mixture to sugar ratio, but we like it less sweet
• Granulated sugar also works, but we prefer cane sugar
• Make sure to sanitize jars before use
• 8 oz mason jars are the right size
• In the future, we'll experiment with spices like saffron, cardamom, star anise, and cinnamon

Distraction is a major cause of waste. I've taken measures to fight distractions in my search experience because my useful work involves lots of searching.

Overview

This is my search experience:

And this is how it works:

• Google Programmable Search Engine
  • Search results only includes links and not multimedia or suggestions
  • Custom filters remove distracting and low-value websites from search results
• uBlock Origin
  • Blocks search ads
  • Custom filter rules hide UI elements that aren't useful to me
  • Custom filter rules restyle UI elements for better usability
• Firefox
  • Toolbar customized to hide distracting UI elements
  • Searchbar customized to disable all suggestions
  • Search engine configured to use my custom search engine by default

Google Programmable Search Engine

I'm surprised Google wants anyone to user a search engine that's not google.com, but Programmable Search Engine exists, somehow. It competes with the likes of Algolia by offering customized website search as a service, but it also works as a standalone search tool.

This is how it compares to google.com:

Pros

• It's free, so there's no need to maintain a subscription like with Kagi
• Search results are instant
• Search results are relevant
• It supports filtering custom domains from search results
• It doesn't have distracting anti-features like multi-media results, related searches, favicons, embedded YouTube player

Cons

• More frequent anti-bot protections, which require manual interaction to bypass
• No support for time range filters, e.g. only show results from past month
• Lack of useful built-in tools like: calculator, dictionary, weather forecast, unit conversions, time and date tools, etc.
• No support for personalized search results, but I don't mind since personalized results usually reinforce my bad habits.

My custom search engine configuration filters out websites that make life worse for various reasons.

• Some websites are too engaging, leading to addiction and doom scrolling. Since we can't have a healthy relationship, it's better to eliminate temptation by removing them from notice.
• Some websites have mostly low quality content, so you're better off looking elsewhere. Sometimes websites are SEO content farms. Other times content is user generated, and the low barrier to entry leads to "the blind leading the blind." I wish there was a way to block all domains in the Medium syndicate.
• Some websites are walled gardens. I'd rather not create an account and sacrifice my privacy to access them.
• Some websites use intrusive dark patterns that render their content not worth the painful user experience.

This is my current blocklist:

*.itnext.io/*
        *.dev.to/*
        *.tiktok.com/*
        *.quora.com/*
        *.w3schools.com/*
        *.facebook.com/*
        *.pinterest.com/*
        *.instagram.com/*
        *.medium.com/*
        *.linkedin.com/*
        *.twitter.com/*
        *.ycombinator.com/*
        *.reddit.com/*
        

I also updated my custom search engine to return 20 results per page instead of the default 10 to reduce the need to click the next page button.

uBlock Origin

The user experience of Google Programmable Search Engine is terrible because ads use up the entire viewport on the results page, forcing users to scroll down to see any results.

Examples like this show how the quest for monetization has rendered the web unusable. Ad blockers exist to defend ourselves from this malpractice.

uBlock Origin is the content blocking web extension. It blocks ads on the search results page by default. This is the result.

Much better! But it can be refined further:

• The blue search button is useless to me because I always submit my searches using the "Enter" key.
• The "x" button is useless to me because I never clear the search query.
• The "About X results (Y seconds)" notice is useless to me. I don't care unless something went wrong.
• The "Sort by relevance/recency" toggle is useless because the recency sort is useless. What I really want is a time range filter.
• There's additional Google branding at the bottom of the page I don't care about.
• Because results are aligned to the left side of the viewport, I have to turn my head or eyes to see them. Center alignment is more ergonomic.

This is the result:

And these are the underlying uBlock Origin filters:

cse.google.com##body:style(width: 580px; margin: auto)
        cse.google.com###cse-search-form:style(width: 580px !important; padding: 0 12px !important)
        cse.google.com##.gsib_b,.gsc-search-button,.gsc-above-wrapper-area
        cse.google.com##.gsc-adBlock,#cse-footer,.gcsc-find-more-on-google-branding
        

Unfortunately, there's no way to get around the reCAPTCHA.

Firefox

Almost all of my searches begin in Firefox, which I use because it best supports uBlock Origin today and in the long term.

I start by typing cmd + L to select Firefox's search bar and begin typing my query. Helpful suggestions appear to save me keystrokes and send me where I want to go.

I lied. The default suggestions UI is terrible for the usual reason, monetization. I actually don't want any suggestions.

• History suggestions are likely to reinforce bad habits as they guide me back toward irresistible, mindless entertainment.
• On the infrequent occasion when I need it, dedicated bookmark search works great.
• Searching for tabs means I have too many tabs open and can't keep track of them.
• Search engine completions are likely to drive me on a tangent, so I'd rather do the work to type the query myself.
• Why are there colorful pictures in my search suggestions? I'm not here for them, but I can't not look.
• How could I not want shopping suggestions from online retailers???

Firefox makes it somewhat tedious, but I was able to disable all suggestions. In the process, I also set the default search engine to my custom search engine and switched to a light browser theme.

What's next?

Mission accomplished. I created a distraction-free search experience.

But it's sad to think about the lost potential of people without the ability or determination to take similar measures. While a few people might come across this post and learn a trick or two to better their search experience, I think it would be far more valuable to the world if there were a more accessible way to apply anti-distraction measures. I shared some thoughts on this in my Web Agency post.

If I could wave a magic wand to change one thing in my setup, it would be for Safari to support the web extension APIs needed by uBlock Origin. I like the minimalism of desktop Safari's controversial compact-mode UI, which embeds the search bar in the active tab widget and themes the toolbar to blend in with the current website.

So you go through this web maze and each page provides its own unique experience usually optimized for keeping you in the network or showing you ads. You've trained yourself to ignore the extraneous, intrusive information and manage to find the information you want. Or maybe you didn't. Perhaps you clicked on an ad interleaved with the content you actually care about (e.g. on Google Search or Reddit) or perhaps you saw a suggestion for a relation question or topic you might be interested in and followed it, or perhaps a video thumbnail caught your attention, and before you know it, you forget the original intent of your search.

Digressing from the negative effects of platform incentives, you might find yourself on a website that is optimized for conveying the information you want. But you still have to understand its UI and the presentation might slow you down; perhaps the text is too small, the font is illegible, the sidebars push in-network content, there's a like count at the top, there are low-value user comments, there's a waste-of-(mental)-space stock image at the top... you get the point. Reader mode is supposed to eliminate these details. But reader mode doesn't work everywhere, it removes features that help you navigate and interact in useful ways, its heuristic algorithm sometimes removes content you care about, and you have to manually enable (and disable) it.

What we really need is a way to standardize the experience of each website to our preference, with ways to remove specific UI elements we don't care about, standardize presentational details like fonts and page alignment, and configurable presets so we don't have to define everything ourselves. Basically, we need a user-agent that gives me agency over the web in a nice usable package. For example, I hate profile images because they steal focus use up valuable space, so I want them all gone across all websites. I want my browser to have a "turn off profile images across all websites" checkbox. This functionality doesn't exist because every website is different and there is no universal way to identify profile images. We actually need to hardcode this logic for every website, which appears infeasible. But is it really?

The 80/20 rule applies but in this case I think it's really a 99.9/0.01 rule. We don't need to hardcode logic for every website on the internet, we just need rules for the tiny minority of websites that account for the majority of use. I prefer not to see profile images at all, but if they're removed from the top 100 websites I visit, then I only risk seeing them rarely, which is practically equivalent to never. Great... so I only need to write rules for 100 websites. I'm lazy, so that's about 100 too many, and I'm unwilling to invest the time unless I'm on an OCD stint.

What I really want is for something or someone else to do the work of writing the rules so I can enjoy the fruits of their labor without lifting a finger myself. I can think of three realistic options:

1. Outsource the work to a hard-working, altruistic benefactor who takes ownership of the rules. This is akin to filter lists used by ad blockers. Tens of someones do the hard, thankless work of writing and updating these lists to save hundreds of millions of people billions of hours annually. In big-tech world where quantifying impact is supposedly important, those numbers are crazy to think about. If you care about effective charity, then maybe these someones deserve something.
2. Crowdsource the rules so that many less hard-working people can contribute a little. Wikipedia, SponsorBlock, and crowd-sources translation platforms are prominent examples. The challenge here is moderation, preventing abuse, and filtering low-quality contributions. SponsorBlock is interesting because there is a right answer: given a video timeline, there are specific objective time segments containing sponsored ads. This makes crowdsourcing a little easier. Wikipedia and crowd-translation platforms are a little tricker and require heavier moderation because there is no right answer. I think website content rules lean toward the objective, but there are still multiple ways to achieve the same result.
3. Have machines devise the rules. What was unachievable just 2 years ago can suddenly be done with the power of AI, particularly gen-AI. Whereas in the past you might attempt to create heuristic algorithms that sort of worked half the time, today you can use gen-AI, and it works great most of the time. You just have to figure out how to get data into the system and pay for GPUs.

These options are not mutually exclusive and in my opinion are all worth exploring. Once you have some sort of system for defining rules, the next question is how do you make them accessible to users. A web extension is the obvious answer, but how should it be designed so that users can easily understand and customize rules to their preference? That will have to wait for my next blog post.

Why not Social Media or Hosted CMSs

Ownership is important to me because I want to control how my content is used and communicated. I don’t want my content presented alongside ads, cookie banners, subscription dialogs, or algorithmic feeds optimized for engagement. I’m fortunate enough to be able to offer my content free of charge or encumbrances and hopefully it’s worth that price. This purism precludes using social media platforms like Medium, Substack, or Dev Community.

This left me with the choice of either a hosted CMS like Wordpress or the tried, old method of a custom website. Unfortunately, the easy hosted option isn’t for me since I prefer managing content in markdown files that can be opened with a local text editor, version-controlled with Git, and understood by many useful tools. The hosted option would also likely involve battling an unfamiliar monolith to achieve my preferred customizations. Fortunately, as a professional software dude, I know how to build a website.

The Static Website Tech Stack

Maintenance is the bane of my existence, so servers that might go down and services that require an up-to-date credit card are out of the question. Luckily GitHub Pages exists, and it offers free static website hosting. It just means my website won’t support dynamic content, which I don’t want anyways because for blogs, dynamic content usually means social features like commenting, liking, and sharing. I think social features, which are usually designed to promote engagement and distribution, are almost always anti-features, since they distract from the main message and consume users’ limited attention.

Knowing I wanted a static website, the next step was to decide on a static website builder. Because TypeScript is my most productive language and I hate wasting time on setup and tooling, I decided to use Deno, a batteries-included TypeScript runtime. I Googled “deno ssg,” found Lume, and never looked back because it just worked.

Lume is just another static site generator. Raw content files go in, beautiful (or, if we’re being honest, usually ugly) website comes out. In my case, I wanted to use markdown to write blog posts and JSX to define its presentation. Markdown is simple, reduces the friction of writing and editing rich(ish) text documents, and is well supported by editors and tooling. JSX is simple, powerful, and saves me from having to learn bespoke templating languages of which Lume has its own. Lume natively supports Markdown and setting up the first-party JSX plugin took me 5 minutes.

Setting up hosting and DNS to actually get the static websites deployed at https://dawoodjee.com was straightforward. I use Namecheap for domain registration since it’s done the job well for me for years, Cloudflare for easy DNS configuration and fast nameservers, and Lume’s stock template for deploying to GitHub Pages with GitHub Actions.

I could have reduced complexity and save an hour by deploying to a github.io subdomain, but using my own domain helps me retain control of my content. I can shift the underlying infrastructure, perhaps by switching from Github pages to GitLab Pages, without breaking the user-facing API: URLs.

Designing the Website

With the basic tech stack decided, the next step was to design the website. I followed this principle: less is more because extraneous content distracts from the core message. There would be no about page, no contacts page, no hamburgers or kabobs; just a home page with basic information and links to blog posts.

Branding helps with recognition, so I added a simple tuxedo cat avatar to the top of the page that also serves as a link to the home page. I created a footer with a few useful tools: a search box (that directs queries to a site-scoped Google search), a link to the page’s underlying file in GitHub in case I want to make a quick edit or someone identifies a typo, and feed links for eccentrics like me.

Blog posts follow this basic structure: first the title, then basic metadata, then tags, and finally the content. I call it “basic metadata” because I think basically everything on the internet should have it. It peeves me when an article is not accompanied by its author or publish date. Why should I trust whatever is written if no accountable name is attached? How do you know whether the information is still relevant or hopelessly outdated? I considered adding time-to-read estimates, but decided not to, since it’s unclear to me if they are actually useful. Email me if you know a compelling reason to include them.

I have mixed feelings about tags because it’s hard to decide on and easy to miss appropriate tags. I chose to include them for now because they help readers quickly identify topics that aren’t of interest. I look forward to a future in which tags are AI-generated based on the content of blog posts. The future is probably the present, but Lume doesn’t have a first-party plugin.

Retrospective

Well, dawoodjee.com is online and I’m on the tail end of my first blog post. Did I succeed? In terms of building a blog in which I retain full control, Yes. In terms of building a compelling blogging experience, Yes. In terms of building a low maintenance website… I’m optimistic but uncertain. I’ll check back again later, perhaps in a year, to see if I’m still happy. Hopefully nothing breaks and Dependabot security alerts skip my inbox.