Categories: sharing » writing

RSS - Atom - Subscribe via email

Adding subheadings and sketches to my blog page navigation

Posted: - Modified: | 11ty, blogging

[2025-04-04 Fri]: Fixed link to onThisPage.cjs. Thanks to John Rakestraw for pointing it out!

Assumed audience:

  • My future self, when I'm trying to figure out where to change things if I want to implement something similar
  • People who like tweaking their blog's CSS, especially if they also use Org Mode or 11ty

Headings help us make sense of longer blog posts. Heading links are like signposts letting you know what's ahead and where you can take a shortcut to get to what you're interested in.

Headings are useful for me too. Sometimes I browse my blog and come across things I've completely forgotten writing about, so the headings can help me remember without having to reread long posts. If I use headings more often, I might be able to work with bigger chunks of thoughts. If I can work with bigger chunks of thoughts, then maybe I can think about more things that are hard to fit within the limits of my working memory. Making headings more navigable also means I might not have to worry about the tangents I go on and the number of different thoughts I try to smoosh together, if people can jump straight to the parts that sound relevant to them.

I particularly like the way Karthik uses a sticky table of contents for long blog posts like The Emacs Window Management Almanac | Karthinks. I also like the way the Read the Docs Sphinx Theme displays a nested table of contents on the left side on wide screens. I use org-html-themes to export Org Mode files with that theme when I want to be fancy, like my Emacs configuration (usually works, although sometimes my config is broken).

The last time I tinkered with my webpage margins, I put my "On this page" list on the left side and the blog post headings (if any) on the right side, mostly because it was easy to do. I just changed the margin and float attributes of the element with the subheadings. I'd like to clear up more space for potential sidenotes or doodles, though. This time, I experimented with nesting the blog navigation inside the "On this page" navigation on the left side.

2025-04-02_14-04-30.png
Figure 1: A screenshot of my blog showing the nested links

Here's how I did it:

Org Mode: where I start writing

I use Org Mode's table of contents directive to include a table of contents in blog posts. I wrap it inside a sticky-toc block to indicate when I want it to be part of the sticky table of contents on my blog. In Org Mode, the syntax looks like this:

#+begin_sticky-toc
#+TOC: headlines 2 local
#+end_sticky-toc

I put it in a yasnippet so that I don't have to remember it. I just type tocs (for TOC, sticky) and then press TAB to complete it.

11ty static site generator: on this page

After exporting individual HTML files from Org Mode, I turn them into my blog using the 11ty static site generator. To simplify my archive pages, I have an onThisPage shortcode which lists the posts on that page. I changed it to include the sticky-toc contents from the items' templateContent attributes.

const { JSDOM } = require('jsdom');

module.exports = function (eleventyConfig) {
  function formatPostLine(item, index) {
    let subtoc = '';
    if (item.templateContent?.match(/sticky-toc/)) {
      const doc = new JSDOM(item.templateContent).window.document;
      const sub = doc.querySelector('.sticky-toc ul, .sticky-toc div, .sticky-toc-after-scrolling div');
      if (sub) {
        if (sub.querySelector('.panzoom')) {
          console.log('remove panzoom');
          sub.querySelector('.panzoom').classList.remove('panzoom');  // don't do panzoom for now
        }
        sub.classList.remove('panzoom');
        subtoc = sub.outerHTML;
      }
    }
    return `<li><a class="toc-link" data-index="index${index}" href="${item.url}">${item.data.title}</a>${subtoc}</li>`;
  }
  eleventyConfig.addShortcode('onThisPage', function (list) {
    return `<nav class="on-this-page">
On this page:
<ul>
${list.map(formatPostLine).join("\n")}
</ul>
</nav>`;
  });
};

And then there's a bunch of CSS in assets/css/style.css:

CSS
/* tables of contents */
.on-this-page > ul > li > ul, .on-this-page > ul > li > div { display: none }

@media only screen and (width >= 95em) {
    html, body { overflow-x: unset; }

    .sticky-toc, .sticky-left, .sticky-right {
        font-size: var(--fs-sm);
        width: calc((100vw - var(--body-max-width) - 5rem)/2);
        position: sticky;
        max-height: calc(100vh - 2rem);
        overflow-y: auto;
        scroll-behavior: smooth;
        background: var(--modus-bg-main);
        top: 0;
        padding: 1rem;
    }

    article .sticky-toc {
        display: none
    }

    .single-post article .sticky-toc {
        display: block;
    }

    .sticky-toc, .sticky-left, .single-post article .sticky-toc {
        margin-left: calc((-100vw + var(--body-max-width))/2);
        float: left;
    }

    .sticky-right {
        margin-right: calc((-100vw + var(--body-max-width))/2);
        float: right;
    }

    /* Hide the TOCs for non-active posts, but only if JS is enabled */
    .js .on-this-page > ul > li > ul, .js .on-this-page > ul > li > div { display: none }
    .on-this-page > ul > li.post-active > ul, .on-this-page > ul > li.post-active > div { display: block }

    .active { background-color: var(--modus-bg-tab-bar) }

    .sticky-toc svg .active rect {
        fill: var(--modus-bg-tab-bar) !important;
        fill-opacity: 1 !important;
        mix-blend-mode: darken;
        stroke-dash-array: unset !important;
        stroke-width: 4px;
    }

    .link-to-nonsticky-toc {
        display: none
    }
}


I also have some Javascript to highlight the active post and show the subheadings for it in assets/js/misc.js.

Javascript
/* Table of contents */

function stickyTocAfterScrolling() {
  const elements = document.querySelectorAll('.single-post .sticky-toc-after-scrolling');
  let lastScroll = window.scrollY;

  elements.forEach(element => {
    const clone = element.cloneNode(true);
    clone.setAttribute('class', 'sticky-toc');
    clone.querySelector('.panzoom')?.classList.remove('panzoom');
    element.parentNode.insertBefore(clone, element.nextSibling);
  });

  const observer = new IntersectionObserver(
    (entries) => {
      const currentScroll = window.scrollY;
      const scrollingDown = currentScroll > lastScroll;
      lastScroll = currentScroll;

      entries.forEach(entry => {
        const element = entry.target;
        const clone = cloneMap.get(element);

        if (!entry.isIntersecting && scrollingDown) {
          clone.setAttribute('class', 'sticky-toc');
          clone.style.display = 'block';
        } else if (entry.isIntersecting && !scrollingDown) {
          element.style.visibility = 'visible';
          clone.style.display = 'none';
        }
      });
    },
    {
      root: null,
      threshold: 0,
      rootMargin: '-10px 0px 0px 0px'
    }
  );

  elements.forEach(element => {
    observer.observe(element);
  });

  window.addEventListener('resize', () => {
    elements.forEach(element => {
      const clone = cloneMap.get(element);
      if (clone.style.display != 'none') {
        // reset didn't seem to work
        svgPanZoom(clone.querySelector('svg')).destroy();
        addPanZoomToElement(clone.querySelector('svg'));
      }
    });
  }, { passive: true });
}

stickyTocAfterScrolling();

function scrollToActiveTocLink() {
  const activeLink = document.querySelector('.sticky-toc .active');
  const tocContainer = document.querySelector('.sticky-toc');
  if (!activeLink || !tocContainer) return;
  const tocRect = tocContainer.getBoundingClientRect();
  const linkRect = activeLink.getBoundingClientRect();
  if (linkRect.top < tocRect.top || linkRect.bottom > tocRect.bottom) {
    const scrollPosition = linkRect.top + tocContainer.scrollTop -
                          (tocRect.height / 2) + (linkRect.height / 2);
    tocContainer.scrollTo({
      top: scrollPosition,
      behavior: 'smooth'
    });
  }
}
function getVisibleArticle() {
  const viewportHeight = window.innerHeight || document.documentElement.clientHeight;
  return [...document.querySelectorAll('article')].find((article) => {
    const rect = article.getBoundingClientRect();
    const visibleTop = Math.max(0, rect.top);
    const visibleBottom = Math.min(viewportHeight, rect.bottom);
    const visibleHeight = Math.max(0, visibleBottom - visibleTop);
    return visibleHeight > 0; // find the first visible one
  });
}

function handleActiveTOCLink() {
  const updateActive = function(links, active) {
    const activeFragment = active.includes('#') ?
          active.substring(active.indexOf('#')) : '';
    links.forEach(link => {
      const href = link.getAttribute('href');
      if (href.includes(window.location.origin)) {
        link.classList.toggle('active', href == active)
      } else if (href.startsWith('#')) {
        link.classList.toggle('active', href == activeFragment);
      }
    });
  };
  const posts = document.querySelectorAll('.post');
  const tocLinks = document.querySelectorAll('.on-this-page .toc-link');
  const options = {
    root: null,
    rootMargin: '-20% 0px -70% 0px',
    threshold: 0
  };
  const observer = new IntersectionObserver((entries) => {
    entries.forEach(entry => {
      if (entry.isIntersecting) {
        const id = entry.target.id;
        const link = document.querySelector(`.toc-link[data-index="${id}"]`);
        document.querySelectorAll('.sticky-toc .active').forEach((o) => o.classList.remove('active'));
        document.querySelectorAll('.post-active').forEach((o) => o.classList.remove('post-active'));
        if (link) {
          link.classList.add('active');
          const item = link.closest('li');
          item.classList.add('post-active');
        }
      }
    });
    scrollToActiveTocLink();
  }, options);
  posts.forEach((post) => { observer.observe(post); });

  const stickyTocLinks = document.querySelectorAll('article .sticky-toc a, .on-this-page a');
  const postTocObserver = new IntersectionObserver((entries) => {
    entries.forEach(entry => {
      if (entry.isIntersecting) {
        const id = entry.target.id;
        const url = window.location.origin + window.location.pathname + '#' + id.replace(/^outline-container-/, '');
        updateActive(stickyTocLinks, url);
      }
    });
    scrollToActiveTocLink();
  }, options);

  document.querySelectorAll('article .sticky-toc, article .sticky-toc-after-scrolling').forEach((toc) => {
    const post = toc.closest('article');
    if (post) {
      post.querySelectorAll('.outline-2, .outline-3').forEach((section) => { postTocObserver.observe(section) });
    }
  });
  const visible = getVisibleArticle();
  const id = visible?.id;
  if (id) {
    const activeLink = document.querySelector(`.toc-link[data-index="${id}"]`);
    if (activeLink) {
      activeLink.classList.add('active');
      activeLink.closest('li').classList.add('post-active');
      scrollToActiveTocLink();
    }
  }
}
handleActiveTOCLink();

I can use a sketch as a map, too

I sometimes want to use sketchnotes as overviews, especially if I've added hyperlinks to them. I used to make the images show up on the right side, but now I want them to show up in the left-side navigation instead. Also, I wanted any links to headings to automatically get recoloured as I scroll to that heading.

2025-04-02-scrolling-svg.gif
Figure 2: Animated GIF showing how the SVG highlights change as you scroll down

I added a special case to the handleActiveTOCLink function to handle anchor hyperlinks (just #anchor) in the SVG. It probably makes sense to make those absolute URLs, which means slightly changing my workflows for hyperlinking SVGs and writing about sketches.

So on both the category page (ex: the Hyperlinking SVGs entry in category - drawing, which might have moved off the first page of results if you're reading this far in the future) and the single-post page (ex: Hyperlinking SVGs), there's a full-sized version of the image in the main blog post, and then a small copy of it in the margin on the left. The sidebar copy is probably too small to read, but it might be enough to get a sense of spatial relationships, and the links also have title attributes that are displayed as tooltips when you hover.

2025-04-03_14-28-48.png
Figure 3: Screenshot of small image in sidebar on the single post page

I use Javascript to duplicate the image and make a small, sticky version because I haven't quite figured out how to properly make it sticky when off-screen with just CSS. Even my JS feels a little tangled. Maybe this would be a good excuse to learn about web components; someone's probably figured out something polished.

I'm curious about using more drawings to anchor my thinking and structure my blog posts.

Progressive enhancement

Some people read my blog using EWW (the Emacs Web Wowser, of course), so I want my blog to be reasonable even without CSS and JS.

A number of people read my blog without Javascript enabled. I installed the Firefox extension Script Switch so that I can test my blog with and without Javascript whenever I remember.

I sometimes look up my blog posts on my phone and there's no space for any of this fanciness there, so it'll only kick in on large screens. My CSS file is littered with various breakpoints I've cargo-culted over the years and I should simplify it at some point. At the moment, if it looks fine on my Lenovo P52, I'm happy.

Other ideas and next steps

Theoretically, the right margin is now available for sidenotes, so I might be able to look at ox-tufte and Eleventufte and get something going. Then I'll have a way to add small notes that are shorter than a paragraph. Longer tangents can go in a details/summary element instead, although I have it on good authority that one can write at length in footnotes. I love the footnotes in the Bartimaeus series, and apparently there are quite a few books where the footnotes are part of the storytelling.)

It might be nice to let tables extend into the right sidebar when I know I won't have a doodle nearby. Incidentally, Sidenotes In Web Design · Gwern.net uses breadcrumbs in the left sidebar instead of a table of contents, so there's more space for tables and sidenotes.

I thought about using CSS breakpoints so that on a medium-sized screen, we can have the left sidebar even if there's no space for something on the right. I haven't gotten around to experimenting with it yet, though. Besides, I don't know yet if I want to prioritize the stuff I want in the right sidebar (side notes, doodles) over fairly-static navigation.

As I mentioned, it might be handy to tweak my SVG linking workflow to use absolute URLs.

Sometimes I look up my notes within Emacs, but surprisingly often, I look them up on the Web. Navigation isn't just cosmetic. I want to get better at using my blog as a tool for thought, so tinkering with layout isn't just window dressing. It's (very slowly) experimenting with scaffolding for my brain. Little things can help!

Using Emacs Lisp to batch-demote HTML headings for my static site

| blogging, 11ty, emacs

Assumed audience: People who have lots of HTML files used as input for a static site generator, might need to do a batch operation on them, and are open to doing that with Emacs Lisp. Which might just be me, but who knows? =)

HTML defines a hierarchy of headings going from <h1> to <h6>, which comes in especially handy when people are navigating with a screenreader or converting web pages to Org Mode. I think search engines might use them to get a sense of the page's structure, too. On my blog, the hierarchy usually goes like this:

  • <h1>: site title,
  • <h2>: blog post titles, since I put multiple blog posts on the main page and category pages (ex: blogging)
  • <h3>: blog post's subheadings, if any
  • <h4>: I rarely need subsubheadings in my main blog posts, but they're there just in case

While fiddling with my blog's CSS so that I could try this fluid type scale, I realized that the subheadings in my exported blog entries started at <h2> instead of <h3>. This meant that the outline was this:

  • Site title
    • Blog post 1
    • Subheading 1
    • Subheading 2
    • Blog post 2
    • Subheading 1
    • Subheading 2
    • Blog post 3

I wanted the outline to be this:

  • Site title
    • Blog post 1
      • Subheading 1
      • Subheading 2
    • Blog post 2
      • Subheading 1
      • Subheading 2
    • Blog post 3

This was because I hadn't changed org-html-toplevel-hlevel during my 11ty export process. To solve this for new posts, I added a new option org-11ty-toplevel-hlevel that defaults to 3 in ox-11ty.el, re-exported one of my long blog posts to test it, and confirmed that my headings now started at <h3>.

I still had all my old HTML files with the wrong levels of headings. I wrote some Emacs Lisp to shift the headings downwards (h5 to h6, h4 to h5, h3 to h4, h2 to h3) in a file if it had an <h2> in it. Regular expressions are usually not a good idea when it comes to HTML because there might be exceptions, but I figured it was a pretty small and low-risk change, so I decided not to use the full XML/DOM parsing functions. I saved all the blog posts under version control just in case I messed things up. Here's my function:

(defun my-html-shift-headings (filename)
  "Shift heading tags in FILENAME."
  (interactive "FFile: ")
  (let ((case-fold-search t)) ; make the search case-insensitive
    (with-temp-buffer
      (insert-file-contents filename)
      (goto-char (point-min))
      ;; Only modify the files where we have an h2
      (when (or (search-forward "<h2" nil t)
                (search-forward "</h2>" nil t))
        (goto-char (point-min))
        ;; Handle both opening and closing tags
        (while (re-search-forward "<\\(/\\)?h\\([2-5]\\)\\>" nil t)
          (let* ((closing-tag (match-string 1))
                 (heading-level (string-to-number (match-string 2)))
                 (new-level (1+ heading-level)))
            (replace-match (concat "<" closing-tag "h" (number-to-string new-level)))))
        (write-file filename)
        filename))))

Running it on all the source HTML files in specific subdirectories was easy with directory-files-recursively.

(dolist (dir '("~/proj/static-blog/blog"
               "~/proj/static-blog/content"))
  (mapc 'my-html-shift-headings
        (directory-files-recursively
         dir
         "\\.html\\'")))

Then I could just rebuild my blog and get all the right heading levels. Spot-checks with Inspect Element show that the headings now have the right tags, and org-web-tools-read-url-as-org now picks up the right hierarchy for the page.

Correcting the input files was easier and more efficient than modifying my 11ty template engine to shift the heading levels whenever I build my site (probably by defining a preprocessor). I could've written a NodeJS script to do that kind of file manipulation, but writing it in Emacs Lisp matched how I might think of doing it interactively. Using Emacs Lisp was also easy to test on one or two files, check the list of files matched by directory-files-recursively, and then run it on everything.

Going forward, the new org-11ty-toplevel-hlevel variable should properly modify the behaviour of Org's HTML export to get the headings at the right level. We'll see!

Moving 18 years of comments out of Disqus and into my 11ty static site

| 11ty, blogging

Assumed audience: Technical bloggers who like:

  • static site generators: this post is about moving more things into my SSG
  • XML: check out the mention of xq, which offers a jq-like interface
  • or Org Mode: some notes here about Org Babel source blocks and graphing

I've been thinking of getting rid of the Disqus blog commenting system for a while. I used to use it in the hopes that it would handle spam filtering and the "someone has replied to your comment" notification for me. Getting rid of Disqus means one less thing that needs Javascript, one less thing that tracks people in ways we don't want, one less thing that shows ads and wants to sell our attention. Comments are rare enough these days, so I think I can handle e-mailing people when there are replies.

There are plenty of alternative commenting systems to choose from. Comentario and Isso are self-hosted, while Commento (USD 10/month) and Hyvor Talk (12 euro/month) are services. Utterances uses Github issues, which is probably not something I'll try as quite a few people in the Emacs community are philosophically opposed to Github. Along those lines, if I can find something that works without Javascript, that would be even better.

I could spend a few years trying to figure out which system I might like in terms of user interface, integration, and spam-filtering, but for now, I want to:

Fortunately, there's 11ty/eleventy-import-disqus (see zachleat's blog post: Import your Disqus Comments to Eleventy)

Exploring my disqus.xml with xq, Org Babel, and seaborn

One challenge: there are a lot of comments. How many? I got curious about analyzing the XML, and then of course I wanted to do that from Emacs. I used pipx install yq to install yq so that I could use the xq tool to query the XML, much like jq works.

My uncompressed Disqus XML export was 28MB. I spent some time deleting spam comments through the web interface, which helped with the filtering. I also deleted some more comments from the XML file as I noticed them. I needed to change /wp/ to /blog/, too.

This is how I analyzed the archive for non-deleted posts, uniquified based on message. I'll include the full Org source of that block (including the header lines) in my blog post so that you can see how I call it later.

#+NAME: analyze-disqus
#+begin_src shell :var rest="| length | \"\\(.) unique comments\"" :exports results
~/.local/bin/xq -r "[.disqus.post[] |
   select(.isDeleted != \"true\" and .message) |
   {key: .message, value: .}] |
  map(.value) |
  unique_by(.message) ${rest}" < disqus.xml
#+end_src

When I evaluate that with C-c C-c, I get:

8265 unique comments

I was curious about how it broke down by year. Because I named the source code block and used a variable to specify how to process the filtered results earlier, I can call that with a different value.

Here's the call in my Org Mode source:

#+CALL: analyze-disqus(rest="| map(.createdAt[0:4]) | group_by(.) | map([(.[0]), length]) | reverse | [\"Year\", \"Count\"], .[] | @csv") :results table output :wrap my_details Table of comment count by year
Table of comment count by year
Year Count
2025 26
2024 43
2023 34
2022 40
2021 55
2020 131
2019 107
2018 139
2017 186
2016 196
2015 593
2014 740
2013 960
2012 784
2011 924
2010 966
2009 1173
2008 1070
2007 98

I tried fiddling around with Org's #+PLOT keyword, but I couldn't figure out how to get the bar graph the way I wanted it to be. Someday, if I ever figure that out, I'll definitely save the Gnuplot setup as a snippet. For now, I visualized it using seaborn instead.

Code for graphing comments by year
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np

df = pd.DataFrame(data[1:], columns=data[0])
df['Count'] = df['Count'].astype(int)
df['Year'] = df['Year'].astype(int)
df = df.sort_values('Year')
plt.figure(figsize=(12, 6))
ax = sns.barplot(x='Year', y='Count', data=df)
plt.title('Comments by Year (2007-2025)', fontsize=16, fontweight='bold')
plt.xlabel('Year')
plt.ylabel('Comments')
plt.xticks(rotation=45)
plt.grid(axis='y')
for i, v in enumerate(df['Count']):
    ax.text(i, v + 20, str(v), ha='center', fontsize=9)
plt.tight_layout()
plt.savefig('year_count_plot.svg')
return 'year_count_plot.svg'
year_count_plot.svg

Ooooooh, I can probably cross-reference this with the number of posts from my /blog/all/index.json file. I used Claude AI's help to come up with the code below, since merging data and plotting them nicely is still challenging for me. Now that I have the example, though, maybe I can do other graphs more easily. (This looks like a related tutorial on combining barplots and lineplots.)

Code for graphing
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import json
from matplotlib.ticker import FuncFormatter
from datetime import datetime

with open('/home/sacha/proj/static-blog/_site/blog/all/index.json', 'r') as f:
    posts_data = json.load(f)

# Process post data
posts_df = pd.DataFrame(posts_data)
posts_df['Year'] = pd.to_datetime(posts_df['date']).dt.year
post_counts = posts_df.groupby('Year').size().reset_index(name='post_count')

# Convert to DataFrame
comments_df = pd.DataFrame(comment_data[1:], columns=comment_data[0])
comments_df['Count'] = comments_df['Count'].astype(int)
comments_df['Year'] = comments_df['Year'].astype(int)

# Merge the two dataframes
merged_df = pd.merge(post_counts, comments_df, on='Year', how='outer').fillna(0)
merged_df = merged_df.sort_values('Year')

# Calculate comments per post ratio
merged_df['comments_per_post'] = merged_df['Count'] / merged_df['post_count']
merged_df['comments_per_post'] = merged_df['comments_per_post'].replace([np.inf, -np.inf], np.nan).fillna(0)

# Create a single figure instead of two subplots
fig, ax1 = plt.subplots(figsize=(15, 8))

# Custom colors
post_color = "#1f77b4"    # blue
comment_color = "#ff7f0e" # orange
ratio_color = "#2ca02c"   # green

# Setting up x-axis positions
x = np.arange(len(merged_df))
width = 0.35

# Bar charts on first y-axis
bars1 = ax1.bar(x - width/2, merged_df['post_count'], width, color=post_color, label='Posts')
bars2 = ax1.bar(x + width/2, merged_df['Count'], width, color=comment_color, label='Comments')
ax1.set_ylabel('Count (Posts & Comments)', fontsize=12)

# Add post count values above bars
for i, bar in enumerate(bars1):
    height = bar.get_height()
    if height > 0:
        ax1.text(bar.get_x() + bar.get_width()/2., height + 5,
                f'{int(height)}', ha='center', va='bottom', color=post_color, fontsize=9)

# Add comment count values above bars
for i, bar in enumerate(bars2):
    height = bar.get_height()
    if height > 20:  # Only show if there's enough space
        ax1.text(bar.get_x() + bar.get_width()/2., height + 5,
                f'{int(height)}', ha='center', va='bottom', color=comment_color, fontsize=9)

# Line graph on second y-axis
ax2 = ax1.twinx()
line = ax2.plot(x, merged_df['comments_per_post'], marker='o', color=ratio_color,
              linewidth=2, label='Comments per Post')
ax2.set_ylabel('Comments per Post', color=ratio_color, fontsize=12)
ax2.tick_params(axis='y', labelcolor=ratio_color)
ax2.set_ylim(bottom=0)

# Add ratio values near line points
for i, ratio in enumerate(merged_df['comments_per_post']):
    if ratio > 0:
        ax2.text(i, ratio + 0.2, f'{ratio:.1f}', ha='center', color=ratio_color, fontsize=9)

# Set x-axis labels
ax1.set_xticks(x)
ax1.set_xticklabels(merged_df['Year'], rotation=45)
ax1.set_title('Blog Posts, Comments, and Comments per Post by Year', fontsize=16, fontweight='bold')
ax1.grid(axis='y')

# Add combined legend
lines1, labels1 = ax1.get_legend_handles_labels()
lines2, labels2 = ax2.get_legend_handles_labels()
ax1.legend(lines1 + lines2, labels1 + labels2, loc='upper left')

# Layout and save
plt.tight_layout()
plt.savefig('posts_comments_analysis.svg')
return 'posts_comments_analysis.svg'
posts_comments_analysis.svg

Timeline notes:

  • In this graph, comments are reported by the timestamp of the comment, not the date of the post.
  • In 2007 or so, I moved to Wordpress from planner-rss.el. I think I eventually imported those Wordpress comments into Disqus when I got annoyed with Wordpress comments (Akismet? notifications?).
  • In 2008 and 2009, I was working on enterprise social computing at IBM. I made a few presentations that were popular. Also, mentors and colleagues posted lots of comments.
  • In 2012, I started my 5-year experiment with semi-retirement.
  • In 2016, A+ was born, so I wrote much fewer posts.
  • In 2019/2020, I wrote a lot of blog posts documenting how I was running EmacsConf with Emacs, and other Emacs tweaks along the way. The code is probably very idiosyncratic (… unless you happen to know other conference organizers who like to do as much as possible within Emacs? Even then, there are lots of assumptions in the code), but maybe people picked up useful ideas anyway. =)

What were my top 20 most-commented posts?

Emacs Lisp code for most-commented posts
(let* ((json-object-type 'alist)
       (json-array-type 'list)
       (comments-json (json-read-file "~/proj/static-blog/_data/commentsCounts.json"))
       (posts-json (json-read-file "~/proj/static-blog/_site/blog/all/index.json"))
       (post-map (make-hash-table :test 'equal)))
  ;; map permalink to title
  (dolist (post posts-json)
    (let ((permalink (cdr (assoc 'permalink post)))
          (title (cdr (assoc 'title post))))
      (puthash permalink title post-map)))
  ;; Sort comments by count (descending)
  (mapcar
   (lambda (row)
     (list
      (cdr row)
            (org-link-make-string
       (concat "https://sachachua.com" (symbol-name (car row)))
       (with-temp-buffer
         (insert (or (gethash (symbol-name (car row)) post-map) (symbol-name (car row))))
         (mm-url-decode-entities)
         (buffer-string)))))
   (seq-take
    (sort comments-json
          (lambda (a b) (> (cdr a) (cdr b))))
    n)))
97 blog/contact
88 Even more awesome LotusScript mail merge for Lotus Notes + Microsoft Excel
75 blog/about
45 How to Learn Emacs: A Hand-drawn One-pager for Beginners / A visual tutorial
42 Planning an Emacs-based personal wiki – Org? Muse? Hmm…
38 Married!
37 Moving from testing to development
36 What can I help you learn? Looking for mentees
33 Lotus Notes mail merge from a Microsoft Excel spreadsheet
30 Nothing quite like Org for Emacs
30 Org-mode and habits
29 zomg, Evernote and Emacs
25 Literate programming and my Emacs configuration file
25 Reinvesting time and money into Emacs
23 The Gen Y Guide to Web 2.0 at Work
22 Drupal: Overriding Drupal autocompletion to pass more parameters
21 Rhetoric and the Manila Zoo; reflections on conversations and a request for insight
20 This is a test post from org2blog
19 Agendas
19 Paper, Tablet, and Tablet PC: Comparing tools for sketchnoting

Top 3 by year. Note that this goes by the timestamp of the post, not the comment, so even old posts are in here.

Emacs Lisp code for most-commented posts by year
(let* ((json-object-type 'alist)
       (json-array-type 'list)
       (comments-json (json-read-file "~/proj/static-blog/_data/commentsCounts.json"))
       (posts-json (json-read-file "~/proj/static-blog/_site/blog/all/index.json"))
       by-year)
  (setq posts-json
        (mapcar
         (lambda (post)
           (let ((comments (alist-get (intern (alist-get 'permalink post)) comments-json)))
             (if comments
                 (cons (cons 'comments (alist-get (intern (alist-get 'permalink post)) comments-json 0))
                       post)
               post)))
         posts-json))
  (setq by-year
        (seq-group-by
         (lambda (o)
           (format-time-string "%Y"
                               (date-to-time
                                (alist-get 'date o))
                               "America/Toronto"))
         (seq-filter (lambda (o) (alist-get 'comments o)) posts-json)))
  (org-list-to-org
   (cons 'unordered
         (seq-keep
          (lambda (year)
            (list
             (org-link-make-string (concat "https://sachachua.com/blog/" (car year))
                                   (car year))
             (cons 'unordered
                   (mapcar
                    (lambda (entry)
                      (list (format "%s (%d)"
                                    (org-link-make-string
                                     (concat "https://sachachua.com" (alist-get 'permalink entry))
                                     (with-temp-buffer
                                       (insert (alist-get 'title entry))
                                       (mm-url-decode-entities)
                                       (buffer-string)))
                                    (alist-get 'comments entry))))
                    (seq-take
                     (sort
                      (cdr year)
                      (lambda (a b) (> (alist-get 'comments a)
                                       (alist-get 'comments b))))
                     n)))))
          (nreverse by-year)))))

As you can probably tell, I love writing about Emacs, especially when people drop by in the comments to:

  • share that they'd just learned about some small thing I mentioned in passing and that it was really useful for this other part of their workflow that I totally wouldn't have guessed
  • point out a simpler package or built-in Emacs function that also does whatever clever hack I wrote about, just in a more polished way
  • link to a blog post or code snippet where they've borrowed the idea and added their own spin

I want to keep having those sorts of conversations.

Deleting spam comments via the Disqus web interface and Spookfox

8000+ comments are a lot to read, but it should be pretty straightforward to review the comments at least until 2016 or so, and then just clean out spam as I come across it after that. I used the Disqus web interface to delete spam comments since the isSpam attribute didn't seem to be reliable. The web interface pages through comments 25 items at a time and doesn't seem to let you select all of them, so I started tinkering around with using Spookfox to automate this. Spookfox lets me control Mozilla Firefox from Emacs Lisp.

(progn
  ;; select all
  (spookfox-eval-js-in-active-tab "document.querySelector('.mod-bar__check input').click()")
  (wait-for 1)
  ;; delete
  (spookfox-eval-js-in-active-tab "document.querySelectorAll('.mod-bar__button')[2].click()")
  (wait-for 2)
  ;; click OK, which should make the list refresh
  (spookfox-eval-js-in-active-tab "btn = document.querySelectorAll('.mod-bar__button')[1]; if (btn.textContent.match('OK')) btn.click();")
  (wait-for 4)
  ;; backup: (spookfox-eval-js-in-active-tab "window.location.href = 'https://sachac.disqus.com/admin/moderate/spam'")
  )

I got to the end of the spam comments after maybe 10 or 20 pages, though, so maybe Disqus had auto-deleted most of the spam comments.

It's almost amusing, paging through all these spammy attempts at link-building and product promotion. I didn't want to click on any of the links since there might be malware, so sometimes I used curl to check the site. Most of the old spam links I checked don't even have working domains any more. Anything that needed spam didn't really have lasting power. It was all very "My name is Ozymandias, king of kings: / Look on my works, ye Mighty, and despair!"… and then gone.

Modifying eleventy-import-disqus for my site

Back to eleventy-import-disqus. I followed the directions to make a contentMap.json and removed the trailing , from the last entry so that the JSON could be parsed.

Modifications to eleventy-import-disqus:

  • The original code created all the files in the same directory, so I changed it to create the same kind of nested structure I use (generally ./blog/yyyy/mm/post-slug/index.html and ./blog/yyyy/mm/post-slug/index.11tydata.json). I decided to store the Disqus comments in index.json, which is lower-priority than .11tydata.json. fs-extra made this easier by creating all the parent directories.
  • Ignored deleted messages
  • Discarded avatars
  • Did some reporting to help me review potential spam
  • Reparented messages if I deleted their parent posts
  • Indent the thread JSON nicely in case I want to add or remove comments by hand

With the thread JSON files, my blog takes 143 seconds to generate, versus 133 seconds without the comments. +10 seconds isn't too bad. I was worried that it would be longer, since I added 2,088 data JSON files to the build process, but I guess 11ty is pretty efficient.

Next steps

It had been nice to have a comment form that people could fill in from anywhere and which shared their comments without needing my (often delayed) intervention. I learned lots of things from what people shared. Sometimes people even had discussions with each other, which was extra cool. Still, I think it might be a good time to experiment with alternatives. Plain e-mail for now, I guess, maybe with a nudge asking people if I could share their comments. Mastodon, too - could be fun to make it easy to add a toot to the static comments from mastodon.el or from my Org Mode inbox. (Update 2025-03-30: Adding Mastodon toots as comments in my 11ty static blog) Might be good to figure out Webmentions, too. (But then other people have been dealing with spam Webmentions, of course.)

Comment counts can be useful social signals for interesting posts. I haven't added comment counts to the lists of blog posts yet. eleventy-import-disqus created a commentsCounts.json, which I could use in my templates. However, I might change the comments in the per-post .json file if I figure out how to include Mastodon comments, so I may need to update that file or recalculate it from the posts.

Many of the blogs I read have shifted away from commenting systems, and the ones who still have comments on seem to be bracing for AI-generated comment spam. I'm not sure I like the way the Internet is moving, but maybe in this little corner, we can still have conversations across time. Comments are such a wonderful part of learning out loud. I wonder how we can keep learning together.

View org source for this post

Old-school blogger

| blogging, writing

Text from sketch

Old-school blogger

[timeline showing different strands braided together]

I started blogging in 2001 (really, more like 2002), as a university student who had started playing around enjoyed learning out loud. Both blogging and Emacs continued through:

  • teaching computer science
  • going on a technical internship in Japan
  • taking up graduate studies
  • working at IBM
  • experimenting with consulting and semi-retirement
  • parenting

directly related to blogging: grad studies, working, experimenting

It's wonderful having such a long archive. I can trace my growth. I've changed a lot over the past 24 years. I miss being so optimistic and energetic, but who I am now and who I'm becoming are also okay.

[drawing of the butterfly life cycle]

  • caterpillar
  • chrysalis: We're in this messy stage where I digest myself and move my insides around
  • butterfly: maybe someday

Learning out loud by blogging:

  • Springboard: Writing as I learn means I can use my notes to pick up from where I left off.
  • Sometimes my notes help other people.
  • Sometimes people share what they've been learning.
  • Writing helps me gather my tribe.

Questions to explore:

  • What do I want to learn? How?
  • What's nearby?
  • What might be useful
    • to my future self
    • to others

Looking forward - I want to…

  • draw more. It's fun.
  • deepen my reflections.
  • learn more.
  • prepare so I can keep doing this.

How can I improve workflows for capturing/thinking/sharing/finding?

What can I do so I can keep learning and writing all my life? How can I get even better at it?

sach.ac/2025-03-16-01

Dave Winer's looking for old school bloggers (also this) so that nudged me to think about how and why I blog.

Still writing

From How to Take Smart Notes (Sönke Ahrens):

If you want to learn something for the long run, you have to write it down. If you want to really understand something, you have to translate it into your own words.

Writing and sharing are part of how I learn. Taking notes helps me learn things that are bigger than my working memory or my uninterrupted time segments. Sharing my notes helps me find them again later on, since I can search the Internet from my phone. Also, if I share my notes, sometimes I get to learn from other people too, and sometimes my notes help people figure out stuff and then they can build on that.

It makes sense to me to share these notes on a blog on my own domain, with a chronological view and an RSS feed that makes it easier for other people to check for updates if they want. Well, some other people. I suppose RSS readers are still a fairly technical sort of thing, and I don't particularly like posting on platforms like Facebook or LinkedIn. Anyway, I'll just keep writing here, and maybe people will come across posts via search engines or figure out how to get updates however they want to.

Summarizing posts
(let* ((annotations '((2001 "university")
                      (2003 "graduated, teaching")
                      (2004 "internship in Japan")
                      (2005 "grad school")
                      (2007 "working at IBM")
                      (2008 "drawing")
                      (2012 "experiment with semi-retirement")
                      (2016 "A+ was born")
                      (2019 "EmacsConf, COVID-19")
                      (2022 "SuperNote A5X")
                      (2023 "even more EmacsConf automation")
                      (2024 "cargo bike")
                      (2025 "added iPad to the mix")))
       (json-array-type 'list)
       (json-object-type 'alist)
       (posts-by-year
        (mapcar
         (lambda (o) (cons (car o) (length (cdr o))))
         (seq-group-by
          (lambda (o) (substring (alist-get 'date o) 0 4))
          (json-read-file "~/proj/static-blog/_site/blog/all/index.json")))))
  (append
   '(("Year" "Posts" "Note") hline)
   (cl-loop
    for i from 2001 to 2025
    collect
    (list (format "[[https://sachachua.com/blog/%d][%d]]"
          i i)
          (alist-get (number-to-string i) posts-by-year nil nil #'string=)
          (or (car (alist-get i annotations)) "")))))
Year Posts Note
2001 3 university
2002 31  
2003 869 graduated, teaching
2004 971 internship in Japan
2005 678 grad school
2006 877  
2007 510 working at IBM
2008 421 drawing
2009 452  
2010 399 Quantified Self
2011 397  
2012 361 experiment with semi-retirement
2013 359  
2014 339  
2015 251  
2016 141 A+ was born
2017 145  
2018 176  
2019 121 EmacsConf, COVID-19
2020 94  
2021 132  
2022 78 SuperNote A5X
2023 122 even more EmacsConf automation
2024 148 cargo bike
2025 49 added iPad to the mix

I don't see myself giving up these tools until I really can't use them any more. I'm keeping an eye out for assistive technology that might help me work around my limitations and the likely cognitive/physical decline I'll eventually run into. I'm encouraged by the fact that quite a few people manage to keep learning and writing even into their 80s and 90s.

Some weeks, Emacs News is all I can squeeze in: a long categorized list of links. When I have more time, I add little bits of code, drawings, reflections.

I love writing about little tweaks. Mostly that's about Emacs. I love the way I can shape it into something that fits me.

I like to summarize books and ideas as sketchnotes so that I have a chance of remembering what I want to learn from them. Also, the drawings are handy for sharing with others, and they're a way of giving back.

I'm slowly learning to write about life in a way that helps me learn more while respecting people's privacy. I like doing little experiments. Even tinier than the ones described in Tiny Experiments. Not "I will write 100 blog posts over the next 100 days," but rather, "What if I postpone fretting about A+'s homework until Saturday? What happens then?"

Writing workflow

After I get the kiddo through the morning routine and ready for virtual school, I usually play piano for about an hour or so. Then it's recess and some more hugs, and then I settle down for some writing or drawing. The weather is getting better, so I'm looking forward to moving some of that outside. Maybe I'll dust off those baby monitor apps so I can hear if A+ needs any help.

I mostly write on my laptop using Org Mode in Emacs. Org Mode is great for literate programming. I can mix my notes and my code however I like.

I don't write in a straightforward way. I jump around. I go on tangents and down rabbit-holes. It helps a little if I've sketched my thoughts beforehand, like for this post, or if I've done some audio braindumping to help me figure out where the interesting thoughts are. Sometimes I capture little thoughts on my phone and then move them to the post I'm working on. I'm trying to figure out how to chunk my thoughts better.

I have a lot of Emacs tweaks to make it easier to link to blog posts, bookmarks, sketches, sites from search results. I like including the text of sketches, too.

I use the 11ty static site generator to make my blog. I switched to it a few years ago because I didn't want to worry about keeping Wordpress secure. I don't have room for many programming languages in my brain at the moment, so I like the fact that 11ty uses JavaScript. It takes me about five minutes to compile my blog.

Reading workflow

From Dan Cullum: The more I read:

There is a strong correlation between the amount I’m reading, and the ideas I have for this blog. When I’m reading a lot, I feel like I have ideas coming out my eyes.

Reading makes me want to write, too.

I love the Toronto Public Library enough to transplant myself from the tropics and learn how to deal with winter. I've been reading more e-books lately. It's easier to highlight e-books compared to paper books. I can pick them up and put them down easily, and keep the pages open when I'm taking notes. I don't have to worry about misplacing them, either. I have some code to grab my highlights as a JSON, and then I can do things with them: include them in blog posts, add them to my personal notes, etc.

Not everything is available as an e-book, though, and sometimes the e-books have long hold times. Paper books are still handy enough.

I like reading blogs. They're much shorter than books are, and much less fluffy. Sometimes I feel like mainstream printed books have a lot of padding because of the considerations of the publishing industry: the book must be a certain size so it doesn't get lost on the bookstore shelf; the book must have a certain weight and thickness so people feel that it's worth $25. Blog posts can just get to the core of the idea instead of belabouring the point. I like the fractal density of hyperlinked text, too, and the conversational possibilities of it. It's a lot easier to bounce an idea back and forth to develop it when you can post in a day instead of waiting for a year for a book to be published.

I like reading on the new iPad. It's smaller than my laptop and bigger than my phone. It's easy to browse through blogs on it, unlike on my Supernote. I'm starting to develop a workflow for reading and writing smaller snippets: (toot)

  1. Read in NetNewsWire.
  2. Open interesting posts in Chrome on the iPad.
  3. Highlight the text.
  4. Use "Copy Link with Highlight".
  5. Tap on the selection again. Use "Share" to send it to Ice Cubes, a Mastodon client that can post to my GoToSocial instance and let me use my full post limit (5,000 characters, mwahahahaha).
  6. Paste the link into the toot, add my own thoughts, and post it.

I like linking to text fragments. Sharing from a webpage on my Android phone does this automatically. "Copy Highlight as Link" works from Chrome on the iPad. It saves people that little bit of scrolling or finding, although I suppose it would be helpful for people to go through the context before that selection. Alternatively, I could share directly from NetNewsWire and just link to the blog post instead of the text.

I like making visual book notes. They help me read a book well, and turning the sketch into a blog post gives me more opportunities to revisit it: when I write the post, and if someone comments or shares it.

Eventually I want to dust off my code for collecting Mastodon posts into a blog post, and maybe also re-establish a weekly review process.

Tangent: Check out Reading more blogs; Emacs Lisp: Listing blogs based on an OPML file for a table of the blogs I'm reading, along with the code I used to make a table of blogs, their latest post (as of the time I wrote my post, of course), and the post date.

Keeping an eye on the future

As the kiddo becomes more independent ("Mom, I'm 9, you don't have to fret about my jacket"), I'll have more time for myself. This is a good time to go bike and walk and explore outside, and to go deep and wide into our interests as a family. I do about 2-4 hours of consulting a week, just the stuff I'm interested in. (TODO: There's a tangent I want to write about interest-based nervous systems, which I notice in both A+ and myself, and probably building on this 2014 reflection on having a buffet of goals.) The rest is life time, divided among the things we want to learn/do/share and the things we do to take care of ourselves.

Even though I have increasing autonomy when it comes to time, and an increasing amount of focused time, I still haven't gotten to the bottom of my idea list or my to-write list. I don't think I'll ever get to the bottom of those lists, actually. I come up with ideas faster than I can do them. That's a good problem to have.

It makes sense to prepare for a couple of changes that will likely come up:

  • Age-related farsightedness: It'll probably get harder to read small text, and I might eventually need to juggle my regular glasses as well as reading glasses. (W- already does this occasionally. He prefers having different pairs of glasses instead of bifocals or progressives, and his reasons seem sound. I don't want to have to adopt different postures to see out of different zones of glasses.) Developing good workflows for reading will probably help here. Also, the cargo vests I wear will probably help me with the "Where are my glasses?" problem.
  • Menopause will probably rewire my brain a lot. I hear brain fog and tip-of-the-tongue can be challenging (see also Brain fog in menopause).
  • My mom is 79 and running into issues with cognitive and physical decline. She has a hard time typing, speaking, remembering, deciding, or feeling good. On the other hand, there are examples of people who have stayed sharp for decades. There are lots of factors that are beyond my control. Still, it would be nice to see if I can stack the deck a little. So yes to:
    • walks, bike rides, exercise, and maybe I can figure out a fun way to improve strength;
    • lots of learning and sharing and connecting
    • and experiments with technological and cognitive aids, like speech recognition to work around typing, text-to-speech interfaces to work around vision, notes to work around working memory, and maybe large language models to work around issues with recall.
    • … and I might as well learn Morse code or explore accessibility tools, just in case I'm limited to twitching cheek muscles or something like that.

The life expectancy at birth for the Philippines for women born in 1983 is ~65 years; in Canada, about ~80 years. I want to keep learning and writing and sharing for as many of those years as I can.

See discussion on Mastodon

View org source for this post

Playing with chunk size when writing

| blogging, writing

How long is a blog post? Some people write short posts with one clear thought. Others write longer essays.

I tend to start out writing a short post and then get distracted by all the rabbit-holes I want to go down. Drafting my thoughts on blogging leads to adding lots of blogs to my reader, writing some code that takes an OPML and makes a table of blogs and their most recent posts, fixing the org-html-themes setup for my Emacs configuration, breaking out this chunk as its own post, drawing a bunch of mindmaps, doing a braindump, tweaking my workflow for processing braindumps to use faster-whisper and whisper-ctranslate2 instead of WhisperX because of this issue, so that I can try the whisper-large-v3-turbo model, experimenting with workflows for reviewing the PDF on the iPad… Definitely lots of yak-shaving (wiktionary definition). I still want to write that post. I already have the sketch I want to include in it. It's like Chilli in the Bluey episode Sticky Gecko (script): "The door: It is right here. All we need to do is walk out of it: it's so easy!" The thought! It's right there! Just get to it, brain! But I wander because I wonder. I suppose that's all right.

It might be fun to play around with the sizes of things I share: shorter when my attention is fragmented or squirrely, longer when I can think about something over several days or years. Here are some ways to tinker with that.

Breaking thoughts down into smaller chunks so I can get them out the door:

  • When I notice that something is a big blog post (like this reflection I've been working on about blogging), I can break out parts of it into their own blog posts and then replace that section with links.
  • I can post interesting quotes and snippets to Mastodon and then round them up periodically or refer to them in blog posts. TODO: It might be good to have a shortcut for an accessible link to a toot using a speech bubble or similar icon.

Taming my tangents and ideas: I'm sometimes envious of blogs with neat side notes, but really, I should just accept that the tangents that my mind wants to go on can take paragraphs and are more suited to, say, collapsible details or a different blog post. Something I can experiment with: instead of gallivanting off on that tangent (soo hard to resist when there's an idea for an Emacs tweak!), I can add a TODO and leave it for my future self. Maybe even two TODOs: one inline, where it makes sense in the text; and one in my Org Mode, with a link to the blog post so that I can go back and update it when (if!) I get around to it. Who knows, maybe someone might comment with something that already exists.

Saving scraps: It's easier to cut out half-fleshed-out ideas if I tell myself I'm just saving them somewhere. Right now I capture/refile them to a scraps heading, but there's probably a better way to handle this. Maybe I can post some thoughts to Mastodon and then save the toot URL. Maybe I can experiment with using Denote to manage private notes.

Connecting thoughts and building them up:

  • I tend to write in small chunks. (TODO: I could probably do some kind of word-count analysis, might be neat.) Sketchnotes and hyperlinks might help me chunk thoughts so I can think about bigger things. I can link to paragraphs and text fragments, so I can connect thoughts with other parts of thoughts instead of trying to get the granularity right the first time around. The shortcuts I made for linking to blog posts and searching the Web or my notes are starting to help.
  • I sporadically work on topic maps or indices. Maybe I'll gradually flesh them out into a digital garden / personal wiki.
  • Sometimes I don't remember the exact words I used. Probabilistic search or vector search might help here, too. I don't need an AI-generated summary, I just want a list of related posts.
  • I can figure out how to add backlinks to my blog, or simplify the workflow for adding links to previous posts. Maybe something based on this guide for 11ty or binyamin/eleventy-plugin-backlinks. I might need to write something custom anyway so that I can ignore the links coming from monthly/weekly review posts.

Connecting to other people's thoughts: For the purposes of conversation, it'll probably be good to let people know if I write something about their blog post. Doesn't happen automatically. Pingbacks and referrer logs got too swamped by spam a long time ago, so I don't think anyone really uses them. Idea: It might be neat to have something that quickly lists all the external links in a post, and maybe a way to save the e-mail addresses or Mastodon handles for people after I look them up so that I can make that even smoother, and some kind of quick template. I can send email and toot from within Emacs, so that's totally doable… (No, I am not going to write it right now, I'm going to add it to my to-do list.)

(Also, there's another thought here about books and The Great Conversation, and blogs and smaller-scale conversations, and William Thurston and mathematicians and understanding, and cafes…)

Hmm. I think that getting my brain to make smaller chunks and get them out the door will be a good thing to focus on. Synthesizing can come later.

Related:

See discussion on Mastodon

View org source for this post

Reading more blogs; Emacs Lisp: Listing blogs based on an OPML file

| emacs, blogging

Nudged by Dave Winer's post about old-school bloggers and my now-nicely-synchronizing setup of NetNewsWire (iOS) and FreshRSS (web), I gave Claude AI this prompt to list bloggers (with the addition of "Please include URLs and short bios.") and had fun going through the list it produced. A number of people were no longer blogging (unreachable sites or inactive blogs), but I found a few that I wanted to add to my feed reader.

Here is my people.opml at the moment (slightly redacted, as I read my husband's blog as well). This list has some non-old-school bloggers as well and some sketchnoters, but that's fine. It's a very tiny slice of the awesomeness of the Internet out there, definitely not exhaustive, just a start. I've been adding more by trawling through indieblog.page and the occasional interesting post on news.ycombinator.com.

It makes sense to make an HTML version to make it easier for people to explore, like those old-fashioned blog rolls. Ooh, maybe some kind of table like indieblog.page, listing a recent item from each blog. (I am totally not surprised about my tendency to self-nerd-snipe with some kind of Emacs thing.) This uses my-opml-table and my-rss-get-entries, which I have just added to my Emacs configuration.

my-opml-table
(defun my-opml-table (xml)
  (sort
   (mapcar
    (lambda (o)
      (let ((latest (car (condition-case nil (my-rss-get-entries (dom-attr o 'xmlUrl))
                           (error nil)))))
        (list
         (if latest
             (format-time-string "%Y-%m-%d" (plist-get latest :date))
           "")
         (org-link-make-string
          (or (dom-attr o 'htmlUrl)
              (dom-attr o 'xmlUrl))
          (replace-regexp-in-string " *|" "" (dom-attr o 'text)))
         (if latest
             (org-link-make-string
              (plist-get latest :url)
              (or (plist-get latest :title) "(untitled)"))
           ""))))
    (dom-search
     xml
     (lambda (o)
       (and
        (eq (dom-tag o) 'outline)
        (dom-attr o 'xmlUrl)
        (dom-attr o 'text)))))
   :key #'car
   :reverse t))

my-rss-get-entries: Return a list of the form ((:title … :url … :date …) …).
(defun my-rss-get-entries (url)
  "Return a list of the form ((:title ... :url ... :date ...) ...)."
  (with-current-buffer (url-retrieve-synchronously url)
    (set-buffer-multibyte t)
    (goto-char (point-min))
    (when (re-search-forward "<\\?xml\\|<rss" nil t)
      (goto-char (match-beginning 0))
      (sort
       (let* ((feed (xml-parse-region (point) (point-max)))
              (is-rss (> (length (xml-get-children (car feed) 'entry)) 0)))
         (if is-rss
             (mapcar
              (lambda (entry)
                (list
                 :url
                 (or
                  (xml-get-attribute
                   (car
                    (or
                     (seq-filter (lambda (x) (string= (xml-get-attribute x 'rel) "alternate"))
                                 (xml-get-children entry 'link))
                     (xml-get-children entry 'link)))
                   'href)
                  (dom-text (dom-by-tag entry 'guid)))
                 :title
                 (elt (car (xml-get-children entry 'title)) 2)
                 :date
                 (date-to-time (elt (car (xml-get-children entry 'updated)) 2))))
              (xml-get-children (car feed) 'entry))
           (mapcar (lambda (entry)
                     (list
                      :url
                      (or (caddr (car (xml-get-children entry 'link)))
                          (dom-text (dom-by-tag entry 'guid)))
                      :title
                      (caddr (car (xml-get-children entry 'title)))
                      :date
                      (date-to-time (elt (car (xml-get-children entry 'pubDate)) 2))))
                   (xml-get-children (car (xml-get-children (car feed) 'channel)) 'item))))
       :key (lambda (o) (plist-get o :date))
       :lessp #'time-less-p
       :reverse t))))

(my-opml-table (xml-parse-file "~/Downloads/people.opml"))
2025-03-19 Flutterby! Bug replicators
2025-03-19 kottke.org A History Professor Answers Questions About Dictators
2025-03-19 Dan's Daily Posting ahead
2025-03-19 Dave Winer (untitled)
2025-03-19 Doc Searls Come from Everywhere
2025-03-19 Jack Baty Fixing the terrible scrolling behavior with Logitech MX Master on macOS
2025-03-19 Jeremy Friesen Crocus
2025-03-19 Matt Maldre Does the word ‘fascinating’ come from ‘facet’?
2025-03-19 Matt Webb An appreciation for the Useless Machine
2025-03-19 Wil Wheaton I made a thing!
2025-03-19 Sketchy Ideas 10 Lessons from The Psychology of Money in Visuals
2025-03-18 Ava cool links VIII: open access and AI, oppression, euro cloud
2025-03-18 Jessica Smith Big Ideas for Little Philosophers
2025-03-18 Maria Popova How to Get Out of Your Own Way: John Berryman on Defeating the Three Demons of Creative Work
2025-03-18 Om Malik Goodbye Torque. Hello TeraFLOPS!
2025-03-18 Tim Bray Long Links
2025-03-18 Warren Ellis status, week of 18mar25
2025-03-18 Sketchnote Lab Real-World Sketching Workshop with Mike Rohde, coming Saturday, April 26, 2025!
2025-03-17 Matthew Haughey A marathon trip
2025-03-17 Michael Lopp The Product Engineer
2025-03-17 Pete Prodoehl Editing a Bear Theme
2025-03-17 Protesilaos Stavrou On the Stoic harmony with nature
2025-03-16 Abhijit's Sketchnotes Second jobs, pay cuts, glass bottles and Oscars
2025-03-16 Illustrated Life Loving a Bent Nib
2025-03-16 QAspire Consulting - Tanmay Vora Thriving in the Age of AI: Head, Hands, and Heart
2025-03-16 The Visual Drawer Motivation Isn't Magic: It's Structure!
2025-03-15 genehack.blog Weeknote #25 (20250309-20250315)
2025-03-15 Jeffrey Zeldman Your opt-innie wants to talk to your opt-outtie.
2025-03-15 oylenshpeegul Ruth
2025-03-15 Mike Monteiro How to hide a painting
2025-03-14 Henrik Karlsson King of the sea snakes
2025-03-14 Kevin Kelly Best Thing Since Sliced Bread?
2025-03-14 Andy Draws Planting Seeds of Kindness
2025-03-13 Chris Hannah I Challenged Myself to Build a Website Using Cursor
2025-03-13 Marie K. Ekeberg Pi Day 2025 - Let’s have fun with numbers!
2025-03-13 David’s Substack Sources For Graphic Nonfiction Online
2025-03-11 Manuel Uberti A sense of belonging
2025-03-10 Nicholas Carr Strong Men and Strong Machines
2025-03-09 kupajo Start With the End in Mind
2025-03-09 Penelope Trunk I hate having to earn money, but I like knowing what makes me valuable
2025-03-06 Clarity Canvas Weekly by Tanmay Vora Thriving in the Age of AI: Head, Hands, and Heart
2025-03-04 LetSketchin’s Newsletter #84 - What's your motivation to join this newsletter
2025-03-03 Keep the Creative Juices Flowing Have you ever had an UN-IDEA?
2025-03-02 but she's a girl… ZSA Voyager
2025-02-25 Rhys Lindmark 2025 Update
2025-02-24 Derek Sivers Why did I move to New Zealand?
2025-02-23 Scott McCloud New for Spring: The Cartoonists Club!
2025-02-07 James Endres Howell What can one person do?
2025-01-03 Anil Dash Understanding DOGE as Procurement Capture
2024-12-21 Arne Bahlo My favorite things of 2024
2024-12-11 The Sketchy Anthropologist Getting Started with Sketchnotes [2]: Eva Lotta-Lamm - Domestika Sketchnoting Course
2024-11-21 Avdi Grimm You’re not selling a solution
2024-11-08 Joi Ito Morning Thick Tea and Yuen
2024-01-15 Blaine Mooers Track daily writing progress by project in 2024 and 2025
2022-11-13 Howard Rheingold Compendium of Podcasts Featuring Howard

I'm rebuilding my feed list from scratch. I want to read more. I read the aggregated feeds at planet.emacslife.com every week as part of preparing Emacs News. Maybe I'll go over the list of blogs I aggregate there, widen it to include all posts instead of just Emacs-specific ones, and see what resonates. Emacs people tend to be interesting. Here is an incomplete list based on people who've posted in the past two years or so, based on this work-in-progress planetemacslife-expanded.opml. (I haven't tweaked all the URLs yet. I stopped at around 2023 and made the rest of the elements xoutline instead of outline so that my code would skip them.)

(my-opml-table (xml-parse-file "~/Downloads/planetemacslife-expanded.opml"))
2025-03-19 Irreal The Power Of Isearch
2025-03-19 James Dyer Ollama-Buddy 0.9.8: Transient Menu, Model Managing, GGUF Import, fabric Prompts and History Editing
2025-03-19 Emacs Redux Relative Line Numbers
2025-03-19 Jeremy Friesen Crocus
2025-03-19 Michal Sapka I stopped writing alt-text to most images here
2025-03-18 Lars Ingebrigtsen WoRdPrEsS ReWrItEs My PoStS
2025-03-18 William Denton Art is the imposition of form on experience
2025-03-18 Will Schenk Knowledge Navigator
2025-03-17 Listful Andrew Hash tables look better in Emacs 30
2025-03-17 Sacha Chua Org Mode: Merge top-level items in an item list
2025-03-17 Protesilaos Stavrou On the Stoic harmony with nature
2025-03-17 Marcin Borkowski Bash script and passwords
2025-03-17 Christian Tietze NSPopover in NSTextView With Links Is Broken: Accessibility Hierarchy Slowdown
2025-03-17 TAONAW Mode (untitled)
2025-03-17 John D. Cook Lessons Learned With the Z3 SAT/SMT Solver
2025-03-16 Grant Rettke Interesting new gptel v0.9.8 features and commits since v0.9.7
2025-03-16 Magnus Using lens-aeson to implement FromJSON
2025-03-16 200ok Atomize: A Simple CLI Tool for Managing Atom Feeds
2025-03-16 Aimé Bertrand Raycast - Activate Entra Role via PIM with Graph
2025-03-15 Tim Heaney Ruth
2025-03-15 Susam Pal MathB 1.3.0
2025-03-14 Bozhidar Batsov Updating my toolbox: Ghostty and Fish
2025-03-14 Matt Maguire Japanese Electronic Dictionary (Casio XD-G9850)
2025-03-13 Alvaro Ramirez Journelly open for beta
2025-03-13 Charles Choi Announcing Casual Make
2025-03-13 Marie K. Ekeberg Pi Day 2025 - Let’s have fun with numbers!
2025-03-11 Eric MacAdie 2025-03 Austin Emacs Meetup
2025-03-11 Manuel Uberti A sense of belonging
2025-03-10 Norm XML Resolver updates
2025-03-09 Andrey Listopadov Dynamic font-lock for Fennel
2025-03-08 Arthur A. Gleckler Backup Sampling
2025-03-08 Alex Popescu TIL Succinct Data Structures
2025-03-07 Kisaragi Hiu Plasma: Avoiding having to type the login password again when first using Git / GPG
2025-03-07 The Emacs Cat Using Emacs Org Mode for Reproducibility Testing
2025-03-06 Mickey Petersen Replacing tmux and GNU screen with Emacs
2025-03-05 Amit Patel Emacs Tree-sitter custom highlighting, part 3
2025-03-05 Kris Carta My Delivery Sheet
2025-03-05 Ben Simon G's Baltimore Adventure - The USS Torsk
2025-03-02 Thanos Apollo Emacs Note Taking & Journaling using org-gnosis [Video]​
2025-03-02 But She's a Girl ZSA Voyager
2025-02-28 Mario Jason Braganza 2025
2025-02-28 Gijs Hillenius The bathwater of our 21st century
2025-02-25 James Cherti Toggling symbol highlighting in Emacs with unique colors for each symbol using built-in functions
2025-02-24 Peter J. Jones Automatic Theme Switching in Emacs
2025-02-24 Benjamin Slade C-c-c-conjecturing, and dealing with recursion in Emacs (more excursus)
2025-02-23 Ruslan Bekenev Emacs: glasses-mode
2025-02-23 J.e.r.e.m.y B.r.y.a.n.t Emacs 30.1 released including which-key
2025-02-23 Vineet Naik Premature automation
2025-02-22 Rahul Juliato Compiling Emacs 30.1 from the source on Debian
2025-02-22 Mark Tomczak Running Sandstorm From a Raid 1 Drive Array
2025-02-22 whatacold Rewrite of a Flask Web App in Clojure
2025-02-22 localauthor Ežerų Dugne
2025-02-18 Peter Povinec Speed Dialing Your Favorite Files
2025-02-18 Rodrigo Morales Compile zathura 0.5.11 in Ubuntu 24.04 LTS
2025-02-17 William Gallard Hatch Don't Ossify Defaults
2025-02-13 Anand Tamariya Emacs Font is wider
2025-02-13 yuri tricys Etymology: From Cupidity to Romance And Roses
2025-02-12 Erik L. Arneson maybe: A command-line tool that succeeds sometimes
2025-02-12 Luke Plant Christ the True and Better Frodo
2025-02-12 Unwound Stack Peppering Passwords in Rust
2025-02-08 Tory Anderson snippets that defy orgmode tangling
2025-02-08 Chris Maiorana From Emacs To Microsoft Word (And Beyond, Really)
2025-02-08 Anything goes PiZero OTG: Host or Peripheral
2025-02-03 Meta Redux Projectile Introduces Significant Caching Improvements
2025-02-02 Gene Goykhman Quickly summing up the whole stack in Emacs Calc
2025-02-02 Bz01 Using spritely hoot on nixos
2025-02-01 Jack Baty FYI: I have a new blog and RSS feed
2025-01-29 Tony Zorman Speeding up LaTeX compilation
2025-01-26 Arialdo Martini Emacs: a peek under Imenu’s hood
2025-01-25 Yi Tang Setup ssh-agent Systemd Service for Emacs
2025-01-23 punchagan Some useful Git configuration for Windows
2025-01-14 Srijan Choudhary 2025-01-15-001
2025-01-10 Isa Mert Gurbuz .emacs.d/.init.el
2025-01-07 Stefan van der Walt Pomodoros with org-timer
2025-01-03 Wai Hon Distinguish Repeated Tasks in Org Agenda
2025-01-01 Karthik Chikmagalur Tool use with gptel: looking for testers!
2024-12-21 Arne Bahlo My favorite things of 2024
2024-12-18 Maryanne Wachter Why is multithreading Selenium lousy on MacOS?
2024-12-13 Lambda Land What's New in Emacs: Last Decade Edition
2024-12-06 Jean-Christophe Helary Building "vanilla" emacs on macOS, with MacPorts, and more…
2024-11-21 JD Gonzales Kamal Tip - Private Network only Database Server
2024-11-15 Jonathan Lamothe Organizing My Life with org-mode
2024-11-14 Hristos N. Triantafillou Void Linux On A Framework Laptop: Two Years Later
2024-11-14 Hanno git-annex: Managing my most ancient data
2024-11-07 Ryan Rix Two Updates: Org+Nix dev streams, and my new DNS resolver
2024-11-03 Emacs Notes Enable completions for `Font Family’ field in `M-x customize-face RET’
2024-11-02 Ben Whitley Denote Project Tasks
2024-10-27 Andrea A useful function to contribute to Scala Metals lsp server with Emacs
2024-10-24 Summer Emacs ERC Flipping Buffers
2024-10-03 Jiewawa Useful Emacs commands for reading
2024-09-11 Sanel Zukan evil-mode in terminal without Alt/Meta
2024-09-08 Troy Hinckley What is the best pointer tagging method?
2024-08-16 Wiktor Gołgowski Org-roam: custom linking during capture
2024-08-14 Jonas Bernoulli Forge 0.4.0 and 0.4.1 released
2024-08-11 Nicolas Martyanoff Controlling link opening in Emacs
2024-07-31 T. V. Raman Emacspeak — A Speech Odyssey
2024-07-30 jao eww to org
2024-07-27 Peter Tillemans Refactoring Emacs Config using Org
2024-07-07 Timo Geusch If you get this error from Time Machine on Samba, check available disk space
2024-06-23 Peter Vágner Emacs A11y Tip #3: Emacs with speechd-el running on Termux for Android
2024-06-05 Zachary Kanfer Less: a Survival Guide
2024-05-23 Jürgen Hötzel Gnome Search Provider: Emacs Integration
2024-05-22 Gretzuni B/logroll
2024-05-14 Bryan Murdock How To Retroactively Annex Files Already in a Git Repo
2024-05-02 Evan Moses Home Assistant: using target in blueprints
2024-04-11 Emacs TIL The Night Before A Coding Interview
2024-03-26 M. Rincón Eat Evil
2024-02-12 Cameron Desautels Chinese Zodiac Time for Emacs
2024-01-19 Corwin Brust Emacs 29.2 Windows Binaries
2023-12-10 Alex Bennée A Systems Programmer's Perspectives on Generative AI
2023-12-09 Peter Prevos Writing Prose with Emacs
2023-12-05 Thomas Fitzsimmons Product Idea: CRT-alike OLED driver
2023-10-21 What the .emacs.d!? buffers.el-01
2023-08-16 Murilo Pereira I just made my first $1 on the internet!
2023-08-07 Phil Newton Updated Pocket highlights bookmarklet
2023-08-06 Shae Erisson How to use Private Storage on Android
2023-07-18 Phil Jackson Using Djblue's portal for tap in Babashka
2023-06-29 Jiacai Liu Embed git commit in Zig programs
2023-05-21 Fritz Grabo Introducing elfeed-webkit
2023-05-01 Tyler Smith Posts

Making this table was fun. It's nice to see a lot of people also writing and learning out loud. This reminded me a little of EmacsConf - 2020 - talks - Sharing blogs (and more) with org-webring. TODO: Could be fun to have a blogroll page again.

I notice I tend to like:

  • posts about adapting technology to personal interests, more than posts about the industry or generalizations
  • detailed posts about things I'm currently interested in (Emacs, personal knowledge management, some Javascript), more than detailed tech posts about things I've decided not to get into at the moment
  • "I" posts more than "You" posts: personal reflections rather than didactic advice
  • curiosity, fun, experimentation

Looking forward to discovering more!

Related:

See discussion on Mastodon

View org source for this post

Through blogging, we discover our thoughts and other people

| connecting, blogging, writing
Text and links from sketch

Through blogging, we discover our thoughts and other people.

Henrik Karlsson's "Advice for a friend who wants to start a blog" nudged me to explore two threads of thought:

Writing helps you refine your thoughts:

  • This reminds me of Sonke Ahrens's How to Take Smart Notes and David Bessis's Mathematica.
    • Everything drives toward writing; writing is how to clarify your thoughts
    • Writing helps you improve your intuition, which feeds your writing.
  • I want to understand:
    • What's possible?
    • What's easier? what's harder (for now?)
    • How can we make things. easier? How can we make more things doable?
  • and also:
    • What am I thinking?
    • what do I want to try?
  • I want to get better at this through practice.

Writing helps you find your tribe:

  • Definitely - and the more idiosyncratic my posts are, the more amazing it is when someone resonates with it, even years later.
  • (I was amused to see him trace his tweet's flow through Stian Håklev, who reached out for a conversation in 2010 about peer-to-peer education because he read my blog.)
  • I deliberately boost my tribe's information flow:
  • I want to get better at this by
    • following my curiosity
    • improving search and serendipity
    • connecting people & ideas with community infrastructure and resources

Both sides: Because it's fun and leads to more awesomeness.

View org source for this post