I have >= 8000 posts in my blog, almost 4000 sketches (mostly public, many with OCR results), and lots of text notes in my Org Mode files, many of which are in a hierarchy and many of which are just a random list of entries.

There's also the episodic memory aspect of things, trying to remember using different cues. Might be fun to figure out what I can do with >= 16,000 text journal entries (usually a sentence, sometimes longer).

I like the idea of being able to search my notes with both exact and approximate matches. Semantic Search for Org roam | lgmoneda might be a good starting point. Khoj has a free self-hosted option that can be configured to use offline models and some support for Org Mode, but it might be too big for me to wrap my head around at this point. I'll probably write my own thing so that I can understand it and so that I can tailor it to the structure (or lack thereof) of what I've got.

I don't mind starting with just the retrieval aspect, since text generation is still a little iffy. I'd rather be able to flip through the titles and jump to the source information for now.

I'm partly looking for the modern equivalent of the Remembrance Agent, which I enjoyed using in Emacs before. It used a bag-of-words approach to look at a few hundred words around your cursor and suggest files/emails/etc. that were relevant in another window.

Slightly more far-off: it might be nice to be able to find something based on more retrieval cues and episodic memory, and to enrich memory.

Someday it would be pretty cool to have something help me remember where I left something. Video (SpotEM, Episodic Memory Question Answering) might be excessive, but maybe I could get into the habit of audio notes?

It would be good to have more conveniences for fixing commonly-misrecognized words, although that might also be handled with simple regex-based replacements (maybe like mbork's mrr-replace-mode).

It would be nice to automatically break up transcripts of braindumps or web conferences into topics. I currently do a bit of light structuring via keywords in my audio braindumps, but topic segmentation is a well-established area, so I should be able to figure this out once I set aside time for it.

I thought it was a little interesting how Dan Shipper used 10 years of journal data for retrieval-augmented text generation and question-answering (source) and how AI can make analyzing your thoughts and actions feel more like a conversation, but I don't think this is quite useful for me yet.

When I'm writing, my challenge is usually resisting going down the rabbit hole of more tweaks and instead reining things in so that I can finish the post. But if I do find myself wanting to write more, I think it might potentially be interesting to ask what kinds of follow-up questions people might ask about something, and then add links or more explanations.

I categorize Emacs News links so that it's easy for people to find the topics they're interested in. Most of the time, a simple regular expression match is enough (ex: Org Mode, Denote, Magit). I can probably make a training data set from recent Emacs News posts so that I can use text classification libraries to suggest a category for links that aren't easily figure out with regular expressions.

I think GPT is useful for figuring out the words to look for. One of the challenges of learning more is figuring out the jargon, like learning how to use M-x apropos to look for things related to buffers. (Apropos, buffers; Emacs uses lots of weird words.) Even when you know the words, it can be hard to remember them: tip of the tongue is a thing, and it'll probably get harder as I get older. (Another good reason to exercise.) It'll be handy if I can just vaguely describe something and then get the words to look for, or use vector search to find things based on similarity rather than exact matches.

I appreciate that the web access in ChatGPT-4 and Microsoft Bing/Copilot allows me to get search results back without having to deal with all the generic SEO content, newsletter popups, and content paywalls that plague the modern web. Sometimes it will even analyze the information and put things into tables or outlines for me. Of course, it is also contributing to the problem by making it easier for people to churn out lots of generic SEO content or newsletter updates, so this feels somewhat self-defeating in the long run. We'll just have to see how it all works out.

I'm also curious about using it to summarize the videos I come across for Emacs News, although people usually set good titles and a quick skim of the transcript usually helps.

I'm ambivalent about generating outlines for blog post ideas. The ones it comes up with from short prompts are pretty generic and intro-focused, and it feels easier to just start with my own outline.

The outlines it generates from my audio braindumps could be an interesting way to structure a large wall of text, if I could just get it to use more of my own words while still correcting misrecognized words.

I could give GPT a blog post or presentation draft and ask it to generate follow-up questions so that I can flesh ideas out further. Although usually adding more stuff isn't a problem for me–it's more like finding a good stopping point instead of getting tempted to squeeze one more hack in…

The kiddo sometimes needs a bit of prompting to elaborate on her writing homework. She responds well to me mind-mapping while we free-associate words, but sometimes it helps to have follow-up questions, and it works out much better when the computer's the one asking the follow-up questions instead of me.

Sometimes I'm not happy with the way a sentence or paragraph flows. Metaphors can be helpful. Mixed metaphors are weird. Maybe a GPT can help me find a smoother way to explain things. Getting writing suggestions seems to be fine on a very small scale (phrases or sentences).

When I ask GPT to write longer things, then things get crazy. ChatGPT is pretty good at getting me to say "That totally doesn't sound like me, I'd better just write it myself," which seems like a common result.

My energy can get pretty low during the afternoon because of the aforementioned 5:30 cat alarm. It can help to have something suggest a list of ideas that we can pick from, like little Minecraft build challenges.

It's a handy way to generate short stories and poems based on the kiddo's interests. I've been getting her to supply a prompt in either the SWBST story summary format ("Somebody… Wanted… But… So… Then") or the beginning/middle/end format so she does some of the work, and then GPT can deliver the payoff in the form of a fleshed-out story that invariably makes us giggle.

This is more of a low-priority medium-term thing for me to explore, probably after I spend some more time getting the hang of livestreaming, transcripts, and vector search. I'll keep an eye out for interesting ideas.

I think I can access the GPT 4 model (or maybe gpt-4-1106-preview) on a pay-per-use basis through the API, which will probably work out better than paying USD 20/month for ChatGPT Plus. I can also check out Gemini Pro's API.

A cursory skim of r/LocalLLaMA gives me the impression that for reasonable non-toy use, local large language models require way better hardware than I have at the moment. Even people with more powerful hardware tend to focus on smaller models (maybe for coding autosuggestions or generating text for roleplaying games), and they use commercial GPTs for things where they want larger context windows or more general knowledge. Might be worth checking on this in a little while as people work on scaling models down and figuring out more things to do with open source models.

Anyway, those are my notes on this so far. Looking forward to reading more about how people are using LLMs in Emacs, considering all the things that Emacs can work with!