Building an "Open Brain"

I’ve been using AI coding tools daily for about a year now. Claude Code, Codex, Gemini, ChatGPT. They’re all capable, and depending on the task, genuinely useful. But somewhere around month three or four of using them seriously, I started to notice something that bothered me more than any hallucination or wrong answer ever did.

Every single conversation started from zero.

Not just from zero in the sense that the model didn’t remember my name or my preferences, but from zero in the sense that none of the thinking I had done yesterday, last week, or last month existed to any of these tools. Claude Code didn’t know what I wrote last week. It didn’t know what decisions I had already made, what architecture I settled on, or what I had been turning over in my head for the past month. Codex didn’t know why I built a particular feature or what I had been experimenting with in Claude. When I used Gemini to help me plan my week, it had no idea what was important from the week before or what I had been tracking.

Each tool existed in its own bubble, and none of them talked to each other.

I want to be honest about what this actually felt like in practice, because I think a lot of people experience this and just accept it as the cost of using these tools. Every time I opened a new session, the first several minutes were spent doing the same thing: re-explaining context, re-sharing decisions, re-stating preferences. The AI was capable but amnesic. And that friction, the constant context engineering just to get back to where I already was, started to feel like the single biggest bottleneck in how I worked. Not the models. Not the tooling. The fact that I was the only thread connecting all of these conversations, and I was doing it manually, every single time.

So when I learned about Gemini’s Embedding 2.0, I spent a weekend building something to fix it. I’ve been calling it the Open Brain, inspired by Nate.

Where My Thinking Actually Lives

My thinking lives across multiple systems, and that in itself isn’t the problem. The problem is that none of those systems are connected to the AI tools I use when I actually need them.

Obsidian is where I do my deeper writing. Source distillations, strategic documents, atomic ideas, maps of context for both work and personal thinking. It’s the closest thing I have to a second brain, and over the past year it’s grown into something I genuinely rely on.
Slack and Teams are where I capture quick thoughts. Things that feel important in the moment but don’t always make it into a formal note. Key conversations with people where we finalize on an approach or a decision without having to meet in order to do so.
Granola and Facilitator are where my meeting notes live. The conversations I’ve had with colleagues and coworkers that often contain the decisions and ideas that matter most. The thing about meeting notes is that they capture what people actually said, not what I remembered them saying a week later, and that distinction turns out to matter a lot.

All of these tools do their job well on their own. But none of them are accessible to the AI models when I need them the most, which is usually in the middle of working on something or right at the start of something brand new when I’ve got a burst of inspiration and I want to move fast.

I’d catch myself thinking, “Nick and I had a great idea about what to build or experiment with next, but what was it?” The note existed. Either I wrote it down or an AI took the note for me during a meeting. But the tool I was using at that moment had no way to access it. I’d have to stop what I was doing, go searching through Obsidian or scroll back through Slack or dig into meeting transcripts, find the relevant context, copy it, and paste it into the conversation. By the time I got back to what I was actually trying to do, the momentum was gone. That friction breaks the flow of thinking, and it happens constantly. Or at least, it used to.

What I Built

This part is a bit more technical, so if you ignore it, I understand

The Open Brain is a semantic search layer that sits between my knowledge and every new AI session I start. Instead of manually re-explaining context, the AI tools can retrieve the relevant pieces of my thinking automatically. The system, when I look at it now, is surprisingly simple. But getting to simple took some iteration.

The foundation is an Obsidian vault sync. A Deno script walks my entire vault and processes each note. It chunks notes by headings, embeds each chunk using Gemini’s embedding model, extracts metadata with GPT-4o-mini, and upserts everything into a database with vector search indexes. The pipeline runs automatically, so as my vault grows and changes, the search layer stays current without me having to think about it. This was important to me because I knew that if maintaining the system required any manual effort, I’d eventually stop doing it.

For the quicker, less structured thoughts, I created a custom Slack channel that acts as a capture point. Short ideas, half-formed observations, things I want to remember but don’t want to write a full note about. These get the same treatment, processed and embedded using OpenAI’s embeddings model. A stray idea I type on my phone at 10pm becomes searchable context by the next morning. Meeting notes from Granola and Facilitator are ingested alongside everything else, which means conversations with colleagues become part of the searchable knowledge base. This is where a lot of the most valuable context actually lives. The things people say in meetings that you half-remember a week later, the decisions that felt clear in the moment but get fuzzy once you’re back at your desk.

All of this is exposed through an MCP server that runs as a database edge function. It exposes my notes, thoughts, and meeting context as tools that any AI client can call. If a tool supports MCP, it can query my brain. And because the server is registered globally, every session, within every project, automatically has access. I don’t have to configure it per-project or remember to connect it. It’s just there, waiting to be asked.

When I ask Claude about something I’ve written about, it generates embeddings for my query, searches my vault, captured thoughts, and meeting notes in parallel, merges the results by similarity, and returns the most relevant chunks along with source context, note titles, and section paths. The whole thing takes seconds.

What Changed

The difference between before and after is not subtle. Before, every AI conversation felt isolated. The model was working with a blank slate, and I was the one responsible for filling it in every time. Now, when I’m working on product strategy, it can reference the source distillations I wrote weeks ago. When I’m making a technical decision, it can surface the framework I already developed instead of me having to reconstruct it from memory. When I’m planning my week, it can remind me about the thing my boss told me was important at the beginning of last week, the thing I completely forgot about after the weekend.

The sync runs automatically. The total cost of the entire system adds up to roughly a cup of coffee over the course of a month.

The Chunking Lesson

The thing that surprised me most while building this was how much the chunking strategy mattered, and how wrong my first instinct was.

Initially, each heading-based chunk was embedded in isolation. That seemed logical. A section is a section, embed it as-is. But what I found was that a section titled “The Agentic Harness” didn’t carry enough context on its own to match a query like “when should I use Claude Code versus Codex?” The content was there. The words were there. But the embedding didn’t understand what the section was actually about because it was missing the surrounding context, the note title, the heading hierarchy, the thread that tied that section to a larger idea.

The fix turned out to be simple. Prepending the note title and heading path to each chunk before embedding it dramatically improved retrieval quality. And here’s the part that still makes me smile: that suggestion came from Codex itself. I was using the system, noticed the retrieval was off, described the problem to Codex, and it recommended the fix. There’s something poetic about the tool you’re building telling you how to build it better.

These are the kinds of things you only discover by using your own system and testing it against real queries across multiple models. The feedback loop between building and using is where the real refinement happens, and I don’t think you can shortcut it.

Why MCP Matters

MCP is what makes this whole thing possible without it turning into a massive infrastructure project. It gives you a standard protocol for exposing tools and context to any AI client that supports it. No fine-tuning, no complex RAG infrastructure, no vendor lock-in. You build the server once and every compatible tool can use it.

The challenges of context management and token consumption still persist, and as agent memory becomes more persistent across platforms, these problems will evolve. But this approach lets you deliberately engineer what information matters rather than hoping the model figures it out from a giant context dump or, worse, having no context at all. It’s the difference between giving someone a library card and handing them the three pages they actually need.

Closing Thought

If you write notes, capture thoughts digitally, or maintain any kind of personal knowledge base, you can build something like this. The gap between “AI that knows nothing about you” and “AI that has access to all the context it needs” is smaller than you think. The tools exist, the protocols are open, and the cost is negligible.

I built this because the friction was driving me crazy. I kept it because once the AI could reference my own thinking back to me, the nature of the conversation changed. It stopped feeling like I was briefing a new contractor every morning and started feeling like I was picking up where I left off with someone who had actually been paying attention.

That’s the difference I didn’t expect, and the one that’s hardest to go back from.