There’s a related question that I’m pondering, Lance:
I recently spoke with a CEO of a large company about their usage of Microsoft’s Copilot and the way that integrates. Because integration or connection to existing knowledge is what we are concerned with here, right?
Well, in the Copilot integration Microsoft accesses everything. The executive literally said, it’s nuts. It reads every document, every email. That’s a massive amount of data in a decent size organization.
Now compare this to our one-man knowledge organizing. While I don’t know, my hunch is that a few well structured documents, well maintained and relevant documentation, kind of curated, will be superior to giving the AI thousand of email and documents. Just the fact how quickly information becomes outdated, or has been discarded while none of the existing data has been removed or updated.
I am working in a very similar way. I think your categories make sense. My thoughts around this are quite similar. In some sense, much of this is having, or creating, and then providing documentation.
And it sounds a bit technical, but really, that is what you'd have to have or come up with, when onboarding a new person to the team, too.
This just one example. There can be info on the project, the task at hand, frameworks to be used, etc.
Then you can mix that with notes on your preferred writing style, an interview transcript, … and so on.
I love your point about naming the files descriptively, then also structuring and formatting the information itself. Related: there was a good post from Ethan Mollick about how most people assume LLMs can read PDFs, but, in reality, that popular file format has a myriad of issues and often give the models a hard time, leaving them with only partial "understanding" of the content. I try and convert a lot into TXT files and check to see that they actually contain what I expect and assume.
All great points! I'm still working out my own knowledge management strategy.
I think the reason this works with Boodle is because they actually vectorize the PDF. I would probably argue that LLMs never understand the content, but we can narrow the guess they are trying to make by structuring content, using knowledge graphs, etc.
There’s a related question that I’m pondering, Lance:
I recently spoke with a CEO of a large company about their usage of Microsoft’s Copilot and the way that integrates. Because integration or connection to existing knowledge is what we are concerned with here, right?
Well, in the Copilot integration Microsoft accesses everything. The executive literally said, it’s nuts. It reads every document, every email. That’s a massive amount of data in a decent size organization.
Now compare this to our one-man knowledge organizing. While I don’t know, my hunch is that a few well structured documents, well maintained and relevant documentation, kind of curated, will be superior to giving the AI thousand of email and documents. Just the fact how quickly information becomes outdated, or has been discarded while none of the existing data has been removed or updated.
I totally agree … at least with the technology we have so far.
Anyone who has tried to develop content for AI at scale eventually runs into problems that are pretty difficult to fix.
But if I curate a more specific, purposeful knowledge base, the problems don’t scale and are more manageable.
Hey Lance, thank you for sharing this.
I am working in a very similar way. I think your categories make sense. My thoughts around this are quite similar. In some sense, much of this is having, or creating, and then providing documentation.
And it sounds a bit technical, but really, that is what you'd have to have or come up with, when onboarding a new person to the team, too.
I have, for example, a file with my general business context (here’s a template and explanation: https://tight-ops.notion.site/Business-Context-6eb37534ad7347abbdaaa247d034263e?pvs=4).
This just one example. There can be info on the project, the task at hand, frameworks to be used, etc.
Then you can mix that with notes on your preferred writing style, an interview transcript, … and so on.
I love your point about naming the files descriptively, then also structuring and formatting the information itself. Related: there was a good post from Ethan Mollick about how most people assume LLMs can read PDFs, but, in reality, that popular file format has a myriad of issues and often give the models a hard time, leaving them with only partial "understanding" of the content. I try and convert a lot into TXT files and check to see that they actually contain what I expect and assume.
All great points! I'm still working out my own knowledge management strategy.
I think the reason this works with Boodle is because they actually vectorize the PDF. I would probably argue that LLMs never understand the content, but we can narrow the guess they are trying to make by structuring content, using knowledge graphs, etc.