I couldn't trust AI, so I asked AI
How I built better-ai skills to make AI research and write in ways I can trust
I thought that AI could be a powerful tool to learn faster. It can go read hundreds of articles at lightning speed, and give me a digestible summary of what it found.
After using it for a while, I found I couldn't take its answers at face value. On topics I knew well, I'd spot missing details or answers that were only partly right. On topics outside my expertise, I had no way to tell.
So I built a set of skills to make agents research, write, and act in ways I can rely on, based on first principles.
Why I couldn't trust AI
Hallucinations, knowledge limits, and laundering
Recent AI models are nothing like those of a year ago, but they still hallucinate, and I don't expect that to fully go away. Researchers disagree on why: some argue it's an intrinsic part of how LLMs work1, others argue that training and benchmarks reward guessing over admitting uncertainty2. Where models do well is verifying their own output when reliable external feedback exists3. For example, they run their own code, write test cases, and check for syntax errors before answering.
Models also have training date cutoffs. They know nothing of what happened after that date, and will confidently misread new technologies. While experimenting with decision/System One models, I struggled to get agents to write useful code. They'd treat System 1 and 2 as different ways to prompt an agent. They thought TypeSafe AI meant passing a JSON Schema to the LLM.
For research, the most damning problem for me was their confirmation bias4: they start from what they already believe. When I asked why LLMs hallucinate, Opus 5.5 would search "Why language models hallucinate Kalai 2025". This embeds an answer direction into the query: it finds the paper it had in mind, and doesn't automatically look for work that disagrees. The results then launder the model's own assumption into what looks like independent evidence.
It's load-bearing, and I'm still waiting on your answer
LLMs tend to overuse certain words and sentences (e.g. claudisms). Multiple papers567 have explored how word distribution has changed due to AI use. These preferences may partly come from reinforcement learning from human feedback (RLHF)8.
LLMs also build a large context over time, as they read through files, make web searches, call tools, etc. They write as if you share that context, when you don't9. If your codebase is entirely generated, you may have no idea about the architecture patterns it used10.
Overused phrasing and missing context make AI output harder to read: you spend effort decoding its words and filling in context you never saw.
Bias towards the verifiable
If we look at the most used AI benchmarks, we see strong biases towards the verifiable. In Fable 5.2's announcement, several of the headline benchmarks are software engineering and security tasks: FrontierCode 2.0, FrontierSWE v2, CyScenarioBench, and CVE Rediscovery.
OpenAI recently published articles on finding security vulnerabilities and mathematical problems. Both categories require extensive reasoning and exploration to find a solution, but their answers are easy to verify.
I've written in the past about how we're dancing with the compiler with Rust, and I suspect many AI projects are using Rust because it gives that clear, explicit feedback. We're also seeing similar trends with formal verification and testing techniques that used to be quite niche.
These days, I mostly use AI to build understanding and find the least-bad11 solution, not to execute tasks. These are harder to verify. Even research benchmarks like WANDR or BrowseComp grade answers that are easy to check. An AI agent can write code for me, then test it. It cannot reliably test research results12.
Working from first principles
After a year of experiments and iterations, I ended up building a set of better AI skills. They're built around six ideas:
- Start from principles. Every skill lays out its principles first. The repository also has a dedicated
PRINCIPLES.mdfile to explain its ground rules. - Rules come from research. There is still so much we don't know about how AI works best, or how our brains work. Instead of relying on beliefs, I ground almost every13 rule in research on where models fail and how people read.
- Cognitive capacity is scarce. As a human, I shouldn't be deciphering the work of agents, or checking what they could have checked themselves. They should verify what they can, say what they couldn't, and only bring me what needs my judgement.
- We're all unique. For cases where preference genuinely matters, the agents should adapt to the preferences the user states, not ones they guess14.
- Honesty goes a long way. Agents should report only what they did and what they verified. They should highlight problems and mistakes.
- Permanent self-improvements. The skills can be used to improve themselves, and the repository provides ways to find improvements from past feedback and memories, and to test them.
Research better
The most important skill for making AI more reliable for me is /research-better. It reduces confirmation bias by starting searches from generic terms instead of the answer the model already expects. AI agents both process natural language and hold a compressed copy of their training data. When researching, I want it reading current sources, not recalling training data that may be out of date or wrong.
The skill goes back to first principles: how would we research something we know little about? It starts with exploratory searches by rephrasing the original question to discover key concepts. It then asks the user to confirm the actual meaning, direction, and goals. This is an important part, because the same term can mean two things in different contexts, as System 1 did for my agents. Knowing the goals also tells the agent when to stop.
Now the real search begins. The main agent switches to a coordinator role: it sets up a shared research board15 and spawns research agents. Each one picks up a question, finds claims and sources to answer it, writes back its findings, and proposes new questions. Another pool of verifier agents checks each claim against the passage it quotes from its source.
You can see that process in action in the video below. The agent started with 10 questions and ended with 39. The map shows where each question came from.
By the way, this visualisation is bundled with the /research-better skill, and you can talk to the coordinator while it runs, to steer the research or ask what's happening.
Write better
The first skill I wrote was /write-better. When AI can produce so much content so fast, our cognitive capacity becomes the bottleneck. I'd often struggle to understand what an AI is asking me.
It starts by identifying the reader and naming the goal. Every reader knows different things, so it checks the conversation's context, memories, and other signs of preference. It then names the goal of the message, because we're always writing for an outcome: building understanding, teaching something, getting an action, deciding, ...
Next, it assumes that its first draft is unclear, because the agent has context the user hasn't seen: all the tool calls, files read, documents searched. Finally, it cuts what's unnecessary, and puts the main point first.
Here is an example where I asked an AI agent to read a repo I hadn't touched in a long time, then propose next steps. The slides show its answer before and after16 calling /write-better.
Just ask me
I'd fix two ingestion bugs on this branch before ingesting anything else.
Should I start porting fixes 1 and 2 from the research branch?
I prefer a clear question over what you'd do.
Tell me what you guessed
Nothing stated.
I've assumed you want to decide what to work on next. Tell me if you wanted something else.
Agents always assume what the user intends.
What branch?
Cause: main's "feat: attribute claims" commit copied the prompt from feat/attribution-citation-mapping …
The research branch already has this fix …
Should I start by porting two fixes from feat/attribution-citation-mapping? That's the research branch with 29 eval iterations.
It explains what the branch contains.
Which 99.92%?
Consequence: the 99.92% mapping accuracy recorded on that branch was measured on its code. It doesn't describe what main runs.
The research branch measured 99.92% mapping accuracy on its own code, so that figure doesn't describe what main runs.
I don't know what the 99.92% number refers to.
Design better
Writing isn't the only way we communicate information. Visual storytelling and usable interfaces are just as important. As you saw with the /research-better skill, several skills come with interactive visualisations.
Like the two previous skills, /design-better starts by naming the goal. It then reads what's already there: design languages, existing interfaces, styles, etc. Depending on the goal, it designs and builds an interface, constructs a design language, or reviews finished work.
Here is an example of building the before/after comparison for /write-better:
Do better
On top of exhausting my cognitive capacity, I'd often get frustrated with AI outputs. I wouldn't have the energy to explain to it what went wrong, especially when it felt like repeating myself.
For these situations, you can call /do-better. It applies the same six principles. It compares what you asked with what it did, in context, to recover your intent and find where the two differ.
There are two failure modes I wanted to guard against. The agent could become overly cautious, asking questions that'd burn even more cognitive capacity17. The agent could also become sycophantic18, agreeing with you even when you're wrong.
Finally, it fixes the issue and reports back to you, leading with the result and how it misinterpreted you.
Try it yourself
These skills don't make AI foolproof. Hallucinations aren't going away anytime soon, and agents still can't reliably verify research results on their own. What changed for me is that they show me where to look: what they verified, what they couldn't, and what needs my judgement.
While I've built these for myself, I hope they can be useful for others too. Try those skills for yourself and feel free to open issues or proposals in the repo.
How I used better-ai in this article
/research-betterfor the background research and fact-checking. It added 10 references, and corrected two claims I had stated too strongly. I had written that hallucinations are an intrinsic part of how LLMs work; researchers actually disagree on why they happen. I had also written that research benchmarks get closer to measuring the work I do with AI; they still grade answers that are easy to verify./write-betterto fix typos and awkward phrasing, and to rework the introduction and conclusion. The introduction now says why I couldn't trust AI and what I built, and the conclusion says what these skills don't fix./design-betterto change the table of contents, build the carousel and before/after components, generate banner images, and create social media cards.
-
Xu, Z., Jain, S., & Kankanhalli, M. (2024). Hallucination is inevitable: An innate limitation of large language models. ↩
-
Kalai, A. T., Nachum, O., Vempala, S. S., & Zhang, E. (2025). Why language models hallucinate. ↩
-
Kamoi, R., Zhang, Y., Zhang, N., Han, J., & Zhang, R. (2024). When can LLMs actually correct their own mistakes? A critical survey of self-correction of LLMs. ↩
-
Jhaveri, A. R., GX-Chen, A., Sucholutsky, I., & Choi, E. (2026). Failing to falsify: Evaluating and mitigating confirmation bias in language models. ↩
-
Liang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., … Zou, J. Y. (2024). Monitoring AI-modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews. ↩
-
Liang, W., Zhang, Y., Wu, Z., Lepp, H., et al. (2025). Quantifying large language model usage in scientific papers. ↩
-
Kobak, D., González-Márquez, R., Horvát, E.-Á., & Lause, J. (2025). Delving into LLM-assisted writing in biomedical publications through excess vocabulary. ↩
-
Juzek, T. S., & Ward, Z. B. (2024). Why does ChatGPT “delve” so much? Exploring the sources of lexical overrepresentation in large language models. ↩
-
Shaikh, O., Gligorić, K., Khetan, A., Gerstgrasser, M., Yang, D., & Jurafsky, D. (2024). Grounding gaps in language model generations. ↩
-
Chou, C., Lee, S. Y.-T., Choudhuri, R., & Ma, K.-L. (2026). Vibe-GUIDE: A graph-based user interface in IDEs for oversight in vibe coding. ↩
-
My take on my position (principal engineer) is that you don't need someone like me if the solution is obvious. You need someone like me when you need a novel solution, or you need to balance multiple trade-offs without a best solution. ↩
-
Verma, A. (2026). Is this citation on point? ↩
-
Because AI is still so new and changing, there are many situations where the research isn't conclusive. If this is the case and I have a personal preference, I embed that preference instead. ↩
-
Jain, S., Park, C., Viana, M., Wilson, A., & Calacci, D. (2025). Interaction context often increases sycophancy in LLMs. ↩
-
Using a blackboard-like approach. ↩
-
I had to forbid it from using skills at first for a fair comparison. Otherwise Claude will invoke those skills automatically. ↩
-
Vijayvargiya, S., Viswanathan, V., & Neubig, G. (2026). Asking what matters: Reward-driven clarification for software engineering tasks. ↩
-
Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., … Perez, E. (2023). Towards understanding sycophancy in language models. ↩