- MIT Media Lab
- Microsoft Research & Carnegie Mellon
- Harvard
- The World Bank
Some authors have lost publishing deals recently over accusations of using AI to write their books. Whether the accusations were true didn’t even matter. The damage was already done.
Substack has also added an AI detector using a software called Pangram. AI detecting AI. Makes sense. This has some concerned though because the track record on these tools aren’t great. For example, a post published in 2008, well before modern AI, was flagged by Pangram as AI-generated. Pangram itself says detection tools can provide useful initial flags but shouldn’t be used as sole evidence.
I know I’m seeing content that is clear AI slop. No one wants that, and those that are using it that way aren’t fooling anyone. That type of usage is 100% giving AI a bad rap. It’s that type of AI slop that has many asking, “is this stuff making us dumber or smarter?”
Researchers have looked into this for us.
And if you’ve read my newest book or other substack articles….you know I love me a research study.
MIT Media Lab
MIT’s Media Lab published a study in June 2025 called Your Brain on ChatGPT. They took 54 people, put EEG headsets on them, and had them write essays across multiple sessions. One group used ChatGPT, one used a search engine, one used nothing but their own head. The brain-only group showed the strongest, most connected brain activity. The ChatGPT group showed the weakest. And when the ChatGPT writers were later asked to write without it, many couldn’t quote sentences from essays they had just finished. The researchers called the effect cognitive debt. Sounds like a good recipe for AI slop.
Microsoft Research & Carnegie Mellon
Second study, from Microsoft Research and Carnegie Mellon. They surveyed 319 knowledge workers and collected 936 examples of AI use on the job. They found that the higher confidence people had in AI, the less they thought critically. AI is making us more stupider.
Harvard
But why stop there? Harvard built an AI tutor for its intro physics course and ran a randomized experiment with 194 undergraduates. Students using the tutor learned roughly twice as much as students covering the same material in class, in less time, and reported being more engaged while doing it. Wait? They didn’t learn take twice as long to learn? Nope. You read that right. They learned twice as much.
Well, this is getting confusing.
The World Bank
Ok. Maybe this will help give us clarity. The World Bank ran a randomized trial in Nigeria. Not the Prince in Nigeria that has a gold mine you inherited. This trial was real. Students used AI for English tutoring over six weeks, with teachers guiding the sessions. The learning gains equaled somewhere between one and a half and two years of ordinary schooling. In six weeks! That puts the program among the most cost-effective education interventions ever measured. Six weeks!
A couple conflicting studies.
Shucks. Actually, it’s my favorite thing. I think there is too much polarization in today’s world and we need to come together. This gives us just the chance to do that.
In the MIT study where those that fully used AI had the worst connected brain activity of the study, the researchers acknowledged that their lab setup didn’t match how people use AI in real life. They also discovered that the participants who wrote from their own heads first and brought in AI afterward showed the strongest brain connectivity of anyone in the study. Think first, then bring in the tool.
In the Microsoft study, they didn’t just find that those that heavily trusted AI had less critical thought. They also found those that had higher self-confidence exhibited more critical thinking. trusted their own skill, the more they checked the work, the higher level of critical thought.
The most important element that ties these studies together, the element that makes these seemingly contradicting studies match is this: who did the thinking?
Who did the thinking?
When people handed the entire job to AI, their brain disengaged and their work got worse. When a human stayed in charge and used AI to support the work, people learned faster and performed better.
AI is exactly like every human consultant I’ve ever paid.
The way I see it, AI is exactly like every human consultant I’ve ever paid. Just a heck of a lot faster, cheaper, and accessible. When I needed help with whatever my business roadblock was, I found someone that knew how to do it better. I would ask them questions. I would listen. I would take their answer and apply it to my problem based on my own principles. I’ve paid people to help me write better. To edit what I’ve written. To create graphics for me. I’ve paid people to create presentations for me. I’ve paid people to advise me on how to have a difficult conversation. All things one can use AI for now.
Which brings me to my AI policy.
I use AI. Regularly. Intentionally. Here is where AI shows up in my work.
Brainstorming
I use AI to think out loud and test ideas. I use AI to give me an excessive number of ideas so I can filter through them myself. I use AI to poke holes in my own thinking. I don’t use AI to give me ideas on how to be funny. I’ve tried that enough times to know AI sucks at that.
Copy editing
I use AI to give me suggestions on how something I wrote could be clearer and easier to understand. AI helps tighten my sentences. Sometimes it tightens too far which takes the soul out of my writing, so I politely, and sometimes annoyingly, tell AI that was a terrible edit. The final decision on every sentence is mine.
Research
I never accept an AI answer as the original truth. When it gives me information, I ask where it found it, then I go read those sources myself and verify. If a claim cannot survive that check, I do not use it. I used AI to find those four studies and their sources that are in this article.
Titles and marketing
I ask AI for title options built on marketing principles I have paid actual humans to teach me. Then I pick the option or combine options to better match what I want to convey.
I’m an open book.
I also use AI to help me as the CEO of Incite Tax. The purpose of this article was to be clear on how I use AI in my writing. But if you have questions about any of this, let me know. I’m an open book.