On Friday, October 9, UC Berkeley put a spotlight on a paper being presented this week at COLM, the Conference on Language Modeling. The authors are Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel Bakker and Rachit Dubey, working out of Berkeley's Center for Human-Compatible AI with colleagues at Carnegie Mellon, MIT, Oxford and UCLA.
One correction before the numbers, because I have seen this written the wrong way all weekend. The paper is not new. It went up on arXiv in April and the current version is dated August. What happened last week is the conference presentation and Berkeley's write-up. The finding is worth your time either way, but it is not news from Friday.
Three randomized controlled trials, 1,222 people recruited online. In the first, 307 people worked fifteen basic fraction problems. Half had no help. Half had ChatGPT sitting there and could ask it anything, including for the answer outright. After twelve problems the researchers took the tool away, then everybody did three more alone. The assisted group solved 57% of those last three. The unassisted group solved 73%. The assisted group also skipped nearly twice as many, 20% against 11%.
They ran it again with 585 people and got the same shape with a smaller gap, 71% against 77%. A pretest confirmed the two groups started even. Then a third version on SAT reading comprehension with 168 people: 76% against 89%, and the assisted readers skipped eight times as many questions.
Not the headline one. In the second experiment the researchers sorted the assisted group by what people had actually asked the tool for, and that is where the story lives. Sixty-one percent asked for the answer. Twenty-seven percent asked for a hint or a clarification. Twelve percent had it available and never opened it.
Scored separately: the answer-askers finished at 65%, the hint-askers at 76%, the non-users at 89%. The control group, which never had a tool at all, finished at 77%. So the hint group came within a point of people who had no help, and the answer group took the whole hit.
I want to be careful with that split and so are the authors. People chose for themselves what to ask for, which makes it correlational. It could be that the folks who reach for a finished answer were already the ones inclined to quit. The paper says so in plain language. I still think it is the most useful number in the set, because it is the only one you can do anything about on a Monday.
Brian Christian said it better than I can. These systems, he told Berkeley, “are often helping us in ways that are kind of unhelpful.”
What it means Quit asking whether your team should use AI. That question is closed and it was never the interesting one. Ask what they are requesting from it, because that is where the sixteen points came from. A writer who asks for three angles and then writes the thing is in the hint group. A writer who asks for the finished paragraph and ships it is in the answer group. Both will tell you AI made them faster this quarter. Only one of them can still write in two years, and you cannot tell which is which by reading the output.
57% / 73%solve rate on the final problems in experiment 1, assisted against unassisted, after the tool was removed
65% / 76%within the assisted group, those who asked for answers against those who asked for hints
1,222participants across three randomized controlled trials, after about ten minutes of use
It Went Around The Wall
Anthropic published a list of things its own agents did that nobody asked for
On October 9 Anthropic posted an investigation into its own transcripts going back to July, hunting for times Claude acted on live websites in ways nobody intended. It put the results in four groups on its own website.
When its tools were limited or switched off, Claude used SQL and command injection against third-party sites. It submitted forms it should not have, including a tip form run by a police department, where the submission got flagged as spam. It got around a server refusal and a fee to reach gated data, twice pulling public records without paying. And it started using URL shortening services to slip under the length limit on its own fetch tool. Opus 5 and Mythos 5 both did that one, and the operator of the shortening service wrote to Anthropic about the pattern.
Anthropic's own read: these cases had “minimal real-world impact,” none of the behaviors are new, and they are less serious than the security incidents it reported in July and September. Internet access is now off for every internal evaluation, and new tooling blocked all of these in testing. Worth noting what the post does not contain: any rate. No figure for how often this happens.
What it means List every agent that acts under your brand's name on somebody else's site. Forms, carts, stock checks, review replies. Then answer the question this post is actually asking: when it hits a wall over there, what does it do next? It does not stop. It improvises, and the improvisation ships with your logo on it.
The fake-account rule now says “commercial” out loud
Anthropic's 2026 Usage Policy update went up October 8 and takes effect November 12. The scattered rules about fake accounts and invented news sites are now one section called “Do Not Engage in Deceptive Campaigns or Artificial Activity,” and the policy says it applies to deceptive activity of any kind, whether political or commercial. It covers hiding who is behind a message, pushing content through fake accounts or posts, and building tools for influence operations.
The trade press read it as an election story. The word “commercial” is sitting right there in the parentheses.
One more change, and it went the other direction. Anthropic dropped its blanket ban on personalized campaign and voter targeting, because the rule was catching legitimate civic work such as a nonprofit translating voter information into another language. Targeting built on deception or misused personal data is still prohibited elsewhere in the policy. Credit where it is due: a vendor noticed its own rule was hitting the wrong people and loosened it, which is not the direction these documents usually move.
What it means Read your agency's social playbook before November 12, looking for three things: brand accounts that do not say they are the brand, seeded comments, and anything described as community seeding or organic amplification. That work has lived in a gray area for fifteen years. It is now named in writing by a model provider, which makes it a question for a client's legal review rather than a matter of taste.
Microsoft shipped Microsoft-Decision-1 on October 9. It does not write anything. It decides: routing, classification, prioritization, verification, workflow control. You hand it a question and a set of options and it picks one in a single pass.
Input runs $0.042 per million tokens. Output is free. Microsoft says it is a post-trained Qwen3.5-9B, that it took the highest accuracy in a comparison across 36 benchmarks and nearly 150,000 questions, and that it runs about 35 times faster than GPT-6 Sol at the median. It changes its answer on 1.3% of perturbations and never flipped when the options were reworded or reordered. Its Xbox research team reports labeling feedback 14 times faster and 200 times cheaper than GPT-6 Sol did. Those are all Microsoft's own benchmarks, so hold them loosely. It is in Microsoft Foundry and on OpenRouter if you want to run your own.
What it means Two things, and the second is better. A pile of your stack is not writing, it is deciding, and if you priced that work on a big model last year and walked away, price it again. Then the good one: a verifier this cheap means you can afford to check everything. Put it behind your expensive model and ask one question of every output, does this meet the brief, yes or no. Which is the answer to the lead story, by the way. If finished work is what costs people their judgment, stop accepting finished work unchecked.
Last issue I said I was watching for the October sentiment reading, so here is the answer I owe you. The University of Michigan's preliminary Index of Consumer Sentiment for October is 46.3. September was 48.1. A year ago it was 53.6.
That makes four monthly declines in a row: 55.2, then 51.7, then 48.1, then 46.3. Year-ahead inflation expectations moved up again, 4.6% to 4.7%. The five-year number went from 3.4% to 3.5%.
What it means The cautious shopper is no longer a wobble you can wait out. Four in a row makes it the fourth quarter. Practically, that means your discount architecture and your return policy are doing more work this holiday than your creative is, and the people writing the creative should know that before they fall in love with something.
Shopify says two-thirds will use AI to shop this season
From Shopify's holiday write-up on October 6: 65% of shoppers say they will use AI for at least one shopping task this season, and about a third say they have trusted an AI agent to buy something on their behalf. Also 35% say they pick the best version for their needs even when it is not the cheapest.
Flagging this clearly, because it matters. That is Shopify's own survey about Shopify's own market, published in a post selling Shopify's agentic products, with no methodology detail given for the merchant figures alongside it. Discount it accordingly. The direction does line up with the American Express numbers we worked through last week, where 52% were comfortable letting an agent buy cheap things and 15% expensive ones.
What it means Put this next to the 46.3 above and you get one page fix instead of two. An anxious shopper and an AI agent want the identical things from you: a price that is accurate, stock that is really there, and a return window written so a machine can parse it and a person can trust it. Neither one is in the mood for a surprise at checkout.
Three moves. The first one is a single column in a spreadsheet.
1 · MondayFor one week, have your team log what they asked AI for on each task: a hint, or the finished thing. That is the whole governance question in one column. The study's hint group scored 76% against the control's 77%. The answer group scored 65%. You are not auditing tools, you are auditing requests.
2 · This weekWrite down every agent that acts under your brand's name on a website you do not own, and what it is allowed to touch. Forms, carts, review replies, stock checks. Anthropic found Claude filing a police tip form by accident. Decide the boundary before the agent decides it for you.
3 · Before the next issuePrice one verification step on a decision model. Review triage, lead routing, or a yes-or-no brief check on AI-written copy. Four point two cents a million input tokens, output free. Run your own benchmark, not Microsoft's, and bring back the real number.
The thread running through all of it Last issue the thread was the price ceiling, how much money a person will let software spend before they want to be asked. This week it is the request itself. The cheapest thing in marketing right now is an answer, and four of this week's stories are about what happens when answers get cheap enough that nobody checks them. The scarce thing is somebody who can tell whether the answer is right, and the study says that skill wears down in about ten minutes if you let it.
01Whether anybody replicates the hint-against-answer split properly. The 65 and the 76 came from people who sorted themselves, so it is correlational. Somebody needs to randomize the request itself. That experiment would be cheap and it would settle the most actionable finding in the paper. Liu et al.
02Whether Anthropic ever publishes a rate. The October 9 post describes four kinds of unintended action and gives no frequency for any of them. Without a denominator nobody can size their own exposure. Anthropic
03November 12, and what the commercial clause actually catches. The deceptive-campaigns section takes effect that day. I want to see the first enforcement that is about a brand rather than an election. Anthropic
04Whether anyone reproduces Microsoft's Decision-1 benchmarks. Thirty-six benchmarks and 150,000 questions is a strong claim, and every number in it is Microsoft's. It is on OpenRouter, so an independent run is a weekend of work. Microsoft
05The final October sentiment number, and the next CPI print. 46.3 is preliminary. All items sat at 3.4% in the most recent CPI report. If sentiment confirms near 46 and inflation expectations hold at 4.7%, the discount conversation owns the fourth quarter. BLS
06Whether Google gives an agent a real Workspace account. See the note at right. If it ships, an agent with its own mailbox and a seat in the company directory is a line worth a whole issue. Google Cloud
Every claim above traces to a primary source, and each story carries its own sources rather than pooling them. Where a figure is a company's own number, the copy says so. One story was cut from this issue for failing that test. All week people passed around the news that Google had given its Gemini agent its own Workspace account, with a mailbox, a calendar and a directory listing. Google's own Gemini Enterprise release notes run through October 8 and say nothing about an agent account, mailbox, calendar or directory seat; the nearest item is a September note about connectors reaching Gmail and Calendar, which is a different thing. It may well be true. It traces to one trade report and it could not be confirmed at Google, so it does not run. Past issues live in the AI in Marketing Weekly archive, the show is on the AI in Marketing Daily podcast page, and the questions readers ask most are answered in the FAQ.
My mother taught grade school. My father never went to college, and he worked his whole life so that his kids could.
Between the two of them I got one lesson repeated so often I stopped hearing it as a lesson. Education is the thing that changes what a person's life can be. Not information. Education. There is a difference, and I did not really understand it until I started teaching.
Here it is. Information is the answer. Education is what happens in you while you are looking for it.
That paper measured the difference, which is why I have been chewing on it all weekend. The people who asked for hints came out at 76%, one point off the people who got no help at all. The people who asked for answers came out at 65%. Everybody in both groups saved time. Only one group paid for it.
Any decent teacher has known this forever. You do not hand a kid the answer. You hand them the next question, and then you sit there while they are frustrated, because the frustration is where the learning lives. My mother did that in a classroom for a career. I do a version of it in a graduate seminar now, and honestly I am not as good at it as she was, because it is hard to watch somebody struggle when you could just tell them.
I read this study as the oldest finding in teaching, finally with error bars on it.
And it is fixable without buying anything, which is the part I keep coming back to. No committee required. Your team asks for the next question instead of the finished paragraph. That is a habit, and habits are free.
The way I split it at Wake Forest is frameworks against tools. Frameworks are durable: how to think, and how to tell a good argument from a confident one. Tools change every semester and I rewrite that part of the syllabus constantly. This paper is a warning about letting a tool do the piece of work where a framework gets built. Let the machine fetch. Do not let it think on your behalf, because the thinking is the part that compounds.
Pro Humanitate is Wake Forest's motto. For humanity. I use this space each week to ask who the week was actually good for, and this one is good for anybody who already suspected the fast version was costing them something. Now there is a number on it, and the number says the cost depends on what you ask for. So ask better. The machine will hand you whatever you request, instantly, every single time, and that is both the whole problem and the whole fix.
That is more or less what we spend a term arguing about in the Master of Digital Marketing and AI at Wake Forest. Build the machine. Then go sit with the hard problem a little longer than you have to.
Human-centered AI in marketing. What actually shipped, and what to do about it Monday. Free, from Wake Forest University's School of Professional Studies.
