Is it even worth trying to block AI or bots anymore?

We publish things on the internet because we want people to find them.

Then someone builds software that finds them on people’s behalf, and suddenly we’re asking everyone to identify a motorcycle.

There’s a business model somewhere in this picture. I’m less convinced it’s the motorcycle.

What are we actually protecting when we block an AI agent?

Our infrastructure? Private information? Something people pay to access? All reasonable answers.

The assumption that every reader must personally load our website before getting any value? That deserves a harder conversation.

The machines have been studying

In September, researchers reported an automated solver completing 463 of 500 reCAPTCHA sessions. That’s 92.6%, with up to 20 challenge attempts allowed per session. It doesn’t prove every security control is defeated, but it makes image puzzles look like an increasingly awkward long-term bet. Research paper

Credential attacks, spam and runaway traffic still need to be stopped. Keep authentication, enforce permissions, rate-limit expensive operations and monitor abuse.

But a customer’s assistant reading a product page and an attacker hammering a login endpoint have different purposes. Calling both “bots” doesn’t tell us what to do about either.

Why is your content public?

A product page helps sell a product. Documentation helps someone use software. An article might build trust, attract subscribers or introduce someone to your work.

If an agent helps accomplish that, blocking it can work against the reason you published the page.

Of course, someone has to fund the content. If the business depends on advertising beside an article, an answer delivered elsewhere may create value for everyone except the person who paid for the reporting.

“Just embrace AI” is a pretty convenient recommendation when you don’t have to make payroll.

And publicly accessible doesn’t mean unlimited permission to copy, train on or resell something. Those uses deserve separate decisions.

Still… where does the value come back?

AIO won’t answer that for you

Call it AIO, AI optimization, or whatever acronym wins this month. Making your content easier for AI systems to discover and reference can help.

Meanwhile, agencies are selling services to get brands into AI answers: creating content, chasing citations and improving how those brands are described. We’ve given it acronyms, GEO and AEO, so naturally there’s an invoice.

While you’re keeping AI out, competitors may be investing in becoming its sources. Blocking access doesn’t stop the conversation about you… it can leave more of that conversation to someone else.

Nobody can guarantee the answer. And the details matter: some platforms let you block training crawlers while allowing search crawlers. Decide which access you’re refusing before treating all of it as the same thing.

You can also become extremely discoverable and earn absolutely nothing.

Before optimizing for mentions inside an answer, decide what a successful mention should produce. A purchase? A subscriber? A qualified enquiry? Paid access to something useful?

Otherwise we’ve invented another dashboard to admire.

Here are a few ways I’d think about changing the product.

1. A travel site could sell the next step.

Let agents discover the destination guides. Build value around live availability, price alerts, personalized comparisons and booking assistance. Earn through disclosed referral arrangements, bookings or a paid planning product.

The article gets someone interested. The service helps them actually go somewhere.

The hard part becomes reliable inventory, useful recommendations and conversion. A thousand generic destination pages won’t do that job.

2. A recipe blog could become a useful weekly tool.

Keep recipes discoverable. Offer a paid product that turns household preferences into meal plans, scales portions, remembers what’s already in the kitchen and builds a shopping list.

An agent can summarize a recipe. An ongoing product can save someone time every week.

That also creates real responsibilities around privacy, accuracy and making the tool useful enough to renew.

3. A specialist publication could sell current data.

Publish analysis that demonstrates expertise. Offer a paid API for fresh prices, benchmarks, availability or historical comparisons, with clear permissions and usage limits.

Now an agent has a supported way to become a customer.

This works when freshness, coverage and reliability matter. Wrapping information everyone already has in an API doesn’t magically make it valuable.

4. A blog could become the front door to a community.

Public essays help people discover the author. A registered or paid community provides expert discussions, introductions, live sessions and contributions from people worth knowing.

A registration wall alone isn’t a product. There has to be a reason to come inside.

Authentication creates a clearer boundary than leaving everything public and guessing which visitor looks human. It also gives you accounts, permissions and the ability to revoke access. It won’t stop every member from copying something.

The stronger protection is that copying an essay doesn’t reproduce the relationships or participation.

5. A site could charge for machine use directly.

Cloudflare’s Pay Per Use beta lets publishers accept AI buyers’ offers for specified uses of their content. Buyers report usage, with billing and payments handled through the system. That gives this idea an actual mechanism. Announcement

Whether it becomes meaningful revenue depends on participation, pricing and trustworthy reporting. “We support payments” and “people will pay us” remain two very different milestones.

6. A store could let the agent bring the sale.

Shopify’s Agentic Storefronts support product discovery and purchasing through AI channels, subject to eligibility and setup. The customer doesn’t have to follow the old browsing journey for the merchant to make a sale. Shopify documentation

The tradeoff is dependence on someone else’s interface. Margins, attribution and the customer relationship still matter.

Pick a business model, then defend it

The awkward middle is a pseudo-public site that needs discovery while spending more and more effort deciding which visitors look sufficiently human.

Sometimes that effort buys meaningful protection. Sometimes it postpones a business decision.

I’d start with three questions:

• What should be discoverable?
• What requires permission or payment?
• What behavior creates measurable harm?

Then measure sales, subscriptions, retention, infrastructure costs and abuse prevented. Also measure legitimate customers you accidentally locked out.

Blocking bots can be worth doing. But if an agent can deliver most of your product’s value by reading a few public pages, the next strategy discussion probably needs to go beyond a better CAPTCHA.