Best SEO tools review 2026: the free and paid tools we actually use in the age of AI search
Nine years after our first review, the core stack is the same three tools with an AI layer on each, plus two categories that did not exist: citation tracking and crawler control. Here is what we use, what it costs, and what to stop paying for.
What are the best SEO tools in 2026, now that AI search is here?
Screaming Frog, Ahrefs, and Google Search Console remain the working core, each with a new AI layer: Screaming Frog's MCP server, Ahrefs Brand Radar, and Search Console's generative AI report. Add one citation tracker (Canonry or Otterly), a fan-out tool (Qforia), and a deliberate robots.txt and CDN policy. Skip llms.txt generators and schema shortcuts.
There are few industries which change as quickly as the world of SEO, and the nine years since we last published this review have been the most disruptive of the lot. Google now answers a growing share of searches itself, ChatGPT and Perplexity cite pages that never ranked, and the crawlers reading your site are no longer only Google's. The tools have changed to match, some beyond recognition, and a fair number of the ones we recommended in 2017 no longer exist.
The core stack has not changed as much as the marketing would have you believe. Screaming Frog, Ahrefs, and Google Search Console are still the three tools we open every day, and after all these years we are still learning new tricks with each of them. What has changed is that each now has an AI layer, that tracking whether an answer engine cites you has become a category of its own, that controlling which bots can read your content has become a job, and that the most useful new tool is not a dashboard at all but an AI assistant wired into the data.
This review covers what we consider to be the best SEO tools currently available, based on years of using them on client work. It would not be feasible (or enjoyable for our readers) to review every piece of software we have tried, so we have picked our favourites, the ones we rely on week in, week out, plus a short list of things we have stopped paying for. Prices were checked on 5 September 2026 and will move; the vendor links are there so you can check.
What has changed since our last review
Four things have changed, and each one changes which tools earn their place.
Clicks are harder to come by. Ahrefs re-ran its AI Overviews study on December 2025 data across 300,000 keywords and found that when an AI Overview appears, the top-ranking page's click-through rate is 58% lower than it would otherwise be. The same study on March 2025 data had put the gap at 34.5%, so the direction of travel is clear. Google's AI Mode, the fuller conversational version, has been rolling out in the UK since 28 July 2025.
Citations do not follow rankings. In an Ahrefs analysis of 15,000 prompts, only 12% of the URLs cited by ChatGPT, Gemini, and Copilot ranked in Google's top 10 for the same question; Perplexity was the outlier at 28.6%, and around 80% of cited pages did not rank at all for the query. A rank tracker on its own now tells you less than it used to.
AI referrals are real but small. Across 3,000 sites, Ahrefs found AI chatbots send 0.17% of the average site's visitors, with ChatGPT accounting for half of that; across 74,752 tracked sites, all chatbots combined sent 3.5 million visitors in March 2026, 0.28% of total traffic. Worth measuring, not worth reorganising your whole budget around.
The crawlers reading your site are not all Google's. Vercel and MERJ analysed hundreds of millions of requests and found that none of the major AI crawlers execute JavaScript: OpenAI's GPTBot made 569 million requests on Vercel's network in a single month, Anthropic's Claude 370 million, and neither renders a page the way Googlebot does. Only Google's Gemini, which rides on Googlebot's infrastructure, sees rendered content. If your important text arrives client-side, most AI systems never see it.
One more thing has not changed, and it matters for the shopping list. Google's own documentation says there are no additional requirements to appear in AI Overviews or AI Mode, no special schema, and no machine-readable AI file. Its May 2026 guide to optimising for generative AI features says the same in plainer words: you do not need llms.txt files, special markup, or Markdown, because Google Search does not use them. Any tool that sells itself as a shortcut into AI answers is selling something Google says does not exist.
How we chose
We kept a tool in this review if we have used it on live client work in the last year, if it does a job nothing free does better, and if we would still pay for it next month. Where a free tool and a paid tool overlap, we say which we would pick if we could only have one. Prices are the vendor's own, on the date given, and quoted in the currency the vendor lists them in, so you will see pounds for Screaming Frog and Ahrefs and dollars for the American tools. We have no affiliate arrangements with anyone listed, so if a tool is here it is because we like it.
Website audits and crawling
(Free and paid)
What is the best SEO tool for performing a website audit? If we could only pick one, it would still be Screaming Frog. It has been on our list since the first version of this review and it has aged better than anything else on it. The free version crawls up to 500 URLs; a licence is £199 per user per year, which for the amount of work it does is close to embarrassing.
What has changed is the AI layer. Version 21 (November 2024) added direct API integration with OpenAI, Gemini, and Ollama, so you can generate alt text, summarise page content, or classify intent while the crawl runs. Version 22 (June 2025) added semantic similarity analysis, which finds pages that are too alike without being exact duplicates, the cause of most cannibalisation we see. Version 24 (May 2026) added an MCP server, which means you can ask an AI assistant to run a crawl, pull the data, and chart it in plain English (more on that below). We run it on a Hetzner server for large crawls and on the desktop for everything else.
(Paid)
Cloud crawlers for large estates. Lumar is what DeepCrawl became in September 2022, and Oncrawl has been part of BrightEdge since February 2022; both still run as products. We use Oncrawl on a 17-site publishing estate where the value is the log-file analysis and the ability to export a slim set of columns across sites. For most B2B sites under 50,000 URLs, Screaming Frog on a decent machine does the job.
Screaming Frog Log File Analyser
(Free and paid)
We said in 2017 that this was one for the more advanced SEO. In 2026 it is for anyone who wants to know which AI bots are reading their site, and Screaming Frog now describes it as a tool for analysing search engine and AI bot crawling. The free version handles 1,000 log events and a licence is £99 a year. Your server logs are the only honest record of whether GPTBot, ClaudeBot, PerplexityBot, and the retrieval bots that answer live questions are fetching your pages, how often, and what they fetch. Analytics will not tell you, because the AI crawlers do not run the JavaScript your analytics tags rely on.
(Free and paid)
The quickest way to find duplicate content on a smaller site, and the free service covers sites of up to 250 pages. Screaming Frog's semantic similarity has taken over this job for us on larger sites, but Siteliner remains a handy five-minute check, and one we still reach for.
What Google and Bing give you for free
(Free)
It really is still a must-have, and it is where the biggest change in this section has happened. Since June 2026 Search Console has had a generative AI performance report showing how often your pages appear in AI Overviews, AI Mode, and the generative features in Discover, broken down by page, country, and date. It went live for every property worldwide on 31 August 2026, alongside a toggle that lets you opt your site out of Google's generative AI features altogether. Two caveats: at launch the report shows impressions, not clicks, and opting out means no traffic or impressions from those features. It remains the best free view of AI exposure you can get, because it is the only one that measures rather than infers.
(Free and paid)
Universal Analytics stopped processing data on 1 July 2023 and access was removed a year later, so if your reports still reference "UA" something has gone wrong. GA4 is where you will see AI referrals arriving, as long as you set up a channel group for chatgpt.com, perplexity.ai, and the rest; by default they hide inside referral traffic.
(Free)
Unchanged in role since 2017, and still one of the more satisfying tools on this list: the way to deploy tracking without waiting in a developer's ticket queue. One warning that did not apply then. Tag Manager injects client-side JavaScript, and AI crawlers do not run it, so never use it to inject content, links, or structured data you want an answer engine to see.
(Free)
Worth setting up for the same reason as always, with a new one added: Bing's index feeds Microsoft Copilot (Microsoft's own documentation describes the agent sending a distilled query to the Bing search service and building its answer from what comes back), so a page Bing cannot see is a page Copilot is unlikely to cite. It takes 10 minutes to verify, and IndexNow submission comes with it.
PageSpeed Insights and Lighthouse
(Free)
Still the reference for speed, now built around Core Web Vitals. Interaction to Next Paint replaced First Input Delay as the responsiveness metric in March 2024, so old audits that quote FID are out of date. Google retired its separate Mobile-Friendly Test in December 2023 and points people to Lighthouse instead, which is fine: a Lighthouse run in Chrome DevTools covers mobile usability, performance, and accessibility in one go.
Rich Results Test and the Schema Markup Validator
(Free)
The Structured Data Testing Tool we recommended in 2017 lives on as the Schema Markup Validator, and Google now points you to the Rich Results Test for anything that affects search features. Use the Rich Results Test to check eligibility for a specific rich result and the Schema validator to check the markup is well formed. Keep expectations in proportion: Google says no special structured data is needed for AI Overviews or AI Mode, so schema is hygiene, not a lever.
Rankings, backlinks, and keyword research
(Paid, with free tools)
If you wanted to know how to check the backlinks of any website in 2017 and could only pick one tool, we said Ahrefs. That is still our answer, and Ahrefs has spent the intervening years becoming the tool we use for most things that are not crawling. UK pricing on 5 September 2026 starts at £99 a month for Lite, which now includes API access, MCP access, and five tracked AI prompts, and rises through Standard (£199) and Advanced (£359). We use it for backlinks, keyword research, and site audits, and its rank tracker for our own site; its published research (the click-through and citation studies quoted above) is also the most rigorous public data on AI search we have found.
(Paid add-on)
Ahrefs' AI visibility product, sold per AI platform index at $199 a month, or $699 a month for all of them, on top of a Lite or higher plan. It reports brand mentions, citations, and share of voice across a pre-collected index of 462 million monthly prompts (308 million of them from AI Overviews), plus custom prompts you define. We track a set of daily ChatGPT prompts for our own site with it. Read the caveat in the AI visibility section before you buy, because the same limitation applies to everyone.
(Paid, with free tools)
Still the jack of all trades, and now owned by Adobe, which completed the acquisition on 28 April 2026 and talks about SEO, "generative engine optimisation", and "agentic search optimisation" in the same breath. Its AI Visibility Toolkit is $99 a month per domain billed annually for 25 custom prompts tracked daily across ChatGPT, Google AI, Gemini, and Perplexity. If your team already lives in Semrush, switch that on before buying a separate tracker.
Google Keyword Planner, Google Trends, and AlsoAsked
(Free / Free / Free and paid)
Keyword Planner still needs an Ads account and still brackets volumes unless you spend money, but it remains the source everyone else's numbers derive from. Google Trends is unchanged, still free, and still one of our favourites for seeing how demand for a term moves over time; its sister tool Google Correlate, which we recommended in 2017, has since been retired. AlsoAsked maps the "People also ask" tree for a query, which is the nearest free thing to seeing how a question fans out (see below).
AI visibility and citation tracking
This category did not exist in 2017. It answers a simple question: when a buyer asks ChatGPT, Perplexity, Gemini, or Google AI Mode about your category, are you mentioned, are you cited, and who is named instead?
Before the tools, the caveat that applies to all of them. Most AI measurement products report "impressions" inside LLMs. Almost none of them can measure that; they infer it from citation rates and use organic search volumes as a proxy for how often a prompt is asked. Treat those numbers as a benchmark, not a fact. Answer engines are also non-deterministic: the same prompt returns different sources on different days, so a single "AI rank" tells you almost nothing. Track a fixed set of buyer questions, re-run them the same way on a schedule, and read the trend.
(Paid, from $29 a month)
The cheapest way in, and the one we point small teams at first. The Lite plan tracks 15 prompts across ChatGPT, Google AI Overviews, Perplexity, and Copilot, with Google AI Mode, Gemini, and Claude sold as add-ons. Enough for a small team to watch its 15 most commercial questions.
(Paid)
Priced by tracked prompts (plans at 50, 150, and 350 prompts, three models each, daily tracking, unlimited users). The one we see most often at mid-sized marketing teams and agencies.
(Paid)
The enterprise end, with self-serve Starter and Growth plans on monthly credits and a custom tier above them, plus "agent analytics" that reads AI-sourced traffic from your CDN or hosting logs (Cloudflare, Vercel, and others). More than most B2B teams need, but the reporting is the most complete we have seen.
(Free, open source, self-hosted)
Our preference for teams with a developer on hand. Canonry tracks what ChatGPT, Claude, Gemini, and Perplexity say about you by sweeping each provider's API on a schedule, joins that to Search Console and server evidence, keeps everything in a local SQLite file on your own server, and exposes MCP tools so your assistant can query it. You pay only the API costs. Of everything in this section, it guesses least.
DataForSEO AI Optimization API
(Paid, pay as you go)
Not a dashboard but the raw data underneath one: LLM mentions, response data, and AI keyword metrics across ChatGPT, Gemini, Google AI Overview, Claude, and Perplexity, on a pay-as-you-go basis. We are evaluating it as a data source for our own citation reporting. If you would rather build than buy, start here.
Query fan-out and content planning
Qforia by iPullRank
(Free, open source; needs a paid Gemini API key)
When Google's AI Mode or an AI Overview answers a question, it first breaks the question into a fan of background sub-queries and synthesises an answer from across the results. Qforia simulates that fan-out so you can see which sub-questions your page needs to answer at passage level. It is probabilistic (two runs disagree) and it is a model of Google's behaviour, not a window into it, but it is the best planning tool we have found for AI search and our own internal fan-out tool is built on it. If you do nothing else new this year, run your five most commercial pages through it and count the gaps.
Crawler control: who can read, train on, and quote your content
Also a category that did not exist in 2017, when a robots.txt checker was the whole story. The choice is no longer "let the AI labs have everything" or "block them all". Three things changed.
Training and retrieval crawlers are now separate. OpenAI runs GPTBot for training data, OAI-SearchBot to find answers for users, and ChatGPT-User to fetch a page on a user's behalf. Anthropic, Perplexity, and others have split theirs similarly. A publisher can block the training bot and still be cited by the retrieval bots. The robots.txt file is the tool, and it is free.
Google-Extended is not a crawler. It is a robots.txt directive that tells Googlebot not to use what it fetches for Gemini training. Block it at the CDN level and Googlebot still walks straight in, sees no directive, and uses your content. This trips people up constantly; we found it in our own research into how UK publishers handle AI access, where only around 15% had any kind of mixed policy.
Snippet controls work passage by passage. Google's documentation confirms that nosnippet, data-nosnippet, max-snippet, and noindex govern what its AI features can show. Wrap a paragraph in data-nosnippet and it stays out of AI Overviews and AI Mode while the rest of the page remains eligible. We built a free WordPress plugin, Search Snippet Control for Blocks, to do this block by block.
Cloudflare sits above all of this. Since 1 July 2025 it has blocked AI crawlers by default for its customers unless they permit them, and runs a Pay Per Crawl marketplace; it carries traffic for around 20% of the web. If your site is behind Cloudflare, check the AI crawler settings before you assume anything about your visibility, in either direction.
Speed, Core Web Vitals, and structured data
The authoritative tools here are Google's own, covered above: PageSpeed Insights and Lighthouse for speed, the Rich Results Test and the Schema Markup Validator for markup. Two supporting tools earn a place alongside them:
(Free and paid)
Still our second opinion after PageSpeed Insights, and still one of our favourites: clearer waterfall charts, and the ability to test from a UK location.
(Free and paid)
The quickest fix for oversized images, which are still the most common reason we see a page fail its Largest Contentful Paint target. Modern image formats (WebP, AVIF) have moved the goalposts since 2017, so check what your CMS can serve before compressing by hand. Frankly, we still struggle to see the difference in the compressed image, which is rather the point.
Redirects, status codes, and monitoring
(Free)
The tool we reach for to check a batch of URLs for status codes and redirect chains, and the one we used to audit our own site migration in 2022. Paste up to 100 URLs, pick a user agent, export.
Redirect Path (Chrome extension)
(Free)
Unchanged: click the icon and the redirect chain and status codes for the page you are on are right there. It does not get easier than this, and it is essential after any migration.
(Free and paid)
Whether you look after one site or 50, you need to know when one goes down. Uptime Robot tells you, tells you again when it comes back, and the free tier (50 monitors) is still enough for most.
The AI assistant layer: MCP
This is the section that would have made no sense in 2017, and it is where our own work has changed most. If you had told us then that we would one day ask a piece of software, in plain English, to crawl a site and tell us what it found, we would have laughed.
The Model Context Protocol (MCP) is an open standard Anthropic released in November 2024 for connecting AI assistants to the systems where data lives. In practice it means an assistant such as Claude can read Ahrefs, run Screaming Frog, query Search Console, and pull Canonry citations in one conversation, then do the analysis you used to do by exporting six CSVs. Ahrefs includes MCP access from its Lite plan. Screaming Frog shipped its MCP server in version 24. There is no official Search Console MCP server from Google, but community-built ones wrap the real API. Canonry exposes MCP tools out of the box.
What this changes is the shape of the work rather than the tools. A technical audit that took a day of exports and pivot tables becomes a scripted, repeatable check. A weekly "which AI bots hit us, what did they fetch, what did we get cited for" report becomes a standing question your assistant answers on Monday morning. We run our own agency this way, with a shared knowledge base our assistants read, and we have written separately about what MCP actually changes for a marketing team and which connectors to wire first.
Two cautions. Give the assistant read-only access wherever you can; it is very good at analysis and only as good at judgement as the person checking its work. And do not let the assistant become the source of truth: the tools above still hold the data, the assistant just reads it.
What we have retired since 2017
Some of this is us moving on. Most of it is the tools moving on without us.
- DeepCrawl. What happened: Rebranded as Lumar, September 2022; What we use instead: Lumar, Oncrawl, or Screaming Frog
- Google Mobile-Friendly Test. What happened: Retired 1 December 2023; What we use instead: Lighthouse, PageSpeed Insights
- Google Structured Data Testing Tool. What happened: Refocused as the Schema Markup Validator (Google, December 2020); Google recommends the Rich Results Test; What we use instead: Rich Results Test, Schema Markup Validator
- Google Analytics (Universal). What happened: Stopped processing 1 July 2023; What we use instead: Google Analytics 4
- Google Correlate. What happened: Retired (2019); What we use instead: Google Trends, Ahrefs
- Moz Open Site Explorer. What happened: Replaced by Link Explorer, 2018; What we use instead: Ahrefs
- GetFiveStars. What happened: Rebranded as GatherUp, September 2018; What we use instead: Google Business Profile review links
- AMP validator. What happened: AMP no longer matters for a B2B site; What we use instead: Core Web Vitals
- SerpTrends, Keyword.io, BROWSEO, Cache Killer, and the various case-conversion tools. What happened: Either gone or replaced by features inside the tools above; What we use instead: Screaming Frog, Ahrefs
Where to start if you are a small team
If you are one marketer with a modest budget, this is the order we would buy in:
- Google Search Console, Google Analytics 4, and Bing Webmaster Tools. Free. Turn on the AI performance report and build a channel group for AI referrals.
- Screaming Frog. £199 per user per year. Crawl the site, fix what it finds, and turn on the semantic similarity report.
- Ahrefs Lite. £99 a month. Backlinks, keywords, rank tracking, and MCP access in one place.
- Qforia. Free plus a Gemini API key. Run your five most commercial pages and fill the gaps before you buy any tracker.
- One citation tracker. Otterly at $29 a month if you want a dashboard; Canonry if you have a developer and want to own the data.
- A robots.txt review and, if you are on Cloudflare, its AI crawler settings. Free, an hour's work, and the most common thing we find wrong.
Everything else in this review is for when the basics are fixed. Most AI visibility programmes stall because they buy a tracker before they fix eligibility.
Common questions
Do AI search tools replace SEO tools?
No. Answer engines run on the same crawl, index, and eligibility as ordinary search, so the crawler, the rank tracker, and Search Console still do most of the work. The AI layer adds citation tracking and fan-out planning on top.
Is there a Google tool for AI Overviews?
Yes, since June 2026: the generative AI performance report in Search Console, global from 31 August 2026. It shows impressions in AI Overviews, AI Mode, and Discover's generative features, but not clicks.
Which SEO tools have MCP servers?
Ahrefs (from the Lite plan), Screaming Frog (version 24 onwards), and Canonry ship them; DataForSEO offers one for its API; Search Console is covered by community servers rather than an official one.
What is the cheapest way to track AI citations?
Canonry if you can self-host (API costs only); Otterly's $29 a month plan if you cannot.
Should we block GPTBot?
That depends on whether your writing is your product. Blocking GPTBot stops OpenAI training on your content; leaving OAI-SearchBot open keeps you citable in ChatGPT answers. Make the decision deliberately, in robots.txt, and check any CDN-level blocking does not contradict it.
So, we know this is quite an expansive article, but trust us when we say that you have saved yourself a lot of time by reading it. New "AI SEO" tools seem to pop up every few minutes, and while some are genuinely useful, the majority hinder rather than help. Get the basics right, make sure your content is retrievable and well structured, measure citations honestly, and then experiment. Follow that order and you can spend your time on what matters: helping your customers find you.
For further help with your technical SEO or AI search visibility, please feel free to book a call and we will be happy to help you.
If you found this article useful, you might also like our guide to how to do AEO properly in 2026.
Want to make this practical?
Book a call and we will walk through your current stack, what is worth wiring first, and what should wait.
Book a call