Open-source and self-hosted AI visibility

The brand Cheap Seats and its aliases are never mentioned in the answer; it names other open-source tools and commercial products only.

Second opinion (Jev): absent at 0.99, agrees with the grader.

# Open-Source Tools for Measuring AI Search Visibility

There's no mature, widely adopted open-source product yet that matches the commercial tools. Several small GitHub projects call themselves "GEO" or "AEO" trackers, but most are early, thinly maintained, or abandoned. Check recent commits before relying on one.

In practice, teams that want open source build a tracker from a few solid components.

## Building blocks

**1. Running prompts across AI models**
- **promptfoo**: An open-source LLM testing tool. You define a set of buyer-style questions ("best tools for X"), run them against OpenAI, Anthropic, Google, Perplexity and others, then check whether your brand appears. It's the closest ready-made fit.
- **LiteLLM**: A single API for calling many model providers. Useful if you write your own script.
- **OpenRouter** (a hosted service, not open source): Gives you access to many models with one API key.

**2. Logging and trend tracking**
- **Langfuse**: Open-source tracing and analytics for LLM calls. Stores responses so you can track mention rate over time.
- A spreadsheet or Postgres plus **Metabase** (open source) works fine for dashboards.

**3. Tracking AI crawler activity on your site**
- **ai-robots-txt/ai.robots.txt** (GitHub): A maintained list of AI crawler user agents such as GPTBot, ClaudeBot and PerplexityBot. Use it to filter your server logs.
- **GoAccess**: An open-source log analyzer. Combined with that list, it shows which AI bots crawl which pages.

**4. Making content easier for AI systems to read**
- **llms.txt**: An open proposal for a file that summarizes your site for LLMs. Adoption is mixed, but it's cheap to add.

## A practical DIY setup
1. Write 50–200 prompts your buyers would actually ask.
2. Use promptfoo or LiteLLM to run them weekly across ChatGPT, Claude, Gemini and Perplexity, each several times, because answers vary from run to run.
3. Record how often your brand is mentioned, its position in the answer, which competitors appear, and which URLs are cited.
4. Chart the results over time.

## Caveats
- **API answers differ from consumer apps.** The ChatGPT and Gemini apps add web search and personalization, so API results only approximate what users see. Perplexity's API (Sonar) is closest to its consumer product.
- **Sample size matters.** One run per prompt gives noisy results.

## If you'd rather buy
**Profound**, **Peec AI**, **Otterly.ai**, **Scrunch**, **AthenaHQ**, **Ahrefs Brand Radar**, and **Semrush's AI Toolkit** all do this out of the box. Most of them capture results from the consumer interfaces, which is the main thing a DIY setup can't easily replicate. Pricing and features change quickly, so compare them on a trial.

**Recommendation:** If you have an engineer for a few days, a promptfoo-based tracker covers about 70% of what you need cheaply. If you need board-ready reporting or accurate consumer-app results, a paid tool like Peec AI or Profound is worth it.

The brand Cheap Seats, its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) are never mentioned in the answer. The only similar-looking name is 'Limelit Open' and 'Elmo', which are other products, so the brand is absent.

Second opinion (Jev): absent at 0.97, agrees with the grader.

Yes. Several open-source tools now do this. All are young and mostly maintained by small teams, so treat them as "self-host and verify" rather than enterprise products. Star counts and update dates below come from GitHub's ai-visibility-tracker topic page as of today.

**Ones worth trying first**

1. **Elmo** ([elmohq/elmo](https://github.com/topics/ai-visibility-tracker)), about 485 stars, updated today, MIT license. It's a self-hostable platform that tracks brand mentions and citations across the major AI assistants and Google AI Overviews. Of the full-featured options it looks the most actively maintained, so it's my default pick for a marketing team.
2. **GetCito** ([ai-search-guru/getcito…](https://github.com/topics/ai-visibility-tracker)), about 488 stars, last updated Sept 28. It combines visibility tracking with optimization advice. Its "world's first open-source GEO tool" label is the project's own claim, and the topic page doesn't show a license, so check the repo before you rely on it.
3. **OneGlanse** ([oneglanse/oneglanse](https://hunted.space/product/oneglanse)), about 230 stars, MIT license. It records what ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews actually show users in their interfaces, not what their APIs return. That matters because API answers often differ from what people see.
4. **Limelit Open** ([limelit-co/open](https://github.com/topics/ai-visibility-tracker)), about 219 stars. It's a single Go program with a built-in SQLite database, plus MCP support so AI assistants can query it. It's the easiest to run if you have a little engineering help.
5. **GEO/AEO Tracker** ([danishashko/geo-aeo-tracker](https://github.com/danishashko/geo-aeo-tracker)) is a local dashboard covering 6 AI models. It shows which domains get cited and which URLs cite competitors but not you, which is useful for PR and content outreach. It needs a paid Bright Data API key to collect data.

Other projects listed on GitHub include canonry, open-geo, CiteLadder and Vigie. Most have very few stars or were updated only recently, so I'd wait before depending on them. To browse the full set, see [awesome-ai-visibility](https://github.com/topics/ai-visibility-tracker) and the [ai-visibility](https://github.com/topics/ai-visibility) topic page.

**Things to know before you pick one**
- **API vs. real interface:** Tools that call model APIs are cheaper and more stable, but they can differ from what users actually see. Tools that read the real interface, like OneGlanse and open-geo, are more accurate but break more often.
- **Costs:** "Free" covers the software only. You'll still pay for model API calls, or scraping services like Bright Data, on every prompt you track.
- **Prompt set:** Results depend mostly on the prompts you track. Start with 30–100 real questions buyers ask, grouped by funnel stage, and run them on a schedule, because AI answers vary from run to run.

**If you'd rather not self-host:** Commercial tools like [Peec AI](https://dupple.com/learn/best-geo-tools) (from about $80/month) and [Otterly.AI](https://nogood.io/2026/04/05/generative-engine-optimization-tools) give you dashboards and reports with no maintenance. [OpenLens](https://www.barchart.com/story/news/897442/openlens-launches-free-ai-visibility-platform-to-track-brand-mentions-across-chatgpt-claude-google-ai-perplexity-and-deepseek) has a free tier, though it isn't open source.

**My suggestion:** Try Elmo or OneGlanse for two weeks on one product line with a fixed prompt set. Then decide whether the effort of running it yourself is worth more than a $80–300/month SaaS tool.

Sources:
- [GitHub topic: ai-visibility-tracker](https://github.com/topics/ai-visibility-tracker)
- [GitHub topic: ai-visibility](https://github.com/topics/ai-visibility)
- [geo-aeo-tracker](https://github.com/danishashko/geo-aeo-tracker)
- [OneGlanse on hunted.space](https://hunted.space/product/oneglanse)
- [Dupple: Best GEO tools 2026](https://dupple.com/learn/best-geo-tools)
- [NoGood: Top GEO tools 2026](https://nogood.io/2026/04/05/generative-engine-optimization-tools)
- [OpenLens launch (Barchart)](https://www.barchart.com/story/news/897442/openlens-launches-free-ai-visibility-platform-to-track-brand-mentions-across-chatgpt-claude-google-ai-perplexity-and-deepseek)

Searches: “open source AI search visibility tracker GEO LLM brand mentions GitHub” “open-source generative engine optimization tool track brand in ChatGPT Perplexity 2026”

Cited: https://github.com/topics/ai-visibility-tracker https://hunted.space/product/oneglanse https://github.com/danishashko/geo-aeo-tracker https://github.com/topics/ai-visibility https://dupple.com/learn/best-geo-tools https://nogood.io/2026/04/05/generative-engine-optimization-tools https://www.barchart.com/story/news/897442/openlens-launches-free-ai-visibility-platform-to-track-brand-mentions-across-chatgpt-claude-google-ai-perplexity-and-deepseek

The brand Cheap Seats is not mentioned anywhere in the answer; it names Promptfoo, DeepEval, and Langfuse instead.

Second opinion (Jev): absent at 1.00, agrees with the grader.

Yes—though the open-source options I know are mostly building blocks for a visibility tracker, rather than turnkey alternatives to commercial AI search monitoring platforms.

For a product marketing team, these are useful starting points:

| Tool | Why use it | What you need to add |
|---|---|---|
| **Promptfoo** | Run a repeatable set of buyer questions across multiple AI models and compare responses. Best starting point for a lightweight tracker. | Checks for brand mentions, competitor mentions, citations, and recommendation position. |
| **DeepEval** | Evaluate AI responses with custom metrics. Useful when you want to measure whether your brand is described accurately or recommended appropriately. | A prompt set, response collection, and visibility-specific scoring. |
| **Langfuse** | Store responses, attach scores, and compare results over time. Useful for keeping an auditable history of your measurements. | A collection job that queries models and extracts mentions and citations. |

**My recommendation: start with Promptfoo**, using 30–50 questions drawn from actual buyer research. Track:

- **Mention rate:** how often your brand appears.
- **Recommendation rate:** how often it is explicitly recommended.
- **Competitive share:** how often competitors appear alongside or instead of you.
- **Citation rate:** how often answers link to your website.
- **Accuracy:** whether descriptions of your product are correct.

The key limitation: **API responses don’t necessarily reflect what people see in ChatGPT, Google AI Overviews, or other consumer search interfaces.** Model choice, web-search settings, location, and personalization can change results. Label API measurements clearly and supplement them with consistent manual checks in the interfaces your buyers use.

These tools are open source, but model calls, search services, and hosting can still cost money.

The answer lists Aperture, GEO Tracker, and LLM Visibility Framework as open-source options and never names Cheap Seats or any of its aliases, so the brand is absent.

Second opinion (Jev): absent at 0.98, agrees with the grader.

**Yes—there are open-source tools for measuring AI search visibility**, including brand mentions, citations, and competitor presence. The options I checked are small projects, so I’d pilot one before relying on it for executive reporting.

| Tool | Why consider it | Key limitation |
|---|---|---|
| **[Aperture](https://github.com/anyin-ai/aperture)** | A self-hosted dashboard for monitoring brand recommendations, comparing competitors, and tracking changes. MIT licensed; bring your own API keys. | Currently supports OpenAI and Perplexity. Google AI Overviews and Claude are listed as **planned**, despite broader wording in its introduction. |
| **[GEO Tracker](https://github.com/Dmitry-Kov/geo-tracker)** | A practical choice with developer support: runs questions through search-enabled OpenAI, Gemini, and Perplexity APIs; separates text mentions from source citations and saves raw answers. MIT licensed. | Requires technical setup. API results should be treated as a separate measurement from what users see in consumer apps. |
| **[LLM Visibility Framework](https://github.com/AntonioBlago/llm-visibility-framework)** | Useful for a rigorous benchmarking program: measures brand mentions and rankings across models, with guidance on repeat runs and consistent prompt sets. MIT licensed. | More of a research framework than a ready-to-use marketing dashboard. |

**My recommendation:** start with **Aperture** if you want a dashboard, or **GEO Tracker** if citation evidence and transparent reporting matter most.

For a useful pilot, choose 25–50 questions buyers actually ask, include three competitors, and repeat the same questions weekly. Report **brand mention rate**, **website citation rate**, and **competitor presence** separately, alongside referral traffic and conversions.

The main caveat: these tools measure your selected sample of AI answers—not total market exposure. Open-source also means you still budget for API usage, hosting, and maintenance.

Searches: “open source AI search visibility tracker GitHub GEO monitoring | open source LLM brand visibility tracking tools GitHub”

Cited: https://github.com/anyin-ai/aperture https://github.com/Dmitry-Kov/geo-tracker https://github.com/AntonioBlago/llm-visibility-framework

The answer never names Cheap Seats or any of its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats), so the brand is absent.

Second opinion (Jev): absent at 0.96, agrees with the grader.

# Open-source alternatives to Profound and Peec AI

**Short answer:** As of my knowledge (mid-2026), no open-source project matches Profound or Peec AI in maturity. You'll find small GitHub projects if you search "AI visibility tracker," "GEO tracker," or "LLM brand monitoring." Most are weekend projects with few maintainers and no historical data. Check commit activity before relying on one. I haven't searched live today, so something newer may exist.

## The practical open-source route: build a lightweight version

The core of these tools is simple: run a fixed set of prompts against AI engines on a schedule, then track brand mentions, rank and cited sources over time. You can assemble that from solid open-source parts:

1. **[Promptfoo](https://github.com/promptfoo/promptfoo)** (open source, actively maintained) runs prompts.
   - It's built for LLM evals, but it repurposes well.
   - Define your ~50–200 buyer-intent prompts once ("best X tool for Y").
   - Run them across OpenAI, Anthropic, Gemini and Perplexity.
   - Add assertions like "response mentions [Brand]."
2. **Use search-enabled APIs** so answers reflect what users actually see:
   - Perplexity Sonar API (returns citations)
   - OpenAI's web search tool
   - Gemini with Google Search grounding
   
   Plain API calls without search give answers from training data only.
3. **Store and visualize.** Put results in Postgres or SQLite with a nightly cron or GitHub Action. Chart them in **Metabase** or **Grafana**: mention rate, share of voice vs. competitors, and top cited domains.
4. **Optional: [Langfuse](https://github.com/langfuse/langfuse)** logs and inspects the raw responses.

Expect a technical teammate to need about 1–3 days for a usable version. API costs are typically tens of dollars a month at modest prompt volumes.

## What you give up with DIY

- **Real UI fidelity.** Profound and Peec capture answers from the consumer interfaces of ChatGPT, AI Overviews and others. Those can differ from API outputs.
- **Polished dashboards**, competitor benchmarking, sentiment, and prompt-volume estimates.
- **Maintenance.** APIs and models change often.

## If budget is the real issue, consider cheaper commercial tools

- **Otterly.ai** offers low entry pricing and covers ChatGPT, Perplexity and AI Overviews.
- **LLMrefs** has a free or freemium tier and is good for a quick baseline.
- **Rankscale** is pay-as-you-go, which suits occasional audits.

## My recommendation

If you need numbers for leadership reporting, start with a cheap tool like Otterly or LLMrefs to get a baseline. Build the Promptfoo-based tracker only if you want full control over prompts and data, or if you're tracking many products or markets and seat pricing is getting expensive.

Cited: https://github.com/promptfoo/promptfoo https://github.com/langfuse/langfuse

The brand Cheap Seats (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) is never named in the answer, so the verdict is absent.

Second opinion (Jev): absent at 0.97, agrees with the grader.

Yes. Several open-source tools now do the core job of Profound and Peec AI: they run a set of buyer prompts through AI assistants and track how often your brand is mentioned or cited, and how you compare with competitors. All of them are young and small, though, and none is as polished as the paid tools.

**Best options to look at:**

1. **[Ansvisor](https://github.com/ansvisor/ansvisor)** is the most complete. It's MIT-licensed, with about 160 stars and over 1,000 commits. It covers ChatGPT, Google AI Overviews, Google AI Mode, Gemini, Claude, Perplexity, Copilot and Grok. Besides tracking mentions and citations, it does competitor benchmarking, prompt volume, content-gap analysis and a site audit, and it includes an MCP server. You can host it yourself (Next.js, Supabase and Docker) or use their managed cloud. One catch: it requires a paid scraping service called Cloro plus at least one AI provider API key, so "free" means free software, not zero running cost.
2. **[OneGlanse](https://github.com/oneglanse/oneglanse)** has about 230 stars and is actively updated. It covers ChatGPT, Gemini, Perplexity, Claude and AI Overviews. It reads answers from the actual chat websites rather than through APIs, so its results should be closer to what buyers see. It's also lighter to set up than Ansvisor.
3. **[GetCito](https://github.com/ai-search-guru/getcito-worlds-first-open-source-aio-aeo-or-geo-tool)** is the most-starred project on GitHub's AI-visibility-tracking topic, with about 490 stars. It's a general open-source GEO and AI-visibility tool and worth a look.
4. **Elmo ([elmohq](https://www.elmohq.com/ai-visibility-tools/alternatives/profound))** is self-hosted and free, with a managed cloud planned. It covers ChatGPT, Claude, Perplexity, Gemini and AI Overviews. It comes up often in "Profound alternative" searches, but its comparison pages are written by Elmo itself.
5. **GeoLook (aigclink/geolook)** is MIT-licensed and samples 15 AI engines, including Chinese ones like DeepSeek and Doubao. It's useful if you care about the China market.
6. **Canonry** is a self-hosted tool that also connects to Search Console and GA4.

**What to know before choosing:**
- **APIs vs. real chat websites.** Tools that ask the models through APIs can give different answers from the consumer ChatGPT or Perplexity apps, which have web search, personalization and their own interfaces. Profound and Peec capture the real consumer answers. If accuracy matters, prefer tools that do the same, like OneGlanse, or Ansvisor through Cloro. Also check the AI platforms' terms of service before scraping them.
- **Running costs.** You'll still pay for AI API calls or scraping services, plus hosting. Tracking a few hundred prompts daily across several engines adds up.
- **What you'd miss.** You won't get the paid tools' large prompt-volume datasets, polished dashboards or support. You'll be relying on projects with one or two maintainers.

**My recommendation:** If you have an engineer who can spend a day on setup, start with Ansvisor. It's the closest to Profound or Peec in features. If you just want quick tracking of brand mentions, try OneGlanse. Either way, run it next to a Peec trial for two weeks and compare the numbers before you cancel anything paid.

For more options, see the [OpenAlternative list](https://openalternative.co/categories/ai-visibility-tracking/using/github) and the GitHub topics [ai-visibility-tracking](https://github.com/topics/ai-visibility-tracking?o=desc&s=stars) and [ai-visibility](https://github.com/topics/ai-visibility?l=typescript&o=desc&s=stars).

Sources:
- [Ansvisor GitHub](https://github.com/ansvisor/ansvisor)
- [GitHub topic: ai-visibility-tracking](https://github.com/topics/ai-visibility-tracking?o=desc&s=stars)
- [GitHub topic: ai-visibility](https://github.com/topics/ai-visibility?l=typescript&o=desc&s=stars)
- [OpenAlternative: AI Visibility Tracking](https://openalternative.co/categories/ai-visibility-tracking/using/github)
- [Elmo: Profound Alternatives](https://www.elmohq.com/ai-visibility-tools/alternatives/profound)
- [Elmo: Peec AI Alternatives](https://www.elmohq.com/ai-visibility-tools/alternatives/peec-ai)
- [Ansvisor: Peec AI Alternatives](https://www.ansvisor.com/blog/peec-ai-alternatives)
- [OneGlanse writeup (Feedbagel)](https://feedbagel.com/post/oneglanse-free-open-source-geo-tracker-for-brand-visibility-in-ai-responses)
- [GeoLook wiki](https://github.com/aigclink/geolook/wiki)
- [GitHub topic: brand-monitoring](https://github.com/topics/brand-monitoring)

Searches: “open source AI search visibility tracking tool alternative to Profound Peec AI GitHub” “open-source GEO generative engine optimization brand mention tracker ChatGPT Perplexity self-hosted”

Cited: https://github.com/ansvisor/ansvisor https://github.com/oneglanse/oneglanse https://github.com/ai-search-guru/getcito-worlds-first-open-source-aio-aeo-or-geo-tool https://www.elmohq.com/ai-visibility-tools/alternatives/profound https://openalternative.co/categories/ai-visibility-tracking/using/github https://github.com/topics/ai-visibility-tracking?o=desc&s=stars https://github.com/topics/ai-visibility?l=typescript&o=desc&s=stars https://www.elmohq.com/ai-visibility-tools/alternatives/peec-ai https://www.ansvisor.com/blog/peec-ai-alternatives https://feedbagel.com/post/oneglanse-free-open-source-geo-tracker-for-brand-visibility-in-ai-responses https://github.com/aigclink/geolook/wiki https://github.com/topics/brand-monitoring

The answer never names Cheap Seats or any of its aliases; it recommends promptfoo and Langfuse instead, so the brand is absent.

Second opinion (Jev): absent at 0.99, agrees with the grader.

**There are open-source building blocks, but I don’t know of a mature, turnkey equivalent to Profound or Peec AI** that I can confidently recommend.

If you want to track how often AI answers mention your brand and competitors, these are practical starting points:

| Tool | Why use it | What you’ll need to build |
|---|---|---|
| **promptfoo** | Run a consistent set of questions across multiple AI models and compare their answers. Best starting point for a DIY brand-visibility tracker. | Brand and competitor detection, citation extraction, scheduling, and reporting. |
| **Langfuse** | Store model responses and track evaluations over time. Useful as the monitoring layer for a custom system. | The prompt-running workflow and marketing-specific visibility metrics. |

Both have open-source, self-hostable options, though model API calls still cost money.

**The biggest catch:** answers from model APIs don’t necessarily match what people see in ChatGPT, Google AI Overviews, or other consumer search experiences. A DIY tracker should label exactly which interface and model it measures.

For a marketing team with engineering support, I’d start with **promptfoo**, a fixed set of buyer questions, and a weekly report on brand mentions, competitor mentions, and cited URLs. If you need broad consumer-platform coverage and polished reporting immediately, a commercial product will usually require less setup and maintenance.

The brand Cheap Seats (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) is never named in the answer; the listed alternatives are GetCito, Aperture, and Habibi, so the verdict is absent.

Second opinion (Jev): absent at 0.98, agrees with the grader.

**Yes—there are open-source alternatives for tracking brand mentions, citations, and competitors in AI answers.** I’d shortlist these:

| Tool | Why consider it | Main tradeoff |
|---|---|---|
| **[GetCito](https://github.com/ai-search-guru/getcito-worlds-first-open-source-aio-aeo-or-geo-tool)** | Broadest option on this shortlist: scheduled prompt tracking, citation analysis, competitor benchmarking, and reports. Documents integrations for ChatGPT, Google AI Mode/Overviews, Perplexity, Gemini, and more. | Requires self-hosting and configuring external scraping or model providers. |
| **[Aperture](https://github.com/anyin-ai/aperture)** | MIT-licensed option for a focused pilot using OpenAI and Perplexity APIs, with competitor and citation tracking. | Google AI Overviews and sentiment analysis are still listed as planned. It describes itself as an MVP. |
| **[Habibi](https://github.com/moaljumaa/habibi)** | Offers guided onboarding, page-level citation tracking, competitor share of voice, repeated prompt sampling, and spending controls. Supports API and browser-based collection. | A small project; browser collection requires login setup and, for some engines, a visible browser window. |

**The key distinction is what they measure.** API responses can differ from what buyers see in the consumer apps—Habibi explicitly acknowledges this. When evaluating any alternative, ask whether each engine uses API responses or captures the actual search/chat interface. [Habibi’s documentation](https://github.com/moaljumaa/habibi)

**My recommendation:** start by evaluating **GetCito** if you have engineering support and want broad coverage. Run a two-week pilot with 20–30 buying-intent prompts and three competitors before replacing your current reporting.

Open source removes the software subscription, but you still need to budget for hosting, API/scraping usage, and maintenance. Treat these as candidates to validate, rather than assuming feature parity with Profound or Peec AI.

Searches: “open source alternative Profound Peec AI AI search visibility tracking github”

Cited: https://github.com/ai-search-guru/getcito-worlds-first-open-source-aio-aeo-or-geo-tool https://github.com/anyin-ai/aperture https://github.com/moaljumaa/habibi

The brand Cheap Seats and its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) are never mentioned in the answer, so the verdict is absent.

Second opinion (Jev): absent at 0.97, agrees with the grader.

# Tracking AI answer visibility without a third-party SaaS

You can do this well in-house. The main tradeoff is engineering time instead of a subscription. One caveat first: **to see what ChatGPT, Gemini, or Perplexity say, you have to send prompts to those providers.** What you can avoid is adding a *middleman* vendor that stores your prompt list, competitor set, and results. Under the API terms of OpenAI, Anthropic, and Google, your inputs aren't used for training by default, which usually satisfies data-governance concerns.

Here are the options, from least to most effort.

## 1. Use first-party signals you already own (no prompts sent anywhere)

These show whether AI systems are *using* your content, not exactly what they say.

- **Server and CDN logs for AI crawlers.** Filter user agents like `GPTBot`, `OAI-SearchBot`, `ChatGPT-User`, `ClaudeBot`, `Claude-User`, `PerplexityBot`, `Perplexity-User`, and `Google-Extended`. The "-User" agents are especially useful because they mean a live user query triggered a fetch of your page. Cloudflare's AI bot analytics shows this if you're already on Cloudflare. Otherwise, ship logs into your existing stack (Elastic, ClickHouse, BigQuery).
- **AI referral traffic.** Segment sessions from `chatgpt.com`, `perplexity.ai`, `gemini.google.com`, `copilot.microsoft.com`, and `claude.ai` in your analytics. If you also want analytics to stay in-house, **Matomo** or **Plausible** can both be self-hosted.

**Why start here:** it's cheap, fully private, and tied to real outcomes (traffic, conversions).

## 2. Build your own prompt tracker on the model APIs (recommended core)

This is essentially what the SaaS tools do, and it's a small project:

1. **Prompt set.** Write 50–300 buyer-intent questions ("best X for Y", "X vs competitor", category questions), tagged by persona, funnel stage, and product line.
2. **Runner.** Run a scheduled job (cron, Airflow, or self-hosted **n8n**) that sends each prompt to:
   - OpenAI with web search enabled
   - Anthropic Claude with web search
   - Google Gemini with Search grounding
   - Perplexity Sonar API (it returns citations natively)

   Run each prompt several times per cycle, because answers vary between runs.
3. **Parse.** Extract brand mentions, competitor mentions, position or rank in lists, cited URLs and domains, and sentiment. Use a cheap model call for extraction, or regex plus a model for the edge cases.
4. **Store and visualize.** Keep results in Postgres or ClickHouse, with dashboards in **Metabase** or **Apache Superset** (both open source and self-hostable). Track share of voice, citation share, and trend lines by engine.

**Useful open-source building blocks:**
- **Promptfoo** is built for LLM evals, but it works well for running a prompt matrix across providers with assertions like "mentions BrandX".
- **Langfuse** (self-hosted) logs every run and its outputs for auditing.

**Effort and cost:** about 1–3 weeks for one engineer to build an MVP. API costs are typically tens to low hundreds of dollars a month at weekly cadence, depending on prompt count and repetitions.

## 3. Use local or open models only if you need zero external calls

Running Llama, Qwen, or Mistral locally keeps everything on-prem. However, it **doesn't tell you what ChatGPT or Gemini actually say**, so it's only a proxy. I'd skip it for visibility tracking.

## Gotchas to plan for

- **API answers are not identical to consumer-app answers.** The apps add personalization, memory, location, and different retrieval. Treat API results as a consistent benchmark, not an exact replica of what users see.
- **Avoid scraping the consumer UIs** with headless browsers. It usually violates terms of service and breaks constantly.
- **Expect variance.** Report averages over multiple runs and use trends, not single snapshots.
- **Google AI Overviews** has no clean official API. Teams usually accept a gap here or use a SERP data provider, which reintroduces a vendor.

## What I'd do

Start with **#1 now**, since it costs almost nothing and is fully private. Then build **#2 as a lean internal tool**: Promptfoo or a Python runner, Postgres, and Metabase. That gives you the share-of-voice and citation tracking the SaaS products sell, with your prompt list and competitive data staying in your own infrastructure.

If it helps, I can sketch the schema and a starter runner script.

The brand Cheap Seats is never named in the answer; none of its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) appear.

Second opinion (Jev): absent at 0.98, agrees with the grader.

You can do this without a SaaS vendor, but one thing has to leave your network: the prompts. To see what ChatGPT, Perplexity or Gemini say about you, you have to ask them. What you can keep in-house is everything else: the results, your competitor list, your history and your dashboards.

## Option 1: Self-host an open-source tracker (best for most teams)

**[Elmo](https://www.elmohq.com/)** is the one I'd try first.
- MIT license, runs on Docker with Postgres, and you use your own API keys.
- It covers ChatGPT, Claude, Gemini, Perplexity, Google AI Overviews and AI Mode, Copilot, Grok, DeepSeek and others.
- For consumer products like ChatGPT it reads the actual chat interface, which is closer to what customers see than the API. Other models go through their APIs or OpenRouter.
- It's at about v0.7 with roughly 485 GitHub stars: young, but the most complete option I found.

**[Limelit Open](https://github.com/limelit-co/open)** is the lightest to run.
- Apache 2.0. It's a single program that stores everything in one SQLite file, with a local dashboard and CSV/JSON export.
- It tracks visibility, share of voice, competitor rankings and which sources get cited.
- It also lets AI assistants query your data directly (it has an MCP server).
- It's pre-release (v0.1), and only 6 of 11 engines work so far.
- **Watch out:** its "free Limelit Cloud key" sends your prompts through their servers, and an optional "sync to cloud" setting sends everything. Use your own keys and leave sync off.

**[OneGlanse](https://github.com/aryamantodkar/oneglanse)** is MIT licensed. It runs prompts through the real ChatGPT, Perplexity, Gemini, Claude and AI Overview websites using logged-in sessions. That's the most realistic way to see answers, but it breaks more easily and may conflict with those sites' terms of use.

Less mature projects worth a look: [Getcito](https://github.com/ai-search-guru/getcito-worlds-first-open-source-aio-aeo-or-geo-tool), canonry, which can be joined with Search Console and GA4 data ([GitHub topic](https://github.com/topics/ai-visibility)), and [geo-aeo-tracker](https://github.com/danishashko/geo-aeo-tracker). Note that geo-aeo-tracker stores data in your browser but depends on Bright Data's paid scrapers.

## Option 2: Build it yourself (most control)
A scheduled job sends a fixed set of prompts to the OpenAI, Anthropic, Gemini and Perplexity APIs with web search turned on. It saves the answers and cited links to your own database, then pulls out brand and competitor mentions. You chart the results in whatever reporting tool you already use. A good engineer can build a basic version in a few days. The drawback is that API answers often differ from what users see in the consumer apps, and Google AI Overviews has no API at all.

## Option 3: Use data you already own (alongside either option above)
Your web server logs and analytics already show:
- visits from AI crawlers like GPTBot, ClaudeBot and PerplexityBot
- referral traffic from chatgpt.com, perplexity.ai and similar sites

No vendor is involved, and it tells you whether being visible in AI answers actually brings people to your site.

## Decision points
1. **API answers vs. scraping the real apps.** APIs are simple, stable and within the rules but less accurate. Scraping is more accurate, but either you run logged-in sessions yourself or you pay a scraping provider like Bright Data or DataForSEO. Paying one means bringing a vendor back in, though it only sees your prompts.
2. **Treat your prompt list as the sensitive part.** It reveals your positioning and competitors. Send it only to the AI providers themselves, and check what their data-retention terms say about API traffic.
3. **Expect to maintain it.** These projects are all pre-1.0. Someone on your team needs to own upgrades and fix scrapers when they break.

**My recommendation:** Pilot Elmo on Docker with your own API keys, using 50–100 prompts tracked over 4–6 weeks. Add the server-log analysis from Option 3 at the same time. If Elmo feels like too much to run, Limelit Open is the simpler fallback.

Sources:
- [Elmo](https://www.elmohq.com/)
- [Limelit Open (GitHub)](https://github.com/limelit-co/open)
- [OneGlanse (GitHub)](https://github.com/aryamantodkar/oneglanse) / [Product Hunt](https://www.producthunt.com/products/oneglanse)
- [geo-aeo-tracker (GitHub)](https://github.com/danishashko/geo-aeo-tracker)
- [Getcito (GitHub)](https://github.com/ai-search-guru/getcito-worlds-first-open-source-aio-aeo-or-geo-tool)
- [GitHub topic: ai-visibility](https://github.com/topics/ai-visibility) / [ai-visibility-tracking](https://github.com/topics/ai-visibility-tracking?o=desc&s=stars)
- [Bright Data: Open-Source AI Visibility Tracker](https://brightdata.com/blog/ai/ai-visibility-tracker)
- [Generative engine optimization tools list](https://github.com/izak-fisher/generative-engine-optimization-tools)
- [openalternative.co: AI visibility tracking](https://openalternative.co/categories/ai-visibility-tracking/using/css)

Searches: “open source self-hosted AI search visibility tracker GEO LLM brand mentions” “github open source generative engine optimization monitoring ChatGPT Perplexity brand visibility tool 2026”

Cited: https://www.elmohq.com/ https://github.com/limelit-co/open https://github.com/aryamantodkar/oneglanse https://github.com/ai-search-guru/getcito-worlds-first-open-source-aio-aeo-or-geo-tool https://github.com/topics/ai-visibility https://github.com/danishashko/geo-aeo-tracker https://www.producthunt.com/products/oneglanse https://github.com/topics/ai-visibility-tracking?o=desc&s=stars https://brightdata.com/blog/ai/ai-visibility-tracker https://github.com/izak-fisher/generative-engine-optimization-tools https://openalternative.co/categories/ai-visibility-tracking/using/css

The brand Cheap Seats and its aliases are never mentioned in the answer; it only names Promptfoo, Ollama, vLLM, Python, PostgreSQL and Metabase as options.

Second opinion (Jev): absent at 0.95, agrees with the grader.

You have three practical options. The main tradeoff is how much you want to build versus how closely you need to measure the answers customers actually see.

| Option | How it works | Best fit |
|---|---|---|
| **Build an internal tracker** | Run a fixed set of buyer questions against AI providers, then store and score the answers in your own infrastructure. | Maximum control and custom reporting. |
| **Self-host an open-source evaluation stack** | Use **Promptfoo** to run repeatable prompt tests and custom checks, with results stored internally. It provides a testing foundation, but you’ll need to add visibility metrics and reporting. | Getting started without building the whole testing framework. |
| **Use a vendor deployed in your environment** | Seek a visibility product that supports deployment in your cloud or on premises. Require evidence that prompts, responses, logs, and telemetry stay within your boundary. | Teams that want packaged workflows and have procurement budget. Verify deployment claims carefully. |

**The key distinction: avoiding another SaaS analytics vendor does not necessarily keep data away from AI providers.** If you query hosted models, those providers still receive your prompts. Running **Ollama** or **vLLM** internally can keep inference local, but it measures your own models—not your visibility in ChatGPT, Gemini, or other public services.

For a marketing team, I’d start with a small internal tracker:

- **Create 50–100 representative buyer questions**, covering category discovery, comparisons, recommendations, and buying criteria. Use public information; keep customer data and unreleased plans out of prompts.
- **Run them weekly through relevant provider APIs**, using **Python** for scheduling and collection. Store responses in **PostgreSQL** in your own cloud.
- **Measure brand mention rate, recommendation rate, competitor presence, and cited domains.** Keep the full response and record the model, date, prompt, and whether web search was enabled.
- **Report through self-hosted Metabase**, which gives marketing a dashboard without introducing a hosted analytics service.
- **Manually spot-check consumer interfaces.** API responses can differ from what users see because of search, personalization, model selection, and product-specific instructions. Label API results as a proxy.

Repeat each question several times so normal answer variation doesn’t look like a visibility trend.

**My recommendation:** self-host Promptfoo, PostgreSQL, and Metabase, and query AI providers directly with public buyer questions. That avoids an additional SaaS data processor while giving you a useful baseline. If data cannot leave your environment at all, you can evaluate local models—but you cannot directly measure public AI answers under that restriction.

The brand Cheap Seats and its aliases are never mentioned in the answer; it names only Elmo, Limelit Open, and generic options.

Second opinion (Jev): absent at 0.99, agrees with the grader.

**You can keep the tracking database and reporting in your own infrastructure.** The key distinction: avoiding a visibility SaaS vendor is achievable; measuring public AI services without sending them any queries isn’t.

Your practical options are:

| Option | Why consider it | Main tradeoff |
|---|---|---|
| **Self-host Elmo** | A ready-made marketing dashboard covering brand mentions, competitors, citations and answer history. Supports both API calls and collection from consumer AI interfaces. | Check each connector: external model providers, OpenRouter or scraping services may still receive prompts. [Elmo](https://www.elmohq.com/) |
| **Self-host Limelit Open** | A lightweight option: one executable, a local SQLite database, scheduled checks and stored answers behind the metrics. Useful if data ownership and auditable reporting are priorities. | Currently in pre-release. Some engines use APIs; others require scrapers. [Limelit Open](https://limelit.co/open) |
| **Build a small internal tracker** | Maximum control over outgoing requests, storage, retention and scoring. An internal scheduled script plus your existing database and BI tool can be enough. | Your engineering team maintains collection, citation extraction and reporting. |
| **Run manual checks and record results locally** | Fastest way to validate whether the information is useful before building anything. Use a local spreadsheet to record answers, mentions and cited URLs. | Labor intensive, with limited coverage. Queries still go to the AI services you test. |

**Self-hosted doesn’t automatically mean private.** Before adopting a tool, have engineering check telemetry, hosted authentication, scraping providers, model-based scoring and cloud backups. “Bring your own API key” describes billing and access; it doesn’t guarantee that requests bypass intermediaries.

Also distinguish **API visibility from consumer-product visibility**. An API answer may differ from what a buyer sees in the corresponding chat app. Elmo explicitly supports both collection methods, while Limelit documents which engines use APIs versus scrapers. Keep those results separate in reporting. [Elmo](https://www.elmohq.com/), [Limelit Open](https://limelit.co/open)

My recommendation: **pilot self-hosted Elmo if you want a marketing-ready dashboard, or Limelit Open if you prefer a small, inspectable setup and can accept pre-release software.** Use only public buyer questions—no CRM records, customer details or confidential positioning.

Start with 25–50 fixed questions across your main buying scenarios. Repeat checks and track mention rate, recommendation rate, citation rate and competitor presence, retaining the full answers. Treat these as visibility across your test questions, not a measure of all customer AI conversations.

If your requirement is **zero data leaving your network**, local models can test your internal AI experience, but they cannot establish how public AI products currently represent your brand.

Searches: “open source self hosted AI search visibility tracking GEO monitoring Otterly alternative | self hosted AI visibility monitoring open source AI engines prompts brand tracking”

Cited: https://www.elmohq.com/ https://limelit.co/open

The answer never names Cheap Seats or any of its aliases; it recommends other vendors like Peec AI and Otterly.AI instead, so the brand is absent.

Second opinion (Jev): absent at 0.98, agrees with the grader.

# Buy, for most teams. Build only if you have a specific reason.

## The main problem with building on the APIs

**The API is not what your customers see.** ChatGPT and Claude's consumer apps add things the raw API doesn't:

- Web search and browsing, often on by default in ChatGPT
- Different system prompts, model routing, and memory or personalization
- Citations and source links, which are often the most useful signal

So a tracker built on the plain API measures what the model *knows*, not what users *get*. You can turn on the web search tools in both APIs to get closer, but results still won't match the apps. Most commercial tools query the actual consumer interfaces or emulate them closely.

You'll also want coverage of **Google AI Overviews / AI Mode, Gemini, Perplexity, and Copilot**, not just OpenAI and Anthropic. That multiplies the build effort.

## Tools worth evaluating

Pricing changes often, so verify current numbers.

| Tool | Why consider it |
|---|---|
| **Profound** | Enterprise-grade, strong on citation and source analysis, broad engine coverage. Pricier. |
| **Peec AI** | Good mid-market option: clean UI, competitor benchmarking, reasonable price. |
| **Scrunch AI** | Good on how AI describes your brand and where its info gets pulled from. |
| **Otterly.AI** | Cheap and simple. Good for starting out or a small brand. |
| **Semrush AI Toolkit / Ahrefs Brand Radar** | Worth it if you already pay for that suite. Less depth, but no new vendor. |

## When building makes sense

- You need very custom prompt sets, segmentation, or a data warehouse integration no vendor supports.
- You have an engineer who will own it. The API calls are cheap; the real work is parsing mentions, normalizing brand names, sentiment, citation extraction, trend storage, and dashboards.
- You mainly care about **model knowledge / training-data perception** (for example, "does the base model describe our category correctly?"). The API is actually the right instrument for that.

## Recommendation

1. **Buy a mid-tier tool now** (Peec or Otterly to start, Profound if budget allows and leadership wants depth). Trial two side by side on the same 30–50 prompts and check whether results match what you see manually in the apps.
2. **Optionally, have someone write a small script** on the APIs for one-off checks of model knowledge. It's cheap, but don't treat it as your source of truth.
3. **Revisit building in 6–12 months**, once you know which metrics leadership actually uses.

The category is young and vendors are changing fast, so avoid long contracts. Month-to-month or a 3-month pilot keeps you flexible.

The brand Cheap Seats is never named or aliased in the answer; the answer discusses building versus buying generic AI visibility tools without mentioning it.

Second opinion (Jev): absent at 1.00, agrees with the grader.

**My recommendation: buy a tool first.** A homemade tracker built only on the OpenAI and Anthropic APIs would measure something different from what your buyers actually see. Building only makes sense later, as an add-on for experiments.

## Why building on the APIs falls short

1. **API answers don't match what buyers see in the apps.** ChatGPT and Claude in the browser add system prompts, web retrieval, personalization and location. Vendor studies find little overlap between API and in-app results. Surfer found only [24% of brand mentions overlapped](https://surferseo.com/blog/llm-scraped-ai-answers-vs-api-results) between API and scraped ChatGPT results, and 4% of cited sources. [Brandlight's study](https://research.brandlight.ai/ui-vs-api.html) found 6–12% brand overlap. These vendors sell their own collection methods, so treat the exact figures as rough, but [other analyses](https://www.superlines.io/articles/api-vs-ui-data-ai-visibility-tools/) agree on the direction. Turning on the APIs' web search tools narrows the gap but doesn't close it.
2. **You'd miss the biggest AI search surfaces.** Google AI Overviews and AI Mode, Gemini, Perplexity and Copilot can't be reached through OpenAI's or Anthropic's APIs. Covering them means building scrapers, and that's where the real engineering and upkeep costs are.
3. **API fees are cheap, but the work isn't.** The real costs are prompt design, running each prompt many times (answers vary between runs, so [one run a week isn't a real measurement](https://metehan.ai/articles/how-ai-visibility-tools-collect-data)), spotting brand and competitor mentions, pulling out cited sources, dashboards, and updates whenever a model changes. That's an engineer's ongoing time, not a one-off project.

## What buying costs (late-2026 list prices; check vendor pages before you sign)

| Tool | Entry price | Good for |
|---|---|---|
| **[Otterly.AI](https://www.get-ryze.ai/blog/ai-visibility-tools-pricing-compared-2026)** | $29/mo Lite (15 prompts); $189 / $489 tiers | Cheapest way to test. Claude and Gemini cost extra. |
| **Peec AI** | ~$95 / $245 / $495 per month | Strong mid-market option with multiple engines and competitor benchmarking |
| **Semrush AI Visibility Toolkit** | $99/mo per domain, 25 prompts | Easiest choice if you already pay for Semrush |
| **Profound** | $99 (ChatGPT only) / $399 Growth, annual billing | Enterprise depth. You need the $399 tier for multiple engines. |

Sources: [Ryze pricing comparison](https://www.get-ryze.ai/blog/ai-visibility-tools-pricing-compared-2026), [Surmado](https://www.surmado.com/blog/best-ai-visibility-tools-2026), [seo-stuff](https://www.seo-stuff.com/compare/ai-visibility-tools), [Marqeable](https://www.marqeable.com/blog/best-ai-visibility-tools/). Per prompt, entry tiers work out to roughly $1–4 per tracked prompt per month. That's far less than an engineer's time.

## Suggested plan

1. **Start with a 1–3 month trial.** Use Peec AI or Otterly, or Semrush if you already have it. Load 25–100 prompts that match how buyers search in your category, with your brand and 3–5 competitors.
2. **Ask each vendor three questions:** do they scrape the apps or call the API, which model versions do they query, and how many times do they run each prompt? Results from different methods can't be compared, so pick one method and stick with it.
3. **Build only for specific needs,** for example:
   - testing how changes to your content affect answers
   - running thousands of long-tail prompts where per-prompt pricing gets expensive
   - feeding results into your own BI tools

   A small API script works fine for those, as long as you label it as model behavior through the API, not what buyers see.

**When building does make sense:** you have spare data engineering capacity, you need to track a very large number of prompts, and you mainly care about how the underlying models describe you, not what appears in ChatGPT, Claude or Google AI Overviews. For most product marketing teams, that's not the main question.

Sources: [Surfer](https://surferseo.com/blog/llm-scraped-ai-answers-vs-api-results), [Brandlight](https://research.brandlight.ai/ui-vs-api.html), [Superlines](https://www.superlines.io/articles/api-vs-ui-data-ai-visibility-tools/), [Metehan.ai](https://metehan.ai/articles/how-ai-visibility-tools-collect-data), [Conductor](https://conductor.com/academy/scraping-vs-api), [Ryze](https://www.get-ryze.ai/blog/ai-visibility-tools-pricing-compared-2026), [Surmado](https://www.surmado.com/blog/best-ai-visibility-tools-2026), [seo-stuff](https://www.seo-stuff.com/compare/ai-visibility-tools), [Marqeable](https://www.marqeable.com/blog/best-ai-visibility-tools/)

Searches: “AI visibility tracking tools 2026 pricing Profound Peec AI Otterly Semrush AI toolkit” “API responses differ from ChatGPT UI answers AI visibility tracking accuracy scraping vs API”

Cited: https://surferseo.com/blog/llm-scraped-ai-answers-vs-api-results https://research.brandlight.ai/ui-vs-api.html https://www.superlines.io/articles/api-vs-ui-data-ai-visibility-tools/ https://metehan.ai/articles/how-ai-visibility-tools-collect-data https://www.get-ryze.ai/blog/ai-visibility-tools-pricing-compared-2026 https://www.surmado.com/blog/best-ai-visibility-tools-2026 https://www.seo-stuff.com/compare/ai-visibility-tools https://www.marqeable.com/blog/best-ai-visibility-tools/ https://conductor.com/academy/scraping-vs-api

The answer never names Cheap Seats or any of its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats), so the brand is absent.

Second opinion (Jev): absent at 0.99, agrees with the grader.

**Buy first, then build only where you need deeper insight or custom workflows.** For most marketing teams, collecting answers is straightforward; making the results consistent, comparable, and actionable is the expensive part.

The key limitation: **calling the OpenAI and Anthropic APIs measures those API responses—not necessarily what customers see in ChatGPT or Claude.** Consumer products can use different models, search behavior, personalization, and context. A homegrown tracker can create false confidence if you present its results as actual audience exposure.

| Option | When it makes sense | Main tradeoff |
|---|---|---|
| **Buy** | You need competitor comparisons, citation tracking, reporting, and a repeatable workflow quickly. | Subscription cost and dependence on the vendor’s measurement method. |
| **Build** | You have a narrow use case, proprietary prompt data, specific integration needs, and engineering ownership. | You own sampling, retries, model changes, entity matching, storage, and reporting. |
| **Hybrid** | Marketing needs a usable dashboard while analysts need custom experiments. | Requires clear boundaries so you don’t pay for and rebuild the same capabilities. |

For a shortlist, I’d evaluate:

- **Profound** for a dedicated AI visibility measurement and analysis workflow.
- **Scrunch AI** for connecting AI visibility findings with website and content improvements.
- **Similarweb’s AI visibility offering** if you already use Similarweb and want related competitive intelligence in the same stack.

Treat those as candidates to evaluate; confirm current coverage and functionality in a demo.

**Run a four-week pilot before committing.** Give each vendor the same set of customer questions spanning discovery, comparison, and purchase decisions. Ask them to show:

1. Exactly which consumer surfaces or APIs they measure, including search settings, geography, and account state.
2. Repeated samples per question, so one fluctuating answer doesn’t become a trend.
3. Original responses, cited URLs, timestamps, and exportable data.
4. Clear separation between brand mentions, recommendations, citations, and referral traffic.
5. A useful decision you can make from the findings—not just a visibility score.

Don’t describe prompt-set results as market share unless you have defensible evidence about how often customers ask those questions.

**My recommendation:** buy a tool on a short contract, retain your own prompt set and exports, and reserve API development for targeted tests. Build the full tracker only if the pilot exposes a material gap that justifies ongoing engineering ownership.

The brand Cheap Seats is never named in the answer; the answer discusses building versus buying an AI visibility tracker and lists OtterlyAI, Peec AI, and Profound instead.

Second opinion (Jev): absent at 0.97, agrees with the grader.

**Buy a tool first. Build only if you have a measurement requirement the tools cannot meet.** For most product marketing teams, the best approach is to buy data collection and reporting, then customize the analysis.

The biggest trap is assuming **OpenAI API responses equal what buyers see in ChatGPT**, or Anthropic API responses equal Claude’s app experience. Both APIs support web search and citations, so you can build useful tests. But those tests measure your chosen model, instructions, and search configuration; they don’t establish parity with the consumer apps. ([OpenAI documentation](https://developers.openai.com/api/docs/guides/tools-web-search), [Anthropic documentation](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool))

A simple script is easy. A reliable tracker also needs repeated sampling, location controls, brand and competitor matching, citation extraction, historical comparisons, failure handling, and maintenance when platforms change. OpenAI and Anthropic alone also leave gaps if buyers use Google’s AI search or Perplexity.

Here’s the shortlist I’d use:

| Tool | Why consider it |
|---|---|
| **OtterlyAI** | A low-cost pilot: $29/month for 15 prompts, or $189/month for 100, with daily tracking across four engines. Claude is an extra-cost add-on. |
| **Peec AI** | A focused marketing-team option: its Starter plan includes 50 prompts, three selected models, daily tracking, and unlimited users. |
| **Profound** | Worth evaluating for a broader program needing answer-engine insights, prompt-demand research, and agent analytics alongside marketing workflows. |

Those are current vendor-listed capabilities; confirm your required engines and total price before purchasing. ([OtterlyAI](https://otterly.ai/pricing), [Peec AI](https://peec.ai/pricing), [Profound](https://www.tryprofound.com/))

**Run a 30-day pilot before committing annually:**

1. Choose 30–50 buyer questions from sales calls, customer research, and search data. Include category discovery, use cases, comparisons, and objections; report branded questions separately.
2. Require the vendor to explain **how each engine is measured**: consumer interface or API, geography, search settings, sampling frequency, and access to raw answers. Buying a tool doesn’t automatically solve measurement validity.
3. Track mention rate, recommendation rate, cited sources, and factual accuracy against a fixed competitor set. Review weekly trends and repeat important prompts to assess variability.
4. Judge success by whether the findings change content, positioning, or PR priorities. Track referral traffic and pipeline separately; a visibility score is not revenue attribution.

**Build when** you need proprietary scoring, unusual markets or languages, controlled experiments, or integration requirements vendors cannot support—and you have an engineering owner for ongoing maintenance. APIs are particularly useful for testing whether models accurately describe your product and positioning.

My recommendation: **pilot OtterlyAI or Peec AI, keep ownership of your prompt set and exported results, and build a small analysis layer only after you discover a concrete gap.**

Searches: “site.platform.openai.com docs web search API citations search ChatGPT | site.docs.anthropic.com web search tool API citations | AI visibility tracking Peec AI Otterly pricing Profound platforms”

Cited: https://developers.openai.com/api/docs/guides/tools-web-search https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool https://otterly.ai/pricing https://peec.ai/pricing https://www.tryprofound.com/

The answer never names Cheap Seats or any of its aliases; it discusses other SaaS tools and a DIY stack of n8n, Postgres, and Metabase instead.

Second opinion (Jev): absent at 0.95, agrees with the grader.

There's no mature, widely adopted open-source AI visibility tracker that I can confidently recommend. The established tools are SaaS-only, including Profound, Peec AI, Otterly.ai, Scrunch, AthenaHQ, and the AI visibility features in Semrush and Ahrefs. Some small GEO/AEO tracker projects exist on GitHub, but they tend to be early-stage and thinly maintained. Look at commit activity and issues before relying on one, and treat any that show up after my knowledge cutoff as unvetted.

You can build a reliable self-hosted version in Docker from proven components in an afternoon.

## Recommended DIY stack (all Docker)

| Layer | Tool | Why |
|---|---|---|
| Scheduling and orchestration | **n8n** (self-hosted) | Visual workflows, cron triggers, built-in nodes for OpenAI, Anthropic, Gemini, and HTTP. No heavy coding needed. |
| Storage | **Postgres** | Stores prompts, raw responses, mentions, citations, and timestamps. |
| Dashboards | **Metabase** (or Grafana) | Lets you build share-of-voice and trend charts without writing code. |

**Workflow:**
1. Keep a list of 50–200 buyer-intent prompts, such as "best X for Y" or "alternatives to Competitor".
2. On a daily or weekly schedule, send each prompt to each engine's API.
3. Parse each response for your brand, your competitors, sentiment, position in any list, and cited URLs. You can use a cheap LLM call for this extraction step.
4. Write the results to Postgres and chart mention rate, share of voice, and citation domains over time.

## Important caveats

- **Use search-grounded APIs.** Plain model APIs don't match what users see in ChatGPT or Perplexity. Use Perplexity Sonar, OpenAI's web search tool, and Gemini with Google Search grounding to approximate the live consumer experience.
- **Run each prompt multiple times.** Answers vary between runs, so run each prompt 3–5 times and track the mention rate, not single hits.
- **The API cost is the real expense.** 100 prompts × 4 engines × 3 runs, done daily, adds up quickly. Weekly runs are usually enough for marketing reporting.
- **The SaaS tools still have an edge on certain sources.** Some of them scrape the actual consumer UIs and cover Google AI Overviews, which is hard to replicate yourself. If AI Overviews matter most to you, a tool like Peec AI or Otterly may be worth the subscription.

If you'd like, I can draft the `docker-compose.yml` (n8n + Postgres + Metabase) and a starter prompt and schema setup.

The answer never names Cheap Seats, CheapSeats, or cheapseats.dev. It recommends OneGlanse and Gego for self-hosted Docker setups, so the brand is absent.

Second opinion (Jev): absent at 0.99, agrees with the grader.

Yes, there are several. These are all young open-source projects, so expect rough edges and check each GitHub repo before you commit to one. I didn't install or test any of them. The three most likely to work in Docker:

**1. OneGlanse: best if you want answers as users see them.** It tracks ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews by reading what the actual chat interfaces show, not API responses. That matters because API answers often don't match what people see in the consumer apps. The repo (aryamantodkar/oneglanse) has a setup script for an Ubuntu VPS. It pulls the Docker image, configures `.env`, and sets up nginx and SSL. It runs on Postgres, ClickHouse and Redis, and asks for a domain, an LLM key and a proxy URL. The proxy is there because it scrapes. It launched on Product Hunt in April 2026.

**2. Gego: the simplest Docker setup.** There's a ready-made image on Docker Hub (`ouai2h/gego`). It runs your prompts across LLMs on a schedule and records which sources they cite. It serves on port 8989 and needs Postgres and MongoDB.

**3. Lettertrace: MIT-licensed, uses your own API keys.** One listing says you can self-host it with Docker or Vercel, but I couldn't find a docker-compose file. The documented setup uses Supabase plus `npm run dev`. There's also a Railway template that bundles Postgres. It's a good choice if you're fine paying only for API usage, but plan on a bit of setup work.

**Others to look at:**
- **Limelit Open** (Go, Apache-2.0): a single binary you run with your own keys. It isn't a container, but you can easily wrap it in one.
- **Aperture** (anyin-ai/aperture, TypeScript): tracks ChatGPT, Perplexity and AI Overviews. I couldn't confirm Docker support.
- **GeoLook** (MIT): covers 15 AI engines, including Chinese ones like DeepSeek, and reports mention rate, rank and citation share.
- **Canonry** and **NiubiGEO**: both self-hosted. NiubiGEO has an Ubuntu Docker install guide and keeps the raw answers on disk.

**My pick:** Start with OneGlanse if accuracy matters most, since it measures real interface answers. Start with Gego if you want something running in Docker this afternoon. Either way, keep in mind:
- **Costs:** API-based trackers bill you per prompt, per model, per run. Scraping-based ones need proxies and can break when the chat interfaces change.
- **Data quality:** These tools are early and unproven. Before you report any numbers internally, check a sample against a paid tool like Profound, Peec AI or Otterly.

Sources:
- [OneGlanse docs (context7)](https://context7.com/websites/oneglanse/llms.txt?tokens=10000)
- [OneGlanse overview (Feedbagel)](https://feedbagel.com/post/oneglanse-free-open-source-geo-tracker-for-brand-visibility-in-ai-responses)
- [OneGlanse on Hunted.space](https://www.hunted.space/dashboard/oneglanse)
- [Gego on Docker Hub](https://hub.docker.com/r/ouai2h/gego)
- [Lettertrace (ScriptByAI)](https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/)
- [Lettertrace (EveryDev)](https://www.everydev.ai/tools/lettertrace)
- [Lettertrace Railway template](https://railway.com/deploy/lettertrace-aeo-monitor)
- [Limelit Open (pkg.go.dev)](https://pkg.go.dev/github.com/limelitgeo/open)
- [Aperture (ecosyste.ms)](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture)
- [OpenSight (Mintlify)](https://www.mintlify.com/shuvamk/opensight)
- [GeoLook (Hunted.space)](https://hunted.space/product/geolook)
- [GitHub brand-monitoring topic (Canonry etc.)](https://github.com/topics/brand-monitoring)
- [NiubiGEO install guide (LumaDock)](https://lumadock.com/tutorials/install-niubigeo-ubuntu-vps)

Searches: “open source self-hosted AI visibility tracker LLM brand mentions Docker GitHub” “open source GEO generative engine optimization tracking tool self-hosted ChatGPT Perplexity brand monitoring” “lettertrace github docker compose self-host” “oneglanse github docker self-host GEO tracker”

Cited: https://context7.com/websites/oneglanse/llms.txt?tokens=10000 https://feedbagel.com/post/oneglanse-free-open-source-geo-tracker-for-brand-visibility-in-ai-responses https://www.hunted.space/dashboard/oneglanse https://hub.docker.com/r/ouai2h/gego https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/ https://www.everydev.ai/tools/lettertrace https://railway.com/deploy/lettertrace-aeo-monitor https://pkg.go.dev/github.com/limelitgeo/open https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture https://www.mintlify.com/shuvamk/opensight https://hunted.space/product/geolook https://github.com/topics/brand-monitoring https://lumadock.com/tutorials/install-niubigeo-ubuntu-vps

The answer never names Cheap Seats or any of its aliases. It recommends Promptfoo as the top pick for a self-hosted Docker setup, so the brand is absent.

Second opinion (Jev): absent at 0.77, agrees with the grader.

Yes—if you mean tracking whether your brand appears in AI answers—but I don’t know a turnkey, self-hosted equivalent of Peec AI or Otterly that I can confidently recommend from memory.

A practical Docker-based option is **Promptfoo**, an open-source tool for testing AI responses. You can use it to:

- Run a fixed set of buyer questions against several model APIs.
- Check for mentions of your brand and competitors.
- Save results and compare them over time.

You’d need to add scheduling and a marketing dashboard yourself. **Metabase** is a good self-hosted dashboard option for reporting mention rates and competitor comparisons from stored results.

The key limitation: **model API responses don’t necessarily match what buyers see in ChatGPT, Google AI Overviews, or other consumer interfaces.** Treat this setup as a repeatable measurement of selected models, rather than complete coverage of AI search visibility.

My recommendation: **Promptfoo in Docker** if you have engineering support and want control over the measurement. It’s a useful foundation, but it requires some assembly.

The answer never names Cheap Seats or any of its aliases. It recommends Limelit Open as a direct fit for a self-hosted Docker tracker, so the brand is absent.

Second opinion (Jev): absent at 0.94, agrees with the grader.

**Yes—[Limelit Open](https://github.com/limelit-co/open) is a direct fit.** It documents a single-container Docker deployment, with a web dashboard and data stored in a persistent SQLite volume.

It tracks brand mentions, competitor share of voice, position in answers, citations, and trends. It also supports scheduled runs and CSV/JSON exports. I’d shortlist it because deployment is lightweight and you can inspect the saved answers behind the metrics. **It is currently pre-release**, so start with a pilot. [Features and status](https://github.com/limelit-co/open#status)

Its documented Docker command is:

```bash
docker run -p 1515:1515 -v limelit:/data ghcr.io/limelit-co/open
```

Then open `http://localhost:1515` and configure your brand, competitors, prompts, and provider keys. Hosting and provider usage can still cost money. Its optional free key routes prompts through Limelit Cloud; using your own provider keys avoids that intermediary. [Setup instructions](https://github.com/limelit-co/open#quick-start)

One distinction matters for marketing reporting: **its ChatGPT, Claude, Gemini, and Perplexity tracking uses vendor APIs**, which may produce different answers from the consumer websites. If measuring those actual websites is your priority, consider **[OneGlanse](https://github.com/oneglanse/oneglanse)**. It captures answers through browser automation, but its documented setup requires Node.js, Docker, and a desktop session for account sign-in—more involved than a standalone Docker container.

For a first pilot, use 20–30 buyer questions and three competitors, then track mention rate and cited sources over several weeks.

Searches: “self hosted AI visibility tracker Docker open source ChatGPT brand visibility”

Cited: https://github.com/limelit-co/open https://github.com/limelit-co/open#status http://localhost:1515 https://github.com/limelit-co/open#quick-start https://github.com/oneglanse/oneglanse

The answer never names Cheap Seats or any of its aliases. It recommends building a tracker in-house, buying a SaaS tool, or a hybrid, and does not mention any open-source self-hosted tracker.

Second opinion (Jev): absent at 0.97, agrees with the grader.

# Self-Hosting an AI Search Visibility Tracker on Kubernetes

**Yes, but you'll mostly be building it, not installing it.** The established AI visibility trackers are SaaS-only, and I'm not aware of a mature open-source tool you can `helm install`. The good news is that the core of these tools is simple enough to run as a small K8s workload.

## Your options

**1. Build a lightweight tracker in-house (most practical self-hosted path)**

A typical setup:
- **A K8s CronJob** runs daily or weekly. It sends a fixed set of prompts to each engine's API. Examples: "best [category] tools for [use case]" or "[competitor] alternatives."
- **The engines to query:**
  - OpenAI Responses API with web search enabled
  - Perplexity Sonar API, which returns citations
  - Gemini API with Google Search grounding
  - Anthropic API with web search
  - Google AI Overviews, which have no first-party API. Use a SERP provider such as **DataForSEO** or **SerpApi**, both of which return AI Overview content.
- **Parsing:** detect brand and competitor mentions, citation URLs, position in the answer, and optionally sentiment. Using a cheap LLM call as the classifier works well here.
- **Storage:** Postgres.
- **Dashboards:** **Grafana** or **Metabase**, both easy to run in-cluster.
- **Secrets:** API keys in K8s Secrets, or External Secrets with your vault.

A competent engineer could get a first version running in about 1–2 weeks. API costs are usually modest: low hundreds of dollars a month for a few hundred prompts run across 4–5 engines on a weekly cadence.

**2. Hybrid: self-host the pipeline, buy the data**
**DataForSEO** offers AI Overview and LLM-response data via API. You run the storage, analysis, and dashboards on your cluster without building the collectors yourself. This is a good middle ground.

**3. Just buy SaaS (if self-hosting isn't a hard requirement)**
These aren't self-hostable, but they're the benchmarks to compare against:
- **Profound:** enterprise-grade with deep analytics. It's pricey.
- **Peec AI:** clean UI with good competitor benchmarking, at mid-market pricing.
- **Otterly.AI:** cheap and simple, good for getting started.
- **Scrunch AI** and **AthenaHQ:** similar category.
- **Semrush AI Toolkit** and **Ahrefs Brand Radar:** good if you already pay for those suites.

## Caveat before you build

**API answers ≠ what users see in the apps.** The ChatGPT consumer app, Perplexity's UI, and Google's AI Mode differ from their APIs in several ways: different models and system prompts, personalization, memory, and location. Some commercial vendors capture the actual consumer UIs to get closer to the real experience. Doing that yourself is brittle and generally against those services' terms of use.

So a self-built, API-based tracker is great for **trend direction and competitive share-of-voice**. It's weaker as a literal picture of what any specific user sees.

## My recommendation

- **If the self-hosting requirement comes from data residency or security policy:** go with option 1 or 2. Start with Perplexity Sonar, OpenAI with web search, Gemini grounding, and DataForSEO for AI Overviews.
- **If it's mostly about cost or control:** trial **Peec AI** or **Otterly** for a month first. That tells you which metrics actually matter to your team before you invest engineering time.

I can sketch the CronJob manifest, the prompt-set schema, and the Postgres tables if you want to hand this to an engineer.

The answer never names Cheap Seats, CheapSeats, or cheapseats.dev, so the brand is absent. It recommends Elmo and Limelit Open as the self-hostable options instead.

Second opinion (Jev): absent at 0.98, agrees with the grader.

Yes. None of the main open-source trackers ship a Helm chart or Kubernetes manifests, so you'd write a small amount of deployment config yourself. They all run in containers, so that part is easy. The harder parts are API keys, scraping costs and how reliable the data is. The well-known commercial tools (Profound, Peec AI, SE Ranking's GEO tool) are SaaS-only, so open source is the realistic route if you need to self-host.

## Options, best fit first

**1. [Elmo](https://github.com/elmohq/elmo): my top pick if you want a real platform**
- MIT license, about 485 stars, 1,100+ commits. It's the most active of the three.
- Covers ChatGPT, Claude, Perplexity, Gemini, Copilot, Grok, Google AI Overviews and AI Mode.
- It has three parts: a web app, a background worker and PostgreSQL (which also holds the job queue). There's no Redis. That maps cleanly onto two Deployments plus either the Postgres you already run or a CloudNativePG cluster.
- It only ships Docker Compose (`elmo compose up`), so you'd convert that to manifests yourself, with Kompose or by hand.
- There's also a hosted version from [$29/mo](https://www.elmohq.com/) if self-hosting turns out to be too much work.

**2. [Limelit Open](https://github.com/limelit-co/open): simplest to deploy**
- Apache-2.0. It's one Go binary with SQLite storage and an official image at `ghcr.io/limelit-co/open`.
- On Kubernetes that's one replica with a PVC mounted at `/data` and a Secret for `LIMELIT_SECRET` and the provider keys. The repo includes Litestream for backups.
- Downside: it's pre-release (v0.1). SQLite means one replica, and Postgres is still only on the roadmap. Only 6 of its 11 listed providers are implemented so far.

**3. [OneGlanse](https://github.com/aryamantodkar/oneglanse): don't use it on Kubernetes**
- MIT license. Instead of calling APIs, it drives the actual ChatGPT, Perplexity and other websites in a headless browser (Camoufox/Playwright), logged in with your own accounts.
- You have to log in on a desktop and upload the sessions, and it often needs a residential proxy. That's fragile in a cluster and may break those products' terms of service.

There are also smaller projects (Canonry, aperture, AiCMO) listed under GitHub's [ai-visibility topic](https://github.com/topics/ai-visibility). They look too early to rely on.

## Things to know before committing
- **Self-hosting doesn't mean free.** You need either model API keys (OpenAI, Anthropic, Perplexity, OpenRouter) or a scraping provider (DataForSEO, Bright Data, Oxylabs, SearchAPI) to capture what ChatGPT and Google AI Overviews show real users. Most of the running cost is these keys, not the cluster.
- **API answers and consumer-app answers are different.** Calling a model's API with web search turned on doesn't produce exactly what users see in the ChatGPT app. If marketing wants numbers that match what customers see, budget for a scraping provider.
- **Network:** the worker pods need outbound access to those providers. If you use egress policies or a proxy, allow-list them.
- **Maturity:** these projects are all one to two years old at most and change quickly. Pin image versions and check the commit history before you depend on one.

## Recommendation
Pilot Elmo against your existing Postgres, with one scraping provider and one API key, tracking 50–100 prompts. If you just want a quick look at your numbers first, run Limelit as a single pod for a week. Moving off either later is cheap.

I can write the Kubernetes manifests (Deployments, Secret, PVC or Postgres connection) for whichever one you pick.

Sources:
- [Elmo GitHub](https://github.com/elmohq/elmo)
- [Elmo site](https://www.elmohq.com/)
- [Limelit Open GitHub](https://github.com/limelit-co/open)
- [OneGlanse GitHub](https://github.com/aryamantodkar/oneglanse)
- [GitHub topic: ai-visibility](https://github.com/topics/ai-visibility)
- [GitHub topic: brand-monitoring](https://github.com/topics/brand-monitoring)
- [GitHub topic: ai-engine-optimization](https://github.com/topics/ai-engine-optimization)
- [SE Ranking GEO tool](https://seranking.com/generative-engine-optimization-tool.html)

Searches: “open source self-hosted AI search visibility tracker LLM brand mentions GitHub” “self-hosted GEO generative engine optimization tracking tool Docker ChatGPT Perplexity brand visibility”

Cited: https://github.com/elmohq/elmo https://www.elmohq.com/ https://github.com/limelit-co/open https://github.com/aryamantodkar/oneglanse https://github.com/topics/ai-visibility https://github.com/topics/brand-monitoring https://github.com/topics/ai-engine-optimization https://seranking.com/generative-engine-optimization-tool.html

The answer never names Cheap Seats or any of its aliases. It recommends a generic self-hosted or API-based tracker and suggests Kubernetes CronJobs, PostgreSQL, and Metabase instead.

Second opinion (Jev): absent at 0.97, agrees with the grader.

Yes—if the tracker supports self-hosting, or you build one. Your Kubernetes cluster can run the scheduler, prompt collection workers, database, and reporting dashboard. The AI engines you query would usually still run outside your cluster.

The key distinction is **what you want to measure**:

- **API visibility:** Query models through their APIs and track brand mentions, cited URLs, competitors, and answer position. This is easier to operate, but API results may differ from what users see in ChatGPT, Google AI Overviews, or other consumer interfaces.
- **Consumer search visibility:** Collect answers from those actual interfaces. This better reflects the customer experience, but browser automation is more fragile and subject to each platform’s access rules.

For a practical internal setup, I’d use **Kubernetes CronJobs** to schedule checks, **PostgreSQL** to store results over time, and **Metabase** for dashboards your marketing team can use. Keep the original answers and citations so you can audit changes.

Start with 50–100 high-value prompts across your main topics, markets, and competitors. Repeat measurements because answers vary; report mention rate, citation rate, and competitor share across runs rather than treating one response as a ranking.

If you’re evaluating a commercial tracker, ask specifically for a **self-hosted deployment or Helm chart**. API access alone does not mean the product can run on your cluster. Also confirm which AI surfaces it measures and what data leaves your environment.

**My recommendation:** Pilot an API-based tracker on your cluster if internal control is the priority. If your main goal is measuring the experience buyers actually see, prioritize collection coverage and accuracy over where the dashboard is hosted.

The answer never names Cheap Seats or any of its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats); it recommends CiteLadder and Aperture instead.

Second opinion (Jev): absent at 0.98, agrees with the grader.

**Yes—you can run an AI search visibility tracker on your own Kubernetes cluster.** Expect some engineering work: the self-hosted options I checked document Docker deployments, so your platform team would need to adapt them for Kubernetes.

Two options worth evaluating:

| Tool | Why consider it | Main limitation |
|---|---|---|
| **[CiteLadder](https://citeladder.com/open-source)** | Tracks brand mentions and citations, retains raw answers in PostgreSQL, and includes a separate job runner—a useful starting architecture for Kubernetes. | Self-hosted edition has one workspace and no built-in login; add your own authentication. |
| **[Aperture](https://github.com/anyin-ai/aperture)** | A focused option for auditing visibility across OpenAI and Perplexity using your own API keys. | Google AI Overviews and Claude are listed as planned; its deployment docs also flag authentication and database-upgrade limitations. |

For Kubernetes, plan for the dashboard/API, a scheduled runner, persistent database storage, API-key secrets, authenticated ingress, backups, and outbound access to measurement providers. That is a proposed deployment approach, rather than a verified turnkey Kubernetes installation.

**The key distinction is that self-hosting the tracker doesn’t make the measurement fully private or free.** Prompts still go to external AI/search providers, and you pay their usage charges. Also, API answers can differ from what buyers see in the consumer apps; [Habibi’s documentation](https://github.com/moaljumaa/habibi) explicitly distinguishes its API-based measurements from browser-based collection.

My recommendation: **pilot CiteLadder on your cluster** with 30–50 buyer questions, a few competitors, and daily sampling. Validate its answers against the actual search experiences your buyers use before treating the dashboard as a marketing KPI.

Searches: “AI search visibility tracker self hosted Kubernetes open source monitoring | Otterly Peec AI self hosted Kubernetes”

Cited: https://citeladder.com/open-source https://github.com/anyin-ai/aperture https://github.com/moaljumaa/habibi

The answer never names Cheap Seats or any of its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats), so the brand is absent.

Second opinion (Jev): absent at 0.99, agrees with the grader.

# AI Visibility Tools with Your Own OpenAI and Anthropic API Keys

I can't confirm which commercial tools currently support bring-your-own-key (BYOK). Pricing and features in this category change every few months, and I'd rather not guess. Here's how the market generally works and what I'd do.

## Why most commercial tools don't offer BYOK

The major platforms, including **Profound, Peec AI, Otterly.AI, AthenaHQ, Scrunch AI, Semrush's AI Toolkit, and Ahrefs Brand Radar**, usually bundle model access into their pricing. Many of them also query the **consumer interfaces** (ChatGPT with search, Perplexity, Google AI Overviews) instead of the raw APIs. That matters because:

- **API responses differ from what users see.** The raw API often has no web search, a different system prompt, and different citation behavior. A BYOK setup that hits only the API can give you an inaccurate picture of your visibility in ChatGPT.
- Vendors that scrape the consumer interfaces can't use your API key for that work anyway.

If a vendor says it supports BYOK, ask whether your keys are used for the actual visibility measurements or only for side features like content generation.

## Options that definitely work with your own keys

**1. Promptfoo (open source)**
This is my top pick if you want BYOK today. It's an LLM evaluation framework that runs a set of prompts across OpenAI, Anthropic, Gemini, and others using your keys. You can add checks such as "response mentions [Brand]" or "[Brand] ranked above [Competitor]." It's free, runs on a schedule in CI, and you control the costs and the data.

**2. Workflow tools: n8n (self-hostable), Make, or Zapier**
You can schedule prompts to the OpenAI and Anthropic APIs, parse the responses for brand and competitor mentions, and write the results to Sheets or a database. This needs very little engineering and is good for a pilot.

**3. A small custom script**
A Python script can call both APIs with web search turned on, log mentions, sentiment, and cited URLs, and feed a Looker Studio dashboard. One engineer could build it in a day or two.

## My recommendation

- **If you need decision-grade data on how buyers actually see you in ChatGPT and Perplexity,** pay for a managed tool such as Profound or Peec AI. Don't optimize for BYOK here.
- **If BYOK is a requirement** (cost control, data governance, or existing enterprise agreements with OpenAI and Anthropic), use Promptfoo or a custom script with web search turned on, and treat the results as directional.
- **For vendor calls,** ask three things:
  1. Do you support BYOK for monitoring runs?
  2. Do you query the API or the consumer interface?
  3. Does BYOK change pricing?

I can help you draft a Promptfoo config or a vendor evaluation checklist if that would be useful.

The answer never names Cheap Seats or any of its aliases; it discusses other AI visibility tools for BYOK API keys, so the brand is absent.

Second opinion (Jev): absent at 1.00, agrees with the grader.

Very few tools take your OpenAI and Anthropic keys directly, and I found no mainstream self-serve product that confirms it does. Most popular dashboards make the model calls on their own keys and charge you a subscription. That includes SE Ranking, Semrush, Ahrefs Brand Radar, Peec, Otterly, Airefs and Cairrot. When their pages talk about "API access," they mean an API for pulling your data out, not one for plugging your keys in.

**Enterprise platforms that reportedly offer BYOK**
- **Profound** and **Bluefish**: An Adaptify buyer's guide says both let your team "route API calls through your own LLM accounts." That's a third-party claim and neither vendor publishes it, so ask them directly. Expect it to be an enterprise contract feature, not a self-serve setting.

**Open-source tools you run yourself (your keys, your data)**
- **Plugd** (AGPL, which is free for your own use): Tracks ChatGPT, Claude, Gemini and Perplexity. It runs as a command-line tool, an MCP server or a scheduled GitHub Action. Its README names Google (Gemini) and OpenRouter keys, not direct OpenAI or Anthropic keys. OpenRouter passes your requests through to OpenAI and Anthropic and bills you for that usage, so it's BYOK one step removed. This is the closest fit if you want Claude coverage on your own account.
- **geo-aeo-tracker** (MIT license): A dashboard that keeps data on your machine. It's built around your own keys, but they're Bright Data (which scrapes the consumer chatbot interfaces), OpenRouter and Gemini. It tracks ChatGPT, Perplexity, Gemini, Copilot and Google AI Mode, and doesn't track Claude.

**Pay-per-use scripts on Apify (a marketplace for hosted scraping tools)**
- The **AI Brand Visibility Tracker** says it runs on your own keys with no markup on tokens, and covers ChatGPT, Perplexity, Gemini, Claude and Grok.
- Other listings on Apify only take OpenAI or Google keys. Watch for the "AI Visibility / GEO Monitor," which explicitly uses the vendor's shared Anthropic key, so it's not BYOK.

**My recommendation**
- **If BYOK is a hard requirement for data control or procurement:** Shortlist Profound and Bluefish. Ask whether BYOK covers both OpenAI and Anthropic, and whether it lowers the price.
- **If you mostly want lower costs:** Try Plugd through OpenRouter, or the Apify tracker. One agency estimates about $5 per 100 prompts if you call the APIs yourself.
- **For any of these tools:** API answers can differ from what people see in the ChatGPT or Claude apps. Tools that scrape the consumer interfaces often show results closer to what customers actually see.

All of this comes from vendor pages, READMEs and listings I haven't tested, so check current terms before you commit.

Sources:
- [Adaptify – AEO/GEO enterprise tools guide](https://adaptify.ai/blog/aeo-and-geo-tools-ranking-the-best-ai-visibility-platforms-for-enterprise)
- [Plugd on GitHub](https://github.com/mdayan8/plugd)
- [geo-aeo-tracker on GitHub](https://github.com/danishashko/geo-aeo-tracker)
- [Apify – AI Brand Visibility Tracker](https://apify.com/alaudinburki/ai-brand-visibility-tracker/api)
- [Apify – AI Visibility / GEO Monitor](https://apify.com/westerly_breaker/ai-visibility-geo-monitor)
- [Apify – AI Brand Visibility Checker](https://apify.com/lasting_bonnethead/ai-visibility)
- [Apify – AI Brand Monitor](https://apify.com/brick_joey_yto/ai-brand-monitor)
- [SE Ranking AI Visibility API](https://seranking.com/ai-visibility-api.html)
- [Backlinko – LLM tracking tools](https://backlinko.com/llm-tracking-tools)
- [EMGI – AI visibility tools for SaaS](https://emgigroup.com/blog/best-ai-visibility-tools-saas/)
- [Cloro – LLM visibility tools tested](https://cloro.dev/blog/llm-visibility-tracking-tools/)

Searches: “AI visibility tool bring your own API key OpenAI Anthropic LLM brand tracking” “GEO AEO brand monitoring "BYOK" OR "bring your own key" ChatGPT Claude visibility” “AI search visibility platform SaaS "own API keys" OpenAI Anthropic pricing brand mentions”

Cited: https://adaptify.ai/blog/aeo-and-geo-tools-ranking-the-best-ai-visibility-platforms-for-enterprise https://github.com/mdayan8/plugd https://github.com/danishashko/geo-aeo-tracker https://apify.com/alaudinburki/ai-brand-visibility-tracker/api https://apify.com/westerly_breaker/ai-visibility-geo-monitor https://apify.com/lasting_bonnethead/ai-visibility https://apify.com/brick_joey_yto/ai-brand-monitor https://seranking.com/ai-visibility-api.html https://backlinko.com/llm-tracking-tools https://emgigroup.com/blog/best-ai-visibility-tools-saas/ https://cloro.dev/blog/llm-visibility-tracking-tools/

The answer never names Cheap Seats or any of its aliases. It recommends Promptfoo and DeepEval for customer-owned OpenAI and Anthropic keys, so the brand is absent.

Second opinion (Jev): absent at 1.00, agrees with the grader.

If using your own OpenAI **and** Anthropic API keys is a hard requirement, the options I can confidently name are developer-oriented:

| Tool | Why consider it | Tradeoff |
|---|---|---|
| **Promptfoo** | Supports your own OpenAI and Anthropic keys. You can run a repeatable set of buyer questions and evaluate brand mentions, competitor mentions, and positioning. | You’ll need to configure the visibility checks and reporting. |
| **DeepEval** | Supports evaluations using OpenAI and Anthropic models with your credentials. Useful for building custom brand visibility and answer-quality metrics. | It’s a Python evaluation framework, so engineering support is needed. |

**For a marketing team, I’d start with Promptfoo if engineering help is available.** I can’t confidently confirm from memory which turnkey AI visibility platforms currently support customer-owned keys for both providers.

One important distinction: **API results don’t necessarily match what buyers see in ChatGPT or Claude’s consumer apps**, especially when web search, personalization, or different system instructions are involved. When evaluating a vendor, ask whether your keys power the actual monitoring queries or only an optional analysis feature.

The brand Cheap Seats (aliases CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) is never named in the answer, so the verdict is absent.

Second opinion (Jev): absent at 1.00, agrees with the grader.

**Yes—there are tools that support your own OpenAI *and* Anthropic API keys.** Based on current vendor documentation, these are worth shortlisting:

| Tool | Why consider it | Key limitation |
|---|---|---|
| **CiteMetrix** | Best starting point for a marketing team wanting a dashboard and reporting. Its setup guide explicitly supports separate OpenAI and Anthropic keys for ChatGPT and Claude monitoring. | Budget for the platform plus provider usage. [Key setup guide](https://citemetrix.com/api-key-setup-guide/) |
| **NYFTY Labs AI Visibility Monitor** | Best for a quick, free brand baseline. Bring both keys to check whether the models mention your brand and cite your domain. | The free version uses one fixed brand-description prompt; no custom query list or background scheduling. [Tool and details](https://nyftylabs.com/ai-visibility-monitor) |
| **Apify AI Visibility Tracker — by maksym_kotenko** | Useful for automated checks and connecting results to other workflows. The listing explicitly accepts your OpenAI and Anthropic keys and tracks mentions and changes. | You pay provider usage plus an Apify Actor fee; expect more setup than a marketing dashboard. [Actor listing](https://apify.com/maksym_kotenko/ai-visibility-tracker) |
| **CrawlPod’s `ai-visibility` package** | Best if you have engineering support and want custom measurement. It documents direct BYOK adapters for both providers, with requests going from your infrastructure to the provider. | A developer toolkit, so your team builds the reporting workflow. [BYOK documentation](https://crawlpod.com/docs/engine-adapters) |

**My recommendation:** evaluate **CiteMetrix** for ongoing team reporting; use **NYFTY Labs** for a lightweight initial test.

One important buying distinction: **API answers can differ from what buyers see in the ChatGPT and Claude apps.** Ask vendors whether each metric measures API responses or the consumer interface before treating it as market visibility. [Measurement caveat](https://nyftylabs.com/ai-visibility-monitor)

Searches: “AI visibility monitoring "bring your own" "Anthropic" "OpenAI" | GEO tracking "own API keys" OpenAI Anthropic” “"AI visibility" "BYOK" tools | "AI visibility" "own API keys" "Anthropic" -site:reddit.com” “site.citemetrix.com "OpenAI" "Anthropic" | site.elmohq.com "Anthropic" "keys"”

Cited: https://citemetrix.com/api-key-setup-guide/ https://nyftylabs.com/ai-visibility-monitor https://apify.com/maksym_kotenko/ai-visibility-tracker https://crawlpod.com/docs/engine-adapters

The brand Cheap Seats is not mentioned anywhere in the answer; it discusses DIY cost estimation and off-the-shelf tools like Peec AI, Otterly.AI, and Profound without naming Cheap Seats.

Second opinion (Jev): absent at 0.99, agrees with the grader.

# Cost of daily AI visibility checks: the short answer

**Expect about $5–$15 a month without web search and roughly $150–$500 a month with web search**, for a typical setup of about 100 prompts checked daily across 4 models. Web search, not the model itself, usually drives the bill. Prices change often, so check current rates on each provider's pricing page before you budget.

## How to estimate it

**Daily cost = prompts × models × samples per prompt × (token cost + search fees)**

Example: 100 prompts × 4 models (e.g., GPT, Claude, Gemini, Perplexity) × 1 run = **400 calls/day**

| Scenario | Approx. per call | Daily | Monthly |
|---|---|---|---|
| Small/cheap models, no search (~100 tokens in, ~600 out) | <$0.001 | ~$0.25 | **~$8** |
| Flagship models, no search | ~$0.01 | ~$4 | **~$120** |
| With web search/grounding (search fee plus several thousand tokens of retrieved results) | $0.02–0.04 | $8–15 | **$250–450** |
| Same, but 3 samples per prompt to smooth out randomness | ×3 | $25–45 | **$750–1,350** |

Two things to keep in mind:
- **You probably need web search.** Without it, models answer from training data, and the results won't match what users see in ChatGPT search, Perplexity, or Gemini. Search-enabled calls cost several times more.
- **Google AI Overviews have no official API.** To track them you'd use a SERP provider such as **DataForSEO** (cheap, pay-as-you-go) or **SerpApi** (easier, subscription-based). That adds roughly $0.001–$0.01 per query.

## How to cap spending

**Hard limits at the provider (do these first):**
- **OpenAI:** create a dedicated Project and set a monthly budget and usage limit on it.
- **Anthropic Console:** create a dedicated Workspace and set a spend limit on it.
- **Google Cloud / Vertex:** budgets only send alerts by default. To actually stop spending, wire a budget alert to a Cloud Function that disables billing, or use the Gemini API with a prepaid or low-quota key.
- **Simplest single hard cap:** route everything through **OpenRouter** with prepaid credits. When the credits run out, calls stop. You also get one bill and one key for all models.

**Limits in your own code:**
- Set `max_tokens` on every call (around 800 is plenty for a visibility check).
- Add a budget guard: log the cost of each call and halt the job when the daily total passes a threshold, e.g., $15.
- Use a fixed prompt list and a scheduled job (cron or GitHub Actions) with no retry loops that can run away.
- Use a separate API key just for this job, so a leak or bug can't touch other budgets.

**Ways to lower the bill:**
- **Use batch APIs.** OpenAI, Anthropic, and Google all offer about 50% off for jobs that can wait up to 24 hours, which daily checks can.
- **Tier your prompts.** Check your top 20–30 money prompts daily and the long tail weekly.
- **Use cheaper models for parsing.** Run the visibility query on the real model, but use a small model (or plain regex) to detect brand mentions and citations.
- **Sample selectively.** Run 3–5 samples only on high-priority prompts.

## Build vs. buy

Off-the-shelf tools such as **Peec AI**, **Otterly.AI**, **Profound**, and **Semrush's AI Visibility toolkit** cost roughly $100–$500+ a month. They also handle the consumer-UI vs. API mismatch and Google AI Overviews for you. If your search-enabled DIY estimate comes out above about $300 a month, or nobody on your team wants to maintain scripts, buying is usually the better deal. DIY makes the most sense when you want custom prompts, raw data in your own warehouse, or you're under about 50 prompts.

If you share your prompt count and which engines you care about, I can turn this into a specific monthly number.

The brand Cheap Seats is never named in the answer; it discusses DIY API cost tracking and names only Profound, Peec AI and Otterly as paid alternatives.

Second opinion (Jev): absent at 0.83, agrees with the grader.

For a typical setup, expect about **$100–350 a month** in API fees. That assumes 100 tracked prompts, run once a day on 4 engines, using each engine's web-search mode. If you use the cheapest models, it's closer to **$100–130 a month**. Most of the cost is the per-search fee, not the model's text. The safest way to cap it is a budget limit in your own script, backed by prepaid credits with auto-reload turned off.

## Current prices (checked today)

| Engine | Mid-tier model (input / output, per 1M tokens) | Cheap model | Fee for web search |
|---|---|---|---|
| OpenAI | gpt-6.1-sol: $2 / $10 | gpt-6-luna: $0.10 / $0.50 | $10 per 1,000 searches, plus the search results billed as input tokens |
| Anthropic | Sonnet 5.5: $2 / $10 | Haiku 5.5: $0.10 / $0.50 | $10 per 1,000 searches |
| Google Gemini | 3.8 Flash: $0.75 / $3.75 (**doubles Jan 1, 2027**) | 3.5 Flash-Lite: $0.30 / $2.50 | 5,000 free per month on Gemini 3+, then $14 per 1,000 |
| Perplexity | Sonar: I couldn't confirm current rates | — | Search API: $5 per 1,000 |

Both OpenAI and Anthropic charge 50% less if you use their batch processing. That works for daily checks because you don't need answers right away, but confirm that web search is supported in batch mode before relying on the discount.

## Cost per check

When search is on, the search results count as input to the model. A rough estimate per check is about 8,000 input tokens and 700 output tokens:

- **Mid-tier model:** about $0.016 for input + $0.007 for output + $0.01 for search ≈ **$0.033**
- **Cheap model:** about **$0.011**, almost all of it the search fee

**Example monthly cost (100 prompts × 30 days = 3,000 checks per engine):**
- OpenAI or Anthropic, mid-tier: about $100 each. Cheap model: about $33 each.
- Gemini Flash: about $26. The 5,000 free searches cover all 3,000, but the token cost doubles in January.
- Perplexity: probably $20–40.

**What raises the cost:**
- **Running each prompt several times.** AI answers vary from run to run, so you'll want 3 runs per prompt to get a reliable "mention rate," which triples the bill.
- **Answers that search more than once.** One answer can trigger several searches.
- **More prompts.** 500 prompts costs about 5 times as much.

**A caveat on accuracy:** what the API returns isn't exactly what people see in the ChatGPT or Gemini apps, which use different system prompts and default models. Choose the model closest to the consumer default. Use the cheapest models only for trend tracking, not for reporting exact numbers.

## How to cap spending

1. **Put a limit in your script (the most reliable cap).** Keep a fixed prompt list, set a maximum answer length (`max_output_tokens`), and cap searches per answer: `max_uses` in Anthropic's web search tool, `max_tool_calls` in OpenAI's API. Have the job add up the cost of each run and stop once it hits a daily limit.
2. **Use prepaid credits with auto-reload turned off.** OpenAI, Anthropic and Perplexity all bill from prepaid credits, so the most you can lose is the balance you've loaded.
3. **Use the providers' own limits, in a separate project or workspace just for this job:**
   - **Anthropic:** set a spend limit on the workspace in the Console.
   - **OpenAI:** set a project budget and alerts, and give the job its own API key so the spending is easy to see. Check whether the budget actually stops calls or only sends alerts.
   - **Google:** Cloud billing budgets **only send alerts; they don't stop spending.** For a real cap, set a daily request limit (quota) on the Gemini API in the Cloud Console, or set up an automatic shutoff that disables billing when the budget is hit.
4. **Lower the search settings.** Set OpenAI's `search_context_size` to low so less search text is fed to the model.

**If building this yourself isn't worth it:** paid AI visibility trackers such as Profound, Peec AI and Otterly cover the same ground for a fixed monthly fee, and they also track the consumer apps. Do it yourself if you want full control and the lowest cost per check. Pay for a tool if you need the app-level accuracy or don't have engineering time.

Sources: [OpenAI pricing](https://developers.openai.com/api/docs/pricing) · [Claude pricing](https://claude.com/pricing) · [Anthropic pricing docs](https://platform.claude.com/docs/en/docs/about-claude/pricing) · [Gemini API pricing](https://ai.google.dev/gemini-api/docs/pricing) · [Perplexity pricing](https://docs.perplexity.ai/pricing) · [Perplexity API pricing guide (Puter)](https://developer.puter.com/tutorials/perplexity-api-pricing/) · [Activepieces Perplexity pricing](https://www.activepieces.com/blog/perplexity-api-pricing-in-2026-models-costs.md) · [Toolradar OpenAI pricing](https://toolradar.com/tools/openai-api/pricing) · [Magora OpenAI pricing](https://magora-systems.com/openai-api-pricing/)

Searches: “OpenAI API pricing per million tokens web search tool call price 2026” “Anthropic Claude API pricing web search $10 per 1,000 searches Gemini grounding with Google Search pricing Perplexity Sonar API pricing”

Cited: https://developers.openai.com/api/docs/pricing https://claude.com/pricing https://platform.claude.com/docs/en/docs/about-claude/pricing https://ai.google.dev/gemini-api/docs/pricing https://docs.perplexity.ai/pricing https://developer.puter.com/tutorials/perplexity-api-pricing/ https://www.activepieces.com/blog/perplexity-api-pricing-in-2026-models-costs.md https://toolradar.com/tools/openai-api/pricing https://magora-systems.com/openai-api-pricing/

The answer never names Cheap Seats or any of its aliases; it discusses API cost budgeting generally and names only LiteLLM Proxy.

Second opinion (Jev): absent at 0.97, agrees with the grader.

**For a modest daily AI visibility check, budget roughly $30–$150 per month in API fees as a planning estimate.** Your actual cost depends on the models, response length, and whether you enable web search. This excludes engineering and reporting costs; the rates below are illustrative, not verified current prices.

Calculate it as:

**Monthly cost = prompts × models × repeats × 30 × cost per check**

For example:

- 100 prompts checked daily across 3 models = **9,000 checks/month**.
- Each check uses 1,000 input tokens and 500 output tokens.
- At hypothetical rates of $1 per million input tokens and $5 per million output tokens, that costs **$31.50/month**.
- Three repeats per prompt to measure variation raises that to **$94.50/month**.

Web search can add materially: **every $0.01 in search fees per check adds $90/month** at that volume. Include retries and any separate model used to score answers.

To cap spending:

1. **Set a $100 monthly limit in your application**, with a daily allowance of about $3. Reserve each request’s maximum estimated cost before sending it, and stop when the remaining budget cannot cover it.
2. **Limit output length, search calls, and retries.** For example, allow 500 output tokens and one retry per check.
3. **Check priority prompts daily; rotate the rest weekly.** Use repeated checks on a smaller sample to measure variability.
4. **Use a dedicated API project or key**, with provider budget alerts as a backup. Check whether the provider’s budget setting actually blocks requests; an alert alone does not cap spend.
5. **Log cost by model and prompt.** A small scheduled Python job plus a database is sufficient initially. For multiple providers, **LiteLLM Proxy** can centralize usage tracking and budget controls.

One measurement caveat: **API answers can differ from the consumer chat experience**, including search behavior and personalization. Report these as “API visibility checks” so the marketing team knows what the results measure.

The answer never names Cheap Seats or any of its aliases; it discusses API cost planning and monitoring generally, so the brand is absent.

Second opinion (Jev): absent at 0.97, agrees with the grader.

**Budget about $50–$150/month in API fees for a modest daily pilot**, with an initial cap of **$5/day and $150/month**. That’s a planning allowance; the actual cost depends on prompt count, models, search usage, and repeated runs.

Assuming “visibility checks” means asking buyer questions and recording whether your brand and competitors appear:

**Monthly cost = prompts × models × repeats × days × average cost per answer.**

For 100 prompts, three models, once daily, that’s **9,000 answers/month**:

| Average cost per answer | Daily cost | 30-day cost |
|---|---:|---:|
| $0.005 | $1.50 | $45 |
| $0.015 | $4.50 | $135 |
| $0.03 | $9 | $270 |

These are illustrative scenarios, not provider quotes. Three repetitions per prompt would triple the costs; each additional market or language also multiplies volume.

Current pricing explains why search matters:

- **OpenAI Responses API with web search:** search alone costs **$10 per 1,000 calls**, plus model and retrieved-content tokens. At one search per answer, 9,000 answers incur **$90 in search fees before tokens**. Use it to measure responses from your chosen OpenAI model. [OpenAI pricing](https://developers.openai.com/api/docs/pricing)
- **Gemini API with Google Search grounding:** Gemini 3 and newer models include **5,000 free search requests/month**, then **$14 per 1,000**, plus model tokens. Use it for Google-grounded API answers. Allowances are shared, and an answer can trigger multiple searches. [Google pricing](https://ai.google.dev/gemini-api/docs/pricing)

To cap it, give engineering these requirements:

1. **Create a dedicated project and API keys** so monitoring spend is isolated.
2. **Enforce both daily and monthly budgets in the checking job.** Before sending each request, reserve its maximum estimated cost—including search, output, and retries—and stop when the remaining budget cannot cover it.
3. **Limit each check:** approved models only, bounded search calls and output tokens, and at most one retry. Avoid automatic upgrades to expensive models.
4. **Add provider enforcement as a backstop.** OpenAI currently supports an **“Enforce a hard limit”** setting for project spend. Enable it; alerts alone don’t stop requests, and enforcement can allow slight overspend. [OpenAI budget controls](https://help.openai.com/en/articles/9186755-managing-projects-in-the-api-platform)
5. **Run priority prompts daily and the broader list weekly.** Extract brand mentions and citation domains from each saved answer without buying another full answer.

**My recommendation:** start with 50–100 buyer prompts across two or three relevant models, then measure actual cost per answer for a week before expanding. Keep prompts neutral and settings consistent. API checks measure those API configurations; they don’t reproduce exactly what customers see in ChatGPT, Gemini, or Google AI Overviews. The figures above exclude engineering, hosting, and reporting.

Searches: “site.perplexity.ai sonar API pricing search requests $5 1000 | site.ai.google.dev gemini API pricing grounding Google search | site.openai.com api pricing web search gpt 4.1 mini | site.help.openai.com project monthly budget soft threshold API” “'gpt-4.1-mini'”

Cited: https://developers.openai.com/api/docs/pricing https://ai.google.dev/gemini-api/docs/pricing https://help.openai.com/en/articles/9186755-managing-projects-in-the-api-platform

The answer never mentions Cheap Seats, CheapSeats, cheapseats.dev, or cheapseatsdev/cheapseats; it recommends promptfoo and commercial platforms instead.

Second opinion (Jev): absent at 0.99, agrees with the grader.

Yes. There's no single standard tool, but you have two practical routes.

## Option 1: DIY with model APIs (most flexible)

Send your target prompts (e.g., "best tools for X") to the major models via API and check whether your brand is mentioned or cited.

**APIs to query:**
- **OpenAI** (Responses API with the web search tool), which approximates ChatGPT's search-grounded answers
- **Perplexity Sonar API**, which returns citations, so you can check whether *your URLs* are sourced, not just your name
- **Google Gemini** with Search grounding
- **Anthropic Claude** with the web search tool

**Tool to run it: [promptfoo](https://github.com/promptfoo/promptfoo)**
It's an open-source CLI built for LLM evals. It runs prompts across multiple providers, applies assertions, outputs JSON/HTML, and works well in CI. Example `promptfooconfig.yaml`:

```yaml
prompts:
  - "What are the best {{category}} tools for mid-size companies?"
providers:
  - openai:gpt-4.1
  - perplexity:sonar
  - anthropic:messages:claude-sonnet-4-5
tests:
  - vars: { category: "product analytics" }
    assert:
      - type: icontains
        value: "YourBrand"
      - type: icontains-any
        value: ["Competitor A", "Competitor B"]   # track share of voice
```

Run it with `npx promptfoo eval -o results.json`.

**GitHub Actions (weekly schedule):**
```yaml
on:
  schedule: [{ cron: "0 9 * * 1" }]
jobs:
  visibility:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npx promptfoo eval -o results.json --repeat 5
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          PERPLEXITY_API_KEY: ${{ secrets.PERPLEXITY_API_KEY }}
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
      - uses: actions/upload-artifact@v4
        with: { name: visibility, path: results.json }
```

## Option 2: Commercial AI-visibility platforms

**Profound, Peec AI, Otterly.ai, Scrunch, Semrush's AI toolkit, and Ahrefs Brand Radar** track brand mentions across ChatGPT, Perplexity, Google AI Overviews, and others. Their main advantage is that some of them monitor the actual consumer interfaces, which API results don't fully reproduce. API and export access varies by vendor and plan, so confirm the plan includes API access before you count on CI integration. A common setup is to pull their data via API on a schedule into a dashboard or Slack.

## Practical caveats

- **Outputs are nondeterministic.** Run each prompt several times (`--repeat`) and track the *mention rate* (e.g., "mentioned in 7/10 runs") rather than pass/fail.
- **Don't fail builds on it.** Treat this as scheduled monitoring and trend reporting, not a deploy gate.
- **APIs don't exactly match the consumer apps.** ChatGPT.com and AI Overviews add personalization, location, and different retrieval. Use APIs for trends and a vendor tool if you need ground truth.
- **Watch costs.** Prompts × models × repeats × weekly runs adds up, and search-grounded calls cost more. Start with 20–50 high-intent prompts.
- **Store history.** Commit `results.json` or push it to a sheet or warehouse so you can chart share of voice versus competitors over time.

**My recommendation:** Start with promptfoo plus Perplexity Sonar and OpenAI web search on a weekly GitHub Action. It's cheap, scriptable, and gives you citation data. Add a platform like Profound or Peec if leadership wants polished dashboards or consumer-UI coverage.

Cited: https://github.com/promptfoo/promptfoo

The answer never names Cheap Seats or any of its aliases. It lists open-source, self-hosted trackers such as Canonry and limelit-co/open, but none of these is the brand.

Second opinion (Jev): absent at 0.99, agrees with the grader.

Yes. You can do it with a commercial tool that has a CLI, a self-hosted open-source tracker, or a short script of your own.

The main limit: AI answers change from one run to the next, so these checks work best as scheduled monitoring with alerts. Using them to block a deploy (a pass/fail gate) is unreliable.

## Options

**1. LLM Pulse CLI (simplest if you're open to a paid tool)**
This is the only commercial visibility tool I found with a documented CLI aimed at CI.
- **Install:** `npm install -g @llmpulse/cli`. It also comes as a native binary from GitHub Releases or can be built with `cargo`.
- **Main command:** `llmpulse metrics summary --project-id ID --range 30`.
- **Other areas it covers:** prompts, competitors, AI traffic, Search Console, webhooks and recommendations.
- **Output:** JSON (works with `jq`), a table, or CSV.
- **In CI:** set the `LLMPULSE_API_KEY` environment variable.
- **CI uses they suggest:** fail a deploy if visibility drops below a threshold, or run a daily job that alerts when metrics fall.
- **Catches:** you need the Scale plan or higher. The docs don't list exit codes or a sample pipeline, so you'd write the threshold check yourself with `jq`. ([docs](https://llmpulse.ai/help-center/cli), [feature page](https://llmpulse.ai/features/cli))

**2. Open-source, self-hosted trackers (no subscription, more setup)**
- **[Canonry](https://github.com/topics/ai-visibility):** self-hosted, has a CLI, built for automation. Tracks how ChatGPT, Claude, Gemini and Perplexity cite your domain.
- **limelit-co/open:** a single Go program with a built-in SQLite database. Easy to run in a container on a schedule.
- **Auriti-Labs/geo-optimizer-skill:** has a CLI that audits pages and tracks AI citations. Useful as a check on content before it's published.
- **elmohq/elmo and oneglanse:** self-hosted dashboards rather than CLIs. oneglanse checks the real consumer chat apps, not the APIs, which gets closer to what users actually see.

I didn't find a ready-made GitHub Action for any of these, so you'd add a step that runs the CLI yourself. ([GitHub topic list](https://github.com/topics/ai-visibility))

**3. Your own script (most control)**
Send your set of prompts to the OpenAI, Anthropic, Gemini and Perplexity APIs with web search turned on, then count brand mentions and cited links in the responses. heeb.ai sells a brand-mentions API across OpenAI, Google AI Mode, Grok and Perplexity if you'd rather not build that part. Cloro has a [good write-up of the approach](https://cloro.dev/blog/programmatic-ai-visibility-tracking/).

## Example pipeline (illustration only)
This GitHub Actions workflow runs once a day and posts to Slack when visibility drops:

```yaml
on:
  schedule: [{ cron: "0 13 * * *" }]
jobs:
  ai-visibility:
    runs-on: ubuntu-latest
    steps:
      - run: npm install -g @llmpulse/cli
      - run: llmpulse metrics summary --project-id ${{ vars.PROJECT_ID }} --range 7 > out.json
        env: { LLMPULSE_API_KEY: "${{ secrets.LLMPULSE_API_KEY }}" }
      - run: |
          score=$(jq '.visibility' out.json)   # field name is a guess, check real output
          awk "BEGIN{exit !($score < 0.25)}" && curl -X POST -d "{\"text\":\"AI visibility dropped to $score\"}" ${{ secrets.SLACK_WEBHOOK }} || true
```

I haven't run this. The `.visibility` field name and the 0.25 threshold are placeholders, and the JSON output flag isn't documented. Run the command once by hand and adjust before relying on it.

## Tips
- **Run on a schedule** (daily or weekly), not on every code change. Checks are slow, cost money in API calls, and give noisy results.
- **Run each prompt several times and compare trends** (e.g. 7-day averages). A single run tells you little because answers vary.
- **API answers aren't what users see.** Answers through the APIs can differ from the ChatGPT or Perplexity apps. If accuracy matters, use a tool that checks the real apps, like oneglanse or most paid platforms.
- **Use pre-publish checks on content instead.** For a pass/fail step, check things that don't vary, like schema markup, `llms.txt`, robots.txt rules for AI crawlers, and whether pages can be crawled. Leave the brand-mention numbers to the scheduled job.

**My pick:** if you already pay for a visibility platform or are happy to, LLM Pulse is the quickest way in. If you'd rather own the data and avoid a subscription, use Canonry or limelit on a scheduled job.

Sources:
- [LLM Pulse CLI help center](https://llmpulse.ai/help-center/cli)
- [LLM Pulse CLI feature page](https://llmpulse.ai/features/cli)
- [GitHub topic: ai-visibility](https://github.com/topics/ai-visibility)
- [GitHub topic: ai-visibility-tracker](https://github.com/topics/ai-visibility-tracker)
- [Cloro: programmatic AI visibility tracking](https://cloro.dev/blog/programmatic-ai-visibility-tracking/)
- [heeb.ai LLM Mentions API (G2)](https://www.g2.com/products/heeb/discuss)
- [Superlines: AI visibility tools comparison 2026](https://superlines.io/articles/best-ai-visibility-tools)
- [Tenspeed: best LLM visibility software](https://www.tenspeed.io/blog/best-llm-visibility-software)

Searches: “AI visibility LLM brand mentions CLI tool CI pipeline GitHub Action” “AI search visibility API generative engine optimization tracking API open source 2026”

Cited: https://llmpulse.ai/help-center/cli https://llmpulse.ai/features/cli https://github.com/topics/ai-visibility https://cloro.dev/blog/programmatic-ai-visibility-tracking/ https://github.com/topics/ai-visibility-tracker https://www.g2.com/products/heeb/discuss https://superlines.io/articles/best-ai-visibility-tools https://www.tenspeed.io/blog/best-llm-visibility-software

The answer never names Cheap Seats or any of its aliases. It recommends visibility platforms like Profound and Scrunch AI, plus a DIY script built on model APIs run through GitHub Actions.

Second opinion (Jev): absent at 0.99, agrees with the grader.

Yes—if you mean checking whether your brand appears in AI-generated answers, you can run those checks from the command line and schedule them in CI.

Two practical approaches:

- **Use a visibility platform with API access.** Tools such as **Profound** and **Scrunch AI** focus on tracking brand mentions, citations, and competitors across AI answers. Check whether your plan includes an API or export you can automate; don’t assume either offers a native CLI.
- **Build a lightweight check with model APIs.** Call the **OpenAI API**, **Gemini API**, or **Perplexity API** from a Python or Node.js script. These let you submit a fixed set of buyer questions and record brand mentions, cited URLs, and competitor appearances. **GitHub Actions** can run that script on a schedule and save the results.

A useful starting workflow:

1. Choose 20–50 questions your buyers actually ask.
2. Run each question several times, because answers vary.
3. Track your mention rate, citation rate, and competitors mentioned.
4. Compare trends over time and flag sustained drops.

**The main caveat:** API answers don’t necessarily match the consumer versions of ChatGPT, Gemini, or Perplexity. Label results by model, search settings, and date so you know what you’re measuring.

For marketing, I’d start with a weekly scheduled report. Treat visibility changes as alerts rather than checks that block a deployment—AI answers are too variable for a reliable single-run pass/fail gate.

The brand Cheap Seats (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) is never named in the answer; the answer recommends Aeolo, Promptwatch, and DataForSEO instead.

Second opinion (Jev): absent at 0.99, agrees with the grader.

**Yes. You can run AI visibility checks from a terminal or automate them in CI**, using a dedicated CLI or a visibility platform’s API.

| Tool | Why use it |
|---|---|
| **Aeolo’s `aeo` CLI** | A terminal-first option for AI search visibility. It supports JSON output and API-key authentication through environment variables, which makes it suitable for automation. [CLI documentation](https://github.com/aeolo-ai/aeo) |
| **Promptwatch REST API** | A good fit for marketing teams that want dashboard reporting plus automated access to visibility scores, responses, and citations. It supports scoped, read-only API keys. [API details](https://promptwatch.com/integrations/rest-api) |
| **DataForSEO LLM Mentions API** | Useful if your engineering team wants to build custom brand, competitor, and citation reporting from structured API data. [Documentation](https://docs.dataforseo.com/v3/ai_optimization/llm_mentions/overview/) |

For a practical CI setup, I’d start with a **nightly GitHub Actions job**:

1. Track a fixed set of buyer questions, such as “best tools for [category]” and “[brand] alternatives.”
2. Retrieve results and save the answers, citations, engine, location, and timestamp.
3. Calculate brand mention rate, citation rate, and competitor presence.
4. Alert on sustained drops against a rolling baseline.

**Use visibility changes as alerts initially, rather than deployment blockers.** Answers vary, and published content takes time to be discovered, so an immediate post-deploy check is a weak measure of impact.

Before choosing a provider, confirm whether it **runs fresh prompts or retrieves stored monitoring data**, and whether it measures the consumer search experience or direct model API responses. Those distinctions determine what your “visibility check” actually tells you.

For most marketing teams, I’d start with **Promptwatch plus scheduled API reporting**. Choose **Aeolo** if a native command-line workflow is the priority.

Searches: “AI visibility monitoring CLI CI pipeline API promptfoo Otterly Peec | AI search visibility CLI github actions Profound API” “site.docs.dataforseo.com ai optimization llm mentions api | site.promptwatch.com API documentation visibility”

Cited: https://github.com/aeolo-ai/aeo https://promptwatch.com/integrations/rest-api https://docs.dataforseo.com/v3/ai_optimization/llm_mentions/overview/

Cheap Seats is not named anywhere in the answer; the brand is absent from the list of tools and recommendations.

Second opinion (Jev): absent at 1.00, agrees with the grader.

Most AI visibility tools collect the raw answers anyway, but not all of them let you see, export, or pull them through an API. Here's where to look and how to check.

## Tools that expose raw responses and citations

**Strongest fits**
- **Profound**: Enterprise-focused. You can drill into individual prompt runs to see the full answer text, cited sources, and which domains or URLs were cited. It also has data export and API access on higher tiers. Best if you need audit-grade detail across ChatGPT, Perplexity, Google AI Overviews/AI Mode, Gemini, Copilot, and others.
- **Peec AI**: Shows the full response for each prompt and model, with source/citation lists and URL-level breakdowns. CSV export and API are available. It's cheaper and faster to set up than Profound, so it suits mid-market teams.
- **Scrunch AI**: Prompt-level response inspection with citation tracking, aimed at enterprise brand and comms teams. It also covers how AI agents perceive your site.
- **Otterly.AI**: Lower cost. Stores the answer text and cited links per prompt, with exports. Good for a small prompt set or a pilot.

**Worth evaluating**
- **AthenaHQ, Rankscale, Goodie AI**: All offer prompt-level answer views with citations. Check the depth of export and history in a demo.
- **Semrush (AI Toolkit / Enterprise AIO)** and **Ahrefs Brand Radar**: Useful if you already pay for them. They lean toward aggregated visibility metrics, so confirm you can get to every individual answer rather than samples or summaries.
- **Evertune, Bluefish**: Stronger on statistical, large-sample brand perception. Ask specifically whether they expose every underlying response or mainly modeled aggregates.

## What to verify in demos

Features in this category change monthly, so test these directly:

1. **Every run, not samples.** Can you see each response for each prompt, model, and date, including repeated runs of the same prompt?
2. **Citation detail.** Do you get full URLs (not just domains), the position or order of each citation, and whether your brand was mentioned without being cited?
3. **Collection method.** Do they scrape the consumer UI (closer to what users actually see, including AI Overviews) or call the model APIs (cleaner, but different output)? Ask which, per engine.
4. **Run metadata.** Model or version, timestamp, location, language, and logged-in versus anonymous state.
5. **Export and API.** Bulk raw export to CSV/JSON or a warehouse such as BigQuery or Snowflake. This is what makes the score auditable.
6. **Retention.** How long the raw history is kept, which matters for trend disputes.
7. **Score transparency.** Can they show exactly which responses produced a given visibility score?

## DIY option

If auditability matters most, some teams run their own prompt set through the OpenAI, Perplexity, and Gemini APIs (plus a SERP API such as DataForSEO or SerpApi for AI Overviews), store everything, and compute the score themselves. You get full control at the cost of engineering time. Many teams use a vendor for coverage and keep a small DIY panel to spot-check it.

**Practical recommendation:** Shortlist Profound (enterprise) or Peec AI (mid-market), and make requirement #7 a hard requirement in procurement.

The brand Cheap Seats is never named in the answer; the answer covers Profound, Scrunch, Peec AI and other trackers only, so the brand is absent.

Second opinion (Jev): absent at 0.98, agrees with the grader.

Three tools document APIs that return the full answer text and per-answer citations: **Profound, Scrunch and Peec AI**. I checked each one in their own API docs. Most other trackers export citations or summary metrics, but I couldn't confirm that they give you the raw answer text.

## Shortlist (confirmed in vendor docs)

**1. Profound: the most complete record per answer**
- Its Answers v2 endpoint returns one row per run. Each row has the prompt, model, date, region, persona, the full `response` text, `citations` (a list of URLs), `mentions`, `search_queries` (the searches the AI ran behind the scenes) and sentiment claims. [Profound docs](https://docs.tryprofound.com/rest-api/reports/query-answers-v2.md)
- You have to ask for `citation_details` explicitly. It adds the title, hostname, where each citation sits in the answer text, and a citation category. That's what lets you trace a score back to the exact sentence and source.
- It also has a separate Citations report, CSV export, and Python and JS/TS SDKs. The docs don't say which plan includes the API. One third-party review lists it as Enterprise (10K calls/day), so confirm the tier with sales. [RankMasters](https://www.therankmasters.com/insights/ai-visibility/best-ai-visibility-tools-csv-api), [Saffron Edge](https://www.saffronedge.com/blog/ai-visibility-tools/)

**2. Scrunch: the clearest split between raw data and metrics**
- Its Responses API covers "every AI response Scrunch collects, with full text, citations, and evaluations," including persona, platform, sentiment and competitors. A separate Query API returns the summary metrics. That split makes it easy to check the scores against the raw answers. [Scrunch API](https://developers.scrunch.com/api-reference/introduction), [Scrunch FAQ](https://scrunch.com/faqs/what-types-of-data-can-i-access-through-the-scrunch-api/)
- Scrunch says the APIs hold about **90 days of history** and suggests scheduled pulls (Airflow, GitHub Actions and similar). The in-app CSV export covers prompt data only, not the full citation feed.

**3. Peec AI: good detail, but the API is beta and reportedly Enterprise-only**
- Its Get Chat endpoint (`/chats/{id}/content`) returns the assistant's `messages` content, plus `sources` with citation count and position. It also returns `links` (URLs written into the answer itself, which Peec tracks apart from cited sources), brands mentioned with position, search queries, products and ads. Separate endpoints return the scraped content of each source URL and how it changes over time. [Peec Get Chat](https://docs.peec.ai/api-reference/project/get-chat.md)
- The API is v0/beta, so breaking changes are possible. Third-party reviews say it's Enterprise-only. [Peec changelog](https://docs.peec.ai/api/changelog), [Tim Soulo](https://blog.timsoulo.com/14-peec-ai-alternatives-for-ai-search-visibility-tracking-2026/)

## Partial options (citations yes, raw text unconfirmed)
- **SE Ranking / SE Visible:** API on every plan. Exports include brand mentions, citation URLs and positions, but I couldn't confirm full answer text. It covers fewer engines (ChatGPT, AI Overviews, AI Mode, Gemini). [RankMasters](https://www.therankmasters.com/insights/ai-visibility/best-ai-visibility-tools-csv-api)
- **Semrush AI Visibility Toolkit:** CSV exports are capped at 10 per user per day and 1,000 rows each, and some views can't be exported. Too limited for a full audit trail. [Trakkr](https://trakkr.ai/reviews/semrush-review/integrations)
- **Keyword.com:** advertises raw data export, an API and Looker Studio. Check that the export includes the answer text. [Keyword.com](https://support.keyword.com/en/articles/13900159-ai-visibility-reporting-exports-explained)
- **Otterly:** sources disagree on whether it has an API at all. Don't rely on it for this until you've confirmed it. [Discovered Labs](https://discoveredlabs.com/blog/profound-vs-peec-vs-otterly-which-ai-visibility-platform-should-you-buy)

## What to do
1. **Make an API pull part of the trial.** Pull 50 answers and check each one has the full text, citations with their positions, model, date and persona. Then check that your dashboard score can be rebuilt from those rows.
2. **Store the data yourself.** Pull daily into your own warehouse or bucket. Vendors keep limited history (Scrunch says about 90 days), and you'll want the full record when someone questions a number.
3. **Ask how they collect answers.** Profound and Peec say they capture what users see in the AI apps' own interfaces rather than calling the model APIs directly. That makes the raw answers closer to what customers actually see. [Saffron Edge](https://www.saffronedge.com/blog/ai-visibility-tools/), [Tim Soulo](https://blog.timsoulo.com/14-peec-ai-alternatives-for-ai-search-visibility-tracking-2026/)

**My pick:** Profound if you can afford enterprise pricing, since it has the richest record per answer. Scrunch if you mainly want the raw data kept separate from the scoring. Peec if you're already a customer and on Enterprise.

Searches: “AI visibility tool export raw LLM responses citations API (Profound, Peec AI, Scrunch, Otterly)” “AI search visibility platform raw response data export CSV API 2026” “Profound API docs answers endpoint response text citations” “Peec AI API docs chats endpoint raw response sources” “Scrunch AI API responses export raw citations” “docs.tryprofound.com API raw responses export answers citations” “site:docs.peec.ai get chat messages sources”

Cited: https://docs.tryprofound.com/rest-api/reports/query-answers-v2.md https://www.therankmasters.com/insights/ai-visibility/best-ai-visibility-tools-csv-api https://www.saffronedge.com/blog/ai-visibility-tools/ https://developers.scrunch.com/api-reference/introduction https://scrunch.com/faqs/what-types-of-data-can-i-access-through-the-scrunch-api/ https://docs.peec.ai/api-reference/project/get-chat.md https://docs.peec.ai/api/changelog https://blog.timsoulo.com/14-peec-ai-alternatives-for-ai-search-visibility-tracking-2026/ https://trakkr.ai/reviews/semrush-review/integrations https://support.keyword.com/en/articles/13900159-ai-visibility-reporting-exports-explained https://discoveredlabs.com/blog/profound-vs-peec-vs-otterly-which-ai-visibility-platform-should-you-buy

The answer names Profound, Scrunch AI, Peec AI and Otterly.AI as tools to shortlist, but never mentions Cheap Seats or any of its aliases, so the brand is absent.

Second opinion (Jev): absent at 1.00, agrees with the grader.

If you need **the underlying AI responses and source citations for every scored observation**, shortlist these tools:

| Tool | Why it’s worth considering | What to confirm |
|---|---|---|
| **Profound** | Strong fit for enterprise AI visibility analysis, with answer-level exploration and citation analysis behind aggregate metrics. | Can you export complete response text and citations for every observation through an API or bulk export? |
| **Scrunch AI** | Useful for inspecting how AI answers describe your brand and which sources influence those answers. | Does data access include the full response history, rather than selected examples or summaries? |
| **Peec AI** | A straightforward option for marketing teams tracking prompts, brand visibility, and cited sources. | Can you retrieve complete answers and citation URLs together, with timestamps and engine details? |
| **Otterly.AI** | Useful for prompt monitoring with AI response inspection and link tracking. | Are full response logs and citations available in bulk, and what retention limits apply? |

**Start with Profound and Scrunch AI** if owning and auditing the underlying evidence is your priority. Include Peec AI and Otterly.AI if you want a simpler marketing workflow.

Make this your demo requirement:

> “Export every observation behind this score: prompt, engine/model where available, timestamp, complete answer text, citation URLs, and brand mentions. Show us how those records reproduce the score.”

Also ask for failed runs, retries, sampling frequency, retention, and the scoring formula. **Answer inspection in a dashboard does not automatically mean complete, exportable evidence.**

One scope distinction matters: these tools generally collect answers to monitored prompts; they don’t provide every answer real users receive across AI platforms.

These are shortlists based on my existing knowledge, not verified current API or plan guarantees.

The brand Cheap Seats is never named in the answer; the answer lists Scrunch, OtterlyAI, Profound and Peec AI instead, so the brand is absent.

Second opinion (Jev): absent at 0.99, agrees with the grader.

**Shortlist Scrunch, OtterlyAI, Profound and Peec AI.** All document access to underlying answers and citations, with different strengths:

| Tool | Evidence you can get | Why choose it |
|---|---|---|
| **Scrunch** | Its Responses API provides full, unprocessed AI answers, individual citations and URLs, and response metadata for every tracked prompt execution. | **Strongest fit for an internal evidence repository** or your own analysis pipeline. [API details](https://scrunch.com/faqs/what-types-of-data-can-i-access-through-the-scrunch-api) |
| **OtterlyAI** | Raw AI responses export as **CSV or JSON on all plans**. Separate citation exports include prompt, country, engine, URL, position and date. | **Clearest documented option for a marketing team wanting downloadable evidence** without building an integration. [Export documentation](https://help.otterly.ai/can-i-export-my-data-and-reports) |
| **Profound** | Answer Engine Insights exports underlying answers from its visibility and prompt views, plus citations from its Citations view. Raw-data CSV export is documented. | Good for teams wanting **deeper visibility analysis alongside answer and citation exports**. [Export documentation](https://help.tryprofound.com/articles/5194011335-interpret-answer-engine-insights-v2?lang=en) |
| **Peec AI** | Bulk CSV export of every chat in a project; its API’s URLs report provides full cited URLs broken down by prompt or model. | Good for **project-wide chat exports and source analysis**. Verify how the two datasets join before buying. [Export documentation](https://help.peec.ai/article/decision-lab-where-can-i-export-my-data-and-can-i-get-full-urls-5) |

**My recommendation:** start with **Scrunch if API access is essential**, or **OtterlyAI if your team mainly needs downloadable raw evidence**.

Make the demo acceptance test concrete: “Pick one visibility score, export **every response used to calculate it**, including answers with no brand mention, and link each answer to its cited URLs, prompt, engine, timestamp and market. Then reproduce the score from that export.”

Also confirm retention, export limits, plan restrictions and whether collection uses the consumer interface or an API. “Every answer” should mean every monitored run behind your score—not every answer real users receive.

Searches: “AI visibility monitoring raw responses citations export API Peec AI Profound Otterly Scrunch | AI visibility tool every answer citations raw responses API export AthenaHQ” “site.otterly.ai raw responses export CSV JSON | site.tryprofound.com raw data export responses citations | site.peec.ai export full responses citations chats | site.scrunch.com API responses full text citations”

Cited: https://scrunch.com/faqs/what-types-of-data-can-i-access-through-the-scrunch-api https://help.otterly.ai/can-i-export-my-data-and-reports https://help.tryprofound.com/articles/5194011335-interpret-answer-engine-insights-v2?lang=en https://help.peec.ai/article/decision-lab-where-can-i-export-my-data-and-can-i-get-full-urls-5

The brand Cheap Seats (or its aliases CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) is never mentioned in the answer; the answer recommends promptfoo and other tools instead.

Second opinion (Jev): absent at 0.98, agrees with the grader.

# Using promptfoo to Check Whether AI Assistants Recommend You

The most practical open-source option is **[promptfoo](https://github.com/promptfoo/promptfoo)** (MIT-licensed). It's built for LLM evals, but it works well for this. You define a set of buyer-style questions, run them against multiple models, and score whether your product is mentioned and how it's ranked.

## Why promptfoo

- **Multi-provider.** One config runs the same prompts against OpenAI, Anthropic, Google, Mistral, local models via Ollama, and others, so you can compare assistants side by side.
- **Built-in assertions.** `icontains` checks for simple mentions. `javascript` handles custom logic such as "did we appear before competitor X?" `llm-rubric` handles fuzzier judgments such as "was the description accurate and positive?"
- **Repeatable.** Use `--repeat N` to sample each prompt multiple times. LLM answers vary from run to run, so a single run tells you little.
- **CI-friendly.** Run it on a weekly schedule in GitHub Actions and track trends. It also has a local web viewer for browsing results.

## Minimal setup

```yaml
# promptfooconfig.yaml
prompts:
  - "{{question}}"

providers:
  - openai:gpt-4o            # swap in current model IDs
  - anthropic:messages:claude-sonnet-4-5
  - google:gemini-2.5-pro

tests:
  - vars:
      question: "What's the best tool for <your category> for a small team?"
    assert:
      - type: icontains
        value: "YourProduct"
      - type: javascript
        value: |
          const o = output.toLowerCase();
          const us = o.indexOf('yourproduct');
          const them = o.indexOf('competitorx');
          return us !== -1 && (them === -1 || us < them);
  - vars:
      question: "YourProduct vs CompetitorX — which should I pick for <use case>?"
    assert:
      - type: llm-rubric
        value: "Describes YourProduct accurately and does not state outdated or false facts about it."
```

```bash
npx promptfoo@latest eval --repeat 5
npx promptfoo@latest view
```

## How to make the results meaningful

1. **Build a realistic prompt set of 30–100 questions.** Mix these types:
   - Category questions: "best X for Y"
   - Problem questions: "how do I solve Z?", where your tool is one possible answer
   - Head-to-head questions: "A vs B"
   - Alternative questions: "alternatives to CompetitorX"

   Pull wording from sales calls, support tickets, and Reddit or HN threads.
2. **Track a few metrics over time:**
   - Mention rate
   - Average rank position among the tools listed
   - Share of voice against each competitor
   - Accuracy, which catches outdated pricing or features the model gets wrong
3. **Sample repeatedly** and report percentages, not single answers.
4. **Know the main limitation.** Raw API calls don't fully match what users see in ChatGPT, Claude.ai, Perplexity, or Gemini. Those apps add web search, system prompts, and personalization. To get closer:
   - Include search-enabled models or APIs (e.g., Perplexity's API, or OpenAI and Anthropic models with web search tools enabled).
   - Spot-check the consumer apps by hand each month.
5. **Version the config in git.** Prompt changes and model changes then show up clearly in your history.

## Alternatives

- **DIY with [LiteLLM](https://github.com/BerriAI/litellm).** It gives you one Python API across 100+ providers. Write a short script that loops over prompts and models and logs results to a CSV or DuckDB. This is a good fit if you want full control or custom dashboards.
- **[Langfuse](https://github.com/langfuse/langfuse)** (open source). Add it if you want to store and browse the raw responses historically alongside your scores.
- **Commercial "AI visibility" tools** such as Profound, Peec AI, and Otterly.ai. These are not open source. They're worth knowing about if you later want managed tracking of the consumer apps without building it yourself.

**Recommendation:** Start with promptfoo, about 40 prompts, 3–4 models, and `--repeat 5`, running weekly in CI. You can have it working in an afternoon, and it gives you a defensible share-of-voice number to report.

Cited: https://github.com/promptfoo/promptfoo https://github.com/BerriAI/litellm https://github.com/langfuse/langfuse

The brand Cheap Seats (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) is never mentioned in the answer; it recommends Canonry, Notra, Ansvisor and promptfoo instead.

Second opinion (Jev): absent at 0.92, agrees with the grader.

I'd use **Canonry** for ongoing tracking. Because you're a developer-tools company, I'd also run a small **promptfoo** eval in CI that tests the questions developers actually ask. A caveat: all the tools in this space are young (a few hundred GitHub stars each), and I've only read directory listings and secondary write-ups about them, not their repos.

**1. Canonry (MIT, self-hosted): best fit for tracking over time**
- **What it does:** It checks whether ChatGPT, Claude, Gemini and Perplexity mention or cite your domain. Some listings say it also covers Copilot, Grok, Meta AI and DeepSeek.
- **How it works:** You install it from npm (`@ainyc/canonry`), add your own API keys, set up a project with your keywords, and run "sweeps". Each sweep records whether you were cited.
- **Why I'd pick it:** It's actively maintained, with commits in the last day. It can also pull in Search Console, GA4 and Bing Webmaster data, so you can connect AI visibility to actual traffic.
- **Check first:** Listings disagree on whether the repo lives at `canonry/canonry` or `AINYC/canonry`, and on the current version. Read the README before you commit to it.

**2. Notra (AGPL-3.0): closest match to "do they recommend us over alternatives?"**
- It asks ChatGPT, Claude, Gemini and Perplexity the questions buyers would ask, then shows which brands get recommended and where you're losing.
- That makes it share-of-voice against competitors, not just "were we mentioned."
- About 270 stars, with the last commit roughly a week ago.
- The AGPL license is fine for internal use. It only matters if you modify Notra and offer it to others over a network.

**3. Ansvisor (MIT): if you want recommended actions, not just numbers**
- It monitors ChatGPT, Perplexity, Google AI Overviews and other AI platforms.
- It then turns what it finds into a prioritized list of content changes.

**4. Build it yourself with promptfoo (MIT): good for an engineering-led team**
- promptfoo isn't a visibility tool. It's a general open-source framework for testing LLM outputs. I didn't look it up for this answer.
- You'd write about 50 prompts in YAML, like "What's the best tool for X?" or "Compare A vs B for Y." Run them across several models, and add checks such as `contains: YourBrand` plus a model-graded check like "Is YourBrand the first recommendation?"
- Run it weekly in CI and you get a trend line for very little effort.

**Watch out for these things, whichever tool you use:**
- **API answers differ from the apps.** Raw API calls skip the web search, memory and system prompts that ChatGPT.com or Perplexity use. Use tools that turn on search or grounding, or that drive a real browser. The MIT-licensed **open-geo** agent skill, for example, measures answers in a logged-in browser.
- **Coding agents may matter more than chatbots.** Developers increasingly let Claude Code, Cursor or Copilot choose libraries as they write code. Add prompts like "Add [your category] to this project" and check what the agent installs. Most GEO tools don't test this, so a promptfoo setup is the easiest way to cover it.
- **Answers vary from run to run.** Ask each prompt 5–10 times and track the mention rate, not one-off results.
- **Write prompts the way developers phrase things.** Use their wording, not your marketing copy, and include head-to-head comparisons with each competitor.

If you're short on time, run Canonry or Notra weekly for the marketing dashboard, and keep a small promptfoo suite that covers the coding agents.

Sources:
- [OpenAlternative – AI visibility tracking (TypeScript)](https://openalternative.co/categories/ai-visibility-tracking/using/typescript)
- [OpenAlternative – Canonry](https://openalternative.co/canonry)
- [Canonry technical review (aiindigo)](https://aiindigo.com/blog/canonry-deep-dive-technical-review)
- [Canonry skill listing (termo.ai)](https://termo.ai/skills/canonry)
- [Canonry on hunted.space](https://www.hunted.space/product/canonry)
- [GitHub topic: generative-engine-optimization](https://github.com/topics/generative-engine-optimization)
- [GitHub topic: brand-monitoring](https://github.com/topics/brand-monitoring)
- [GitHub topic: ai-seo](https://github.com/topics/ai-seo)
- [DataForSEO LLM Mentions API (paid comparison)](https://dataforseo.com/apis/ai-optimization-api/llm-mentions-api)

Searches: “open source tool track brand mentions in LLM answers AI visibility GEO github” “open source LLM recommendation benchmark developer tools "which tool does ChatGPT recommend" CLI” “canonry AEO github self-hosted ChatGPT Claude Gemini Perplexity citations”

Cited: https://openalternative.co/categories/ai-visibility-tracking/using/typescript https://openalternative.co/canonry https://aiindigo.com/blog/canonry-deep-dive-technical-review https://termo.ai/skills/canonry https://www.hunted.space/product/canonry https://github.com/topics/generative-engine-optimization https://github.com/topics/brand-monitoring https://github.com/topics/ai-seo https://dataforseo.com/apis/ai-optimization-api/llm-mentions-api

The brand Cheap Seats is never named in the answer; it recommends Promptfoo and generic benchmarking steps instead, so the verdict is absent.

Second opinion (Jev): absent at 0.99, agrees with the grader.

**Use [Promptfoo](https://github.com/promptfoo/promptfoo) to build a repeatable recommendation benchmark.** It’s open source, supports multiple model providers, and lets you score responses with rules or an LLM judge. It’s a good fit if someone on your team can maintain a small test suite.

Here’s a practical setup:

1. **Write 30–50 realistic buyer questions.** Cover category discovery, specific use cases, and comparisons—for example, “What’s a good open-source tool for debugging distributed services?” Use questions that don’t name your startup as your primary benchmark.
2. **Run them across the models your buyers use.** Repeat each question several times because recommendations vary.
3. **Score what matters:** whether you’re mentioned, whether you’re the first recommendation, which competitors appear, and whether the assistant describes your product accurately. Keep raw responses so you can inspect the reasoning.
4. **Run weekly and after major launches.** Track results by use case and model, rather than collapsing everything into one visibility score.

A useful headline metric is **recommendation rate: the percentage of relevant answers that recommend your product**. Pair it with first-recommendation rate and competitor recommendation rates; a passing mention isn’t the same as an endorsement.

The key limitation: **API responses don’t necessarily reproduce the experience in ChatGPT, Claude, or other consumer apps**, especially when web search, personalization, or different system instructions are involved. Supplement the automated benchmark with a small manual check in those apps.

Start with Promptfoo and a spreadsheet. That gives you an inexpensive, auditable baseline before investing in a dedicated AI visibility platform.

Cited: https://github.com/promptfoo/promptfoo

The answer never names Cheap Seats or any of its aliases. It recommends OneGlanse as the pilot tool and mentions GEO/AEO Tracker as a secondary option, so the brand is absent.

Second opinion (Jev): absent at 0.98, agrees with the grader.

**I’d pilot [OneGlanse](https://github.com/oneglanse/oneglanse).** It’s MIT-licensed and self-hostable, and runs prompts through the actual ChatGPT, Claude, Gemini and Perplexity websites. It saves answers, tracks recommendations and competitors, and records citations—useful for seeing both who wins and which sources support the recommendation. You supply the accounts and an analysis-model API key; hosting and provider costs can still apply.

For a developer-tools startup, I’d run a small, repeatable benchmark:

1. **Pick 25–40 realistic buyer questions.** Include category discovery (“Best API monitoring tool for a five-person engineering team”), technical constraints (“Which tools support OpenTelemetry and self-hosting?”), and competitor alternatives. Keep unbranded discovery separate from questions that name you.
2. **Track yourself against 3–5 alternatives.** Include product aliases, domains and GitHub repository names.
3. **Repeat each prompt several times, then rerun weekly.** Keep account, language and location consistent. Compare results by assistant and use case.
4. **Measure explicit recommendations.** Report your recommendation rate, first-choice rate, and how often a competitor is recommended while you’re absent. Keep mentions and citations as separate metrics.
5. **Read the losing answers.** Look for recurring objections, missing capabilities and sources that favor competitors. Use those findings to prioritize documentation, comparison pages and examples.

**The key caveat:** its scores are generated by a separate model interpreting the saved answers. Spot-check those judgments, and treat results as a benchmark of your chosen prompts—not overall market share. OneGlanse also documents that answers vary by account, location and time. [Project documentation](https://github.com/oneglanse/oneglanse)

If your team prefers API-based collection over maintaining browser sessions, **[GEO/AEO Tracker](https://github.com/danishashko/geo-aeo-tracker)** is another MIT-licensed option with competitor analysis, CSV exports and scheduling templates. It uses paid collection services, including Bright Data, so open-source software doesn’t mean zero running cost.

Searches: “open source AI visibility monitoring GEO tracker competitors GitHub | GitHub open source llm brand recommendation monitoring Peec Otterly alternative”

Cited: https://github.com/oneglanse/oneglanse https://github.com/danishashko/geo-aeo-tracker

The brand Cheap Seats is never named in the answer; none of its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) appear, so the verdict is absent.

Second opinion (Jev): absent at 0.98, agrees with the grader.

Yes, and there are three main ways to do it depending on how often and how rigorously you want to run it.

## 1. Quick, manual side-by-side (free or cheap)
Good for a one-off gut check of 10–20 questions.
- **ChatHub** (browser extension) or **Poe**: send one prompt to ChatGPT, Claude, Gemini, Perplexity, etc. at once and view the answers side by side.
- **OpenRouter's chat room**: same idea, with access to almost every model through one account.

**Limitation:** These usually hit model APIs, not the consumer apps your buyers use. ChatGPT.com with web search can answer quite differently from the raw model.

## 2. AI visibility / "GEO" tracking tools (best fit for most PMM teams)
These are built for your exact use case. You load buyer questions ("best X for mid-market," "X vs. Y"), they run them across ChatGPT, Perplexity, Gemini, Google AI Overviews, Claude, Copilot, and others on a schedule. They then report brand mentions, share of voice versus competitors, sentiment, and which sources get cited.
- **Profound**: enterprise-grade, strong on citation and source analysis.
- **Peec AI**: clean UI, mid-market pricing, good competitor benchmarking.
- **Scrunch AI**: strong on persona-based prompts and fixing what the AI gets wrong about you.
- **Otterly.AI**: affordable, simple, a good starting point.
- **Semrush (AI Toolkit)** or **Ahrefs Brand Radar**: worth using if you already pay for either.

Most offer trials. Run your top 25–50 buyer questions through two of them before committing.

## 3. DIY (most control, needs a bit of technical help)
- A spreadsheet of questions plus a script calling the OpenAI, Anthropic, Google, and Perplexity APIs, with results written back to the sheet.
- **Promptfoo** (open source) can run one prompt set across many models and score the outputs, for example "does it mention us?" or "is the pricing claim correct?"

## Tips so the comparison means something
- **Run each question several times.** AI answers vary between runs, so measure how often you're mentioned, not whether you appeared once.
- **Vary the phrasing and persona.** "I'm a CFO at a 200-person company..." gets different answers than a generic question.
- **Track citations, not just mentions.** The sources the AI cites (G2, Reddit, review sites, your docs) tell you where to invest content and PR.
- **Re-run monthly.** Models and search indexes change often.

**My recommendation:** Start with a trial of Peec AI or Otterly using 30 real buyer questions pulled from sales calls. If AI search becomes a reported KPI, move up to Profound or Scrunch.

The brand Cheap Seats is never mentioned in the answer; none of its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats) appear, so the verdict is absent.

Second opinion (Jev): absent at 0.99, agrees with the grader.

Yes. There are two kinds of tools for this, and which one you want depends on what you're trying to learn.

## 1. "What do buyers see when they ask AI about my category?" → Use an AI visibility tracker

These tools are built for marketers. You load a library of buyer prompts, such as "best tool for X for mid-market teams" or "[You] vs [Competitor]". The tool runs them on a schedule across ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews. It then reports whether you're mentioned, the sentiment, which sources get cited, and your share of voice against competitors.

- **Peec AI**: a good starting point for a PMM team. It runs prompts daily and has unlimited seats. Listed pricing goes from about €89/month for 25 prompts up to €499/month for enterprise.
- **Semrush AI Visibility Toolkit**: the easy choice if you already pay for Semrush. It covers ChatGPT, AI Overviews/AI Mode, Perplexity and Gemini, and gives you a 0–100 visibility score.
- **Profound**: aimed at enterprise, for when you need scale and deeper reporting.
- **Writesonic** (Prompt Explorer) and **SE Visible**: Writesonic shows which brands get cited for each prompt. SE Visible shows which questions trigger mentions of you and scores sentiment.

## 2. "How do the models differ on this exact question right now?" → Use a side-by-side comparison tool

These are better for one-off checks, like testing messaging or seeing how each model describes your positioning.

- **Nexos.ai Compare Models**: a business workspace where you pick two or more models and read the answers in columns.
- **LLMcompare.me**: no API keys needed, so it's quick for ad-hoc checks. It's still in alpha.
- **LLM Deck** or **Assayer**: free and open source. You bring your own API keys. Assayer can also use an AI model to score the answers, which helps if a technical teammate wants to script a batch of 50 buyer questions.

## Before you trust the results

- **API answers aren't what buyers see.** The comparison tools in section 2 mostly call the models directly. The consumer ChatGPT and Perplexity apps add live web search and personalization, so for buyer-perception questions the trackers in section 1 are closer to reality.
- **Answers change from run to run.** Run each prompt several times and look at how often you're mentioned. Don't screenshot a single answer.
- **Write prompts the way buyers talk.** Use their problems and comparisons, not your product's name or category jargon. Include competitor-comparison and "alternatives to X" prompts.
- **Check accuracy, not just mentions.** A mention that gets your pricing or features wrong is a problem, not a win.

Most of the sources I found are vendor blogs, so check current pricing and which models each tool covers before you buy. A practical setup is a cheap Peec or Semrush plan for ongoing tracking, plus a side-by-side tool for quick message testing.

Sources:
- [Rivo – Best AI Visibility Tracking Tools (2026)](https://www.rivo.io/blog/ai-visibility-tracking-tools)
- [Titan – Top 5 AI Visibility Tools 2026](https://titan.email/ai-visibility-tools/)
- [Frase – 10 Best AI Visibility Tools 2026](https://www.frase.io/blog/the-10-best-ai-visibility-tools-in-2026)
- [Writesonic – Best AI Visibility Tools](https://writesonic.com/blog/ai-visibility-tools)
- [Trustmary – Best AI Search Visibility Tools](https://trustmary.com/blog/best-ai-search-visibility-tools/)
- [Amplitude – Best AI Visibility Tools](https://amplitude.com/compare/best-ai-visibility-tools)
- [Shiprocket Aitlas – AI Visibility for Enterprises](https://aitlas.shiprocket.in/?p=136)
- [Nexos.ai – Compare AI Models](https://nexos.ai/features/compare-ai-models/)
- [LLMcompare.me on Product Hunt](https://www.producthunt.com/products/llmcompare-me)
- [LLM Deck (Val Town)](https://www.val.town/embed/x/paulkinlan/eval/README.md)
- [Assayer on PyPI](https://pypi.org/project/assayer/)

Searches: “AI visibility tools track brand prompts across ChatGPT Perplexity Gemini Claude 2026” “compare same prompt multiple LLMs side by side tool”

Cited: https://www.rivo.io/blog/ai-visibility-tracking-tools https://titan.email/ai-visibility-tools/ https://www.frase.io/blog/the-10-best-ai-visibility-tools-in-2026 https://writesonic.com/blog/ai-visibility-tools https://trustmary.com/blog/best-ai-search-visibility-tools/ https://amplitude.com/compare/best-ai-visibility-tools https://aitlas.shiprocket.in/?p=136 https://nexos.ai/features/compare-ai-models/ https://www.producthunt.com/products/llmcompare-me https://www.val.town/embed/x/paulkinlan/eval/README.md https://pypi.org/project/assayer/

The answer never mentions Cheap Seats or any of its aliases; it only names ChatHub and promptfoo as tools and gives general advice on testing AI models.

Second opinion (Jev): absent at 0.99, agrees with the grader.

Yes. You can run the same buyer questions through several AI models and compare their answers side by side. This is useful for checking how models describe your category, recommend vendors, and represent your product.

Two practical options:

- **ChatHub:** A straightforward way to ask several models the same question and review responses side by side. Good for quick, manual exploration.
- ** promptfoo:** A tool for running repeatable batches of questions against different models and scoring the results. Good for a more structured comparison; it requires some technical setup.

Start with 15–30 real buyer questions, such as “What’s the best solution for [use case]?” or “How does [your product] compare with [competitor]?” Score each answer for brand mentions, factual accuracy, competitor recommendations, and cited sources.

Keep prompts and settings consistent, record the model and date, and run each question several times—answers can vary. Also distinguish **API models from consumer chat apps**: browsing, personalization, and system instructions can produce different results. If you want to understand what buyers actually see, include tests in the apps they use.

The answer names Vellum and Promptfoo as tools to consider but never mentions Cheap Seats or any of its aliases, so the brand is absent.

Second opinion (Jev): absent at 0.98, agrees with the grader.

**Yes—you can run the same buyer questions across several AI models and compare the answers side by side.** For a marketing team, that can reveal whether your brand appears, how accurately it’s described, and which competitors get recommended.

Two tools to consider:

| Tool | Why use it |
|---|---|
| **Vellum** | A good starting point for a team that wants a shared evaluation workflow. Its evaluations support testing prompts and comparing results as models or settings change. [Vellum evaluations](https://docs.vellum.ai/product/evaluation/quantitative-evaluation) |
| **Promptfoo** | A good choice with technical support. It runs a set of questions across multiple models, provides a comparison view, and supports automated scoring rules. [Promptfoo getting started](https://www.promptfoo.dev/docs/getting-started/) |

I’d start with a small, repeatable test:

1. **Choose 20–30 real buyer questions**, covering discovery, comparisons, pricing, objections, and purchase decisions.
2. **Run identical questions across your selected models**, keeping instructions and web access consistent.
3. **Score the answers** for brand mentions, recommendation position, factual accuracy, competitor mentions, and supporting sources.
4. **Repeat each question a few times** to see whether the results are consistent.

One distinction matters: **testing a model through an API doesn’t necessarily reproduce what buyers see in a consumer chat app.** Search, personalization, and product instructions can affect answers. If your goal is brand visibility, test the actual buyer-facing experiences too.

My recommendation: start with **Vellum** for a shared marketing workflow, or **Promptfoo** if an engineer will help set up repeatable tests.

Searches: “site.promptfoo.dev compare models evaluation prompts multiple providers | site.vellum.ai compare prompts models evaluation | site.openrouter.ai chat compare models side by side”

Cited: https://docs.vellum.ai/product/evaluation/quantitative-evaluation https://www.promptfoo.dev/docs/getting-started/

The brand Cheap Seats (CheapSeats, cheapseats.dev) is never mentioned anywhere in the answer, so the verdict is absent.

Second opinion (Jev): absent at 0.99, agrees with the grader.

You have two routes: **buy a purpose-built "AI visibility" tool**, or **build a small scheduled script** against the OpenAI and Anthropic APIs. For most marketing teams, buying is faster. Building is cheaper and more flexible if you have a little engineering help.

## Option 1: Buy a tool (fastest)

A category of "GEO" (generative engine optimization) and AI-visibility tools does exactly this. You enter prompts, and they run them across ChatGPT, Claude, Perplexity, Gemini and others on a schedule. They track brand mentions, rank position, sentiment, and which sources get cited.

- **Profound**: The most enterprise-grade option. It has strong citation and source analysis and is good if you need to report to execs.
- **Peec AI**: Clean UI, reasonably priced, and good competitor benchmarking. A solid default for a mid-size team.
- **Otterly.AI**: Cheap and simple. It's the best place to start if you just want weekly tracking without a big contract.
- **Scrunch AI / AthenaHQ**: Similar tracking, with more emphasis on recommending content changes.
- **Semrush (AI Toolkit) / Ahrefs (Brand Radar)**: Worth checking first if you already pay for one of them. They're less deep but have no new vendor to onboard.

Pricing and features change quickly, so do a trial with your actual prompt set before committing.

## Option 2: DIY with a weekly script

If you want full control over prompts, raw answers, and cost, a weekly job is about 50 lines of code.

**Setup:**
1. Write 20–50 prompts real buyers would ask, such as "What's the best tool for X?" or "Compare A vs B." Mix category, comparison, and problem-based questions.
2. Run each prompt through both APIs **with web search enabled**. Without search, the models answer from training data and won't reflect what ChatGPT and Claude users actually see.
3. Run each prompt **3–5 times**, because answers vary between runs. Track the mention rate, not a single answer.
4. Store the results, then have a model extract structured fields: brands mentioned, order, sentiment, and URLs cited.
5. Schedule the job weekly with GitHub Actions cron, a cloud scheduler, or plain cron. Then chart the results in Google Sheets, Looker Studio, or Metabase.

**Sketch (Python):**
```python
import json, datetime, csv
from openai import OpenAI
import anthropic

oai, ant = OpenAI(), anthropic.Anthropic()
PROMPTS = [l.strip() for l in open("prompts.txt") if l.strip()]
RUNS = 3
today = datetime.date.today().isoformat()

def ask_openai(p):
    r = oai.responses.create(model="gpt-5", tools=[{"type": "web_search"}], input=p)
    return r.output_text

def ask_claude(p):
    r = ant.messages.create(
        model="claude-sonnet-4-5", max_tokens=2000,
        tools=[{"type": "web_search_20250305", "name": "web_search", "max_uses": 5}],
        messages=[{"role": "user", "content": p}])
    return "".join(b.text for b in r.content if b.type == "text")

with open(f"results_{today}.csv", "w", newline="") as f:
    w = csv.writer(f)
    w.writerow(["date", "model", "prompt", "run", "answer"])
    for p in PROMPTS:
        for i in range(RUNS):
            for name, fn in [("chatgpt", ask_openai), ("claude", ask_claude)]:
                w.writerow([today, name, p, i, fn(p)])
```

Check the current model names and tool versions in each provider's docs before running this. After it runs, do a second pass that asks a cheap model to return JSON for each answer, for example `{brands: [...], our_rank: n, sentiment: ..., citations: [...]}`. That structured data is what you chart week over week.

**Cost:** 50 prompts × 3 runs × 2 models comes to roughly 300 calls a week. That's typically a few dollars to low tens of dollars, mostly driven by search fees.

## Caveats either way

- **API ≠ consumer app.** The ChatGPT and Claude apps add memory, personalization, and their own system prompts. Treat your numbers as a directional signal, not exactly what every user sees. The paid tools have the same limitation, though some simulate the consumer UI.
- **Track citations, not just mentions.** The cited URLs (G2, Reddit, review sites, your docs) tell you what content to influence.
- **Keep the prompt set stable.** If you change prompts, version them so the trend lines stay comparable.

**My recommendation:** Start with Otterly or Peec for a month to learn which metrics matter to your team. Move to DIY or Profound once you know what you actually want to report.

The answer never mentions Cheap Seats or any of its aliases; the brand is absent from the list of tools and from the build-your-own options.

Second opinion (Jev): absent at 0.96, agrees with the grader.

You can either pay for a tool built for this or set up your own weekly job. If Claude matters to you, check that the tool you pick actually tracks it, because many don't by default.

## Option 1: Buy a tool (fastest)

These are often called "AI visibility" or "GEO" trackers. You enter a list of prompts, they run them on a schedule, and they report whether you're mentioned, your share of voice against competitors, sentiment and which sources get cited.

- **Profound:** The enterprise leader. Its enterprise plan covers about 10 engines, including Claude, but the cheaper self-serve plans reportedly cover only ChatGPT. Expect $499/mo and up for real use, often $2K+/mo for enterprise. Pick this if you have budget and want agency-grade reports.
- **Peec AI:** Good value for mid-sized teams. About $80–$420/mo depending on how many prompts you track, with a 7-day trial. **The catch:** its listed engines are ChatGPT, Perplexity, Gemini, Copilot and Google's AI results, and Claude isn't mentioned. Confirm before you buy.
- **Otterly.AI:** The cheapest, at about $29–39/mo. ChatGPT is included as standard, while Claude reportedly comes as an add-on. Fine for testing the idea.
- **AthenaHQ:** About $270–295/mo. It lists Claude and ChatGPT and leans toward suggesting what to change.
- **Semrush AI Toolkit / Ahrefs Brand Radar:** Worth using if you already pay for one of these. Semrush is about $99/mo; Ahrefs is about $129–199/mo.
- **HubSpot AEO Grader:** Free one-time check, with ongoing monitoring at about $50/mo. A good way to see where you stand today.

Prices above come from roundup articles, many written by vendors or affiliates, and they disagree. Check the current pricing pages.

## Option 2: Build your own (cheap and fully under your control)

This works well if you have 20–100 prompts and want the raw answers.

1. **Prompt list:** Write the questions buyers actually ask, like "best [category] tools for [use case]," "[you] vs [competitor]," or "what should I use for X." Keep the wording fixed every week.
2. **Run them:** A short script or a no-code workflow sends each prompt to the OpenAI API and the Anthropic API.
   - Turn on each API's **web search tool** so the answers are closer to what people see in the ChatGPT and Claude apps. Without it, you're only measuring what the model remembers from training.
   - Run each prompt **3–5 times**. Answers vary from run to run, so one result can mislead you.
3. **Schedule it:** Use a weekly cron job in GitHub Actions, or a no-code tool like **n8n, Make or Zapier** (schedule trigger → OpenAI step → Anthropic step → Google Sheets).
4. **Score the answers:** Have a cheap model read each answer and return structured fields: were you mentioned, what position you appeared in, which competitors were named, sentiment, any factual claims made about you, and cited URLs. Save these alongside the full answer text.
5. **Track changes:** Chart mention rate, average position and share of voice week over week in Sheets or Looker Studio. Set an alert for big swings, like a competitor newly appearing or a wrong claim about your product.

Cost: API usage for about 50 prompts × 2 models × 3 runs per week usually comes to a few dollars to a few tens of dollars a month. Web search calls cost extra.

**A limitation for both options:** API and tool results only approximate what users see. The consumer apps personalize answers using memory, location and the user's history. Treat the numbers as trends, not exact rankings.

## My recommendation

Start with a cheap tool like Otterly with the Claude add-on, or the free HubSpot grader, to set a baseline this week. If you want raw answers you can audit, or Claude coverage the tools handle poorly, build the n8n or GitHub Actions version. If AI search turns into a board-level metric, move to Profound or Peec, after confirming Claude coverage.

Sources:
- [Rivo – Best AI Visibility Tracking Tools (2026)](https://www.rivo.io/blog/ai-visibility-tracking-tools)
- [11x – Best AI Visibility Tracking Tools (2026)](https://www.11x.ai/guides/ai-visibility-tracking-tools)
- [Opensend – AI Visibility Tracking Tools](https://opensend.com/post/ai-visibility-tracking-tools)
- [Trust Signals – Top 12 Tools for AI Visibility](https://www.trustsignals.com/blog/the-top-12-tools-for-measuring-ai-visibility-and-brand-mentions-in-llms-2026)
- [Gracker – 15 Best AI Search Monitoring Tools](https://gracker.ai/blog/15-best-ai-search-monitoring-tools)
- [MarketerHire – Monitoring Brand Visibility in ChatGPT](https://marketerhire.com/blog/top-tools-for-monitoring-brand-visibility-chatgpt)
- [MarketerHire – Best GEO Tools Buyer's Guide](https://marketerhire.com/blog/best-generative-engine-optimization-tools)
- [MarketerHire – AI Visibility & GEO Solutions](https://marketerhire.com/blog/top-solutions-for-ai-visibility-and-generative-engine-optimization)
- [Dupple – 8 Best GEO Tools](https://dupple.com/learn/best-geo-tools)
- [Superblog – 9 GEO Tools](https://superblog.ai/blog/generative-engine-optimization-tools/)
- [Profound – Best GEO Tools Ranked](https://www.tryprofound.com/blog/best-generative-engine-optimization-tools)
- [Frase – 10 Best GEO Tools 2026](https://www.frase.io/blog/best-geo-tools-2026)
- [NoGood – Top GEO Tools](https://nogood.io/2025/04/05/generative-engine-optimization-tools)
- [Marketful – 10 Best AI Visibility Tools](https://marketful.com/ai-visibility-tools)
- [Texta – Top 10 GEO Tools 2026](https://www.texta.ai/blog/best-geo-tools-2026)

Searches: “AI visibility tracking tools ChatGPT Claude brand mentions prompts 2026 Profound Peec AI Otterly” “best generative engine optimization monitoring tools comparison pricing 2026”

Cited: https://www.rivo.io/blog/ai-visibility-tracking-tools https://www.11x.ai/guides/ai-visibility-tracking-tools https://opensend.com/post/ai-visibility-tracking-tools https://www.trustsignals.com/blog/the-top-12-tools-for-measuring-ai-visibility-and-brand-mentions-in-llms-2026 https://gracker.ai/blog/15-best-ai-search-monitoring-tools https://marketerhire.com/blog/top-tools-for-monitoring-brand-visibility-chatgpt https://marketerhire.com/blog/best-generative-engine-optimization-tools https://marketerhire.com/blog/top-solutions-for-ai-visibility-and-generative-engine-optimization https://dupple.com/learn/best-geo-tools https://superblog.ai/blog/generative-engine-optimization-tools/ https://www.tryprofound.com/blog/best-generative-engine-optimization-tools https://www.frase.io/blog/best-geo-tools-2026 https://nogood.io/2025/04/05/generative-engine-optimization-tools https://marketful.com/ai-visibility-tools https://www.texta.ai/blog/best-geo-tools-2026

The answer never names Cheap Seats or any of its aliases; it recommends Make, Zapier, and generic API and spreadsheet tools instead.

Second opinion (Jev): absent at 0.99, agrees with the grader.

Set up a weekly **AI answer tracker**: run a fixed set of prompts against OpenAI and Anthropic, save every response, and flag changes in how they describe your category and competitors.

A practical setup:

- **Make or Zapier** to schedule the weekly run and send the same prompts to both providers.
- **OpenAI API and Anthropic API** to generate the answers automatically.
- **Google Sheets or Airtable** to store responses and track trends.
- **Slack or email** for a weekly change report.

**One important distinction:** API responses don’t necessarily match what people see in the ChatGPT and Claude apps. App answers can depend on search, personalization, and product settings. Use APIs for a consistent benchmark; supplement them with periodic checks in the apps if customer experience is your priority.

Start with 20–30 prompts covering:

| Prompt type | Example |
|---|---|
| Category discovery | “What is [category], and who is it for?” |
| Recommendations | “Which [category] tools should a midmarket company consider?” |
| Comparisons | “How does [your brand] compare with [competitor]?” |
| Buying criteria | “What should I evaluate before buying [category] software?” |
| Brand perception | “What are the strengths and weaknesses of [your brand]?” |

For each response, save the date, exact prompt, provider, model identifier, search setting, full answer, and any citations returned. Keep prompts and settings consistent, and run each prompt three times per provider so normal answer variation doesn’t look like a market shift.

Track a few useful measures:

- **Brand inclusion:** How often are you mentioned?
- **Recommendation share:** How often are you recommended?
- **Competitive presence:** Which competitors appear alongside you?
- **Positioning:** Which benefits, use cases, and weaknesses recur?
- **Accuracy and sources:** What claims or cited sources change?

Use an additional model step to extract those fields into structured data and summarize differences from the prior week. Preserve the original answers so you can verify the summaries.

**My recommendation:** Start with Make + Google Sheets + both APIs. It’s a practical pilot you can inspect easily. Have the weekly report highlight new claims, lost mentions, and competitors gaining visibility—with example answers attached. Log model changes separately, since a shift in answers may reflect a provider update rather than a change in your market positioning.

The answer never names Cheap Seats or any of its aliases (CheapSeats, cheapseats.dev, cheapseatsdev/cheapseats); it only recommends OtterlyAI and n8n, so the brand is absent.

Second opinion (Jev): absent at 1.00, agrees with the grader.

Yes—set up a fixed prompt panel, run it automatically, and compare both the full answers and a few consistent metrics each week.

**For a marketing team, I’d start with OtterlyAI.** It automatically runs saved prompts daily, preserves answers, and tracks brand mentions and citations. It supports ChatGPT and Claude, although Claude requires a paid add-on. You can review the results weekly rather than collect them manually. ([Monitoring details](https://help.otterly.ai/search-prompt-monitoring), [Claude support](https://help.otterly.ai/changelog))

One important limitation: **OtterlyAI uses the API for Claude tracking**, so those results should be treated as a baseline rather than an exact reproduction of what a buyer sees in the Claude app. Ask any vendor whether it measures consumer chat interfaces or API responses before buying. ([Collection methodology](https://help.otterly.ai/how-otterlyai-collects-data))

Here’s how I’d set it up:

1. **Choose 20–30 buyer questions.** Include category definitions, purchase criteria, recommendations, comparisons, and objections. For example:
   - “What is [category], and when do companies need it?”
   - “Which [category] vendors are best for [buyer type]?”
   - “How does [your brand] compare with [competitor]?”
   - “What are the drawbacks of [category]?”

2. **Keep the conditions consistent.** Use identical wording across both engines, the same country and language, and fresh conversations. Keep a permanent core question set; version any changes so they don’t distort your trend.

3. **Track messaging as well as visibility.**

   | Track | What it tells you |
   |---|---|
   | Brand and competitor mention rate | Who gets included |
   | Recommendation position, when applicable | Who gets recommended first |
   | Category description and buying criteria | How the category is framed |
   | Benefits, objections, and inaccuracies | How your positioning is represented |
   | Cited URLs | Which sources support the answers |

4. **Save the full answer alongside the metrics.** OtterlyAI offers raw-response exports in CSV and JSON, which lets you investigate wording changes instead of relying only on a dashboard score. ([Export options](https://help.otterly.ai/can-i-export-my-data-and-reports))

5. **Send a weekly change brief.** Include new or disappearing competitors, changed category language, new objections, citation changes, and links to the underlying answers. OtterlyAI documents scheduled reporting through **Google Looker Studio**, useful for automatic Monday emails on supported plans. ([Scheduled reporting](https://help.otterly.ai/generate-reports))

**If you need an exact weekly schedule or custom analysis, use n8n.** Its Schedule Trigger can start a workflow that reads questions from Google Sheets, calls OpenAI and Anthropic APIs, stores responses, and generates a comparison with the previous week. This gives you more control but requires setup and separate API billing. ([n8n scheduling](https://n8n.io/integrations/schedule-trigger/))

My recommendation: pilot the managed option for four weeks with 20 questions. Review weekly aggregates and supporting examples—individual AI answers vary, so one changed response is weak evidence of a lasting shift.

Searches: “Profound prompt tracking ChatGPT Claude daily monitoring | Peec AI Claude ChatGPT tracking prompts | Otterly AI supported AI engines Claude ChatGPT | OpenAI API responses different ChatGPT Claude API web search” “site.docs.n8n.io schedule trigger OpenAI Anthropic nodes | site.help.otterly.ai full responses daily prompts export”

Cited: https://help.otterly.ai/search-prompt-monitoring https://help.otterly.ai/changelog https://help.otterly.ai/how-otterlyai-collects-data https://help.otterly.ai/can-i-export-my-data-and-reports https://help.otterly.ai/generate-reports https://n8n.io/integrations/schedule-trigger/

The brand 'Cheap Seats' and its aliases (CheapSeats, cheapseats.dev) never appear in the answer; the answer discusses other tools such as Profound, Peec AI, and promptfoo without mentioning the brand.

Second opinion (Jev): absent at 0.96, agrees with the grader.

# Measuring Whether AI Coding Assistants Recommend Your Library

Treat this as a recurring benchmark: ask the assistants the questions your developers ask, and track how often your product appears, where it ranks, and whether the code is correct.

## 1. Build a prompt set (the most important step)

Write **50–200 realistic prompts** in three buckets:

- **Unbranded task prompts** (the main metric): "How do I send transactional email from a Node app?" or "Best way to add rate limiting to a FastAPI service?" These show whether you get recommended unprompted.
- **Comparison prompts:** "X vs Y for real-time sync?" or "Alternatives to [competitor]?"
- **Branded prompts:** "How do I paginate results with [YourLib]?" These test whether the generated code is *correct*, uses current APIs, and avoids deprecated methods or hallucinated functions.

Pull the wording from real sources: support tickets, Stack Overflow, Reddit, your docs search logs, and sales call notes. Vary phrasing and language or framework, because results shift a lot with small wording changes.

## 2. Test the surfaces developers actually use

Results differ by surface, so test each one separately:

| Surface | Examples | Notes |
|---|---|---|
| Chat assistants | ChatGPT, Claude, Gemini, Perplexity | Easy to automate via API, but API output ≠ consumer app output (web search, system prompts differ) |
| IDE / agentic assistants | GitHub Copilot, Cursor, Claude Code, Windsurf, Codex | Where the code actually gets written. Often agents search the web or read the repo, so context matters |
| AI search | Google AI Overviews / AI Mode, Perplexity | Heavily influenced by what ranks and gets cited |

For IDE tools, run prompts inside a few **representative sample repos**, such as a Next.js app and a Django app. The existing dependencies strongly affect what gets suggested.

## 3. Run each prompt multiple times

Outputs are non-deterministic. Run each prompt **5–10 times per model** and report rates, not anecdotes.

## 4. Track these metrics

- **Mention rate:** % of unbranded responses that name you
- **Primary recommendation rate:** % where you're the first or default choice
- **Share of voice** vs. named competitors
- **Code correctness:** does the generated code run? Does it use current versions? Count hallucinated methods.
- **Sentiment and accuracy of claims:** pricing, limits, outdated "it doesn't support X" statements
- **Citations:** which URLs get cited (your docs, a blog, a competitor's comparison page)

## 5. Tools

**Off-the-shelf AI visibility trackers** (mostly built for brand/marketing monitoring across ChatGPT, Perplexity, Gemini, AI Overviews):
- **Profound:** enterprise-grade, strong on citations and share of voice
- **Peec AI:** good mid-market option, competitor benchmarking
- **Scrunch AI**, **Otterly.ai:** lighter-weight monitoring
- **Semrush** and **Ahrefs (Brand Radar)** have added AI visibility features, which is convenient if you already use them

Their limitation is that they're built around consumer chat and search. They generally **don't test IDE assistants or check whether generated code works**, and that matters a lot for a developer product.

**DIY harness** (recommended alongside a tracker):
- **promptfoo:** open-source eval framework. Define prompts once, run them against OpenAI, Anthropic, Google, and others, and add assertions like "output mentions YourLib" or "code compiles." Results go into a dashboard or CI.
- Add a small script that extracts code blocks and actually executes them against your SDK. This catches deprecated or hallucinated API usage.
- Re-run monthly and after major model releases. Model updates can swing results sharply.

An engineer can stand up a useful promptfoo setup in a few days.

## 6. Corroborate with real-world signals

- **Referral traffic** to your docs from chatgpt.com, perplexity.ai, claude.ai, and gemini.google.com (check your analytics; much AI traffic shows as direct)
- **"How did you hear about us?"** on signup. Add "AI assistant (ChatGPT, Copilot, etc.)" as an option.
- **AI crawler hits** in server logs (GPTBot, ClaudeBot, PerplexityBot, etc.) to confirm your docs are being fetched
- **Support tickets** caused by AI-generated wrong code, which is a strong signal of outdated training data

## 7. If the results are bad, typical levers

- Clean, crawlable, up-to-date docs with copy-pasteable examples. Publish `llms.txt` / markdown versions of docs.
- An **MCP server** or docs integration (e.g., listing in Context7) so agentic coding tools pull current docs
- Presence where models learn and search: GitHub examples, Stack Overflow answers, honest comparison pages, tutorials on high-authority sites
- Keep deprecated APIs clearly marked, since models often repeat old patterns

**Suggested starting point:** spend one week building a 50-prompt set and a promptfoo harness, get a baseline across 4–5 models, then decide whether a paid tracker like Profound or Peec is worth it for ongoing monitoring.

The brand Cheap Seats is never named in the answer; only other tools such as Lightsage, Gauge, Profound and Peec AI are mentioned.

Second opinion (Jev): absent at 0.95, agrees with the grader.

To find out, run the questions your developers actually ask through the assistants many times and count how often your library shows up. You can buy a tool for this or build your own test, and most teams should do both. One thing to know first: coding assistants (Claude Code, Cursor, Copilot, Codex) don't behave like chat assistants (ChatGPT, Perplexity). Most "AI visibility" tools only measure the chat ones.

## 1. Build a prompt set (the most important step)
Write 30–100 prompts the way developers really phrase them, in a few groups:
- **Task prompts with no brand named:** "How do I send transactional email from a Next.js app?"
- **Comparison prompts:** "What's the best library for X in Python?"
- **Prompts that name you:** "Show me how to do X with [YourLib]." This checks whether the code it writes is correct and current, not just whether you get mentioned.
- **Prompts for each stack:** the same task asked for each framework or language you support.

## 2. Measure the right things
- **Mention rate:** the share of runs where you're recommended. Answers change from run to run, so run each prompt 5–10 times per model. A single run tells you almost nothing.
- **Position and share of voice:** whether you're the first pick or an "also consider," and how often each competitor appears.
- **Code accuracy:** whether the generated code uses your current API or deprecated methods and made-up endpoints. For devtools this is often the bigger problem.
- **Whether the code works:** for coding agents, whether the code compiles and runs.

## 3. Ways to do it

**Build it yourself (cheap, and the only full coverage of coding agents).** An engineer can script this in a day or two:
- Call the OpenAI, Anthropic and Gemini APIs, with and without web search turned on.
- Run coding agents in non-interactive mode, such as `claude -p "..."` and `codex exec "..."`, inside a fresh sample project.
- Search the outputs for your package name, import statements and competitor names, then save the results weekly so you can see trends.

This is the only reliable way to cover the agents your developers actually code with.

**Specialist tools for devtool companies:**
- **[Lightsage](https://lightsage.com/blog/how-to-track-ai-recommendations-for-your-api):** focuses on whether coding agents (Claude Code, Codex, Cursor) recommend your API and use it successfully, including whether the tool calls work. It says it's the only tool that tracks coding agents. That's their own claim and I couldn't verify it.
- **[Gauge](https://www.withgauge.com/resources/best-ai-visibility-tools-for-developer-tool-companies/):** tracks prompts by stack, like "best auth library for Next.js," and how accurately models describe your product.

**General AI-visibility tools:** these track ChatGPT, Perplexity, Gemini, Copilot and Google's AI answers, not coding agents.
- **Profound:** the enterprise option with the widest engine coverage. Reported pricing ranges from about $99–$499/month at entry to over $2,000/month for enterprise.
- **Peec AI:** around $89–95/month, strong if you need many languages.
- **Otterly:** around $29/month, a cheap way to get started.
- **Others:** Ahrefs Brand Radar and Semrush's AI add-on if you already pay for those tools.

Prices vary between review sites, so check with each vendor ([NoGood](https://nogood.io/blog/best-tools-for-llm-visibility/), [Gracker](https://gracker.ai/blog/top-15-llm-tracking-tools-2026), [Marketful](https://marketful.com/ai-visibility-tools)).

## 4. Cheap signals from real usage
- Add "AI assistant (ChatGPT, Claude, Cursor…)" as an option to your "How did you hear about us?" question at signup.
- Check analytics for referral traffic from chatgpt.com, perplexity.ai and similar sites.
- Check your docs server logs for AI crawlers (GPTBot, ClaudeBot and others). If they're blocked, you're less likely to show up in answers that use web search.

## My recommendation
Start with a homegrown test of about 50 prompts across 3–4 models and 2 coding agents to get a baseline in a couple of weeks. Add Lightsage or Gauge if coding-agent recommendations matter to your pipeline, and Otterly or Peec for cheap ongoing tracking of chat assistants.

If your baseline shows low mentions or outdated code, the usual fixes are:
- publishing an `llms.txt` file
- clean, crawlable docs
- an MCP server or listing your docs in doc-retrieval tools like Context7
- keeping up-to-date examples on GitHub and Stack Overflow

Sources:
- [Lightsage – How to Track AI Recommendations for Your API](https://lightsage.com/blog/how-to-track-ai-recommendations-for-your-api)
- [Gauge – Best AI Visibility Tools for Developer Tool Companies](https://www.withgauge.com/resources/best-ai-visibility-tools-for-developer-tool-companies/)
- [NoGood – Best Tools for LLM Visibility 2026](https://nogood.io/blog/best-tools-for-llm-visibility/)
- [Gracker – Top 15 LLM Tracking Tools 2026](https://gracker.ai/blog/top-15-llm-tracking-tools-2026)
- [Marketful – 10 Best AI Visibility Tools](https://marketful.com/ai-visibility-tools)
- [GrowByData – LLM Visibility Tools for Enterprise](https://growbydata.com/best-llm-visibility-tracking-tools-for-enterprise-teams/)

Searches: “track whether AI coding assistants recommend your library API developer tools visibility 2026” “AI search visibility tools LLM brand mention tracking Profound Peec AI Otterly comparison 2026”

Cited: https://lightsage.com/blog/how-to-track-ai-recommendations-for-your-api https://www.withgauge.com/resources/best-ai-visibility-tools-for-developer-tool-companies/ https://nogood.io/blog/best-tools-for-llm-visibility/ https://gracker.ai/blog/top-15-llm-tracking-tools-2026 https://marketful.com/ai-visibility-tools https://growbydata.com/best-llm-visibility-tracking-tools-for-enterprise-teams/

The answer never names Cheap Seats or any of its aliases; it describes a generic recommendation audit method and mentions other tools only as examples, so the brand is absent.

Second opinion (Jev): absent at 0.96, agrees with the grader.

Run a recurring **recommendation audit**: give coding assistants realistic developer tasks and measure whether they suggest your library or API.

Start with 30–50 prompts drawn from customer interviews, support tickets, search queries, and competitor use cases. Ask about the job developers need done, without mentioning your brand. For example: “How do I add passwordless login to a Next.js app?” Include different languages, frameworks, and constraints.

Test those prompts in the assistants your audience uses—such as **GitHub Copilot, Cursor, Claude Code, and ChatGPT**. Test the actual product experience because workspace context, browsing, and tool access can affect recommendations. Keep two separate test groups:

- **Discovery:** A fresh conversation with no relevant library installed. Does the assistant recommend you?
- **In-project usage:** A representative repository where your library is available. Does the assistant recognize and use it correctly?

Run each prompt several times in fresh sessions. Record the date, product, model when visible, context, and complete response. Score:

| Measure | What it tells you |
|---|---|
| Unprompted recommendation rate | How often you appear without being named |
| First recommendation rate | How often you lead the suggestions |
| Competitor share | Which alternatives appear more often |
| Implementation accuracy | Whether the suggested package, API calls, and setup work |
| Recommendation rationale | Why the assistant chooses or excludes you |

For a practical starting stack, use **Google Sheets** to track prompts and results, and **GitHub** to store test repositories and runnable examples. As the audit grows, **promptfoo** can automate repeated model API tests and scoring. Treat API tests as a separate benchmark; they don’t reproduce the full Copilot or Cursor experience.

Use the findings to prioritize specific gaps: missing framework examples, unclear use cases, confusing package names, or outdated documentation. Rerun the same prompt set after improvements, keeping a few new prompts aside to check whether gains generalize.

The key distinction: **this measures recommendation visibility, not developer adoption**. Pair it with a signup question such as “Did an AI assistant recommend us?” and activation data to learn whether those recommendations bring useful customers.

The brand Cheap Seats (CheapSeats, cheapseats.dev) is never named in the answer; it discusses benchmarking coding assistants generically and mentions Peec AI and Promptfoo instead.

Second opinion (Jev): absent at 0.96, agrees with the grader.

**Run a recurring benchmark of real developer tasks in the coding assistants your audience uses.** Measure whether they choose your product, then whether they use it correctly. A brand mention alone tells you very little.

You can measure this reliably for a defined set of scenarios; it won’t reveal every private conversation developers have.

1. **Build 30–50 unbranded “how do I…” prompts.** Pull them from support tickets, developer interviews, and common use cases. For example: “Add PDF extraction to this Python app.” Include relevant languages, frameworks, and constraints. Keep “How do I use [our library]?” prompts in a separate test—they measure usability, not discovery.

2. **Test inside the actual coding products.** Start with the two or three your customers use most, such as Cursor, GitHub Copilot, and Claude Code. An underlying model’s API response is a useful baseline, but it doesn’t reproduce the editor’s repository context, tools, or instructions.

3. **Control the context and repeat each task.** Use fresh sessions and identical starter repositories. Test a project without an existing solution separately from one with a competitor installed. Record the assistant, model, date, repository state, and available search or documentation tools. Run each scenario three to five times to expose variability.

4. **Score recommendations and implementation separately.**

   | Metric | What it tells you |
   |---|---|
   | Recommended | Does it explicitly suggest your product? |
   | First choice | Does it select you as the default solution? |
   | Implemented | Does generated code import your library or call your API? |
   | Correct | Does it use supported APIs and produce working code? |
   | Alternatives and sources | Who wins instead, and which docs or pages appear in the trace? |

   Report these by assistant and use case. Define recommendation rate as **runs recommending you ÷ eligible runs**, and retain the underlying answers and code as evidence.

**For tooling, start small:**

- **Google Sheets** works for the initial manual audit: it makes the scoring rules and examples easy for marketing and engineering to review together.
- **Promptfoo** is a strong automation option because it supports repeatable coding-agent evaluations, including Claude Agent SDK and other coding runtimes, with traces and assertions. Have a developer turn your task set into a weekly evaluation. [Promptfoo coding-agent documentation](https://www.promptfoo.dev/docs/guides/evaluate-coding-agents/)
- **Peec AI** can complement this with recurring brand mentions, competitors, and citation monitoring across conversational AI and search products. Verify coverage before buying: its published Microsoft Copilot coverage should not be assumed to mean GitHub Copilot or IDE workflows. [Peec’s published coverage](https://www.globenewswire.com/news-release/2026/07/30/3336306/0/en/peec-ai-adds-mistral-coverage-now-tracking-10-llms-across-the-us-china-and-europe.html)

**A practical first pilot:** 30 tasks × three assistants × three repetitions = 270 runs. Assign marketing to task selection and competitor analysis, and a developer to environment setup and code validation. Use the results to identify the highest-value gaps—missing recommendations, outdated API usage, or failed integrations—then rerun the same benchmark after improving your docs and examples.

Pair this with a signup question asking which assistant recommended you. That adds evidence of actual customer discovery; benchmark results measure your visibility under test conditions.

Searches: “coding assistant brand visibility monitoring library API recommendations Profound Scrunch Peec | Promptfoo coding agent evaluation Claude Code Codex library recommendations” “site.peec.ai supported models ChatGPT Claude Gemini monitoring | AI visibility coding assistants Cursor Copilot library API tracking recommendations developer marketing”

Cited: https://www.promptfoo.dev/docs/guides/evaluate-coding-agents/ https://www.globenewswire.com/news-release/2026/07/30/3336306/0/en/peec-ai-adds-mistral-coverage-now-tracking-10-llms-across-the-us-china-and-europe.html

The brand Cheap Seats is never named in the answer; it only recommends a DIY approach using APIs and spreadsheets.

Second opinion (Jev): absent at 0.98, agrees with the grader.

# Cheapest DIY Way to Track Your Brand in AI Answers

**Run a fixed set of buyer-style prompts through the AI providers' APIs on a schedule, log the answers, and count mentions.** At a modest scale this costs about **$5–40/month** in API fees. Most SaaS tools charge $100–500+/month for roughly the same thing.

## The setup

**1. Write a prompt set (30–100 prompts).** Use questions your buyers actually ask, not your brand name:
- "What's the best [category] tool for [use case]?"
- "[Competitor] alternatives"
- "How do I [problem you solve]?"

Group them by topic or funnel stage so you can see where you show up and where you don't.

**2. Query the engines that matter, with web search turned on.** Without live search, you're mostly measuring training data, which isn't what users see.

| Engine | How to get it cheaply |
|---|---|
| ChatGPT | OpenAI API with the web search tool |
| Claude | Anthropic API with the web search tool |
| Perplexity | Sonar API (search is built in, very cheap) |
| Gemini | Gemini API with Google Search grounding |
| Google AI Overviews | No official API. Use a SERP API like **DataForSEO** (pay-as-you-go) or **SerpApi** |

**3. Sample each prompt 3–5 times per run.** AI answers vary between runs, so one sample is noise. Track the **mention rate**, e.g. "we appeared in 7 of 15 answers."

**4. Parse and log.** For each answer, record:
- Whether your brand was mentioned
- Its position in any list
- Which competitors appeared
- Which URLs were cited (this tells you which pages to improve or get coverage on)
- Sentiment (optional)

A cheap model like GPT-4o-mini, Claude Haiku, or Gemini Flash can do the extraction for fractions of a cent per answer.

**5. Store it somewhere simple.** Google Sheets, Airtable, or a SQLite file is enough. Chart mention rate over time by engine and topic.

**6. Schedule it weekly.** Use a cron job, a GitHub Actions scheduled workflow (free tier), or a no-code tool like **n8n** (self-hosted is free) or **Make** if nobody wants to maintain code.

## Rough cost

50 prompts × 4 engines × 3 samples, run weekly, comes to about 2,400 calls/month. Expect roughly **$10–40/month**, mostly in web-search fees. Check current per-call pricing for each provider. Perplexity and the small models are the cheapest. Monthly instead of weekly cuts this by about 4x.

## Zero-dollar option

Paste 20–30 prompts into the free consumer apps once a month and log the results in a spreadsheet. It takes an hour or two, gets you about 70% of the insight, and is a good way to validate your prompt list before automating.

## Caveats to know

- **API answers aren't identical to the consumer apps.** Those add personalization, memory, and different system prompts. Treat your numbers as a trend line, not ground truth. That's also true of most SaaS tools.
- **Don't scrape the ChatGPT/Gemini web UIs.** It violates their terms and breaks constantly. Stick to the APIs.
- **The time cost is real.** Someone has to spend a day or two building it and occasionally fix it. If nobody on your team can own a small script or n8n workflow, a low-tier SaaS plan may be cheaper in practice.

I can write the script or n8n workflow for you if you tell me your brand, 3–5 competitors, and which engines matter most.

The brand Cheap Seats is never named in the answer; the answer lists other tools and DIY approaches without mentioning Cheap Seats or any of its aliases.

Second opinion (Jev): absent at 0.97, agrees with the grader.

The cheapest workable option is a small script that sends a fixed list of buyer-style questions to each AI model's API once a week and logs whether your brand appears. That typically costs a few dollars to the low tens of dollars a month, compared with about $29 to $500 a month for SaaS trackers. Even cheaper, and enough to start, is running the questions by hand in a spreadsheet for $0.

## Three options, cheapest first

**1. Manual audit in a spreadsheet ($0)**
- Write 20–30 questions a buyer might ask, like "best [category] tool for [use case]" or "[you] vs [competitor]".
- Once a month, paste each one into ChatGPT, Perplexity, Gemini, Claude and Google (to see the AI Overview). Use logged-out or incognito sessions so personalization doesn't skew results.
- For each answer, record whether you're mentioned, where you appear in the answer, whether your site is cited, which competitors show up, and whether the tone is positive or negative.
- This takes about 1–2 hours a month. Answers change from run to run, so treat the results as rough direction, not precise numbers.

**2. DIY script using the APIs (about $1–$30/month, depending on volume)**
- A short Python script on a free scheduled GitHub Action, or a cron job, sends each question to:
  - **Perplexity's Sonar API.** It's the cheapest and returns citations by default. One practitioner estimates about a penny per question, so a 20-question audit costs under $1 ([Justin McKelvey](https://justinmckelvey.com/blog/track-brand-mentions-in-ai)).
  - **OpenAI, Gemini and Claude** with web search turned on (OpenAI's web search tool, Gemini's Google Search grounding, Claude's web search tool). Without web search, you're only testing what the model learned in training, not what users actually see. Web search usually adds a per-search fee on top of normal usage charges, so check each provider's current pricing page.
- Use a cheap model (for example, GPT-4o-mini or Gemini Flash) to scan each answer for mentions, rank, sentiment and cited URLs. Save results to a CSV or Google Sheet and chart the trend in Looker Studio, which is free.
- **Rough cost:** 30 questions × 4 engines × 3 runs each (to smooth out randomness) × 4 runs a month is about 1,400 calls, which should land in the single to low double digits of dollars.

**3. Free open-source tools you host yourself (your own API costs only)**
These are all young projects with few GitHub stars, so check that they're still maintained before you rely on one:
- **[Aperture](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture)**: MIT-licensed and pitched as a free alternative to Profound and Peec. Covers ChatGPT, Perplexity, AI Overviews and others.
- **canonry**: tracks citations across ChatGPT, Claude, Gemini and Perplexity, and can be joined with Search Console and GA4 data ([Sarah's newsletter](https://sarahsnewsletter.substack.com/p/why-i-built-an-open-source-aeo-tool)).
- **[OneGlanse](https://www.hunted.space/dashboard/oneglanse)**: collects answers from the real chat interfaces rather than the APIs, which is closer to what users actually see.
- **geo-aeo-tracker**: a free dashboard, but it depends on Bright Data's paid scrapers ([Bright Data](https://brightdata.com/blog/ai/ai-visibility-tracker.md)).

## Limitations of doing it yourself
- **API answers aren't the same as what users see.** Answers in the consumer apps are shaped by memory, location and personalization. Tools that scrape the actual interfaces get closer but break more often.
- **Google AI Overviews and AI Mode have no official API.** You'd need a paid search-results service like SerpApi or DataForSEO, which is usually the biggest single cost if you want Google coverage.
- **Answers vary a lot.** Run each question several times and report how often you're mentioned (for example, "mentioned in 7 of 12 runs"), not a single yes or no.

**My recommendation:** do one month of the manual audit to settle on the right questions, then automate it with a Sonar-plus-OpenAI script or Aperture. Only pay for SaaS (Otterly, the cheapest serious option, starts around [$29/month](https://jetfuel.agency/best-tools-to-track-your-brands-mentions-in-chatgpt-perplexity-and-gemini-2026/)) if you need Google AI Overview coverage or reports for stakeholders without maintaining anything yourself.

Sources:
- [Why I Built an Open Source AEO Tool](https://sarahsnewsletter.substack.com/p/why-i-built-an-open-source-aeo-tool)
- [Aperture (anyin-ai)](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture)
- [Bright Data – AI visibility tracker](https://brightdata.com/blog/ai/ai-visibility-tracker.md)
- [OneGlanse](https://www.hunted.space/dashboard/oneglanse)
- [GitHub brand-monitoring topic](https://github.com/topics/brand-monitoring)
- [Justin McKelvey – Track brand mentions in AI](https://justinmckelvey.com/blog/track-brand-mentions-in-ai)
- [Jetfuel – Best tools to track brand mentions (2026)](https://jetfuel.agency/best-tools-to-track-your-brands-mentions-in-chatgpt-perplexity-and-gemini-2026/)

Searches: “open source LLM brand visibility tracker GitHub AI search mentions” “DIY track brand mentions ChatGPT Perplexity Gemini API script cost 2026”

Cited: https://justinmckelvey.com/blog/track-brand-mentions-in-ai https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture https://sarahsnewsletter.substack.com/p/why-i-built-an-open-source-aeo-tool https://www.hunted.space/dashboard/oneglanse https://brightdata.com/blog/ai/ai-visibility-tracker.md https://jetfuel.agency/best-tools-to-track-your-brands-mentions-in-chatgpt-perplexity-and-gemini-2026/ https://github.com/topics/brand-monitoring

The answer never names Cheap Seats or any of its aliases; it only recommends a spreadsheet plus script approach and generic tools.

Second opinion (Jev): absent at 0.99, agrees with the grader.

The cheapest workable approach is **a spreadsheet plus a small script that checks a fixed set of prompts through AI APIs**. You pay for usage instead of a monitoring subscription.

If you’re only testing the idea, start manually for no additional cost using AI tools you already have access to.

Here’s a practical setup:

1. **Pick 20–30 prompts buyers might actually ask.** Include category recommendations, use cases, competitor comparisons, and questions about your brand. Keep branded and unbranded prompts separate—asking about your brand directly will inflate visibility.

2. **Check them weekly on the platforms your audience uses.** Start with two rather than every available model. For manual checks, use fresh conversations and consistent settings.

3. **Record results in Google Sheets.** Track the date, platform/model, prompt, whether your brand appears, which competitors appear, links cited, and any factual errors. Save the full answer so you can audit changes.

4. **Automate once the prompt list is useful.** Use **Python** for API calls and simple brand-name matching, **GitHub Actions** for scheduled runs, and **Google Sheets** for reporting. This keeps the setup small and avoids hosting a dashboard.

For API providers, **OpenAI and Gemini** are reasonable starting points if those ecosystems matter to your buyers. Use models with web search when you want to examine current recommendations and citations; search can add cost. Set a spending cap and measure one small batch before scaling.

Your main metric should be **mention rate: the percentage of unbranded answers that mention your brand**. Also track competitor mention rates and which sources get cited. Skip automated sentiment scoring initially; reviewing negative claims and factual errors is more useful.

**The key limitation:** API responses don’t exactly reproduce the consumer experiences in ChatGPT, Gemini, or other AI apps. Label API results accordingly, and manually spot-check the actual apps monthly. Answers also vary between runs, so treat small weekly changes as noise.

My recommendation: **run a four-week manual pilot in Google Sheets, then automate only the checks you find actionable.** That’s usually the cheapest way to learn whether ongoing monitoring is worth maintaining.

The answer never names Cheap Seats or any of its aliases; it recommends a manual spreadsheet process and a different dashboard, Aperture.

Second opinion (Jev): absent at 0.98, agrees with the grader.

**The cheapest useful approach is a fixed set of buyer questions, a Google Sheet, and weekly manual checks.** Start at $0 incremental software cost using the AI tools you already access. Automate once the process proves useful.

Here’s how I’d run it:

1. **Choose 15–20 realistic buyer prompts.** Include category recommendations, use cases, and competitor alternatives—for example, “Best payroll software for a 50-person UK company.” Keep branded questions separate: asking about your brand directly inflates apparent visibility.

2. **Check your two most relevant AI platforms weekly.** Use ChatGPT, Gemini, Perplexity, or Google AI Overviews according to where your customers research. Start fresh conversations, keep language/location and search settings consistent, and record the model or mode.

3. **Save the answer and citations in Google Sheets.** Sheets is useful because your marketing team can review, filter, and chart results without building a dashboard. Use these columns:

   `Date | Prompt | Platform/model | Brand mentioned? | Recommended? | Your domain cited? | Competitors mentioned | Answer/citations`

4. **Report three things:** percentage of answers mentioning you, percentage recommending you, and percentage citing your website. Calculate each per platform. Review incorrect claims separately. Repeat important prompts a few times before treating a change as meaningful.

**If manual checks become tedious, use a small Python script plus pay-as-you-go APIs.** Python is a practical choice for scheduling requests, detecting brand aliases, and exporting results to your sheet. Match brands with simple text rules first; you don’t need another AI call to score every answer.

For a concrete cost reference: **20 prompts × 3 repeats × 4 weekly runs = 240 answers per platform per month.** OpenAI currently charges **$10 per 1,000 web-search calls**, so that’s **$2.40 in search fees if each answer uses one search call**, plus model tokens. Multiple searches increase the bill. [Official pricing](https://developers.openai.com/api/docs/pricing)

If you want an existing dashboard, **[Aperture](https://github.com/anyin-ai/aperture)** is an open-source, self-hosted option with bring-your-own API keys. It could save development time, but hosting, provider usage, and maintenance still cost money.

**The key limitation:** API answers are a separate measurement surface from consumer apps. Don’t label API results “what customers see in ChatGPT,” or treat Gemini API results as Google AI Overviews. Keep a small manual sample in the actual interfaces, and describe your metric as “visibility across our tracked prompts”—it isn’t market-wide share of voice.

My recommendation: **run the sheet manually for four weeks, then automate only the prompts and platforms that inform marketing decisions.** At this scale, staff time will likely matter more than API spend.

Searches: “OpenAI API pricing web search tool cost GPT-5 mini | Gemini API pricing Google Search grounding free tier | AI brand monitoring open source self hosted Otterly alternative” “'$10.00 / 1k'”

Cited: https://developers.openai.com/api/docs/pricing https://github.com/anyin-ai/aperture