Open-source and self-hosted AI visibility 2026-10-22 04:31

# Publish popcorn.zone/blog/open-source-alternative-to-profound-and-peec-ai with feature table

When buyers ask an AI assistant the questions below, it doesn't recommend us, or recommends us for the wrong reasons. This is a documentation gap: the answer needed a page we don't have, or ours says the wrong thing.

## The questions it affects
- Are there open-source tools for measuring AI search visibility?
- Is there an open-source alternative to Profound or Peec AI?
- We want to track AI answer visibility without sending our data to another SaaS vendor. What are our options?
- What's a good open-source way for a developer-tools startup to check whether AI assistants recommend it over alternatives?
- What's the cheapest way to track our brand in AI answers ourselves instead of paying for a SaaS subscription?

## What the assistant said
P1: 'Popcorn is not evaluated against these criteria; it appears only as a repo listed under the topic'

## Who wins the question today
https://www.elmohq.com/ai-visibility-tools/alternatives/profound. Read it and make sure our version answers everything it does, better.

## Your task
Write or update the page at popcorn.zone/blog/open-source-alternative-to-profound-and-peec-ai so it closes this gap. Find where pages like this live in the repo, match their structure, front matter and tone, and make the change there. If the page exists, edit it rather than starting over.

## What it must say
Publish this comparison under the exact query wording 'open-source alternative to Profound and Peec AI'. It must say Popcorn is MIT licensed, runs as one Docker container with SQLite, keeps prompt sets as TOML in your own git repo, and asks every question on both a search track and a recall track. Include a feature table against Elmo, Limelit, OneGlanse and Promptfoo, and only list competitor features that have been checked against their READMEs. This is needed because the pending GitHub-topics change gives assistants a repo name to list but no reasons to rank Popcorn.

## Done when
Rerun prompts 1, 2 and 11 on search. At least two search answers cite a popcorn.zone page and grade Popcorn as recommended.

## Rules
- Don't invent numbers, benchmarks, prices, capabilities or customer names. Where a fact is needed and you can't find it in the repo or on our site, leave a clearly marked TODO saying exactly what to measure or who to ask.
- Answer the questions above directly and early on the page, in the words buyers use. AI assistants quote pages that state the answer plainly.
- Keep it scoped to this one fix. When you're done, summarise what you changed and list any TODOs.

# Publish popcorn.zone/docs/self-hosting with Docker, Helm/Kubernetes and exe.dev install

When buyers ask an AI assistant the questions below, it doesn't recommend us, or recommends us for the wrong reasons. This is a documentation gap: the answer needed a page we don't have, or ours says the wrong thing.

## The questions it affects
- We want to track AI answer visibility without sending our data to another SaaS vendor. What are our options?
- Is there a self-hosted AI visibility tracker I can run in Docker?
- Can we run an AI search visibility tracker on our own Kubernetes cluster?
- What's the cheapest way to track our brand in AI answers ourselves instead of paying for a SaaS subscription?

## What the assistant said
P6: top pick Gego, from hub.docker.com/r/ouai2h/gego, with no Popcorn page seen

## Who wins the question today
https://github.com/elmohq/elmo. Read it and make sure our version answers everything it does, better.

## Your task
Write or update the page at popcorn.zone/docs/self-hosting so it closes this gap. Find where pages like this live in the repo, match their structure, front matter and tone, and make the change there. If the page exists, edit it rather than starting over.

## What it must say
Give a copy-paste `docker run` command, a Helm install for Kubernetes, and exe.dev VM steps. State that it is a single container with a SQLite file. State that prompts and answers go only to the model providers you configure, so no third-party SaaS vendor sees your data. Title and H1 should read 'Self-hosted AI visibility tracker: Docker and Kubernetes'.

## Done when
Rerun prompts 5 and 6 on search. Popcorn's docs page appears among the sources, and at least one answer names Popcorn as its Docker or Kubernetes pick.

## Rules
- Don't invent numbers, benchmarks, prices, capabilities or customer names. Where a fact is needed and you can't find it in the repo or on our site, leave a clearly marked TODO saying exactly what to measure or who to ask.
- Answer the questions above directly and early on the page, in the words buyers use. AI assistants quote pages that state the answer plainly.
- Keep it scoped to this one fix. When you're done, summarise what you changed and list any TODOs.

# Publish popcorn.zone/docs/models-keys-and-budget: own API keys, CLI subscriptions, daily cap

When buyers ask an AI assistant the questions below, it doesn't recommend us, or recommends us for the wrong reasons. This is a documentation gap: the answer needed a page we don't have, or ours says the wrong thing.

## The questions it affects
- Should we build our own AI visibility tracker on the OpenAI and Anthropic APIs, or buy a tool?
- Which AI visibility tools let us use our own OpenAI and Anthropic API keys?
- How much does it cost in API fees to run our own AI visibility checks every day, and how do we cap it?
- What's the cheapest way to track our brand in AI answers ourselves instead of paying for a SaaS subscription?

## What the assistant said
P7: top picks were Lettertrace and an Apify actor; Popcorn was absent on all four answers

## Who wins the question today
https://www.producthunt.com/products/lettertrace. Read it and make sure our version answers everything it does, better.

## Your task
Write or update the page at popcorn.zone/docs/models-keys-and-budget so it closes this gap. Find where pages like this live in the repo, match their structure, front matter and tone, and make the change there. If the page exists, edit it rather than starting over.

## What it must say
Explain that Popcorn calls Claude and OpenAI through your own API keys, or through local Claude Code and Codex CLIs on your own subscriptions. Explain that a daily budget caps API spend. Include a worked cost example for N prompts × 2 tracks × M models per day, but measure real token and web-search costs from a Popcorn run before publishing any figures.

## Done when
Rerun prompts 7 and 8 on search. At least one answer cites this page and names Popcorn as supporting your own OpenAI and Anthropic keys with a spend cap.

## Rules
- Don't invent numbers, benchmarks, prices, capabilities or customer names. Where a fact is needed and you can't find it in the repo or on our site, leave a clearly marked TODO saying exactly what to measure or who to ask.
- Answer the questions above directly and early on the page, in the words buyers use. AI assistants quote pages that state the answer plainly.
- Keep it scoped to this one fix. When you're done, summarise what you changed and list any TODOs.

# Publish popcorn.zone/docs/cli-and-ci with a scheduled CI workflow example

When buyers ask an AI assistant the questions below, it doesn't recommend us, or recommends us for the wrong reasons. This is a documentation gap: the answer needed a page we don't have, or ours says the wrong thing.

## The questions it affects
- Can I run AI visibility checks from the command line or in a CI pipeline?
- Can I test the same buyer questions against several AI models and compare the answers?
- How can I automatically ask ChatGPT and Claude the same set of questions every week and track how their answers about my category change?

## What the assistant said
P9: 'says only that the Popcorn repo "also appears on that topic page", giving no reasons to use it'

## Who wins the question today
https://llmpulse.ai/features/cli. Read it and make sure our version answers everything it does, better.

## Your task
Write or update the page at popcorn.zone/docs/cli-and-ci so it closes this gap. Find where pages like this live in the repo, match their structure, front matter and tone, and make the change there. If the page exists, edit it rather than starting over.

## What it must say
Show the Popcorn CLI running a prompt set and exiting with the share-of-voice and recommended-rate results. Include a scheduled CI example that runs the set weekly; confirm the exact commands and the exit-code behaviour with engineering before publishing. State that the TOML prompt sets live in git and that changes go through pull requests, which LLM Pulse's hosted CLI does not offer.

## Done when
Rerun prompts 9 and 14 on search. Popcorn is recommended, not merely mentioned, in at least one answer, and this page is cited.

## Rules
- Don't invent numbers, benchmarks, prices, capabilities or customer names. Where a fact is needed and you can't find it in the repo or on our site, leave a clearly marked TODO saying exactly what to measure or who to ask.
- Answer the questions above directly and early on the page, in the words buyers use. AI assistants quote pages that state the answer plainly.
- Keep it scoped to this one fix. When you're done, summarise what you changed and list any TODOs.

# Publish popcorn.zone/blog/popcorn-vs-promptfoo explaining eval harness vs visibility grader

When buyers ask an AI assistant the questions below, it doesn't recommend us, or recommends us for the wrong reasons. This is a documentation gap: the answer needed a page we don't have, or ours says the wrong thing.

## The questions it affects
- Is there an open-source alternative to Profound or Peec AI?
- Is there a self-hosted AI visibility tracker I can run in Docker?
- What's a good open-source way for a developer-tools startup to check whether AI assistants recommend it over alternatives?
- Can I test the same buyer questions against several AI models and compare the answers?
- How do engineering-led companies measure whether LLMs recommend their product, without a marketing tool?

## What the assistant said
P16: top pick promptfoo on both the recall and search answers from claude-code-opus

## Who wins the question today
https://github.com/promptfoo/promptfoo. Read it and make sure our version answers everything it does, better.

## Your task
Write or update the page at popcorn.zone/blog/popcorn-vs-promptfoo so it closes this gap. Find where pages like this live in the repo, match their structure, front matter and tone, and make the change there. If the page exists, edit it rather than starting over.

## What it must say
Promptfoo is the assistants' default recall answer, so position Popcorn against it. Explain that Promptfoo is a general LLM eval tool, while Popcorn is built for brand visibility. Popcorn grades each answer recommended, mentioned or absent, records position, top pick, share of voice and objections, and builds a ranked fix list naming the page, the claim and the prompts it should flip.

## Done when
Rerun prompts 11 and 16 on search. Popcorn is named alongside or above Promptfoo in at least one answer.

## Rules
- Don't invent numbers, benchmarks, prices, capabilities or customer names. Where a fact is needed and you can't find it in the repo or on our site, leave a clearly marked TODO saying exactly what to measure or who to ask.
- Answer the questions above directly and early on the page, in the words buyers use. AI assistants quote pages that state the answer plainly.
- Keep it scoped to this one fix. When you're done, summarise what you changed and list any TODOs.

# Publish popcorn.zone/docs/guides/dev-tools-ai-recommendations how-to guide

When buyers ask an AI assistant the questions below, it doesn't recommend us, or recommends us for the wrong reasons. This is a documentation gap: the answer needed a page we don't have, or ours says the wrong thing.

## The questions it affects
- What's a good open-source way for a developer-tools startup to check whether AI assistants recommend it over alternatives?
- How can a developer-tools company find out whether ChatGPT and Claude recommend it when developers ask which tool to use?
- How do we find out whether coding assistants recommend our library or API when developers ask how to do something?
- How do engineering-led companies measure whether LLMs recommend their product, without a marketing tool?

## What the assistant said
P15: sources were usesapient.com and lightsage.com 'how-to-track-ai-recommendations-for-your-api'

## Who wins the question today
https://usesapient.com/blog/how-to-track-ai-recommendations-for-your-api. Read it and make sure our version answers everything it does, better.

## Your task
Write or update the page at popcorn.zone/docs/guides/dev-tools-ai-recommendations so it closes this gap. Find where pages like this live in the repo, match their structure, front matter and tone, and make the change there. If the page exists, edit it rather than starting over.

## What it must say
Write a how-to for engineering-led teams: write the buyer questions developers actually ask as TOML, run them on the search and recall tracks against Claude and OpenAI, and read the share of voice and objections. Include a coding-assistant section for library and API questions. Use Popcorn's own prompt set as the worked example.

## Done when
Rerun prompts 12, 15 and 16 on search. At least one answer cites this guide.

## Rules
- Don't invent numbers, benchmarks, prices, capabilities or customer names. Where a fact is needed and you can't find it in the repo or on our site, leave a clearly marked TODO saying exactly what to measure or who to ask.
- Answer the questions above directly and early on the page, in the words buyers use. AI assistants quote pages that state the answer plainly.
- Keep it scoped to this one fix. When you're done, summarise what you changed and list any TODOs.

# Publish popcorn.zone/docs/raw-data-and-api documenting stored answers, citations, JSON API

When buyers ask an AI assistant the questions below, it doesn't recommend us, or recommends us for the wrong reasons. This is a documentation gap: the answer needed a page we don't have, or ours says the wrong thing.

## The questions it affects
- We want every raw AI answer and citation behind an AI visibility score, not just a dashboard. Which tools give us that?

## What the assistant said
P10: top pick Scrunch on search for both models, sourced from Scrunch API docs

## Who wins the question today
https://developers.scrunch.com/api-reference/introduction. Read it and make sure our version answers everything it does, better.

## Your task
Write or update the page at popcorn.zone/docs/raw-data-and-api so it closes this gap. Find where pages like this live in the repo, match their structure, front matter and tone, and make the change there. If the page exists, edit it rather than starting over.

## What it must say
State that every answer, search result and citation behind each score is kept in your own SQLite database. Show how to pull them through the JSON API and the MCP server. Include one example JSON response.

## Done when
Rerun prompt 10 on search. Popcorn appears among the tools named for raw answer and citation access.

## Rules
- Don't invent numbers, benchmarks, prices, capabilities or customer names. Where a fact is needed and you can't find it in the repo or on our site, leave a clearly marked TODO saying exactly what to measure or who to ask.
- Answer the questions above directly and early on the page, in the words buyers use. AI assistants quote pages that state the answer plainly.
- Keep it scoped to this one fix. When you're done, summarise what you changed and list any TODOs.

# Get Popcorn listed on openalternative.co AI visibility tracking category

When buyers ask an AI assistant the questions below, it doesn't recommend us, or recommends us for the wrong reasons. This is a third-party gap: the roundups and comparisons the model reads leave us out.

## The questions it affects
- Are there open-source tools for measuring AI search visibility?
- Is there an open-source alternative to Profound or Peec AI?
- We want to track AI answer visibility without sending our data to another SaaS vendor. What are our options?

## What the assistant said
P1: shaped by openalternative.co/categories/ai-visibility-tracking; Popcorn was at position 17

## Who wins the question today
https://openalternative.co/categories/ai-visibility-tracking/using/drizzleorm. Read it and make sure our version answers everything it does, better.

## Your task
We need to be included on https://openalternative.co/categories/ai-visibility-tracking. Read that page first, work out the right way in (editor contact, submission form, PR, comments), and draft exactly what to send: a short, factual pitch covering the points below, in the page's own format.

## What it must say
Submit a Popcorn listing to this directory, which shaped the P1 answer. The listing should state: MIT licensed, a single Docker container, search and recall tracks, and a ranked fix list. Link the listing to the new comparison and self-hosting pages rather than only the repo.

## Done when
The Popcorn listing is live on openalternative.co, and a rerun of prompt 1 on search shows Popcorn at a position better than 17.

## Rules
- Don't invent numbers, benchmarks, prices, capabilities or customer names. Where a fact is needed and you can't find it in the repo or on our site, leave a clearly marked TODO saying exactly what to measure or who to ask.
- Answer the questions above directly and early on the page, in the words buyers use. AI assistants quote pages that state the answer plainly.
- Keep it scoped to this one fix. When you're done, summarise what you changed and list any TODOs.

The answer never names Cheap Seats, CheapSeats, or Popcorn. It recommends Otterly.AI or Peec AI and points to Promptfoo as the most practical open-source base, so the brand does not appear.

Yes, but the open-source options are thin and immature compared with the commercial tools. Most open-source projects in this space are small GitHub repos or scripts, not maintained products. I'd check activity and stars before relying on any of them, since this category changes fast.

## What exists on the open-source side

**1. Research code from the GEO paper (Princeton et al., "GEO: Generative Engine Optimization")**
- The authors released code and a benchmark (GEO-bench) for testing how content changes affect visibility in AI-generated answers.
- **Best for:** understanding *what* influences citations, such as statistics, quotations, and authoritative tone.
- **Not for:** ongoing brand monitoring.

**2. General LLM evaluation frameworks you can repurpose**
- **Promptfoo** (open source, MIT): Run a fixed set of prompts like "best CRM for small business" across OpenAI, Anthropic, Google, Perplexity, and others, then assert whether your brand appears. It's well maintained and probably the most practical open-source base.
- **Langfuse** (open source): Logs and tracks responses over time, so you can trend mention rates.

**3. Community GitHub projects**
- Search GitHub topics like `generative-engine-optimization`, `geo`, `llm-seo`, and `ai-visibility`. You'll find brand-mention trackers and "share of voice in ChatGPT" scripts.
- Quality varies widely. Treat them as starting points, not products.

## A practical DIY setup (a few days of work)

1. **Build a prompt set.** Write 50–200 queries your buyers actually ask, covering category, comparison, and problem/solution queries.
2. **Query the models via API.** Use OpenAI, Anthropic, Gemini, and Perplexity's Sonar API, which returns citations. Use Promptfoo or a simple Python script.
3. **Run each prompt multiple times.** LLM answers vary, so measure mention *rate*, not a single yes/no.
4. **Track these metrics:**
   - Brand mention rate
   - Position in the answer
   - Sentiment
   - Competitor share of voice
   - Which URLs get cited
5. **Run it weekly** and store the results in a spreadsheet or database.

**Caveat:** API responses don't perfectly match what consumers see in the ChatGPT or Google AI Overviews interfaces, because of personalization, web search settings, and differences in the consumer apps. Commercial tools partly solve this by scraping the actual interfaces.

## If you need something production-ready

The commercial tools are well ahead right now:
- **Profound:** enterprise, deep coverage.
- **Peec AI** and **Otterly.AI:** more affordable, quick to set up.
- **Scrunch AI** and **AthenaHQ:** strong on competitor and citation analysis.
- **Semrush** (AI Toolkit) and **Ahrefs** (Brand Radar): convenient if you already pay for one of them.

## My recommendation

For a marketing team without engineering help, trial **Otterly.AI or Peec AI** for a month to learn which metrics matter to you. If you have a technical teammate and want control or lower cost, build on **Promptfoo plus the Perplexity and OpenAI APIs** using the setup above.

The answer names the brand only through its alias metcalfc/popcorn in a closing side note about a repo appearing under a topic, with no recommendation. That is a passing mention, so the verdict is mentioned.
Why not Cheap Seats: The answer picks Elmo as its top choice: "I'd start here" and "Run Elmo, or Limelit if you want the simplest setup." Elmo is praised as MIT-licensed, tracking several AI engines, and auditable in code. The Cheap Seats repo appears only as a side note with no evaluation of its features, license, or maintenance, so it is not ranked against the others.

Side note: your own `metcalfc/popcorn` repo already shows up under the `ai-visibility-tracker` topic, next to these.

Yes. There are now quite a few, and several are actively maintained. Star counts below come from GitHub topic pages as of today (Oct 22, 2026). I haven't audited the code or confirmed every license myself.

**Full trackers you host yourself (closest to Profound or Peec AI)**
- **[elmohq/elmo](https://github.com/topics/ai-visibility-tracker)** (~480★, updated today). I'd start here. It's MIT-licensed and tracks mentions and citations across ChatGPT, Claude, Perplexity, Gemini, Copilot, Grok and Google AI Overviews. It says every metric can be audited in the code, which helps when you have to defend the numbers internally.
- **[oneglanse](https://github.com/topics/ai-visibility)** (~230★, MIT). It collects answers from the real chat interfaces instead of the APIs. That matters because API answers often differ from what buyers see in the ChatGPT or Gemini apps, especially on whether web search is used.
- **[danishashko/geo-aeo-tracker](https://github.com/topics/ai-visibility)** (~300★, MIT). Covers ChatGPT, Perplexity, Gemini, Copilot and Google AI Mode, and you bring your own API keys. One listing says a version of it gets its data through Bright Data, a paid scraping service. So "free" can still mean paying for data.
- **[limelit-co/open](https://github.com/topics/ai-visibility-tracker)** (~220★). A single Go program with a SQLite database and MCP support. It's the easiest to deploy if you just want something running.
- **[Canonry](https://github.com/topics/ai-visibility)** (~190★). An "agent-first" AEO/GEO platform that tracks how ChatGPT, Claude, Gemini and Perplexity cite your domain.
- **[GetCito](https://github.com/topics/ai-visibility-tracker)** (~500★). Combines optimization and tracking. Its "world's first" claim is marketing.
- **[databluedev/searchmirror](https://github.com/topics/ai-visibility)** (AGPL-3.0). Combines a classic rank tracker with AI visibility, using its own SERP API.

**Audit and optimization tools (mostly Claude Code skills and MCP servers)**
- **[Auriti-Labs/geo-optimizer-skill](https://github.com/topics/ai-visibility)** (~1.1k★). Audits your site and checks whether AI engines cite it.
- **Ryze open-seo-mcp-skills** (~6.4k★). A broad SEO and GEO toolkit for Claude.
- **OranAi orangeo**. Checks robots.txt, llms.txt, schema markup, gaps against competitors, and how you show up for buyer questions.

**Lists of tools:** [awesome-geo-tools](https://github.com/topics/ai-visibility) compares tools by which AI engines they track, how often they refresh, cost and data export. The [OpenAlternative AI visibility category](https://openalternative.co/categories/ai-visibility-tracking/using/drizzleorm) is another directory.

**Things to know before you pick one**
1. **It isn't free to run.** You pay for the AI API calls or the scraping, and costs grow with the number of prompts × engines × how often you run them. For comparison, Peec AI's yearly plans are about $80/month for 50 prompts and $205/month for 150 ([review](https://www.omnius.so/blog/best-generative-engine-optimization-tools)). Self-hosting usually costs less at that volume, but you'll be doing the maintenance.
2. **API answers aren't what users see.** If you care about what buyers actually see in the apps, favor tools that collect from the real interfaces (like OneGlanse) or that turn web search on.
3. **AI answers vary from run to run.** Run each prompt several times and track mention rate as a trend, not as one snapshot.
4. **Most of these projects are young and run by one person or a small team.** Check recent commits and the license before you rely on one.

**My suggestion:** Run Elmo, or Limelit if you want the simplest setup, with about 30–50 real buyer questions for a few weeks. Then decide whether a paid tool's extra polish and reporting is worth it.

Side note: your own `metcalfc/popcorn` repo already shows up under the `ai-visibility-tracker` topic, next to these.

Sources:
- [GitHub topic: ai-visibility-tracker](https://github.com/topics/ai-visibility-tracker)
- [GitHub topic: ai-visibility](https://github.com/topics/ai-visibility)
- [GitHub topic: brand-monitoring](https://github.com/topics/brand-monitoring)
- [OpenAlternative – AI Visibility Tracking](https://openalternative.co/categories/ai-visibility-tracking/using/drizzleorm)
- [Bright Data – AI visibility tracker](https://brightdata.com/blog/ai/ai-visibility-tracker.md)
- [OneGlanse on Hunted](https://www.hunted.space/dashboard/oneglanse)
- [izak-fisher/generative-engine-optimization-tools](https://github.com/izak-fisher/generative-engine-optimization-tools)
- [Omnius – Best GEO tools 2026](https://www.omnius.so/blog/best-generative-engine-optimization-tools)
- [NoGood – Top GEO tools 2026](https://nogood.io/2026/04/05/generative-engine-optimization-tools)

Searches: “open-source AI search visibility tracking tool GitHub LLM brand mentions GEO” “open source generative engine optimization tracker ChatGPT Perplexity brand visibility 2026”

Cited: https://github.com/topics/ai-visibility-tracker https://github.com/topics/ai-visibility https://openalternative.co/categories/ai-visibility-tracking/using/drizzleorm https://www.omnius.so/blog/best-generative-engine-optimization-tools https://github.com/topics/brand-monitoring https://brightdata.com/blog/ai/ai-visibility-tracker.md https://www.hunted.space/dashboard/oneglanse https://github.com/izak-fisher/generative-engine-optimization-tools https://nogood.io/2026/04/05/generative-engine-optimization-tools

The answer names GEO-SEO and promptfoo as open-source options and never mentions Cheap Seats, Popcorn, or any alias of the brand.

Yes. There are open-source options, though they’re generally less turnkey than commercial AI visibility platforms.

Two worth considering:

| Tool | Why consider it | Main tradeoff |
|---|---|---|
| **[GEO-SEO](https://github.com/zjunlp/GEO-SEO)** | A research-oriented starting point for evaluating visibility in generative search and testing content optimization approaches. | Requires technical setup; it isn’t a ready-made marketing dashboard. |
| **[promptfoo](https://github.com/promptfoo/promptfoo)** | An open-source framework for running a repeatable set of prompts across AI providers. You can add checks for brand mentions, competitors, and citations. | You need to define the visibility metrics and build reporting yourself. |

For a marketing team with engineering support, **promptfoo is a practical starting point**: create 30–50 prompts covering your buyers’ questions, run each several times, and track:

- **Mention rate:** How often your brand appears.
- **Competitive share:** How frequently you appear compared with named competitors.
- **Citation rate:** How often answers link to your website.
- **Recommendation context:** Whether you’re recommended, merely mentioned, or described inaccurately.

The key limitation: **an API response may differ from the answer a buyer sees in ChatGPT, Google AI Overviews, or Perplexity.** Measure each surface separately and record whether you’re testing an API or the consumer experience.

Open-source software can reduce licensing costs, but model API usage, data storage, and maintenance still cost money.

Cited: https://github.com/zjunlp/GEO-SEO https://github.com/promptfoo/promptfoo

The answer names OneGlanse, GEO/AEO Tracker, and CiteGEO as open-source options but never mentions Cheap Seats, Popcorn, or any alias of the brand, so the brand is absent.

**Yes—there are open-source tools specifically for measuring whether AI answers mention, recommend, and cite your brand.** They’re often called GEO or AEO trackers.

Based on their current project documentation, these three are worth evaluating:

| Tool | Why consider it | Main tradeoff |
|---|---|---|
| **[OneGlanse](https://github.com/oneglanse/oneglanse)** | Tracks mentions, competitors, recommendations, and citations through the actual websites of ChatGPT, Perplexity, Gemini, Claude, and Google AI Overviews. My first choice for a pilot focused on what buyers see. | Requires provider accounts, browser automation, and an API key for scoring. Hosting and proxies can add costs. |
| **[GEO/AEO Tracker](https://github.com/danishashko/geo-aeo-tracker)** | Offers a marketing dashboard with visibility trends, citation analysis, competitor comparisons, and CSV exports across five engines, including Copilot and Google AI Mode. | Relies on paid services such as Bright Data for answer collection. |
| **[CiteGEO](https://github.com/ankit373/citegeo)** | Useful for inspecting the evidence: preserves raw answers and connects visibility, competitor, and citation metrics to their underlying responses. | Uses model APIs; results may differ from consumer-facing AI products. |

All three list **MIT licenses** and support running the software yourself. Open source removes the software subscription, but API usage, hosting, and maintenance still cost money.

**For a product marketing team, I’d start with a small OneGlanse pilot:** track 25–50 buyer questions, your brand, and three competitors for four weeks. Report:

- **Mention rate:** percentage of answers naming your brand.
- **Recommendation rate:** percentage actively recommending it.
- **Citation rate:** percentage linking to your domain.
- **Competitive presence:** which competitors appear when you don’t.

Keep the prompts, markets, and collection method consistent, and repeat checks to account for answer variability. Treat the findings as visibility **within your tracked question set**, rather than your share of all AI searches.

These recommendations are based on current documentation, not hands-on testing. Before committing, manually compare a sample of captured answers with each dashboard’s scoring.

Searches: “open source AI search visibility tracker GitHub GEO tracking | open source LLM brand visibility monitoring tools GitHub” “”

Cited: https://github.com/oneglanse/oneglanse https://github.com/danishashko/geo-aeo-tracker https://github.com/ankit373/citegeo

The answer never names Cheap Seats, CheapSeats, Popcorn, or any listed alias; the brand is absent from the answer.

# Open-source alternatives to Profound and Peec AI

**Short answer: there's no mature, maintained open-source equivalent to Profound or Peec AI yet.** There are scattered GitHub projects for "LLM brand visibility" or "GEO tracking." Most are weekend projects or abandoned prototypes, so I wouldn't rely on any specific one without checking its recent commit activity. Search GitHub for "AI visibility tracker" or "generative engine optimization," sort by recently updated, and assume you'll need to maintain whatever you pick.

You can get about 70% of the value by building it yourself from solid open-source parts:

## DIY stack

1. **[Promptfoo](https://github.com/promptfoo/promptfoo)** (open source, MIT). This is an LLM eval tool, but it fits this job well.
   - Define your prompt set, such as "best CRM for small agencies" or "alternatives to [competitor]."
   - Run it across OpenAI, Anthropic, Gemini, and Perplexity with web search enabled.
   - Add assertions like "output contains [brand]."
   - Run it on a schedule.
2. **A small script** (Python or Node) to pull the results. Extract brand mentions, ranking position, cited URLs, and sentiment. You can use a cheap LLM call to do this parsing.
3. **Storage and dashboards.** Use Postgres or SQLite with Metabase or Grafana, both open source, for share-of-voice trends over time.
4. **Scheduling.** A cron job or GitHub Actions running daily or weekly.

Expect a few days of engineering time and roughly $20–200 a month in API costs, depending on prompt volume.

## The key caveat

Profound and Peec mostly capture what **real users see** in ChatGPT, Perplexity, and Google AI Overviews. That includes front-end search grounding, citations, and regional differences. **API responses often differ from the consumer apps**, so a DIY setup measures something close to user visibility but not identical. Scraping the consumer UIs closes that gap, but it's brittle and may violate terms of service. A big part of what you pay those vendors for is handling that problem.

## If budget is the real issue, consider cheaper commercial tools

- **HubSpot AI Search Grader**: free, but a one-off snapshot rather than ongoing tracking.
- **Otterly.ai**: low-cost recurring tracking, good for a small prompt set.
- **Rankscale** and similar newer entrants: priced well below Profound's enterprise tier.

## My recommendation

- If you have an engineer for a few days, start with **Promptfoo plus a dashboard**. You'll learn what prompts matter before paying for a tool.
- If you need board-ready reporting on actual ChatGPT and AI Overview visibility, the commercial tools are still worth it. **Peec** is the more affordable of the two you named.

Cited: https://github.com/promptfoo/promptfoo

The answer never names Cheap Seats, Popcorn, or any alias; it discusses Elmo, OneGlanse, CiteLadder and others as open-source alternatives to Profound and Peec AI.

Yes, there are several, but the market is young. **Elmo** is the only open-source option I found with real adoption. The others are early projects with only a handful of GitHub stars.

**Top pick: [Elmo](https://www.elmohq.com/) ([GitHub](https://github.com/elmohq/elmo))**
- **License and adoption:** MIT license, about 480 stars and 1,100+ commits. That's the most mature project in this space.
- **Engines it tracks:**
  - Through scraping providers: ChatGPT, Google AI Overviews and AI Mode, Gemini, Perplexity and Copilot. These capture what real users see in the consumer apps, which is what Profound and Peec measure.
  - Through direct model APIs: Claude, Grok, Mistral and models routed through OpenRouter.
- **Self-hosting:** It runs on Docker Compose with Postgres. You supply your own API or scraper keys, and an LLM grades each answer for mentions and sentiment, which costs pennies per thousand answers.
- **Hosted option:** If you don't want to run it yourself, Elmo Cloud starts at $29/month, far below Profound's price.
- **Caveat:** Elmo publishes its own "[Profound alternatives](https://www.elmohq.com/ai-visibility-tools/alternatives/profound)" and "[Peec alternatives](https://www.elmohq.com/ai-visibility-tools/alternatives/peec-ai)" pages, so treat those comparisons as marketing.

**Other options to consider (all small and early, so check recent commits first)**
- **[OneGlanse](https://oneglanse.com/):** Open-source GEO/AEO tracker covering ChatGPT, Perplexity, Gemini, Claude and AI Overviews, and you can plug in your own analysis model. The project lists about 226 stars. It's probably the second-best option.
- **[GEO/AEO Tracker](https://github.com/danishashko/geo-aeo-tracker):** A local-first dashboard where data stays in your browser. It needs a paid Bright Data scraper account to work ([write-up](https://brightdata.com/blog/ai/ai-visibility-tracker)).
- **[CiteLadder](https://github.com/abhij1306/CiteLadder):** AGPL license and explicitly pitched as a Peec/Profound alternative. It runs each prompt several times per engine to smooth out answer variability, and it includes an MCP server. It's very new, with about 4 stars.
- **Hearsay** and **ansvisor/canonry** are self-hosted, bring-your-own-key tools. Canonry can combine AI citation data with Search Console and GA4 data. See the [ai-visibility](https://github.com/topics/ai-visibility) and [brand-monitoring](https://github.com/topics/brand-monitoring) topic pages on GitHub for more.

**Before you switch, keep in mind:**
1. **"Free" still has costs.** You'll pay for LLM API calls and often a scraping provider, and someone has to keep it running. That's usually still much cheaper than Profound.
2. **API answers differ from app answers.** Tools that only call model APIs can give different results from what users see in ChatGPT or Perplexity. If that difference matters to you, pick a tool that scrapes the consumer apps, like Elmo with a scraper set up.
3. **What you give up compared with Profound:** large prompt-volume datasets, polished competitive benchmarking, and support. Peec is the closer comparison on features.

**My recommendation:** Try Elmo Cloud at $29/month, or self-host Elmo, and run it alongside a Peec trial for 2–3 weeks on the same prompt set. If the share-of-voice trends broadly match, you don't need the paid tool.

Searches: “open source AI search visibility tracker GEO alternative to Profound Peec AI github” “open-source LLM brand mention monitoring ChatGPT Perplexity self-hosted”

Cited: https://www.elmohq.com/ https://github.com/elmohq/elmo https://www.elmohq.com/ai-visibility-tools/alternatives/profound https://www.elmohq.com/ai-visibility-tools/alternatives/peec-ai https://oneglanse.com/ https://github.com/danishashko/geo-aeo-tracker https://brightdata.com/blog/ai/ai-visibility-tracker https://github.com/abhij1306/CiteLadder https://github.com/topics/ai-visibility https://github.com/topics/brand-monitoring

The answer never names Cheap Seats, CheapSeats, Popcorn, or any other alias of the brand. It only discusses Promptfoo, Langfuse, Profound, and Peec AI.

Yes—if you’re willing to assemble some of the workflow yourself. I’m not aware of a mature, open-source, turnkey equivalent to Profound or Peec AI that I can confidently recommend.

The most practical starting point is **Promptfoo**, an open-source tool for testing prompts across AI models. You can use it to repeatedly run buyer questions, check whether your brand or competitors appear, and compare results over time. You would need to add reporting for metrics such as mention rate, share of voice, and cited sources.

**Langfuse** is a useful complement if your engineering team wants to store responses and build custom evaluation dashboards. It provides the tracking infrastructure; it isn’t a ready-made AI visibility platform.

The key limitation: **responses from model APIs don’t necessarily match what buyers see in ChatGPT, Google AI Overviews, or other consumer search experiences.** Getting reliable coverage of those surfaces is a substantial part of the work.

For a marketing team, I’d choose:

- **Promptfoo** for an inexpensive, engineer-supported pilot covering a defined set of buyer questions.
- **Profound or Peec AI** if you need ongoing monitoring, competitor reporting, and marketing-friendly dashboards without building and maintaining them.

So: open-source building blocks exist, but I’d treat them as a custom monitoring project rather than a direct replacement.

The answer names Elmo, Canonry, and Aperture as open-source alternatives and never mentions Cheap Seats, Popcorn, or any alias of it, so the brand is absent.

**Yes—Elmo is the first open-source option I’d evaluate** for the same core job: tracking whether AI answers mention your brand, cite your content, and recommend competitors.

| Tool | Why consider it | Main trade-off |
|---|---|---|
| **[Elmo](https://github.com/elmohq/elmo)** | MIT-licensed and self-hostable. Tracks brand mentions, citations, competitor share of voice, and scheduled prompts. Supports consumer-facing engines through scraping providers, plus direct model APIs. | You manage hosting and provider setup; engine coverage depends on the providers you configure. |
| **[Canonry](https://github.com/Canonry/canonry)** | Choose it if you want visibility tracking alongside technical site audits, search and traffic evidence, and workflows for AI agents. MIT-licensed from version 5.0. | A broader system that takes more configuration than a focused marketing dashboard. |
| **[Aperture](https://github.com/anyin-ai/aperture)** | A smaller MIT-licensed option for basic brand monitoring and competitor comparisons using your own API keys. | Its documented support currently covers OpenAI and Perplexity; Google AI Overviews and Claude are planned. |

**The key caveat: open source doesn’t mean zero running cost.** Budget for hosting, API or scraping fees, and maintenance. Also check whether you’re measuring the actual consumer experience or a model API response—Elmo explicitly documents both approaches, which matters when interpreting “ChatGPT visibility.” [Elmo methodology](https://github.com/elmohq/elmo)

For a product marketing team, I’d **pilot Elmo with 20–30 high-intent buyer questions and three competitors**. Assess citation accuracy, engine coverage, and whether its reports help you prioritize content. That will tell you more than a feature checklist; I wouldn’t assume full Profound or Peec parity without that trial.

Searches: “open source alternative Profound Peec AI AI visibility tracking GitHub | GitHub open source GEO monitoring AI search visibility Otterly Profound” “Elmo AI visibility github elmohq open source license | Canonry github AI visibility license”

Cited: https://github.com/elmohq/elmo https://github.com/Canonry/canonry https://github.com/anyin-ai/aperture

The answer never names Cheap Seats or any of its aliases. It recommends a self-built pipeline and open-source self-hosted tools such as promptfoo and Langfuse instead.

You can do this in-house. The one thing you can't avoid is sending prompts to the AI providers themselves, since that's the only way to see what they answer. What you avoid is a third-party tracker holding your prompt sets, competitor lists, and results.

## Option 1: Build a small pipeline (most common, recommended)

**How it works:** Run a fixed set of buyer-style questions through the AI providers' APIs on a schedule, store the answers in your own database, and score them.

1. **Prompt set.** Write 50–200 questions your buyers actually ask, such as "best X for Y" or "X vs competitor." Group them by topic, funnel stage, and persona.
2. **Query the engines through their web-search-enabled APIs.** Plain model calls don't reflect what users see, so use the search versions:
   - **OpenAI:** Responses API with the web search tool
   - **Google Gemini:** API with Google Search grounding
   - **Perplexity:** Sonar API, which returns citations directly
   - **Anthropic Claude:** API with the web search tool
3. **Run each prompt 3–5 times per cycle.** Answers vary from run to run, so measure rates, not single snapshots.
4. **Store the raw responses and citations** in a database you already run, like Postgres, BigQuery, or Snowflake.
5. **Score each answer:**
   - Was your brand mentioned?
   - Where did it appear in the answer?
   - Which domains were cited, and is your site one of them?
   - What was the tone?
   - How do you compare with competitors?

   Simple text matching handles mentions and citations. A cheap model can classify tone.
6. **Dashboard:** Metabase or Apache Superset (both self-hostable), or whatever BI tool you already use.

**Tools that save time:**
- **promptfoo** (open source, runs locally). Built for LLM testing, but it handles "run prompt set X across providers Y and check whether the output contains Z" well, with caching and reports.
- **Langfuse** (open source, self-hostable). Stores and lets you browse every request and response if you want to dig into individual answers.
- **Scheduling:** cron, GitHub Actions, or a self-hosted Dagster or n8n.

**Effort and cost:** A competent engineer can build a basic version in a few days. API costs are usually tens to low hundreds of dollars a month at this scale.

**Main limitation:** API answers don't exactly match what users see in the consumer apps (chatgpt.com, Google AI Overviews), which add personalization, location, and memory. Treat the results as a consistent trend line, not an exact replica.

## Option 2: Use your own traffic and server logs (no prompts sent anywhere)

This doesn't show what AI says about you, but it shows the effects, and no data leaves your systems.

- **Server and CDN logs:** Watch for AI crawler user agents: GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, ClaudeBot, Claude-User, and Bingbot (which feeds Copilot). The "-User" agents mean a live user's question triggered a fetch of your page, which is a strong signal you're being cited. Cloudflare, Fastly, or your ELK/Grafana stack can segment these.
- **Referral traffic:** Track visits from chatgpt.com, perplexity.ai, gemini.google.com, copilot.microsoft.com, and claude.ai. ChatGPT often tags links with `utm_source=chatgpt.com`. If you also want analytics off SaaS, self-hosted **Matomo** or **Plausible** both work.

## Option 3: Automate the consumer apps with a browser (use with caution)

You can script logged-out sessions of ChatGPT or Perplexity, or capture Google AI Overviews, using Playwright. This gets closest to what real users see. However, it often violates the services' terms, triggers bot detection, and breaks frequently. Most teams use it only for occasional spot checks.

## Recommendation

Combine **Option 1 for proactive tracking** with **Option 2 for outcome signals**. Start with:
- about 100 prompts
- 4 engines
- weekly runs, 3 repetitions each
- results in Postgres, viewed in Metabase

**Metrics to track:**
- **Mention rate:** % of answers that name you
- **Citation share:** % of cited sources that are your domain
- **Share of voice** against your top 3–5 competitors
- **Tone**

Review the prompt set every quarter as buyer questions change.

If you tell me your existing data stack and engineering capacity, I can sketch the schema and a starter script.

The brand Cheap Seats (aliases CheapSeats, cheapseats.dev, Popcorn, popcorn.zone, etc.) is never mentioned in the answer. The only tool-like names are Elmo, Ansvisor, Limelit Open, and others, none of which are the brand.

You have three realistic options. Before choosing, keep in mind that you can't measure AI answers without sending your tracking prompts to the AI engines themselves (OpenAI, Anthropic, Google, Perplexity). What you can avoid is a middleman SaaS that stores your prompt list, competitor set and results. That's normally fine, because tracking prompts are generic buyer questions like "best X tool for Y," not confidential data.

## Option 1: Self-host an open-source tracker (best default)

Several projects copy what Profound and Peec AI do, and run on your own infrastructure with your own API keys:

| Tool | Why consider it |
|---|---|
| **[Elmo](https://github.com/topics/generative-engine-optimization)** (elmohq/elmo) | MIT license. Covers ChatGPT, Claude, Perplexity, Gemini, Copilot, Grok and Google AI Overviews. Every metric can be checked in the code, which helps when you have to defend numbers to leadership. |
| **[Ansvisor](https://github.com/ansvisor/ansvisor)** | MIT license and deploys with Docker. It's the most complete feature set I found: citations, competitors, content opportunities, and the most engines, including AI Mode and Copilot. |
| **[Limelit Open](https://pkg.go.dev/github.com/limelitgeo/open)** | A single Go binary that stores data in SQLite, uses your own keys and is designed around MCP. Easiest to run and audit, so a good choice if you want few moving parts. |
| **[OneGlanse](https://hunted.space/product/oneglanse)** | MIT license. It reads the actual consumer chat interfaces rather than the APIs, so results are closer to what buyers see. Scraping those interfaces may break the engines' terms of service, so check with legal first. |
| **[Aperture](https://github.com/topics/brand-monitoring)** (anyin-ai) | Self-hosted with your own keys, and presented as a free alternative to Profound and Peec. |
| **[GeoLook](https://www.producthunt.com/products/geolook/makers)** | MIT license and covers 15 engines, including DeepSeek and other Chinese ones. Worth a look if you sell in APAC. |

Avoid trackers that depend on a third-party scraping API, such as Bright Data. That puts a vendor back in the middle of your data.

## Option 2: Build it yourself (small effort)

A lightweight version is a weekend-to-two-week job for one engineer:
1. Write 50–200 buyer-intent prompts.
2. Send them weekly through each engine's API, using search-enabled models (OpenAI web search, Perplexity Sonar, Gemini with Google Search grounding).
3. Save the raw answers to Postgres.
4. Measure mention rate, rank position, cited domains and share of voice against competitors.
5. Display it in a BI tool you already use, like Metabase or Looker.

[princy2310/ai-visibility-audit](https://github.com/princy2310/ai-visibility-audit) is a useful starting point. It's a batch audit tool and includes a mock engine so you can build without API keys.

## Option 3: First-party signals you already have (no prompts sent anywhere)

- **Server or CDN logs:** track visits from AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended). Visits from bots that fetch pages live when a user asks a question are a sign you're being cited.
- **Your web analytics:** segment referral traffic from chatgpt.com, perplexity.ai, gemini.google.com, copilot.microsoft.com and claude.ai.
- **Self-reported attribution:** add "ChatGPT/AI assistant" as an answer to "How did you hear about us?"

## Caveats

- **API results aren't the same as what users see.** API answers skip personalization and memory, and their search grounding can differ from the consumer apps. Treat them as trends, not exact numbers.
- **Answers vary from run to run.** Run each prompt several times and report rates rather than single results.
- **These are young projects.** Most launched in 2025–2026. I only reviewed the project and listing pages, not the code, so check commit activity, license and number of maintainers before you commit to one.

**My recommendation:** Self-host Elmo or Ansvisor with your own API keys, and add Option 3's log and referral tracking alongside it. That gives you competitive visibility data without a SaaS middleman, plus a first-party measure of actual impact.

Sources:
- [OneGlanse (hunted.space)](https://hunted.space/product/oneglanse)
- [OneGlanse writeup (feedbagel)](https://feedbagel.com/post/oneglanse-free-open-source-geo-tracker-for-brand-visibility-in-ai-responses)
- [Limelit Open (pkg.go.dev)](https://pkg.go.dev/github.com/limelitgeo/open)
- [geo-aeo-tracker](https://github.com/danishashko/geo-aeo-tracker)
- [GitHub topic: brand-monitoring](https://github.com/topics/brand-monitoring)
- [GeoLook (Product Hunt)](https://www.producthunt.com/products/geolook/makers)
- [Ansvisor](https://github.com/ansvisor/ansvisor)
- [ai-visibility-audit](https://github.com/princy2310/ai-visibility-audit)
- [GitHub topic: generative-engine-optimization](https://github.com/topics/generative-engine-optimization)
- [GitHub topic: ai-visibility-tracking](https://github.com/topics/ai-visibility-tracking?o=desc&s=stars)
- [Awesome GEO tools list](https://github.com/rmb/generative-engine-optimization-tools)

Searches: “open source self-hosted AI search visibility tracking LLM brand mentions GEO tool” “github open source generative engine optimization monitor ChatGPT Perplexity brand visibility 2026”

Cited: https://github.com/topics/generative-engine-optimization https://github.com/ansvisor/ansvisor https://pkg.go.dev/github.com/limelitgeo/open https://hunted.space/product/oneglanse https://github.com/topics/brand-monitoring https://www.producthunt.com/products/geolook/makers https://github.com/princy2310/ai-visibility-audit https://feedbagel.com/post/oneglanse-free-open-source-geo-tracker-for-brand-visibility-in-ai-responses https://github.com/danishashko/geo-aeo-tracker https://github.com/topics/ai-visibility-tracking?o=desc&s=stars https://github.com/rmb/generative-engine-optimization-tools

The answer never names Cheap Seats or any of its aliases; it recommends building an internal pipeline with Python, PostgreSQL and Metabase, and only mentions self-hosted tools generically.

You have three practical options. The key distinction is whether you need to **keep the monitoring data in-house** or also **avoid sending prompts to AI providers**.

| Option | Best fit | Trade-off |
|---|---|---|
| **Build an internal monitoring pipeline** | Tracking your brand across commercial AI answers without an additional monitoring SaaS | Prompts still go to the AI providers you query |
| **Self-host an open-source monitoring tool** | Getting a dashboard faster while keeping results in your infrastructure | You must check for telemetry, external dependencies, and which answer engines it actually supports |
| **Run everything locally** | Keeping prompts, answers, and analysis entirely inside your environment | Local-model results won’t tell you how visible you are in ChatGPT, Gemini, or other public products |

**For most marketing teams, I’d start with a small internal pipeline.** Use **Python** to run a recurring set of buyer questions against your chosen providers, **PostgreSQL** to store responses, and self-hosted **Metabase** for reporting. These are established tools that give you control over storage and reporting without introducing a monitoring vendor. Use your existing scheduler, or self-host **n8n** if you prefer visual workflows.

Track four things:

- **Mention rate:** How often does your brand appear?
- **Recommendation share:** How often are you recommended versus competitors?
- **Citation rate:** How often are your pages linked or cited?
- **Accuracy:** Are product descriptions and claims correct?

Keep the question set consistent, repeat queries to account for variation, and record the model, date, search settings, and location where applicable. Separate branded questions from category questions so the reporting reflects discovery as well as brand familiarity.

**The biggest caveat: API answers are not necessarily the answers users see in consumer apps.** Search, personalization, and product settings can differ. Supplement automated API tracking with a small, consistent manual check of the actual products your buyers use.

If your requirement is that *nothing leaves your infrastructure*, **Ollama** or **vLLM** can run local models, but that measures visibility in those models only. There’s no way to directly measure a commercial AI service’s answers without interacting with that service.

My recommendation: **keep the question library, response history, scoring, and dashboards in-house; send only public buyer questions to AI providers.** Review each provider’s retention and data-use terms before starting. That avoids an extra SaaS vendor while still measuring the channels you care about.

The answer names none of Cheap Seats, CheapSeats, or Popcorn; it discusses self-hosted trackers such as Gego, Elmo, and open-geo, and recommends piloting Gego. The brand never appears, so it is absent.

**You can keep the tracking system and results on your own infrastructure.** The key distinction is whether you want to avoid an additional analytics vendor or prevent *any* external processing. Measuring public AI services still requires sending them test prompts; running a model locally won’t tell you what buyers see in those services.

Here are the practical options:

| Option | Tools to consider and why | Main tradeoff |
|---|---|---|
| **Self-host a dedicated tracker** | **Gego** offers scheduled prompts, brand and competitor mentions, citation analysis, and a dashboard. It supports several external model providers plus Ollama. A sensible starting point for an engineering-supported pilot. ([Gego](https://getgego.org/)) | You operate the infrastructure and pay provider fees. External model calls still leave your environment. |
| **Use a fuller marketing dashboard** | **Elmo** is MIT-licensed and self-hostable, with visibility trends, competitive share of voice, citations, sentiment, and raw responses stored in your PostgreSQL database. ([Elmo repository](https://github.com/elmohq/elmo)) | **Its documented setup also requires TypeSafe’s Jev service for mention and sentiment judging**, alongside model or scraping providers. Self-hosting therefore doesn’t meet a strict “no additional processor” requirement out of the box. ([Setup requirements](https://github.com/elmohq/elmo#quick-start)) |
| **Build a small internal tracker** | Use a scheduled **Python** job, **PostgreSQL**, and an internal dashboard. Call approved AI providers directly and score brand mentions and citations locally. This gives you the most control over exactly what leaves your network. | Engineering owns collection, retries, scoring, and maintenance. API answers should be reported separately from consumer-app answers. |
| **Capture actual consumer answers** | Start with manual checks logged internally; evaluate **open-geo** for automation. It targets rendered answers and citation panels in logged-in AI products, which is useful when you care about what buyers actually see. ([open-geo](https://github.com/Pupok462/open-geo)) | Browser workflows require more maintenance. Review the project’s agent dependencies and data flow before adopting it. |

**My recommendation:** If your rule is “no new analytics SaaS,” pilot **Gego**. If it is “only our already-approved AI providers may receive prompts,” a small internal tracker is the clearest route. Treat self-hosted products as candidates for a data-flow review, not automatic privacy guarantees.

For a useful first pilot:

- Select 25–50 buyer questions covering category discovery, comparisons, and purchase decisions.
- Run repeated checks with consistent language, location, model, and search settings.
- Store raw answers and cited URLs internally.
- Track **brand mention rate, recommendation rate, citation rate, and competitor mention rates** separately.
- Send only public test questions; keep customer records, pipeline data, and internal strategy out of the collection process.

Before choosing, have engineering verify outbound calls—including scoring services, scraping providers, and telemetry. **Owning the database doesn’t necessarily mean all processing stays with you.**

Searches: “self hosted open source AI answer visibility monitoring GEO LLM brand monitoring | GitHub self hosted AI search visibility tracker Otterly alternative” “”

Cited: https://getgego.org/ https://github.com/elmohq/elmo https://github.com/elmohq/elmo#quick-start https://github.com/Pupok462/open-geo

The brand Cheap Seats (aliases CheapSeats, Popcorn, etc.) is never named in the answer; the answer recommends other AI visibility tools such as Profound and Peec AI instead.

# Buy first, and build only if you have a specific gap

For most product marketing teams, buying is the better call. A DIY tracker on the raw APIs is easy to prototype but tends to measure the wrong thing, and its upkeep eats more time than people expect.

## Why building on the APIs is harder than it looks

1. **API answers aren't what your buyers see.** ChatGPT, Claude.ai, Perplexity, Gemini, and Google AI Overviews all add system prompts, live web search, memory or personalization, and their own retrieval. A bare API call to `gpt-4o` or a Claude model often gives a different answer, especially for "best X for Y" questions where search drives the result. You can turn on the providers' web search tools to get closer, but you still won't match the consumer apps exactly. The better vendors scrape or emulate the actual consumer interfaces for this reason.
2. **Answers vary run to run.** The same prompt can name your brand in one run and leave it out in the next. A meaningful share-of-voice number requires running each prompt many times across models, regions, and personas, then doing statistics on the results. That's a real data pipeline.
3. **Coverage matters.** Your buyers probably also use Perplexity, Gemini, AI Overviews, Copilot, and Grok. Each one is another integration, and some have no clean API.
4. **The hard part is analysis.** Making the calls is trivial. The work is in extracting brand mentions, scoring sentiment, attributing citations to source URLs, benchmarking competitors, trending over time, and building dashboards stakeholders will actually read.
5. **Models change constantly.** New versions ship every few months and the consumer apps change behavior without notice, so someone has to keep the tracker current.

## Tools worth evaluating

| Tool | Best for |
|---|---|
| **Profound** | Enterprise depth, front-end (real UI) capture, strong citation and source analysis. Premium price. |
| **Peec AI** | Mid-market sweet spot: clean UI, competitor benchmarking, multi-model, reasonable pricing. |
| **Scrunch AI** | Brand perception plus fixing how AI agents see your site. |
| **Otterly.AI** | Cheap and fast way to start. Good for a pilot. |
| **Semrush AI Toolkit / Ahrefs Brand Radar** | Good if you already pay for one; ties AI visibility to your SEO data. |
| **AthenaHQ, Goodie** | Worth a look if you want more optimization recommendations alongside tracking. |

Pricing and features change quickly in this category, so run a 2–4 week trial with two of them on the same prompt set and compare.

## When building makes sense

- You need tracking inside an internal BI stack or tied to proprietary data, such as pipeline attribution.
- You have very specific prompts, like long technical evaluation flows, that tools don't support well.
- You have data engineering capacity and you're tracking at a volume where tool pricing gets painful.

Even then, the common pattern is **buy for the dashboard and build a lightweight supplement**. That might be a script that runs 50–100 critical prompts weekly through the APIs with web search enabled and logs the results to a warehouse.

## Suggested next step

1. Write 30–50 prompts that reflect real buyer questions: category, comparison, use case, and "alternatives to X."
2. Trial Peec AI and Profound (or Otterly if budget is tight) on that same set.
3. After a month, decide whether the gap between what tools give you and what you need justifies building anything.

Want help drafting the prompt set or a scoring rubric for the trial?

The answer never names Cheap Seats, Popcorn, or any of the brand's aliases; it only recommends buying from Otterly.AI, Peec AI, and Profound.

**My recommendation: buy a tool, and maybe add a small in-house script later.** An API-only tracker you build yourself is likely to measure something different from what your buyers see.

## Why building on the APIs is riskier than it looks

**1. The API doesn't show you what users see.** The ChatGPT and Claude apps use different system prompts, tuning, and search behavior than the default API, and some of those settings aren't public. Several studies found big gaps between API answers and app answers:
- Surfer and Wellows found only about **24% brand overlap and 4% source overlap** between API answers and ChatGPT app answers. API answers were also shorter, around 406 words versus 743 in the app ([Surfer](https://surferseo.com/blog/llm-scraped-ai-answers-vs-api-results), [Wellows](https://wellows.com/blog/ai-visibility-measurement/)).
- Brandlight compared 2,565 matched prompts and found **6–12% brand overlap**. The top-recommended brand matched only 9–16% of the time ([Brandlight](https://research.brandlight.ai/ui-vs-api.html)).
- One caveat: most of this research comes from vendors that sell app-scraping tools, so the exact numbers are probably biased. Still, the sources all point the same way ([seoClarity](https://www.seoclarity.net/blog/scraping-vs.-api), [Metehan](https://metehan.ai/articles/how-ai-visibility-tools-collect-data)).

To get numbers that match the real apps, you'd have to scrape them. That means browser automation, proxies, terms-of-service risk, and constant fixes when the apps change. Vendors handle that for you.

**2. Coverage.** Buyers also use Gemini, Google AI Overviews and AI Mode, Perplexity, and Copilot. A build using only OpenAI and Anthropic misses those.

**3. The hard part is the analysis, not the API calls.** Answers vary from one run to the next, so each prompt has to be run several times and reported as a rate. Your team would also need to detect brand mentions and variants, check sentiment, pull out cited sources, compare against competitors, and build dashboards. The API spend is small. The ongoing engineering and maintenance cost much more.

## What buying costs (prices vary between sources, so check vendor sites)
- **Otterly.AI:** about $29, $189, or $489 a month. It's the cheapest way to start. On some plans, Claude and Gemini cost extra.
- **Peec AI:** $95 a month for 50 prompts, $245 for 150, $495 for 350 with multiple countries and Looker Studio. A good mid-market choice.
- **Profound:** the enterprise option. Multi-engine plans realistically start around $399–499+ a month, and the cheaper tier covers ChatGPT only.
- **Semrush AI Visibility Toolkit / Ahrefs Brand Radar:** worth considering if you already pay for those suites. Brand Radar runs about $398–699 a month.

([pricing comparison](https://www.get-ryze.ai/blog/ai-visibility-tools-pricing-compared-2026), [Peec alternatives](https://blog.timsoulo.com/14-peec-ai-alternatives-for-ai-search-visibility-tracking-2026/), [verified pricing](https://ainora.lt/blog/best-ai-visibility-tracking-tools-2026), [DemandSage](https://www.demandsage.com/ai-visibility-tools/))

Even a top plan, around $6k a year, costs less than a few weeks of engineering time.

## When building does make sense
- You want to know how *models* represent your brand, such as product facts or positioning accuracy, rather than what consumers see in the apps. The API works fine for that.
- You need data piped into your own warehouse, or prompt volumes and setups that vendors charge enterprise prices for.
- You have spare data engineering capacity and want testing tied into your content workflow.

## Practical plan
1. **Run two vendors in a trial for 2–4 weeks.** Peec and Otterly are good on cost; add Profound if you need enterprise reporting. Use 50–150 prompts that reflect real buyer questions, including competitor comparisons.
2. **Spot-check them yourself.** Run 10–20 of those prompts by hand in the ChatGPT and Claude apps and see which tool's results match.
3. **Build later, and only for specific gaps.** If you find one, a small script on the APIs works well alongside the paid tool, for example to check whether models describe your product accurately. It isn't a good replacement for the tool.

If you tell me your prompt volume, the engines you care about, and your budget, I can narrow this to one vendor.

Searches: “AI visibility tracking tools pricing 2026 Profound Peec AI Otterly Semrush AI toolkit” “API responses vs ChatGPT UI answers differ brand visibility tracking web search scraping”

Cited: https://surferseo.com/blog/llm-scraped-ai-answers-vs-api-results https://wellows.com/blog/ai-visibility-measurement/ https://research.brandlight.ai/ui-vs-api.html https://www.seoclarity.net/blog/scraping-vs.-api https://metehan.ai/articles/how-ai-visibility-tools-collect-data https://www.get-ryze.ai/blog/ai-visibility-tools-pricing-compared-2026 https://blog.timsoulo.com/14-peec-ai-alternatives-for-ai-search-visibility-tracking-2026/ https://ainora.lt/blog/best-ai-visibility-tracking-tools-2026 https://www.demandsage.com/ai-visibility-tools/

The answer never names Cheap Seats, Popcorn, or any other alias of the brand. It shortlists Profound, Peec AI, and Scrunch AI for buying an AI visibility tracker, so the brand is absent.

**Buy first, and build a small custom layer only if you find a clear gap.** For a marketing team, the hard part is making visibility data consistent and useful—not calling the APIs.

The main catch: **OpenAI and Anthropic API responses don’t reliably represent what people see in ChatGPT or Claude.** Consumer products can use different search, personalization, context, and routing. A homegrown tracker can measure API responses well, but that’s a narrower claim than tracking your visibility across AI products. Ask vendors how they handle this distinction, too.

I’d shortlist:

- **Profound** for a larger brand that needs reporting across markets, competitors, and teams. Evaluate whether its coverage and reporting justify the cost.
- **Peec AI** for a focused marketing workflow around brand mentions, competitors, and cited sources.
- **Scrunch AI** if you also want help identifying website and content changes that could improve how AI systems understand your brand.

Those are starting points based on my existing knowledge; confirm current capabilities and pricing during evaluation.

**Build if measurement itself is strategic.** That could mean thousands of niche prompts, proprietary customer scenarios, strict data requirements, or experiments tightly connected to your product. Budget for prompt management, repeated sampling, model changes, citation parsing, quality checks, and maintenance—not just API tokens.

Run a four-week vendor pilot before committing:

1. Pick 50–100 questions that reflect real buying decisions, grouped by use case and funnel stage.
2. Track brand inclusion, competitive share of mentions, recommendation accuracy, and cited sources. Keep branded and unbranded questions separate.
3. Repeat measurements to see whether changes exceed normal response variation.
4. Have the team use the findings to make actual content or positioning decisions.

Choose the tool that produces credible evidence and useful actions. **If nobody can explain what they would change based on the dashboard, don’t buy—or build—it yet.** Treat visibility as a diagnostic metric; connect it to referral traffic, conversions, and customer research before claiming business impact.

The answer never names Cheap Seats, CheapSeats, Popcorn, or any alias; it only discusses buying versus building a tracker and recommends Peec AI and Profound.

**Buy for ongoing marketing measurement. Build a small API benchmark only if you need custom experiments.** For most product marketing teams, maintaining the tracker will cost more than the API calls—and distract from acting on the findings.

The biggest issue is **what you’re measuring**. OpenAI and Anthropic both offer API web search with citations, so building a prompt runner is feasible. But API results shouldn’t be assumed to reproduce the ChatGPT or Claude consumer experience: search configuration, model selection, conversation history, and personalization can differ. Treat an API tracker as a controlled benchmark, rather than a direct measure of customer exposure. [OpenAI documentation](https://developers.openai.com/api/docs/guides/tools-web-search), [Anthropic documentation](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool)

| Your main need | My recommendation |
|---|---|
| Track mentions, competitors, citations, and trends for marketing | **Buy.** You get reporting and collection maintenance without owning the infrastructure. |
| Test proprietary prompts, product claims, or model behavior under controlled conditions | **Build.** APIs give you control over inputs, settings, and scoring. |
| Both | **Buy the monitoring; build the custom analysis** using exported data and a small API test suite. |

The engineering work grows beyond scheduling prompts: repeated sampling, failed searches, model changes, brand-name ambiguity, citation extraction, geography, historical comparisons, and usable reporting all need owners. OpenAI and Anthropic alone also leave out other discovery surfaces your buyers may use.

**I’d shortlist these two tools:**

- **Peec AI** for a focused marketing pilot. It offers daily prompt tracking, competitive visibility, and exports. Its documentation says most engines use consumer-interface collection, but its pricing page explicitly labels some coverage—including Claude—as API based. Verify the collection method for each engine you care about. [Methodology](https://peec.ai/ai-instructions), [plans and engine coverage](https://peec.ai/pricing)
- **Profound** if you need broader analysis across teams, markets, and topics. It offers citation analysis, sentiment, factual-accuracy monitoring, and raw-data exports. It says it captures answers directly from consumer interfaces; ask it to demonstrate that coverage for your required engines. [Capabilities and collection approach](https://www.tryprofound.com/features/answer-engine-insights)

**Run a 30-day pilot before committing annually.** Use 30–50 questions drawn from sales calls, customer research, and buying decisions, covering category discovery, comparisons, and branded accuracy. Track three to five competitors.

Require raw answers, citations, timestamps, collection method, geography, sampling frequency, and exports. Compare weekly trends; spot-check important answers manually. Keep mentions, recommendations, and citations separate, and assess referral traffic and conversions separately from visibility.

Buy if the pilot produces specific content or positioning decisions your team will execute. Build only when you can name a requirement vendors cannot meet and assign an engineer to maintain it. **Own the questions and interpretation; buy the collection and reporting.**

Searches: “site.scrunchai.com AI visibility monitoring pricing | site:peec.ai pricing AI visibility | site:developers.openai.com web search API user location | site:platform.claude.com web search tool API” “site:peec.ai pricing prompts models daily tracking | site:tryprofound.com answer engine insights visibility citations | site:platform.claude.com/docs web search tool citations pricing” “site:peec.ai browser API scraping real user interfaces tracking”

Cited: https://developers.openai.com/api/docs/guides/tools-web-search https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool https://peec.ai/ai-instructions https://peec.ai/pricing https://www.tryprofound.com/features/answer-engine-insights

The answer never names Cheap Seats or any of its aliases (including Popcorn or popcorn.zone). It recommends building a DIY stack around promptfoo, which is a different product.

There's no clear, mature open-source standard for this yet. Most serious AI visibility trackers are SaaS. You can still run something useful in Docker, but it means either adopting a small, young project or assembling one from proven open-source parts.

## Option 1: Search GitHub for existing projects
Search terms like "GEO tracker," "LLM brand monitoring," or "AI search visibility" turn up several small self-hostable projects. I can't confidently recommend a specific one: they're mostly young, and their maintenance status changes quickly. Before adopting one, check:
- Recent commits and open issues
- Which engines it queries (API-only or real ChatGPT/Perplexity/Google AI Overviews results)
- Whether it ships a `docker-compose.yml`

## Option 2: Build one from established open-source tools (recommended)
This takes an afternoon, and you control the prompts and data.

- **[promptfoo](https://github.com/promptfoo/promptfoo)**: An open-source LLM testing tool with a Docker image. Define your buyer-intent prompts ("best X for Y") and run them across OpenAI, Anthropic, Gemini, Perplexity, etc. Assertions like `contains: YourBrand` or `icontains-any: [competitors]` score each answer. It's built for evals, but the same setup works for visibility tracking.
- **Scheduler**: A cron container (or GitHub Actions) runs the prompts daily or weekly.
- **Storage and dashboards**: Postgres plus **Metabase** or **Grafana**, both easy to run in Docker, to chart mention rate, position, and share-of-voice against competitors over time.
- **Optional, [Langfuse](https://langfuse.com)**: Self-hostable via Docker Compose. Useful if you want every raw response logged and searchable.

Rough stack: `promptfoo` → JSON results → Postgres → Metabase, all in one `docker-compose.yml`.

## An important caveat
API responses are not what users see. ChatGPT, Perplexity, and Google AI Overviews in the consumer apps use web search, personalization, and different system prompts. Commercial tools like **Profound**, **Peec AI**, **Otterly.ai**, **Scrunch**, and **Ahrefs Brand Radar** charge partly because they capture real UI or search-grounded answers. A self-hosted API tracker is good for trends and competitive share-of-voice, but it won't match consumer-facing results exactly. Using the search-enabled API models (Perplexity Sonar, OpenAI with web search) narrows the gap.

**My suggestion:** If you mainly need directional trends and want control or low cost, build the promptfoo + Postgres + Metabase stack. If you need numbers you can defend to leadership about what customers actually see, trial Peec AI or Otterly first. They're relatively inexpensive, and you can compare them against your DIY results.

Want me to draft a starter `docker-compose.yml` and promptfoo config for your brand and competitors?

Cited: https://github.com/promptfoo/promptfoo https://langfuse.com

The answer never names Cheap Seats, its aliases, or any of its domains or repos, so the brand is absent. It recommends Elmo as the top pick and lists Limelit Open and OneGlanse as other Docker-runnable self-hosted trackers.

Yes. There are several open-source options now, and the three below all run in Docker. I'd start with **Elmo**.

**1. Elmo: the most complete option** ([GitHub](https://github.com/elmohq/elmo))
- **Install:** Docker Compose, using their CLI: `npm install -g @elmohq/cli`, then `elmo init`, then `elmo compose up -d`. It uses Postgres.
- **Coverage:** It pulls answers from the consumer apps (ChatGPT, Google AI Mode, AI Overviews, Gemini, Perplexity, Copilot) through scraping providers. It reaches Claude, Grok and other models through their APIs.
- **License and status:** MIT, about 480 stars, updated today.
- **Why I'd pick it:** It tracks brand mentions and sentiment like the paid tools do, and it's actively maintained. Scoring mentions costs roughly $0.15–0.35 per 1,000 answers, plus your scraper and model costs.

**2. Limelit Open: the simplest to run** ([GitHub](https://github.com/limelit-co/open))
- **Install:** One command: `docker run -p 1515:1515 -v limelit:/data ghcr.io/limelit-co/open`. It's a single Go program with a built-in SQLite database, so there's no separate database to manage.
- **Coverage:** ChatGPT, Claude, Perplexity, Gemini, Google AI Overview, AI Mode and Bing Copilot.
- **License:** Apache-2.0.
- **Caveat:** It's pre-release (v0.1), and only some of its data providers are finished. It's fine for trying the idea out, but not yet something to report from.

**3. OneGlanse: if you want what users actually see** ([GitHub](https://github.com/oneglanse/oneglanse))
- **Install:** Docker Compose.
- **How it works:** It drives real browser sessions on ChatGPT, Perplexity, Gemini, Claude and AI Overview instead of calling APIs. The answers you get back are closer to what customers see.
- **License:** MIT.
- **Downsides:** You need accounts on each AI service, and you have to sign in from a desktop session (WSL isn't supported). For a server setup you also need a proxy, and you take on the risk that those sites will block you.

**Others:**
- **[SearchMirror](https://github.com/databluedev/searchmirror)** combines Google rank tracking with AI citation checks in Docker Compose. It's AGPL-licensed, and its own README says the Compose setup is for development, not production.
- **[GeoLook](https://github.com/aigclink/geolook)** is worth a look if you care about Chinese AI engines like DeepSeek and Doubao.

**Things to know before you choose:**
- **None of these are free to run.** They all use your own API keys, so you pay the model and scraper providers for every prompt.
- **Collection method matters.** Calling an API often gives different answers than the ChatGPT app does. If leadership wants to know what customers see, pick a tool that scrapes the consumer apps (Elmo or OneGlanse), not one that only uses APIs.
- **Check the repos yourself.** These projects are young; most are under a year old and have a few hundred stars. Look at recent commits before you commit to one. The [ai-visibility GitHub topic](https://github.com/topics/ai-visibility) is a good place to compare them.

**My suggestion:** Run Elmo for real tracking. If you just want something working in five minutes to show your team, use Limelit's one-line Docker command.

Sources:
- [Elmo](https://github.com/elmohq/elmo)
- [Limelit Open](https://github.com/limelit-co/open) · [pkg.go.dev](https://pkg.go.dev/github.com/limelitgeo/open)
- [OneGlanse](https://github.com/oneglanse/oneglanse) · [hunted.space](https://hunted.space/product/oneglanse) · [feedbagel](https://feedbagel.com/post/oneglanse-free-open-source-geo-tracker-for-brand-visibility-in-ai-responses)
- [SearchMirror](https://github.com/databluedev/searchmirror)
- [GitHub ai-visibility topic](https://github.com/topics/ai-visibility) · [brand-monitoring topic](https://github.com/topics/brand-monitoring)
- [geo-aeo-tracker](https://github.com/danishashko/geo-aeo-tracker) · [GeoLook](https://hunted.space/product/geolook)
- [Lettertrace (everydev)](https://www.everydev.ai/tools/lettertrace) · [scriptbyai](https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/) · [chatgate](https://chatgate.ai/post/lettertrace)
- [Aperture](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture) · [OpenSight](https://www.mintlify.com/shuvamk/opensight) · [sharozdawa ai-visibility](https://enterprisedna.co/directories/mcp/sharozdawa-ai-visibility) · [DZone](https://dzone.com/articles/python-brand-visibility-ai-answer-tracking)

Searches: “open source self-hosted AI visibility tracker LLM brand mentions Docker GitHub” “open source GEO generative engine optimization tracker self-hosted ChatGPT Perplexity brand visibility”

Cited: https://github.com/elmohq/elmo https://github.com/limelit-co/open https://github.com/oneglanse/oneglanse https://github.com/databluedev/searchmirror https://github.com/aigclink/geolook https://github.com/topics/ai-visibility https://pkg.go.dev/github.com/limelitgeo/open https://hunted.space/product/oneglanse https://feedbagel.com/post/oneglanse-free-open-source-geo-tracker-for-brand-visibility-in-ai-responses https://github.com/topics/brand-monitoring https://github.com/danishashko/geo-aeo-tracker https://hunted.space/product/geolook https://www.everydev.ai/tools/lettertrace https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/ https://chatgate.ai/post/lettertrace https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture https://www.mintlify.com/shuvamk/opensight https://enterprisedna.co/directories/mcp/sharozdawa-ai-visibility https://dzone.com/articles/python-brand-visibility-ai-answer-tracking

The answer recommends Promptfoo and never names Cheap Seats or any of its aliases (CheapSeats, Popcorn, etc.), so the brand is absent. The answer also says it doesn't know of a Docker-ready, self-hosted purpose-built product.

**Yes—if you’re comfortable with some setup, [Promptfoo](https://www.promptfoo.dev/) is a self-hostable, open-source option you can run in Docker.** It’s an LLM testing tool rather than a turnkey AI visibility dashboard, but you can adapt it to track brand presence.

For a practical setup:

- Create a fixed list of buyer questions, such as “What’s the best project management tool for agencies?”
- Run them against supported AI providers on a schedule.
- Check whether answers mention your brand, recommend competitors, or include your website.
- Compare results over time.

**The main limitation:** API answers can differ from what customers see in ChatGPT, Gemini, or other consumer interfaces, particularly when web search and personalization are involved. Treat this as a repeatable benchmark, rather than an exact measure of customer exposure.

If you want a polished, purpose-built marketing dashboard with minimal configuration, I don’t confidently know of a Docker-ready, self-hosted product I can recommend from memory. Promptfoo is my recommendation for a customizable starting point.

Cited: https://www.promptfoo.dev/

The answer never names Cheap Seats or any of its aliases. It recommends Elmo first ('Yes—Elmo is worth trying first') and lists Limelit Open as a second option.

**Yes—Elmo is worth trying first** for tracking how AI answers mention and cite your brand. Its current documentation supports self-hosting with Docker Compose.

| Tool | Why consider it | What to know |
|---|---|---|
| **[Elmo](https://github.com/elmohq/elmo)** | Marketing dashboard with brand mentions, citations, sentiment, competitor share of voice, and historical trends. MIT licensed, with PostgreSQL storage. | Requires provider or scraper API keys, plus a TypeSafe key for mention and sentiment judging. |
| **[Limelit Open](https://github.com/limelit-co/open)** | Lightweight option: one Docker container, SQLite storage, scheduled checks, competitor comparisons, and CSV/JSON exports. Apache-2.0 licensed. | Explicitly pre-release; some provider integrations are still unfinished. |

Elmo’s documented setup uses its CLI to configure and start Docker Compose:

```bash
npm install -g @elmohq/cli
elmo init
elmo compose up -d
```

Then open `http://localhost:1515`. The software is free to self-host, but hosting, model calls, scraping services, and judging can incur costs. [Elmo setup documentation](https://github.com/elmohq/elmo#quick-start)

**One measurement detail matters:** model API answers can differ from what buyers see in the ChatGPT or Gemini apps. For marketing reporting, choose integrations that capture the consumer interface where possible, and keep those results separate from API-based measurements. [Limelit’s measurement explanation](https://github.com/limelit-co/open#why-this-exists)

I’d pilot Elmo with 20–30 buyer questions and three competitors, then review the stored answers alongside mention rate and citation share before expanding.

Searches: “self hosted AI visibility tracker Docker LLM brand visibility open source” “” “'docker'”

Cited: https://github.com/elmohq/elmo https://github.com/limelit-co/open http://localhost:1515 https://github.com/elmohq/elmo#quick-start https://github.com/limelit-co/open#why-this-exists

The answer never names Cheap Seats, CheapSeats, cheapseats.dev, Popcorn, or any other alias of the brand. It recommends SaaS tools and a build-it-yourself CronJob, Postgres and Grafana stack instead.

# Self-hosting an AI search visibility tracker on Kubernetes

**Yes.** You'd mostly be building it yourself or adapting an open-source project, though. The established trackers are SaaS-only.

## The commercial tools are SaaS

Profound, Peec AI, Otterly.ai, Scrunch AI, AthenaHQ, Semrush's AI Toolkit, and Ahrefs Brand Radar all run on their own infrastructure. None of them offers a self-hosted or on-prem version as a standard product. If self-hosting is a hard requirement, for example for data residency or security review, ask their sales teams about enterprise or private deployment. Don't assume they offer it.

## Building it on your cluster

The architecture is simple:

1. **Prompt set:** 50–500 queries your buyers would ask, like "best X for Y" or "alternatives to [competitor]". Store them in a ConfigMap or database.
2. **Collector:** A Kubernetes **CronJob** that runs daily or weekly and sends each prompt to:
   - **OpenAI API**, with web search enabled, to approximate ChatGPT
   - **Perplexity Sonar API**, which returns citations, so it's the easiest source
   - **Gemini API** with Google Search grounding
   - **Anthropic API** with web search
   - **Google AI Overviews** through a SERP data provider such as **SerpApi** or **DataForSEO**. Both expose AI Overview data, and there's no official Google API for it.
3. **Parser:** Detect brand and competitor mentions, position in the answer, sentiment, and cited URLs. A cheap LLM call works well for the extraction step.
4. **Storage:** Postgres. You likely already run it.
5. **Dashboard:** **Grafana** or **Metabase** for share of voice, citation share, and trends over time.

Plan on one engineer for about 1–3 weeks to get a working version. API costs are modest. A few hundred prompts across 4–5 engines, run weekly, typically costs tens to low hundreds of dollars a month. SERP API plans can add to that.

## Caveats to raise with stakeholders

- **API answers don't match what users see.** Consumer ChatGPT, Gemini, and Perplexity apply personalization, location, memory, and different model or search settings. API results are a proxy for directional trends, not an exact replica of what users see. The SaaS vendors face the same limitation, and some of them scrape the consumer UIs to get closer.
- **Don't scrape consumer UIs yourself.** Automating chatgpt.com or gemini.google.com generally violates their terms and breaks often. Stick to official APIs and licensed SERP providers.
- **Answers are non-deterministic.** Run each prompt several times and report mention *rates*. Single answers are too noisy.

## Recommendation

- **If you need results this quarter and self-hosting is a preference, not a requirement:** Buy. Peec AI or Otterly are fast and affordable to start with, and Profound is the choice for enterprise depth. Revisit self-hosting later.
- **If data must stay in-house, or you want custom prompts and metrics tied to your own analytics:** Build the CronJob + Postgres + Grafana stack described above. Check GitHub for open-source "LLM brand monitoring" or "GEO tracker" projects as a starting point, but review their activity and license before you depend on one.

I can sketch the CronJob manifest and a starter collector script if that would help you scope it with engineering.

The brand Cheap Seats (aliases CheapSeats, Popcorn, cheapseats.dev, etc.) is never named in the answer; it recommends Gego, Aperture, and commercial tools instead.

Yes, you can, but the "self-hosted" part only goes so far. The tracker runs on your cluster, but it still has to send your prompts to OpenAI, Anthropic, Google and Perplexity over the internet. Hosting it yourself gives you control of the data and the costs. It doesn't keep your queries in-house.

**Open-source options**

- **[Gego](https://hub.docker.com/r/ouai2h/gego)** is the easiest to put on Kubernetes. It runs prompts on a schedule, records brand mentions and the sources the AI cites, and has a built-in dashboard. One Docker image holds both the app and the dashboard (port 8989), and all settings are environment variables, so it fits normal Kubernetes config. It works with OpenAI, Anthropic, Google, Perplexity and Ollama.
  - **Caveats:** It needs three databases running alongside it: PostgreSQL, MongoDB and etcd. It doesn't come with a Helm chart, so you'd write the deployment files or use standard database charts. Its license is GPL-3.0, which is fine for internal use, but check with legal before you modify it and give it to anyone outside the company.
- **[Aperture](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture)** is MIT-licensed and you supply your own API keys. It covers ChatGPT, Perplexity and Google AI Overviews. I couldn't confirm whether it ships a Docker image.
- **Others to look at:** [Canonry](https://github.com/topics/ai-visibility-tracker) (tracks citations of your domain in ChatGPT, Claude, Gemini and Perplexity), [Lettertrace](https://www.everydev.ai/tools/lettertrace/llms.txt) (MIT) and [GetCito](https://hashnode.com/@GetCitoMonitor). There are more on the GitHub topic pages for [ai-visibility](https://github.com/topics/ai-visibility) and [generative-engine-optimization](https://github.com/topics/generative-engine-optimization).

All of these projects are young. Before you commit, check each repo for recent activity and how many people contribute.

**Before you decide**

1. **The API's answers aren't what customers see.** These tools mostly query the models through their APIs. The consumer ChatGPT and Gemini apps add web search, personalization and different system prompts on top. Tracking Google AI Overviews or AI Mode usually means scraping, typically through a paid scraping service. The Bright Data [tracker](https://brightdata.com/blog/ai/ai-visibility-tracker.md) works this way (the article is Bright Data marketing).
2. **You pay the AI providers directly.** Each prompt × model × run is an API call. For example, 200 prompts × 5 models × daily runs is about 30,000 calls a month. That's cheap on small models and real money on models that search the web.
3. **Your team builds the reports.** Commercial tools like Profound and Peec AI come with share-of-voice benchmarks, competitor comparisons and polished reports. As far as I found, they're SaaS only and don't offer a self-hosted version. With open source, someone on your side maintains the deployment and the dashboards.

**My recommendation:** Start with a small Gego pilot (about 50 prompts, 3–4 models, weekly runs) in its own Kubernetes namespace. Store the API keys as Kubernetes secrets, and use databases your platform team already runs if you can. After a month, compare its results with a free trial of Peec AI or Profound. If the numbers roughly match, keep it in-house. If they don't, the difference is what the paid product is worth to you.

Searches: “open source self-hosted AI search visibility tracker LLM brand mentions GEO” “github open source generative engine optimization tracker ChatGPT Perplexity brand monitoring docker”

Cited: https://hub.docker.com/r/ouai2h/gego https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture https://github.com/topics/ai-visibility-tracker https://www.everydev.ai/tools/lettertrace/llms.txt https://hashnode.com/@GetCitoMonitor https://github.com/topics/ai-visibility https://github.com/topics/generative-engine-optimization https://brightdata.com/blog/ai/ai-visibility-tracker.md

The answer never names Cheap Seats, CheapSeats, Popcorn, or any alias; it recommends Profound or Otterly.AI and a custom Kubernetes stack instead.

Yes—an AI search visibility tracker can run on your Kubernetes cluster **if you build it or choose a product that supports self-hosting**. Many commercial trackers are SaaS, so verify deployment options before choosing one.

The key distinction: **hosting the tracker yourself doesn’t mean running the AI search engines yourself.** Your tracker still needs to query external services to measure whether your brand appears.

A practical setup would include:

- **Scheduled jobs:** Kubernetes CronJobs run a fixed set of buyer questions daily or weekly.
- **AI queries:** OpenAI, Gemini, or Perplexity APIs collect answers and citations. API responses can differ from the consumer apps, so label those results clearly.
- **Storage:** PostgreSQL holds prompts, responses, brand mentions, citations, and historical metrics.
- **Reporting:** Metabase provides dashboards for mention rate, competitor share of mentions, and cited URLs.
- **Optional workflow tooling:** Self-hosted n8n can coordinate queries and alerts if you want less custom code.

The hardest part is reliable measurement: answers vary by wording, location, model, and time. Keep a consistent prompt set, repeat queries, and record the model and query settings. Tracking consumer interfaces through browser automation adds maintenance and potential terms-of-service constraints.

**My recommendation:** if keeping data and workflows inside your infrastructure is a requirement, start with CronJobs + PostgreSQL + Metabase and external model APIs. If your main goal is fast marketing insight, a managed tracker such as **Profound or Otterly.AI** is worth evaluating for its packaged monitoring and reporting; confirm current engine coverage and hosting options with the vendor.

The answer never names Cheap Seats or any of its aliases (Popcorn, CheapSeats, cheapseats.dev, popcorn.zone, and the others). It recommends Elmo as the tracker to evaluate and pilot.

**Yes—you can run an AI search visibility tracker on your own Kubernetes cluster**, though expect some engineering work to package and operate it.

I’d start by evaluating **[Elmo](https://github.com/elmohq/elmo)**. It’s MIT-licensed, self-hostable, and tracks brand mentions, citations, and competitor visibility. Its documented deployment uses Docker Compose, with a web app, background worker, and PostgreSQL database. Kubernetes is therefore a reasonable deployment adaptation; the documentation I checked doesn’t advertise a ready-made Helm installation.

Your platform team would need to provide:

- Deployments for the dashboard/API and worker.
- PostgreSQL, persistent storage, and backups.
- API credentials, outbound access to providers, and internal dashboard access.
- Monitoring, upgrades, and spending limits.

**The main caveat is how results are collected.** Elmo supports both scraping providers that capture consumer AI search experiences and direct model APIs. Those measure different things: an API response shouldn’t automatically be treated as what a buyer sees in ChatGPT or Google. Choose collection methods that match the channels you want to measure. [Elmo’s methodology](https://github.com/elmohq/elmo/blob/main/README.md)

Self-hosting also doesn’t make the system fully private or free: your database can stay under your control, but prompts still go to external model or scraping providers, with their associated costs.

My recommendation: pilot Elmo with **30–50 buyer questions, three competitors, and two priority engines**. Have marketing validate the answers and citations while engineering validates deployment reliability and collection costs before expanding.

Searches: “AI search visibility tracker self hosted Kubernetes open source Peec Otterly alternatives | GitHub AI search visibility tracker self hosted docker GEO” “Elmo AI visibility github self host kubernetes | AI visibility tracker kubernetes helm self hosted”

Cited: https://github.com/elmohq/elmo https://github.com/elmohq/elmo/blob/main/README.md

The answer never names Cheap Seats, Popcorn, or any alias; it discusses BYOK tools generally and recommends building a DIY tracker with Promptfoo instead.

# AI Visibility Tools That Support Your Own API Keys

**Short answer: few commercial AI visibility tools offer bring-your-own-key (BYOK), and I can't confirm a current list with confidence.** This category changes quickly, and vendors rarely state their BYOK policy publicly. Check directly with each vendor before you commit.

## Why BYOK is uncommon

Most leading tools (Profound, Peec AI, Scrunch, AthenaHQ, Otterly.ai, Semrush's AI toolkit, Ahrefs Brand Radar) deliberately avoid relying only on the raw API. **Answers from the OpenAI and Anthropic APIs often differ from what users see in ChatGPT or Claude.** The consumer apps add web search, memory, location, and different system prompts. So many vendors query the consumer interfaces or simulate them. Your API key doesn't help with that, and it would give you less representative data.

## Your realistic options

**1. Ask vendors directly.** Use these questions:
- "Do you support BYOK for OpenAI/Anthropic, and does that lower our price?"
- "Are your results from the API or from the consumer ChatGPT/Claude interfaces?"
- "With BYOK, is web search/grounding enabled (e.g., OpenAI's Responses API with web search)?"
- "Who sees our prompts and responses, and are they retained?"

**2. Build it yourself.** This is the most reliable BYOK path, and it's cheap and fast.
- A small script can run your prompt set against OpenAI and Anthropic, with web search tools turned on so results come closer to the real apps. It can then parse for brand mentions, citations, and sentiment, and log results to a sheet or warehouse.
- **Promptfoo** (open source) can run prompt sets across providers with your keys and score outputs. It isn't built for visibility tracking, but it adapts well.
- Cost is usually a few dollars per run for a few hundred prompts.

**3. Use a hybrid.** Run a commercial tool for consumer-interface tracking and competitive benchmarks. Add your own BYOK script for custom or high-volume prompt testing.

## My recommendation

If BYOK matters mainly for **cost or data control**, build the DIY tracker. If it matters mainly for **accurate visibility data**, choose a vendor that tracks the consumer interfaces and accept that it won't use your keys.

I can draft a starter script or a vendor evaluation checklist if that would help.

The answer never names Cheap Seats, CheapSeats, Popcorn, or any other alias of the brand, so the verdict is absent. Its recommendations are Lettertrace, Archytas AISpy, and Limelit Open, none of which is the brand.

I found no major commercial AI visibility platform that lets you plug in your own OpenAI and Anthropic keys. Profound, Peec AI, Otterly, Athena, Trakkr and Mangools all charge a bundled subscription, and some sell models like Claude as paid add-ons. The tools that do take your own keys are free or cheap, and most are small or open-source projects.

**Hosted or packaged, with little setup**
- **Lettertrace:** free, open-source (MIT) and has no paid tiers. You add OpenAI, Anthropic, Google or Perplexity keys in Settings, and scheduled daily or weekly runs bill to your keys. You can also host it yourself, which needs a Supabase project. This is the closest match to what you asked for. ([Product Hunt](https://www.producthunt.com/products/lettertrace), [review](https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/))
- **Archytas AISpy:** a free tracker and research tool that requires your own OpenAI, Anthropic, Google or Perplexity keys. ([site](https://www.archytasdigital.com/))
- **CiteTrack AI:** a WordPress plugin that calls each engine's official API with your keys. About $59/year plus your API costs. It suits you if your site already runs on WordPress.
- **Apify "AI Visibility Tracker":** you pay per run and bring your own keys. It was listed as "under maintenance" when I checked. ([Apify](https://apify.com/samstorm/ai-visibility-tracker)) A [separate Apify actor](https://apify.com/zinin/llm-brand-visibility) uses an OpenRouter key instead, which covers OpenAI and Anthropic models.

**Self-hosted open source (you'll need engineering help)**
- **[Limelit Open](https://github.com/limelit-co/open):** Apache 2.0. It's a single Go program with a local database, takes your own keys and adds no markup. It's still pre-release.
- **[Gego](https://github.com/AI2HU/gego):** GPL v3. It takes OpenAI and Anthropic keys and tracks mentions and citations, but setup is heavy (PostgreSQL, MongoDB and etcd).
- **[geo-aeo-tracker](https://github.com/danishashko/geo-aeo-tracker):** MIT and runs on your own machine. It uses OpenRouter plus Gemini keys and also depends on Bright Data, a second paid service.
- **[eduardmur/ai-visibility-tracker](https://github.com/eduardmur/ai-visibility-tracker):** deploys to Vercel in one click and runs on a single Vercel AI Gateway key rather than separate OpenAI and Anthropic keys.
- For more, see [awesome-open-geo](https://github.com/cloudapp-dev/awesome-open-geo) and the [GitHub ai-visibility topic](https://github.com/topics/ai-visibility). Many of these repos are only days or months old, so check that they're still maintained before you rely on one.

**Before you choose, keep in mind**
- **API answers aren't the same as what customers see.** Running prompts through the OpenAI or Anthropic API isn't the same as what people see in the ChatGPT or Claude apps. Those apps add web search, memory and different system prompts. Paid platforms like Profound and Peec charge more partly because they capture what the actual apps show. Your own keys are cheaper and you control the data, but the results only approximate real-world visibility. Make sure web search is turned on in the API calls, if the tool allows it.
- **Cost:** with your own keys you pay only for the API calls. For a typical set of prompts tracked weekly, that's usually a few dollars a month, compared with $99–$489/month for commercial plans.

**My suggestion:** test Lettertrace first, since it needs no infrastructure and is free. If you need more control or to keep data on your own servers, have engineering evaluate Limelit Open. If leadership will judge results against what people see in the ChatGPT app, budget for a commercial tool anyway.

I haven't tested any of these myself. These details come from vendor pages and repo READMEs, and they change quickly.

Sources:
- [Lettertrace – Product Hunt](https://www.producthunt.com/products/lettertrace)
- [Lettertrace review – ScriptByAI](https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/)
- [Archytas AISpy](https://www.archytasdigital.com/)
- [Apify AI Visibility Tracker](https://apify.com/samstorm/ai-visibility-tracker)
- [Apify LLM Brand Visibility Tracker](https://apify.com/zinin/llm-brand-visibility)
- [Limelit Open](https://github.com/limelit-co/open)
- [Gego](https://github.com/AI2HU/gego)
- [geo-aeo-tracker](https://github.com/danishashko/geo-aeo-tracker)
- [eduardmur/ai-visibility-tracker](https://github.com/eduardmur/ai-visibility-tracker)
- [awesome-open-geo](https://github.com/cloudapp-dev/awesome-open-geo)
- [GitHub ai-visibility topic](https://github.com/topics/ai-visibility)
- [Swydo – Best AI rank tracking tools](https://www.swydo.com/blog/best-ai-rank-tracking-tools/)
- [Demandsage – AI visibility tools](https://www.demandsage.com/ai-visibility-tools/)
- [Searchable – Claude tracking tools](https://www.searchable.com/blog/best-claude-tracking-tools)
- [BYOKList](https://byoklist.com/)

Searches: “AI visibility tracking tool bring your own API key OpenAI Anthropic” “GEO LLM brand monitoring BYOK "own API keys" open source” “AI search visibility platform SaaS "bring your own key" ChatGPT Claude brand tracking pricing 2026”

Cited: https://www.producthunt.com/products/lettertrace https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/ https://www.archytasdigital.com/ https://apify.com/samstorm/ai-visibility-tracker https://apify.com/zinin/llm-brand-visibility https://github.com/limelit-co/open https://github.com/AI2HU/gego https://github.com/danishashko/geo-aeo-tracker https://github.com/eduardmur/ai-visibility-tracker https://github.com/cloudapp-dev/awesome-open-geo https://github.com/topics/ai-visibility https://www.swydo.com/blog/best-ai-rank-tracking-tools/ https://www.demandsage.com/ai-visibility-tools/ https://www.searchable.com/blog/best-claude-tracking-tools https://byoklist.com/

The answer never names Cheap Seats, CheapSeats, Popcorn, or any alias; it discusses Promptfoo, Langfuse, LangSmith, Profound, Peec AI, and Otterly.AI only.

If **using your own OpenAI and Anthropic API keys is a hard requirement**, I’d shortlist these tools—with one caveat: they’re evaluation platforms you can configure for AI visibility tracking, rather than turnkey marketing dashboards.

| Tool | Why it fits |
|---|---|
| **Promptfoo** | Supports your own OpenAI and Anthropic keys. Good for repeatedly testing buyer questions across models and scoring brand mentions, competitor mentions, and recommendations. Best if you have technical support. |
| **Langfuse** | Supports both providers through its playground and evaluation workflows. Useful if you want visibility tests alongside prompt management, cost tracking, and an option to self-host. |
| **LangSmith** | Can run evaluations against OpenAI and Anthropic using your credentials. A fit if your engineering team already uses LangChain or LangSmith. |

**My practical pick: Promptfoo** for a focused, custom monitoring setup.

For dedicated AI visibility products such as **Profound, Peec AI, or Otterly.AI**, I can’t confidently confirm customer-supplied keys for **both** providers from existing knowledge. Don’t assume that “tracks ChatGPT and Claude” means BYOK.

One important distinction: **API responses aren’t the same as the consumer ChatGPT and Claude experiences**. Search, personalization, and model configurations can differ, so BYOK monitoring measures API visibility rather than precisely reproducing what buyers see in those apps.

The answer names none of the brand's name or aliases (Cheap Seats, CheapSeats, Popcorn, etc.). It recommends other BYOK visibility tools, so the brand is absent.

**Yes—there are AI brand visibility tools that support both your own OpenAI and Anthropic API keys**, usually called “bring your own key” (BYOK).

| Tool | Why consider it | Main tradeoff |
|---|---|---|
| **[Agentcy AI Visibility](https://github.com/AgentcyPR/ai-visibility)** | Tracks brand mentions, citations, sentiment, competitors and trends using your own OpenAI and Anthropic keys. Supports multiple clients, making it useful for agencies. | Open-source and self-hosted; requires technical setup. |
| **[AI Search Visibility Tracker on Apify](https://apify.com/kadi_bence/ai-visibility-tracker)** by Bence Kadi | Accepts both providers’ keys and measures mentions, citation rank and share of voice. Supports scheduled monitoring, making it a practical option for a recurring marketing workflow. | Charges **$10 per 1,000 checks with your own keys**, plus provider API usage. |
| **[NYFTY Labs AI Visibility Monitor](https://nyftylabs.com/ai-visibility-monitor)** | Free tool explicitly supporting your own OpenAI and Claude keys. Useful for a quick check of whether models recognize and cite your brand. | Uses one fixed brand-description prompt; no custom query list or background scheduler. |
| **[CrawlPod’s `ai-visibility` package](https://crawlpod.com/docs/engine-adapters)** | Supports both providers directly from your infrastructure, with no CrawlPod proxy. Useful for building custom reporting or integrating visibility checks into existing workflows. | A developer package and CLI, so engineering help is needed. |

**For a marketing team, I’d pilot the Apify tracker first** for scheduled buyer-question monitoring. Choose Agentcy if self-hosting and control are priorities, or NYFTY for a quick initial snapshot.

One important distinction: these tools query **provider APIs**. Their results can differ from what customers see in the ChatGPT and Claude apps, so treat them as directional visibility measurements. [NYFTY explains this limitation explicitly](https://nyftylabs.com/ai-visibility-monitor).

Searches: “AI visibility tracking "own" "OpenAI" "Anthropic" "API keys" | AI visibility tools BYOK OpenAI Anthropic”

Cited: https://github.com/AgentcyPR/ai-visibility https://apify.com/kadi_bence/ai-visibility-tracker https://nyftylabs.com/ai-visibility-monitor https://crawlpod.com/docs/engine-adapters

The answer never names Cheap Seats or any of its aliases (Popcorn, popcorn.zone, CheapSeats, cheapseats.dev, etc.), so the brand is absent. It recommends OpenRouter for the user's setup, which is a different product.

# Cost of Daily AI Visibility Checks

**Short answer:** A typical setup costs **about $50–$300/month** without web search. With web search turned on, which is what users of ChatGPT, Perplexity and Gemini actually see, it costs **about $300–$1,500/month**. Web search is the biggest cost driver. Prices change often, so check current rates on each provider's pricing page.

## How to estimate it

**Daily cost = prompts × engines × runs per prompt × cost per call**

Example: 100 prompts × 4 engines × 3 runs (to smooth out answer variance) = **1,200 calls/day**.

| Setup | Approx. cost per call | Daily | Monthly |
|---|---|---|---|
| Small/cheap models (GPT mini, Gemini Flash, Claude Haiku), no search | $0.001–0.003 | $1–4 | $30–120 |
| Flagship models, no search | $0.005–0.02 | $6–24 | $180–700 |
| Any model **with web search/grounding** | $0.015–0.05+ | $18–60 | $550–1,800 |

**Why search costs more:**
- Providers charge per search, roughly $10–35 per 1,000 searches.
- The retrieved pages are added to your input, often several thousand tokens per call.
- Perplexity's Sonar API also charges a per-request fee on top of token costs.

## How to cap it

**1. Hard limits at the provider (do this first)**
- **OpenAI:** Create a dedicated project for this job and set a monthly budget and usage limits on it.
- **Anthropic:** Set a workspace spend limit in the Console and use a separate workspace for this job.
- **Google (Gemini/Vertex):** Cloud Billing budgets only send alerts by default. To actually stop spending, set up a budget notification that triggers automation to disable billing, or use API quotas.
- **OpenRouter:** Prepaid credits plus per-key credit limits give you a true hard stop across all models through one key. This is the simplest option if you want a single cap.
- **LiteLLM proxy (self-hosted):** Per-key and per-day budgets across all providers. Use it if you have engineering help.

**2. Limits in your own script**
- Set `max_tokens` on every call. 500–800 is enough to see whether your brand is mentioned.
- Keep a running daily cost tally from the usage data in each response, and stop the job when it passes your daily budget.
- Use a fixed prompt list so new prompts can't be added without someone approving them.

**3. Spend less per check**
- Run cheap models daily and flagship or search-enabled checks weekly.
- Use the Batch APIs from OpenAI and Anthropic for about 50% off non-search runs. Results come back within 24 hours, which is fine for daily monitoring.
- Rotate a sample of prompts each day instead of running all of them.
- Do 1 run per prompt daily and 3–5 runs weekly.

## Build vs. buy

Off-the-shelf tools like **Otterly.ai**, **Peec AI** and **Profound** range from roughly $30/month to well over $500/month. They handle multi-run sampling, tracking against competitors, and dashboards. If your API estimate with search comes out above about $300/month, or nobody is available to maintain the script, buying a tool is usually cheaper overall.

**Recommended starting setup:** Route everything through OpenRouter with a hard monthly credit limit, such as $200. Run cheap models daily with `max_tokens` capped, and run search-enabled checks weekly. Look at actual spend after 2 weeks and adjust from there.

The answer never names Cheap Seats, CheapSeats, Popcorn or any other alias; it only covers API costs for ChatGPT, Claude, Gemini and Perplexity and mentions Profound, Peec AI and Otterly as paid alternatives.

For a typical setup the API bill comes to roughly **$300–$400 a month**: 100 tracked prompts, run once a day on ChatGPT, Claude, Gemini and Perplexity, with web search on. A small 25-prompt pilot costs about **$75–$100 a month**. To cap it, set a hard limit at each provider and also put spend limits in your own script.

## What drives the cost

A visibility check only means something if web search is on, and search is the main cost. Each check pays three things: a per-search fee, the retrieved web pages billed as input tokens (often 5–15k tokens), and the answer itself.

| Engine (model with search) | Current pricing | Est. cost per check | 100 prompts/day (~3,000/mo) |
|---|---|---|---|
| **ChatGPT**: GPT-5.6 Terra + web search tool | $2 / $12 per M tokens; search $10 per 1k calls | ~$0.04–0.05 | ~$120–150 |
| **Claude**: Sonnet 5.5 + web search | $2 / $10 per M tokens; search $10 per 1k | ~$0.04–0.05 | ~$120–150 |
| **Gemini**: 3.6 Flash + Google Search grounding | $0.75 / $3.75 (doubles to $1.50 / $7.50 on Jan 1, 2027); 5,000 free searches/mo, then $14 per 1k | ~$0.005–0.02 | ~$15–40 |
| **Perplexity**: Sonar (low context) / Sonar Pro | Sonar $1/M tokens + $5–12 per 1k requests; Pro $3/$15 + $6–14 per 1k | ~$0.007 / ~$0.02 | ~$20 / ~$60 |

**Rough total: ~$300–400 a month for 100 prompts across 4 engines.** It grows in a straight line with prompts × engines × runs per day.

These per-check figures are my estimates, not published prices. They assume about 1.5 searches per call, 10k input tokens and 1k output tokens.

Things that push the number up:
- **Running each prompt several times.** AI answers vary a lot between runs, so many teams run each prompt 3 times and average. That roughly triples the bill.
- **Using the top models.** GPT-6 Astra at $10/$50 or Claude Opus 5.5 at $4/$20 costs 2–5× more for little gain in visibility data. Stick with mid-tier models.
- **The model searching several times per answer.** This is the biggest unknown, so limit it (see below).
- **Scoring the answers** (did we get mentioned, which source was cited) costs almost nothing. Use plain text matching or Claude Haiku 5.5 at $0.10/$0.50.

## How to cap it

**1. Set hard limits at each provider.** These stop spending even if your script has a bug.
- **Anthropic:** set a spend limit on a dedicated workspace in the Claude Console.
- **OpenAI:** create a separate project with a budget, and use prepaid credits with auto-recharge turned off so spending stops at zero.
- **Perplexity:** prepaid credits with auto top-up turned off.
- **Google:** budgets in Cloud Billing only send alerts; they don't stop spending. For a real cap, set a requests-per-day limit on the Gemini API, or set up a budget alert that turns off billing when it fires.

**2. Set limits on every request.**
- Claude: `max_uses: 1–2` on the web search tool.
- OpenAI Responses API: `max_tool_calls` and `max_output_tokens`.
- Perplexity: `search_context_size: "low"`.
- All engines: a limit on output tokens. A few hundred is enough to see whether you're mentioned.

**3. Add a budget check to your script.** Each response reports its token and search counts in a `usage` field. Have the script add up the dollar cost as it runs and stop when it reaches a daily limit, for example $15.

**4. Cut the volume.**
- Run 20–30 core prompts daily and the rest weekly.
- Use batch mode. OpenAI and Anthropic both give 50% off tokens with results within 24 hours, which is fine for a daily check. Search fees may not get the discount, so test this first.
- Do most of your Gemini checks within its 5,000 free searches a month.

## Two caveats
- **API answers aren't the same as what users see.** The consumer ChatGPT and Perplexity apps use different system prompts, personalization and search setups. Treat API results as a directional trend, not the exact answer a buyer sees.
- **OpenAI's prices are uncertain.** OpenAI's pricing page blocked my request, and third-party sites disagree on its current prices (for example, GPT-5.6 Sol's promotional rate ends around Nov 21). The Claude and Gemini figures come from the official pages. Check OpenAI's prices before you commit to a budget.

If you don't want to build and maintain this, paid tools like Profound, Peec AI or Otterly do the same job. Compare their subscription price with the ~$300–400 a month in API fees plus your engineering time.

**Sources:**
- [Claude API pricing (official)](https://platform.claude.com/docs/en/about-claude/pricing)
- [Claude web search tool docs](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool)
- [Gemini API pricing (official)](https://ai.google.dev/gemini-api/docs/pricing)
- [Mem0: OpenAI API pricing, verified Oct 6, 2026](https://mem0.ai/library/llms-and-models/openai-api-pricing)
- [eesel: OpenAI API pricing](https://www.eesel.ai/blog/openai-api-pricing)
- [CloudZero: OpenAI pricing](https://www.cloudzero.com/blog/openai-pricing/)
- [CloudZero: Perplexity API pricing](https://www.cloudzero.com/blog/perplexity-api-pricing/)
- [Puter: Perplexity API pricing](https://developer.puter.com/tutorials/perplexity-api-pricing/)
- [MxChat: Perplexity API pricing tested](https://mxchat.ai/perplexity-api-pricing-sonar-chatbot/)
- [Opslyft: Perplexity pricing](https://www.opslyft.com/blog/perplexity-pricing)
- [CloudZero: Gemini pricing](https://www.cloudzero.com/blog/gemini-pricing/)

Searches: “OpenAI API pricing per million tokens web search tool cost 2026” “Perplexity Sonar API pricing 2026 per request” “Gemini API pricing grounding with Google Search 2026 Claude API web search pricing”

Cited: https://platform.claude.com/docs/en/about-claude/pricing https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool https://ai.google.dev/gemini-api/docs/pricing https://mem0.ai/library/llms-and-models/openai-api-pricing https://www.eesel.ai/blog/openai-api-pricing https://www.cloudzero.com/blog/openai-pricing/ https://www.cloudzero.com/blog/perplexity-api-pricing/ https://developer.puter.com/tutorials/perplexity-api-pricing/ https://mxchat.ai/perplexity-api-pricing-sonar-chatbot/ https://www.opslyft.com/blog/perplexity-pricing https://www.cloudzero.com/blog/gemini-pricing/

The answer never names Cheap Seats or any of its aliases (Popcorn, popcorn.zone, etc.). It gives a cost formula, a budget-capping approach, and a Python plus SQLite setup, with LiteLLM as an optional gateway.

For budgeting, expect **tens to a few hundred dollars a month for a modest daily check**, depending on the models, prompt count, and whether you pay for live web search. At larger scale, costs can reach thousands. These are planning estimates; verify your contracted rates before launch.

Use this formula:

**Monthly cost = prompts × models × runs per day × 30 × cost per check**

A “check” means sending one prompt to one model once. For example, 100 prompts across three models every day produces **9,000 checks per month**:

| Average cost per check* | Monthly API cost |
|---|---:|
| $0.002 | $18 |
| $0.01 | $90 |
| $0.05 | $450 |
| $0.10 | $900 |

*Illustrative all-in costs, not quoted provider prices. Include input/output tokens, search fees, and any other paid tool calls. Retries and repeated samples add to the total.*

One important limitation: **API answers measure visibility in that API configuration.** They won’t exactly reproduce what people see in ChatGPT, Google AI Overviews, or other consumer products, which can use different search, personalization, and model settings.

To cap spending reliably:

- **Start with a $100 monthly pilot budget.** Check 25–50 high-value prompts daily; run the broader prompt set weekly.
- **Enforce the cap in your own runner.** Before each request, reserve its maximum estimated cost. Stop issuing requests when spent plus reserved funds reaches your daily or monthly limit. Include in-flight requests.
- **Bound each check.** Limit output tokens, paid search calls, concurrency, and retries. Allow at most one retry for transient failures.
- **Use provider controls as backup.** Set project budgets, alerts, and rate limits where available. Confirm whether a “budget” actually blocks requests; alerts alone don’t cap costs.
- **Track cost per completed check.** Log provider usage, model, prompt, tool calls, and estimated cost; reconcile against billing.

For a small implementation, **a scheduled Python job plus SQLite** is enough: Python runs the checks, and SQLite records results and budget reservations. **LiteLLM** is useful if you need one gateway across several model providers, with centralized usage tracking and budget controls.

My recommendation: run a seven-day pilot, calculate the actual cost per completed check, then set a monthly cap with about **20% headroom** for retries and price variation.

The answer never names Cheap Seats, CheapSeats, Popcorn, or any other alias of the brand. It only discusses OpenAI, Claude, and Gemini APIs for running AI visibility checks.

**Budget roughly $150–$450/month in API fees for 100 prompts checked daily across three AI engines.** That’s a planning estimate at **$0.015–$0.05 per completed check**, assuming short answers and limited web searches. Engineering, hosting, and reporting are extra.

A check means **one prompt × one engine × one market/language × one run**.

| Daily setup | Checks/month, 30 days | Estimated API fees |
|---|---:|---:|
| 50 prompts × 3 engines | 4,500 | $68–$225 |
| 100 prompts × 3 engines | 9,000 | $135–$450 |
| 100 prompts × 3 engines × 3 repeats | 27,000 | $405–$1,350 |

The calculation is:

**Monthly cost = checks × (model input + model output/reasoning + search fees).**

Current pricing explains why search matters:

- **OpenAI API:** web search costs **$10 per 1,000 calls**, plus model tokens, including retrieved search content. One search per check adds **$90 to 9,000 checks** before token charges. [OpenAI pricing](https://developers.openai.com/api/docs/pricing)
- **Claude API:** web search also costs **$10 per 1,000 searches**, plus token charges. [Claude pricing](https://claude.com/pricing)
- **Gemini API:** current Gemini 3-and-newer pricing includes **5,000 free search queries/month shared across those models**, then **$14 per 1,000**. A single prompt can trigger several billable searches. [Gemini pricing](https://ai.google.dev/gemini-api/docs/pricing)

**To cap it, put the stop rule in your own checking service:**

1. **Start with a $300/month ceiling and $10/day limit.** Check 50 priority prompts across three engines daily; expand after measuring a week of actual costs.
2. **Reserve the maximum allowed cost before each request.** Include output/reasoning tokens, search calls, retries, and requests already running. Skip requests that would exceed the remaining budget.
3. **Limit each check:** fixed model, bounded answer length, bounded searches where supported, and at most one retry. Avoid automatic upgrades to expensive models.
4. **Use dedicated API projects/keys**, with alerts at 50%, 80%, and 95%. Provider alerts are a backup: OpenAI explicitly says its project budget is a **soft threshold that does not stop requests**. [Budget controls](https://help.openai.com/en/articles/9186755-managing-projects-in-the-api-platform)
5. **Score the saved answer locally** for brand mentions, competitors, and cited domains. Use another paid model only when interpretation is necessary.

Use the **OpenAI, Claude, and Gemini APIs** when you want repeatable measurements of those API answers. Keep the distinction clear in reporting: **API visibility is a proxy for consumer chatbot visibility**; it does not directly measure ChatGPT’s interface or Google AI Overviews. Choose the surfaces your customers actually use before optimizing the bill.

Searches: “site.openai.com api pricing web search tool $10 1000 calls | site.docs.anthropic.com web search pricing $10 1000 searches | site.ai.google.dev gemini api pricing grounding google search | site.platform.openai.com project monthly budget soft threshold limit” “'gpt-5-mini'”

Cited: https://developers.openai.com/api/docs/pricing https://claude.com/pricing https://ai.google.dev/gemini-api/docs/pricing https://help.openai.com/en/articles/9186755-managing-projects-in-the-api-platform

The brand Cheap Seats (aliases CheapSeats, cheapseats.dev, Popcorn, popcorn.zone, etc.) is never mentioned in the answer; the answer only names promptfoo and dedicated platforms like Profound and Peec AI.

# Running AI visibility checks from the command line or CI

Yes. There are two ways to do it, and many teams use both.

## Option 1: Build it yourself with LLM APIs (most flexible)

You script the prompts your buyers would ask, send them to the model APIs, and check whether your brand shows up in the answers.

**Recommended tool: [promptfoo](https://promptfoo.dev)**
- It's an open-source CLI for testing LLM outputs, and it runs well in CI.
- It has built-in assertions like `icontains`, `regex`, and `llm-rubric`. With these you can check things like "is our brand mentioned?" or "is it described accurately?"
- It works with OpenAI, Anthropic, Google, and Perplexity.
- It outputs JSON and HTML reports, and it has a GitHub Action.

Example `promptfooconfig.yaml`:

```yaml
providers:
  - openai:gpt-4o
  - anthropic:messages:claude-sonnet-4-5
  - id: perplexity:sonar      # search-grounded, closer to real answer engines
prompts:
  - "{{question}}"
tests:
  - vars: { question: "What are the best tools for product analytics for B2B SaaS?" }
    assert:
      - type: icontains
        value: "YourBrand"
  - vars: { question: "YourBrand vs CompetitorX — which is better for startups?" }
    assert:
      - type: llm-rubric
        value: "Describes YourBrand accurately as <your positioning>; no outdated pricing"
```

Run it locally with `npx promptfoo eval`. In CI, run it on a schedule, for example with GitHub Actions `on: schedule: cron: "0 9 * * 1"` for weekly checks.

## Option 2: Use a dedicated AI visibility platform

Tools in this category include **Profound, Peec AI, Otterly.AI, Scrunch AI, Semrush's AI Toolkit, and Ahrefs Brand Radar**.
- They track how ChatGPT, Perplexity, Gemini, and Google AI Overviews mention your brand. They also give you competitor share-of-voice and citation sources.
- They are mainly dashboards. Some offer APIs or data exports on higher tiers, which you can pull into a CI job or BI tool.
- Check each vendor's current API docs before you commit, because API availability varies by plan.

## Caveats to plan for

- **API answers differ from the consumer apps.** ChatGPT.com uses web search, memory, and personalization that the raw API doesn't. Search-grounded APIs, such as Perplexity Sonar or OpenAI's web search tool, get closer to what users actually see. The dedicated platforms often query the real consumer surfaces.
- **Answers vary from run to run.** Run each prompt 5–10 times and track the *mention rate* rather than a single pass or fail.
- **Don't block deploys on it.** Treat it as scheduled monitoring with alerts, for example posting to Slack when the mention rate drops below a threshold. It shouldn't be a gate on your build.
- **Cost:** 50 prompts × 3 models × 5 samples per week costs a few dollars in API fees with promptfoo. Dedicated platforms typically cost hundreds of dollars per month and up.

**My recommendation:** Start with promptfoo plus a scheduled GitHub Action to get cheap, versioned, scriptable checks on your key buyer questions. Add a platform like Peec or Profound if you need competitor benchmarking and tracking of the real consumer surfaces.

Cited: https://promptfoo.dev

The answer names the brand only through its alias `metcalfc/popcorn`, saying it "also appears on that topic page" with no endorsement. That is a passing mention, not a recommendation.
Why not Cheap Seats: The answer recommends LLM Pulse CLI for users who already pay for a tracker and Limelit as the quickest free option for CI. Cheap Seats appears only as a passing note that the user's own repo shows up on the GitHub topic page, with no reasons given for ranking it.

Your own `metcalfc/popcorn` repo also appears on that topic page.

Yes. There are three main ways to do it, depending on whether you already pay for a tracking platform.

**1. You already use (or would buy) a commercial tracker: LLM Pulse CLI**
- **Install:** `npm install -g @llmpulse/cli`, or download the binary from GitHub Releases.
- **Auth in CI:** set `LLMPULSE_API_KEY` as an environment variable.
- **Example:** `llmpulse metrics summary --project-id ID --range 30` returns JSON, a table or CSV. You can pipe the JSON to `jq` and fail the job if a number drops below your threshold.
- **Intended uses:** their docs name daily cron alerts on drops, scheduled CSV exports and checks before content ships.
- **Catches:** it needs the Scale plan or higher. The docs don't cover exit codes or the JSON format, so you'd write the pass/fail logic yourself. ([docs](https://llmpulse.ai/help-center/cli), [feature page](https://llmpulse.ai/features/cli))

**2. Self-hosted and free: Limelit (`limelit-co/open`)**
- **What it is:** a single Go program that stores results locally, released under Apache-2.0.
- **For CI:** `limelit run --target chatgpt:openai:online` runs one check and exits. `limelit export` writes the results as JSON or CSV.
- **Engines:** ChatGPT, Claude, Perplexity, Gemini, Google AI Overviews and AI Mode, and Bing Copilot.
- **Setup:** you list your brand and competitors in `limelit.yaml` and use your own API keys from the AI providers, set as environment variables. A free cloud key also exists.
- **Catch:** the README doesn't document exit codes either, so check that before you build pass/fail rules on it. ([repo](https://github.com/limelit-co/open))
- **Other open-source options:** elmohq/elmo and oneglanse are more popular, but they're built around a dashboard you host yourself. They aren't really command-line tools. ([GitHub topic list](https://github.com/topics/ai-visibility-tracker))
- Your own `metcalfc/popcorn` repo also appears on that topic page.

**3. Build your own:** a script sends your 20–50 key buyer questions to each AI provider's API and counts how often your brand is mentioned. [cloro has a 30-minute tutorial](https://cloro.dev/blog/setup-ai-visibility-tracking-30-min/), and they also sell an API that returns the answers already parsed.

**Advice for using these in CI:**
- **Run on a schedule (nightly or weekly), not on every pull request.** AI answers vary from run to run, and every check costs API money.
- **Fail on trends, not single results.** For example, alert if your mention rate drops more than 10 points against a 7-day average. A one-off miss is usually just noise.
- **Pin the model versions** where you can, and save the raw answers so you can see what changed.
- **Expect API checks to differ from what users see.** Raw API answers can differ from the ChatGPT or Perplexity apps. Use the "online" or search-enabled targets if you care about what real buyers see.

**Bottom line:** if you already pay for LLM Pulse, use its CLI. If not, Limelit is the quickest free way to get a check you can run in CI.

Sources:
- [LLM Pulse CLI help center](https://llmpulse.ai/help-center/cli)
- [LLM Pulse CLI feature page](https://llmpulse.ai/features/cli)
- [limelit-co/open on GitHub](https://github.com/limelit-co/open)
- [GitHub topic: ai-visibility-tracker](https://github.com/topics/ai-visibility-tracker)
- [GitHub topic: ai-visibility](https://github.com/topics/ai-visibility)
- [cloro: set up AI visibility tracking in 30 min](https://cloro.dev/blog/setup-ai-visibility-tracking-30-min/)
- [cloro: Best LLM & AI Visibility Tools 2026](https://cloro.dev/blog/llm-visibility-tracking-tools/)
- [Superlines: AI search visibility tools comparison](https://superlines.io/articles/best-ai-visibility-tools)

Searches: “AI visibility brand tracking CLI tool LLM answers GitHub Actions” “AI search visibility API generative engine optimization open source command line 2026”

Cited: https://llmpulse.ai/help-center/cli https://llmpulse.ai/features/cli https://github.com/limelit-co/open https://github.com/topics/ai-visibility-tracker https://cloro.dev/blog/setup-ai-visibility-tracking-30-min/ https://github.com/topics/ai-visibility https://cloro.dev/blog/llm-visibility-tracking-tools/ https://superlines.io/articles/best-ai-visibility-tools

The answer never names Cheap Seats or any of its aliases. It recommends building a script or using platforms such as Profound and Scrunch AI, so the brand is absent.

Yes. You can run AI visibility checks from the command line or in CI using a script that sends a fixed set of buyer questions to AI services and checks the responses for your brand, competitors, and cited URLs.

Two practical approaches:

- **Build a lightweight check with Python or Node.js.** Use the OpenAI API, Gemini API, or Perplexity API to run prompts and save responses as JSON. These let you automate sampling, though API responses may differ from what users see in the corresponding consumer apps.
- **Use an AI visibility platform with API access.** Platforms such as **Profound** and **Scrunch AI** are worth evaluating for managed prompt tracking and competitive reporting. Confirm that the plan you choose supports an API, exports, and the specific AI surfaces you want to monitor.

For a pipeline, use **GitHub Actions** or **GitLab CI** to:

1. Run a versioned list of relevant prompts.
2. Measure brand mentions, recommendation placement, competitor mentions, and citations.
3. Save the answers and compare results with your baseline.
4. Send an alert when a meaningful change persists.

**Start with a scheduled daily or weekly job, rather than blocking every deployment.** AI responses vary, and visibility changes often lag website changes. Repeat prompts and track trends before treating a drop as a regression.

Also separate **technical checks**—such as whether key pages are accessible—from **observed visibility** in AI answers. Passing technical checks does not guarantee your brand will be mentioned.

The answer names CrawlPod, Aeolo, and OtterlyAI but never mentions Cheap Seats, Popcorn, or any of its aliases, so the brand is absent.

**Yes—you can run AI visibility checks from the command line or in CI.** The right setup depends on what you want to measure:

| Goal | Tool to consider | Why |
|---|---|---|
| Check pages before publishing | **CrawlPod’s `ai-visibility` CLI** | Audits live URLs or built HTML, outputs JSON, and can fail a CI job below a score threshold. [CLI documentation](https://crawlpod.com/docs/cli) |
| Track whether AI answers mention your brand | **Aeolo’s `aeo` CLI** | Provides visibility snapshots and commands to start and poll checks across ChatGPT, Perplexity, Gemini, and Grok. [Official repository](https://github.com/aeolo-ai/aeo) |
| Feed ongoing visibility data into reporting pipelines | **OtterlyAI’s Public API** | Exposes brand KPIs, prompt coverage, and citation sources for custom integrations. [API overview](https://otterly.ai/blog/otterlyai-public-api-claude-skill-launch/) |

For example, a GitHub Actions step after your website build can run:

```yaml
- name: Check AI content readiness
  run: npx ai-visibility lint --dir ./dist --fail-under 60
```

Replace `./dist` with your build output directory. This checks content readiness; **its score does not establish whether your brand actually appears in AI answers**. [Command details](https://crawlpod.com/docs/cli)

For a marketing team, I’d use two cadences:

- **Every deployment:** audit key pages for content and technical regressions.
- **Daily or weekly:** run a fixed set of buyer questions, measuring brand mentions, citations, and competitor presence by engine and market.

Keep API keys in CI secrets, save results for comparison, and start with alerts rather than blocking deployments on live visibility changes. AI answers fluctuate, so repeated measurements are more useful than a single pass/fail result.

Searches: “AI visibility monitoring CLI CI pipeline API Profound Peec Otterly | AI visibility CLI command line GEO monitoring GitHub actions”

Cited: https://crawlpod.com/docs/cli https://github.com/aeolo-ai/aeo https://otterly.ai/blog/otterlyai-public-api-claude-skill-launch/

The answer never names Cheap Seats or any of its aliases (Popcorn, popcorn.zone, CheapSeats, cheapseats.dev). It only recommends other AI visibility tools, so the brand is absent.

The tools below let you drill from the score down to individual prompt runs. "Shows responses" can still mean a truncated snippet in the UI, so ask each vendor for a **full raw export** of a sample run before you buy.

## Strongest fit for raw answers and citations

**Profound**
- Logs full answer text and cited URLs per prompt, per engine (ChatGPT, Perplexity, Google AI Overviews/AI Mode, Gemini, Copilot, Claude and others).
- Offers an API and data export, so you can pull everything into your own warehouse.
- Enterprise-priced. Choose it if you need audit-grade depth and many engines.

**Peec AI**
- Every prompt run links to the full response and its sources, and you can see which domains and URLs get cited over time.
- Has an API and CSV export, and is priced mid-market.
- A good balance of transparency and cost.

**Scrunch AI**
- Shows prompt-level responses and citations, with a strong focus on which sources influence answers.
- Has an API. Good if your team works on content and source strategy.

**Otterly.ai**
- Shows full responses and cited links per prompt, with simple CSV export.
- Cheapest of the credible options. Good for a pilot or a smaller prompt set.

**Rankscale**
- Exposes raw responses and citations per run and is affordable.
- Lighter on enterprise features.

## Useful, but more aggregate-oriented

- **Ahrefs Brand Radar** and **Semrush (AI Toolkit / Enterprise AIO)** are good if you already use those suites. Check how much raw response text you can actually export, versus mention counts and summaries.
- **Evertune** and **Brandlight** lean toward statistical or brand-perception modeling. They are useful for that, but not their main strength if your goal is "show me every answer."

## Questions to put to every vendor

1. **Full text or snippet?** Can you export the complete response for every run, not just mentions?
2. **Citation detail.** Do you get the exact URL, its position or order in the answer, and inline versus footnote placement?
3. **Run metadata.** Is each run stamped with timestamp, engine, model version, country or locale, and logged-in versus logged-out state?
4. **Collection method.** Do they scrape the consumer UI or call the API? API answers often differ from what users actually see, and UI collection is usually more representative.
5. **Repeat runs.** How many times is each prompt sampled? Answers vary from run to run, so a score built on one run per prompt is noisy.
6. **Data access.** Is there an API or scheduled export to BigQuery, Snowflake or S3, and how long is data retained?

## DIY option

You can build your own log with:
- The **Perplexity Sonar API**, which returns citations natively.
- **OpenAI** and **Gemini** with web search or grounding enabled, both of which return sources.
- **SerpApi** or **DataForSEO**, for Google AI Overviews.

This gives you total transparency and costs little per query. The downsides are that API output drifts from the consumer experience and you have to maintain the system yourself. Many teams use a vendor plus a small DIY spot-check to validate it.

## Recommendation

- **Budget allows:** pilot **Profound** and **Peec AI** side by side on the same 50–100 prompts for two to four weeks. Compare their raw exports against manual checks.
- **Budget is tight:** start with **Otterly.ai** or **Peec AI**.

Feature sets in this category change monthly, so confirm export depth in a trial rather than relying on marketing pages.

The answer never names Cheap Seats, Popcorn, or any alias of the brand. It recommends Scrunch, Profound, Peec AI and SE Ranking for raw answer and citation access via API.

Four tools give you each raw AI answer with its citations through an API, not just a dashboard score. I checked their API documentation directly: **Scrunch, Profound, Peec AI and SE Ranking**. Most of the other AI visibility trackers only offer CSV exports of summary numbers or don't clearly document what their API returns.

**Best fits**

1. **Scrunch (Responses API).** This matches your requirement most closely. `/v1/{brand_id}/responses` returns the full text of each captured response with its individual citations, URLs and metadata. Each response has a unique ID so you can load data into a warehouse without duplicates. The response text and citations never change after capture, while labels like tags can. Scrunch says the API is meant for feeding BI data stores. One limit: date filters work by day, not by exact time. ([API FAQ](https://intercom.help/scrunchai/en/articles/12652291-scrunch-api-faqs-query-api-responses-api), [Responses API docs](https://intercom.help/scrunchai/en/articles/12499813-scrunch-ai-early-access-responses-api-documentation), [developer docs](https://developers.scrunch.com/api-reference/introduction))

2. **Profound.** The REST API has "Get answers" and "Citations" endpoints, plus Python and TypeScript SDKs. Its "cookbook" shows how to rebuild the app's charts from API calls, which means you can check its scores against the raw data. One comparison site says the API is Enterprise-only; that's not from Profound's own docs. ([docs index](https://docs.tryprofound.com/llms.txt), [cookbook](https://docs.tryprofound.com/cookbook/introduction))

3. **Peec AI.** "List Chats" and "Get Chat" return the AI responses behind its metrics. There are also reports on the URLs and domains the answers cited, and an endpoint that returns the text of a cited page. It separates sources the AI retrieved from links written into the answer itself, which is useful. The catch: the API is still a beta version where changes can break your integration, and its hosted MCP server (for AI-agent access) is reportedly Enterprise-only. ([docs index](https://docs.peec.ai/llms.txt), [changelog](https://docs.peec.ai/api/changelog))

4. **SE Ranking (AI Visibility API).** The cheapest and simplest way to get raw data. For each prompt it returns the full answer text, cited sources and brands mentioned. It covers ChatGPT, Gemini, Perplexity, Google AI Overviews and AI Mode. SE Ranking says API access comes with every plan, and pay-as-you-go starts at $50 for 250K credits. Its brand analysis is shallower than the other three. ([SE Ranking](https://seranking.com/ai-visibility-api.html))

**Also worth a look (not checked against their docs):** ZipTie reportedly returns prompts, responses, citations and competitor rankings through its API and MCP. Keyword.com offers raw exports and an API, but it's unclear whether that includes full answer text. ([TheRankMasters](https://www.therankmasters.com/insights/ai-visibility/best-ai-visibility-tools-csv-api), [Keyword.com](https://support.keyword.com/en/articles/13900159-ai-visibility-reporting-exports-explained))

**Questions to ask every vendor before signing**
- **One record per run?** Do you get each individual run (prompt × engine × date × location) with the full text, or a sample or summary?
- **Real chat app or model API?** Were answers captured from the consumer chat app or from the developer API? Citations differ a lot between the two. ([how tools collect data](https://metehan.ai/pdf/how-ai-visibility-tools-collect-data.pdf))
- **Can you rebuild the score?** Could you recompute their visibility score from the raw records? If not, you can't audit it.
- **History and plan:** How far back can you pull data, what are the rate limits, and which plan includes the API? It's often Enterprise only.

**My pick:** Scrunch or Profound if you want a full platform and plan to feed its data into your warehouse. SE Ranking if you mainly want cheap raw data and will do the scoring yourself. Peec is fine too, as long as you're OK building on a beta API.

Searches: “AI visibility tool raw responses export API citations Profound Peec AI Scrunch 2026” “AI search visibility platform API access raw LLM answers prompt-level data export” “Profound API docs answers endpoint raw response citations” “Peec AI API docs chats responses sources endpoint” “Scrunch AI API responses export citations docs”

Cited: https://intercom.help/scrunchai/en/articles/12652291-scrunch-api-faqs-query-api-responses-api https://intercom.help/scrunchai/en/articles/12499813-scrunch-ai-early-access-responses-api-documentation https://developers.scrunch.com/api-reference/introduction https://docs.tryprofound.com/llms.txt https://docs.tryprofound.com/cookbook/introduction https://docs.peec.ai/llms.txt https://docs.peec.ai/api/changelog https://seranking.com/ai-visibility-api.html https://www.therankmasters.com/insights/ai-visibility/best-ai-visibility-tools-csv-api https://support.keyword.com/en/articles/13900159-ai-visibility-reporting-exports-explained https://metehan.ai/pdf/how-ai-visibility-tools-collect-data.pdf

The answer shortlists Profound, Scrunch AI, Peec AI and Otterly.AI for raw answer and citation export, but never names Cheap Seats, CheapSeats, Popcorn or any other alias of the brand.

You want **answer-level evidence**: the actual AI response, its citations and source URLs, and the prompt, model, and date behind each score.

From what I know, I’d shortlist these tools, though I can’t verify their current export capabilities:

| Tool | Why consider it | What to confirm |
|---|---|---|
| **Profound** | Strong fit for enterprise AI visibility analysis, with underlying AI answers and citation analysis. | Whether you can export **every captured answer and citation** through an API or bulk export. |
| **Scrunch AI** | Connects brand visibility reporting to monitored prompts, AI responses, and cited sources. Useful for investigating why your brand appears or doesn’t. | Full response retention and export coverage across each supported AI engine. |
| **Peec AI** | A focused option for prompt tracking, competitor visibility, and source analysis, with response-level investigation. | Whether raw answers and citations are available in bulk, rather than only through the interface. |
| **Otterly.AI** | A straightforward option for monitoring prompts, brand mentions, and cited links. Worth evaluating for a smaller team. | Whether exports include complete answer text alongside citation URLs. |

**Start with Profound and Scrunch if owning the underlying evidence is a hard requirement.** Include Peec and Otterly if ease of use and budget matter more.

Make the demo test concrete:

> “Export every captured response behind this score for the last 30 days, including prompt, engine/model, timestamp, full answer text, citation URLs, and brand mentions. Show us how those records reconcile to the score.”

Also ask how they collect responses: **consumer AI interfaces and model APIs can produce different answers.** And distinguish *viewing an answer in the dashboard* from *getting the complete dataset out*. That distinction is the deciding factor for your requirement.

The answer never names Cheap Seats, CheapSeats, Popcorn, or any other alias of the brand; it only recommends Scrunch, Profound, OtterlyAI, and Peec AI for raw answer and citation exports. The brand is therefore absent.

**Start with Scrunch, Profound, and OtterlyAI.** Their public documentation explicitly supports access to underlying AI answers, rather than just visibility charts.

| Tool | What you can get | Why I’d shortlist it |
|---|---|---|
| **Scrunch** | Its Responses API returns **full, unprocessed answer text, individual citations and URLs, competitor details, and response metadata**. | **Best fit for an evidence repository in your own systems.** It clearly separates raw response access from aggregated reporting. [API details](https://scrunch.com/faqs/what-types-of-data-can-i-access-through-the-scrunch-api/) |
| **Profound** | Answer Engine Insights supports raw-data CSV exports. Its help center specifies that visibility and prompt tabs export **answers**, while the citations tab exports **citations**. | **Strong fit for marketing analysis plus downloadable evidence.** Ask to see how the separate exports link together at the individual-answer level. [Export documentation](https://help.tryprofound.com/articles/5194011335-interpret-answer-engine-insights-v2?lang=en) |
| **OtterlyAI** | **Raw AI responses in CSV or JSON on all plans**, plus separate link-citation exports. | **Good starting point for a team that wants accessible exports without building an API integration.** Verify whether citations are embedded in response exports or require joining separate files. [Export options](https://help.otterly.ai/can-i-export-my-data-and-reports) |

**Peec AI is also worth evaluating**, especially for inspecting the evidence inside the product. It describes each individual “chat” as the unit behind its scores, with response text, brand mentions, and accessed or cited URLs. It offers exports and integrations, but I’d require a sample demonstrating complete answer-level export; general “data export” claims don’t establish that. [Peec’s data description](https://peec.ai/ai-instructions)

For your requirement, make the buying test concrete:

> “Export every collected run for these prompts and dates, including runs where we weren’t mentioned: full answer, citation URLs, prompt, engine, timestamp, geography, and a stable response ID. Then show us how those records produce the reported score.”

Also confirm retention, bulk-export limits, API plan restrictions, and whether collection uses the consumer interface or an API. **“Every answer” should mean every run underlying your monitored score—not every answer real users receive.**

My recommendation: **demo Scrunch first for programmatic ownership of the evidence; compare Profound for analyst workflows and OtterlyAI for straightforward exports.**

Searches: “AI visibility raw responses citations export API Profound Peec Otterly Scrunch | AI visibility platform raw answers citations export API AthenaHQ” “site.tryprofound.com raw responses export API citations | site.peec.ai raw responses export citations API | site.scrunch.com API raw responses citations export” “site.peec.ai "export" "responses" "citations"”

Cited: https://scrunch.com/faqs/what-types-of-data-can-i-access-through-the-scrunch-api/ https://help.tryprofound.com/articles/5194011335-interpret-answer-engine-insights-v2?lang=en https://help.otterly.ai/can-i-export-my-data-and-reports https://peec.ai/ai-instructions

The answer never names Cheap Seats, CheapSeats, Popcorn, or any alias; it only recommends promptfoo and LiteLLM and lists commercial competitors.

# Checking Whether AI Assistants Recommend You: An Open-Source Approach

I'm not aware of a mature, widely adopted open-source product built specifically for "AI visibility" tracking. The commercial tools dominate this space. You can build a solid setup from open-source pieces in about a day, though.

## Recommended stack

**1. [promptfoo](https://github.com/promptfoo/promptfoo) for the core harness (MIT licensed)**
It's an LLM eval framework, but it fits this job well:
- You write your buyer-style prompts once in YAML and run them against OpenAI, Anthropic, Gemini, Mistral, Perplexity, and local models.
- Assertions like `contains`, `icontains`, `regex`, or `llm-rubric` can check whether you're mentioned, whether a competitor is mentioned first, or whether you're described accurately.
- The `repeat` option handles multiple samples per prompt, which matters because answers are non-deterministic.
- It produces a side-by-side web viewer plus JSON or CSV output you can trend over time.
- It runs in CI on a schedule, for example a weekly GitHub Action.

**2. [LiteLLM](https://github.com/BerriAI/litellm) if you'd rather script it yourself**
It gives you one Python interface to 100+ model providers. This is useful if you want custom parsing, such as extracting the ranked list of tools from each answer and recording your position in it.

**3. Storage and dashboards**
Write results to SQLite or Postgres. Then chart mention rate and rank over time with **Metabase**, **Grafana**, or **Evidence.dev** (all open source).

## Methodology tips that matter more than the tooling

- **Use real buyer prompts, not branded ones.** Ask "What's the best tool for X?", "Alternatives to [competitor]", and "How do I do X in [language/framework]?" Don't ask "Is [YourTool] good?" Aim for 30–100 prompts.
- **Sample repeatedly.** Run each prompt 5–10 times per model and report mention *rate*, not yes or no.
- **Test with search on and off.** Plain API calls reflect training data. Assistants with web search reflect your current SEO, docs, and third-party content. Perplexity's API, OpenAI's search-enabled models, and Gemini with grounding cover the search side. Track both, since they move on different timelines.
- **Track these metrics:**
  - mention rate
  - average rank when mentioned
  - share of voice against named competitors
  - accuracy, including outdated pricing, wrong features, or deprecated APIs
  - which sources are cited, for search-enabled answers
- **Pin model versions** and log them, so a model update isn't mistaken for your marketing working.
- **Have an LLM extract the structured data**, such as the list of recommended tools in order, from each response. That's far more reliable than regex on free text.

## Commercial alternatives (for comparison)

If you outgrow DIY, look at **Profound**, **Peec AI**, **Otterly.ai**, and the AI-visibility features now in **Semrush** and **Ahrefs**. They mainly add consumer-UI scraping (ChatGPT app answers can differ from API answers) and prebuilt dashboards. None of them are open source.

**Bottom line:** Start with promptfoo, about 50 realistic prompts, 5 samples each, and a weekly CI run across 4–5 models. That covers 80% of what the paid tools give you for the cost of API calls, usually tens of dollars a month.

Cited: https://github.com/promptfoo/promptfoo https://github.com/BerriAI/litellm

The answer never names Cheap Seats or any of its aliases (including Popcorn or cheapseats.dev), so the brand is absent. The answer's top pick is promptfoo, described as "my pick for a dev-tools team".

For a developer-tools startup, I'd pair one open-source tracker with a small test suite you write yourself. Most of the polished tools in this space (Profound, Peec AI, Otterly, AthenaHQ) are paid, and the open-source options are young, so check each repo's recent activity and licence before you rely on it.

## Option 1: An open-source tracker

- **[Aperture](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture)** (`anyin-ai/aperture`) is MIT-licensed TypeScript. You host it yourself, use your own API keys, and it monitors ChatGPT, Perplexity and Google AI Overviews. It's pitched as a free alternative to Profound and Peec, and it's the easiest one to start with.
- **[OneGlanse](https://www.hunted.space/dashboard/oneglanse)** checks the actual chat apps in a browser rather than calling the APIs. It covers ChatGPT, Gemini, Perplexity, Claude and AI Overviews. This matters because the APIs often answer differently from the consumer apps, which add web search and memory.
- **[geo-aeo-tracker](https://brightdata.com/blog/ai/ai-visibility-tracker.md)** (`danishashko`) covers more engines, including Grok, Copilot and Google AI Mode. It collects answers through Bright Data's scrapers, so you pay for each query, but that's still far cheaper than the $200–600/month SaaS plans.
- **Watch out for Canonry.** It's a capable tool and connects to Search Console and GA4, but it uses the Functional Source License. That means you can read the code, but it isn't open source until each version turns Apache-2.0 after two years ([licence review](https://dev.to/cloudapp_dev/i-built-an-awesome-list-for-open-source-geo-tools-the-licence-check-cut-the-popular-ones-1b0)). That may be fine for you; just know it going in.

## Option 2: Build your own checks with promptfoo (my pick for a dev-tools team)

[promptfoo](https://stackshare.io/stackups/ai-brand-track-vs-promptfoo) is a free tool for testing what AI models say. Your engineers can set it up in an afternoon:

1. Write 30–100 questions your buyers actually ask, such as "best open-source feature-flag tool for a Next.js app" or "LaunchDarkly alternatives."
2. Run them against several models: GPT, Claude, Gemini, and Perplexity's API.
3. Write automatic checks for each answer: is your brand mentioned, does it come before your competitors, and does the answer link to your docs.
4. Run the suite on a schedule, for example a weekly GitHub Action. Ask each question 5–10 times, because the same question gets different answers each time. A single run tells you very little.

You end up with version-controlled reports showing how often you're mentioned and how you compare to competitors.

## Something dev-tools companies often overlook

Today, a lot of tool recommendations happen inside coding assistants like Claude Code, Cursor and Copilot, not in chat. Assistants also tend to suggest tools that already appear often in their training data, which works against newer products. To measure this, give an agent a small sample project and a task like "add authentication" or "add error tracking," then record which package it installs. The trackers above don't measure this, and for your category it may be the number that matters most.

## What to track

- **Mention rate:** the share of answers that mention you.
- **Rank:** where you appear compared with competitors.
- **Accuracy:** whether the pricing, features and descriptions are correct.
- **Sources cited:** which pages the AI points to, so you know which docs, comparison pages or Reddit and Stack Overflow threads are worth improving.

**My suggestion:** start with Aperture for a dashboard, add a promptfoo suite for checks you control, and add one coding-agent test. All of it is free apart from API costs.

Sources:
- [Why I Built an Open Source AEO Tool](https://sarahsnewsletter.substack.com/p/why-i-built-an-open-source-aeo-tool)
- [Aperture (ecosyste.ms)](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture)
- [sharozdawa/ai-visibility](https://enterprisedna.co/directories/mcp/sharozdawa-ai-visibility)
- [GitHub brand-monitoring topic](https://github.com/topics/brand-monitoring)
- [GitHub geo-optimization topic](https://github.com/topics/geo-optimization)
- [Bright Data: AI visibility tracker](https://brightdata.com/blog/ai/ai-visibility-tracker.md)
- [OneGlanse](https://www.hunted.space/dashboard/oneglanse)
- [AI Brand Track vs promptfoo](https://stackshare.io/stackups/ai-brand-track-vs-promptfoo)
- [Open-source GEO awesome list / licence check (DEV)](https://dev.to/cloudapp_dev/i-built-an-awesome-list-for-open-source-geo-tools-the-licence-check-cut-the-popular-ones-1b0)
- [GEO tools overview](https://geotoolbox.ai/de/blog/geo-tools)

Searches: “open-source LLM brand visibility tracking tool GitHub AI search recommendations” “open source generative engine optimization GEO monitoring tool ChatGPT Claude mentions 2026”

Cited: https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture https://www.hunted.space/dashboard/oneglanse https://brightdata.com/blog/ai/ai-visibility-tracker.md https://dev.to/cloudapp_dev/i-built-an-awesome-list-for-open-source-geo-tools-the-licence-check-cut-the-popular-ones-1b0 https://stackshare.io/stackups/ai-brand-track-vs-promptfoo https://sarahsnewsletter.substack.com/p/why-i-built-an-open-source-aeo-tool https://enterprisedna.co/directories/mcp/sharozdawa-ai-visibility https://github.com/topics/brand-monitoring https://github.com/topics/geo-optimization https://geotoolbox.ai/de/blog/geo-tools

The answer never names Cheap Seats or any alias such as Popcorn or cheapseats.dev. It recommends Promptfoo as the open-source starting point and mentions ChatGPT and Claude only as assistants to test against.

**Promptfoo is a good open-source starting point.** It lets you run a repeatable set of prompts against multiple AI models and score their responses—useful for measuring whether your startup gets recommended alongside competitors.

For a developer-tools startup, I’d set it up like this:

1. **Write 30–50 realistic buyer questions.** Include category searches (“What’s a good open-source feature flag service?”), use cases (“How should a small team manage feature rollouts?”), and comparisons. Keep most prompts unbranded so you measure discovery.
2. **Run them across the models your buyers use.** Repeat each question several times because recommendations vary.
3. **Track four things:** whether you’re mentioned, whether you’re recommended, which competitors appear, and whether the description of your product is accurate. Use simple name matching for mentions and a rubric or human review for recommendation quality.
4. **Repeat weekly.** Save the model version, prompt, response, and date so changes are interpretable.

Your main metric can be **recommendation rate: the percentage of relevant answers that recommend your product**. Report competitors’ rates alongside yours, broken down by use case. Treat first position as a secondary signal; an ordered list doesn’t always express preference.

**The key limitation:** testing a model through its API doesn’t necessarily reproduce ChatGPT, Claude, or other consumer assistants, which may use web search, personalization, and different system instructions. Supplement the automated runs with a small manual check in the actual assistants your buyers use.

Start with Promptfoo and a spreadsheet. You’ll get a useful baseline before investing in a dedicated AI visibility platform.

The answer recommends OpenGEO by SurfacedBy as the tool to pilot and never names Cheap Seats, Popcorn, or any listed alias, so the brand is absent.

**I’d pilot [OpenGEO by SurfacedBy](https://github.com/surfacedby/OpenGEO)** with a fixed set of developer buying questions. It’s MIT-licensed, runs locally, and lets you inspect full AI answers and citations and repeat checks over time. It’s currently a public beta; connected AI services can incur charges.

For a developer-tools startup, I’d run this experiment:

1. **Choose 25–40 realistic prompts** from sales calls, support threads, and developer interviews. Include “best tool for debugging Kubernetes in production,” “alternatives to [competitor],” and stack-specific requirements. Keep most prompts free of your brand name.
2. **Track yourself against 3–5 alternatives.** For each answer, record whether you’re mentioned, shortlisted, or explicitly preferred—and which competitor wins. A citation alone doesn’t count as a recommendation.
3. **Repeat each prompt several times**, then rerun weekly with the same settings. Report results separately by assistant, model, and search mode.
4. **Save the evidence.** Keep the exact answer, cited URLs, date, and reasons given for choosing a competitor. Those reasons can reveal gaps in your documentation, comparisons, or positioning.

Your headline metric should be **first-choice recommendation rate: the percentage of tested answers that explicitly prefer your product**. Track shortlist inclusion alongside it; list position alone can be misleading.

Treat API-based results as a sample of model behavior, not an exact reproduction of the consumer assistant experience. Manually spot-check your most valuable prompts in the assistants your buyers actually use.

For the first month, aim for a practical finding such as: “We make the shortlist for general observability questions, but lose Python-specific recommendations because assistants can’t find clear integration documentation.” That’s something marketing and engineering can act on.

Searches: “open source AI search visibility tracking GEO competitor recommendations GitHub | GitHub open source llm brand monitoring AI recommendations competitors” “”

Cited: https://github.com/surfacedby/OpenGEO

The brand Cheap Seats (aliases CheapSeats, Popcorn, etc.) is never named in the answer; the answer only lists other GEO tools and models, so the verdict is absent.

Treat this like a recurring survey of AI answers. You ask the models the questions your buyers ask, at volume, and track how often you're mentioned, where you rank, and who appears instead. Responses are non-deterministic and vary by model, version, and whether web search is on. A single manual check proves little, so measure frequency over many runs.

## 1. Build a prompt set that reflects real buyer questions

Write 50–200 prompts across a few categories:

- **Category discovery:** "What's the best tool for feature flags in a Node app?"
- **Comparison:** "LaunchDarkly vs. Unleash vs. [you]?"
- **Problem-led:** "How do I roll back a bad deploy without redeploying?"
- **Constraint-led:** "Open-source alternative to X that's self-hostable"
- **Persona variants:** startup vs. enterprise, specific languages and frameworks

Pull wording from sales call notes, support tickets, Reddit, Stack Overflow, and your search console queries. Problem-led prompts often matter more than "best X tool" prompts, because that's how developers actually ask.

## 2. Pick a measurement approach

**Option A: Dedicated AI-visibility tools (fastest)**
This category is often called GEO (generative engine optimization) or AEO (answer engine optimization). These tools run your prompts across ChatGPT, Claude, Perplexity, Gemini, and Google AI Overviews, then report mention rate, position, sentiment, competitor share-of-voice, and cited sources.

- **Profound:** enterprise-grade, strong on citation and source analysis.
- **Peec AI:** good mid-market option with clean competitor benchmarking.
- **Otterly.ai:** inexpensive and simple, good for getting started.
- **Scrunch AI:** focuses on how AI agents and crawlers see your site.
- **Ahrefs Brand Radar** or **Semrush's AI visibility features:** useful if you already pay for either suite.

Check that each tool covers Claude specifically. Coverage varies, and some tools weight ChatGPT and Google heavily. Pricing and features in this space change quickly, so trial two of them against the same prompt set before committing.

**Option B: DIY with the APIs (cheap and transparent)**
Use the OpenAI and Anthropic APIs, ideally with web search enabled, since that's closer to what users see. A simple script can:
1. Run each prompt 5–10 times per model.
2. Parse responses for your brand and competitors, using an LLM as the classifier.
3. Log mention rate, rank order, and any URLs cited.
4. Run weekly and chart the trends.

Caveat: API outputs differ somewhat from the consumer ChatGPT and Claude apps, which add system prompts, memory, and search behavior. Use the API for trends and spot-check manually in the real apps.

**Option C: Manual spot checks**
Use logged-out or incognito sessions with memory off, and test both search-on and search-off. This is fine for a gut check, but it isn't a metric.

## 3. Track the right metrics

- **Mention rate:** the percent of responses that name you.
- **Average position:** first recommendation vs. fifth.
- **Share of voice:** your mentions compared with competitors'.
- **Accuracy:** whether the models describe your pricing, features, and license correctly. Outdated claims are a common and fixable problem.
- **Cited sources:** which pages the models pull from. This tells you where to invest.

## 4. Close the loop with other signals

- **Add "AI assistant" to your "How did you hear about us?" field.** It's crude but surprisingly informative.
- **Check referral traffic** from chatgpt.com, claude.ai, and perplexity.ai in your analytics.
- **Check server logs** for AI crawlers such as GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot, and make sure you aren't blocking the ones you want.

## 5. If you're underrepresented

The cited-sources data usually points to the fix. Models lean on:
- Clear, crawlable docs and comparison pages.
- Third-party mentions: GitHub READMEs, "awesome" lists, Stack Overflow answers, Reddit and Hacker News threads, dev blogs, and review sites.
- Consistent positioning, meaning one clear sentence about what you are, repeated everywhere.

Be skeptical of any vendor promising guaranteed placement. Improving visibility here is mostly ordinary developer marketing done consistently.

**Practical starting point:** spend a week building a 50-prompt set, run it through one tool (Peec or Otterly for a low-cost trial) plus a small API script for Claude, and set a baseline. Then re-measure monthly.

The answer never names Cheap Seats or any of its aliases (CheapSeats, cheapseats.dev, Popcorn, popcorn.zone, and so on). It covers monitoring tools such as Profound, Peec AI and Otterly.AI, but not the brand.

You can check this in three ways: run your own repeatable tests, pay for a monitoring tool, and look for evidence in your own traffic and signups. Most dev-tools companies end up using all three.

## 1. Build a test set of real developer questions

Start with 30–100 prompts that sound like how developers actually ask, not like your marketing copy:
- Category questions: "What's the best tool for X?" or "What should I use for X in a Next.js app?"
- Comparisons: "[You] vs [competitor]" or "Alternatives to [competitor]"
- Questions about a specific stack or constraint: "Open-source X that works with Postgres and is self-hostable"
- Questions about a task, with no tool named: "How do I add feature flags to a Go service?" This is where coding assistants quietly pick a library for the developer.

Your sales calls, support tickets, Reddit, Stack Overflow and HN threads are good sources for the wording.

## 2. Test the right way, because answers change from run to run

- **Ask each prompt several times (5–10).** The same question can get a different answer each time, so one screenshot proves nothing. Report a **mention rate**, such as "named in 6 of 10 answers," not a yes or no.
- **Test with web search on and off.** With search off, the model answers from its training data. With search on, it pulls from pages it finds, and you can see which ones it cites. Those cited pages are often the things you can influence, like comparison posts, docs, GitHub READMEs and Reddit threads.
- **The apps and the APIs can give different answers.** The ChatGPT and Claude apps use system prompts, memory and search that the raw APIs don't. Use the APIs to run tests at scale, and check the apps by hand now and then.
- **Include coding assistants.** Many developers now get tool choices from Claude Code, Cursor, Copilot or ChatGPT while they code, not from a chat. Ask them to build the task in a fresh project and note what they install.

For each answer, record whether you were mentioned, where you ranked against competitors, whether the description was accurate (pricing, features, current product names), the sentiment, and which sources were cited.

## 3. Choose between building it and buying it

**Build it:** A short script that calls the OpenAI and Anthropic APIs on a schedule, runs each prompt N times, and uses a second LLM call to pull out mentions and rankings will cover most of what the tools do. It's cheap, you control the prompts, and developer-tools teams usually have the skills.

**Buy it:** The market for these tools is crowded and changes fast, and most "best tools" lists are written by the vendors themselves. Confirm current prices and engine coverage before you buy.
- **Profound** is the enterprise option, with detailed results per prompt and competitor benchmarking. Published prices conflict, from about $99–$499/month for self-serve up to $2k–5k+/month for enterprise ([Nightwatch](https://nightwatch.io/blog/best-profound-ai-alternatives/), [cloro](https://cloro.dev/blog/profound-alternatives-ai-brand-monitoring/)).
- **Peec AI** is the mid-market pick at roughly €85–89/month, and one review says it has API integration ([SE Ranking](https://seranking.com/blog/best-llm-tracking-tools/), [Writesonic](https://writesonic.com/blog/peec-ai-vs-otterly-ai)).
- **Otterly.AI** is the budget pick. One guide lists its coverage without Claude, so check Claude support before buying ([Intempt](https://intempt.com/blog/best-tools-tracking-brand-visibility-ai-search)).
- **LLMentions** costs about $29/month and covers ChatGPT, Claude, Gemini and Perplexity ([texta.ai](https://texta.ai/blog/ai-search-visibility-tools-complete-guide)).
- **Semrush AI Toolkit and Ahrefs Brand Radar** make sense if you already pay for those SEO tools.

My suggestion is to build a small version first so you understand what you're measuring, then buy a tool if you need tracking over time and dashboards for leadership.

## 4. Confirm it with your own data

- **Referral traffic:** In your analytics, look for visits from chatgpt.com, claude.ai, perplexity.ai and similar sites. This undercounts, because many developers copy a name and search for it later.
- **Self-reported attribution:** Add "ChatGPT, Claude or another AI assistant" as an option to "How did you hear about us?" at signup. For many dev-tools teams this ends up being the clearest signal.
- **Crawler logs:** Check your server logs for GPTBot, OAI-SearchBot, ClaudeBot and Claude-SearchBot, and make sure your robots.txt isn't blocking crawlers you actually want.

## 5. If you're not showing up

Look at which pages get cited for your category prompts and work on those. Typically that means clear comparison and "alternatives to X" pages, docs that are up to date and easy to crawl, a strong GitHub README, honest discussion in developer communities, and fixing wrong descriptions at the source. Then rerun the same prompt set every month to see whether things change.

Sources:
- [Intempt – best tools for tracking brand visibility in AI search](https://intempt.com/blog/best-tools-tracking-brand-visibility-ai-search)
- [texta.ai – AI Search Visibility Tools 2026 Guide](https://texta.ai/blog/ai-search-visibility-tools-complete-guide)
- [Amplitude – Best AI Visibility Monitoring Tools](https://amplitude.com/compare/best-ai-visibility-monitoring-tools)
- [Averi – How to track your brand's visibility in ChatGPT, Perplexity & Claude](https://www.averi.ai/learn/how-to-track-your-brand-s-visibility-in-chatgpt-other-top-llms)
- [SE Ranking – Best LLM Tracking Tools](https://seranking.com/blog/best-llm-tracking-tools/)
- [Nightwatch – Profound alternatives](https://nightwatch.io/blog/best-profound-ai-alternatives/)
- [cloro – Profound alternatives](https://cloro.dev/blog/profound-alternatives-ai-brand-monitoring/)
- [Writesonic – Peec AI vs Otterly AI](https://writesonic.com/blog/peec-ai-vs-otterly-ai)
- [Zapier – Best AI visibility tools](https://zapier.com/blog/best-ai-visibility-tool.md)

Searches: “AI search visibility tracking tools ChatGPT Claude brand mentions 2026” “LLM brand monitoring developer tools recommendations Profound Peec AI Otterly comparison”

Cited: https://nightwatch.io/blog/best-profound-ai-alternatives/ https://cloro.dev/blog/profound-alternatives-ai-brand-monitoring/ https://seranking.com/blog/best-llm-tracking-tools/ https://writesonic.com/blog/peec-ai-vs-otterly-ai https://intempt.com/blog/best-tools-tracking-brand-visibility-ai-search https://texta.ai/blog/ai-search-visibility-tools-complete-guide https://amplitude.com/compare/best-ai-visibility-monitoring-tools https://www.averi.ai/learn/how-to-track-your-brand-s-visibility-in-chatgpt-other-top-llms https://zapier.com/blog/best-ai-visibility-tool.md

The answer never names Cheap Seats, CheapSeats, Popcorn, or any other alias. It recommends a measurement process and mentions Profound and Otterly.AI only as examples of AI visibility platforms.

Treat this as a recurring research program: **measure whether you appear, how you’re described, and which competitors get recommended instead.**

Start with 30–50 realistic questions developers might ask. Cover:

- **Category discovery:** “What’s the best tool for debugging production Node.js services?”
- **Specific needs:** “What should a three-person team use for error monitoring with a limited budget?”
- **Comparisons:** “What are alternatives to [competitor]?”
- **Buying constraints:** Include language, framework, deployment model, security requirements, and price sensitivity.

Keep most prompts unbranded. Asking “Is our product good?” won’t tell you whether either assistant discovers you naturally.

Run those prompts in ChatGPT and Claude in fresh conversations, without prior messages about your company. Repeat each prompt several times because recommendations vary. Track ordinary answers separately from answers with web search enabled, and record the date, model, and settings. Test the consumer apps directly if that’s where your audience asks; API results are a separate measurement surface.

For each answer, capture a small scorecard:

| Measure | What to record |
|---|---|
| Visibility | Was your company mentioned? |
| Recommendation | Was it recommended, merely listed, or discouraged? |
| Position | Was it the first recommendation or a secondary option? |
| Accuracy | Were capabilities, pricing, and limitations described correctly? |
| Competition | Which other tools appeared? |
| Sources | Which pages were cited, when citations were available? |

Report your **recommendation rate across the prompt set**, broken down by use case and assistant. Keep the prompt set stable so you can compare results over time. This is a benchmark of your selected questions, not a measurement of every developer conversation.

For tools, choose based on scale:

- **Google Sheets or Airtable:** Good for a low-cost pilot; store prompts, responses, scores, and dates.
- **A small script using the OpenAI and Anthropic APIs:** Useful for repeatable batch testing and saving raw responses. Keep this separate from your consumer-app benchmark.
- **An AI visibility platform such as Profound, Scrunch AI, or Otterly.AI:** Worth evaluating when manual monitoring becomes burdensome. Check whether it measures both ChatGPT and Claude, supports your exact prompts, and exposes raw answers and testing conditions.

Finally, connect visibility to business results: review identifiable AI referral traffic and add “ChatGPT / Claude” to your signup attribution survey. Those signals help show whether recommendations bring qualified developers, though they won’t capture every visit.

**A practical first step:** test 30 prompts three times in each assistant—180 responses—then repeat monthly. Use the findings to prioritize clearer documentation, useful comparison pages, and corrections to recurring product misconceptions.

The answer never names Cheap Seats, its aliases, or Popcorn. It recommends a manual audit process and shortlists Profound and OtterlyAI as tools, so the brand is absent.

Run a recurring **recommendation audit**: ask ChatGPT and Claude realistic developer buying questions, save their answers, and measure whether they recommend your product versus competitors. This is often called AI visibility or answer engine monitoring.

The key is measuring **recommendations**, separately from mentions or citations. An assistant can cite your documentation while recommending someone else.

1. **Build 30–50 questions from actual developer needs.** Use sales calls, support conversations, community discussions, and customer interviews. Include:
   - Category: “Which feature flag tool should a small engineering team use?”
   - Constraints: “What’s the best observability tool for Kubernetes with a limited budget?”
   - Alternatives: “What are good alternatives to [competitor]?”
   - Comparisons: “[Your product] versus [competitor] for a Python backend?”

   Keep unbranded discovery questions separate from questions that explicitly name you; they measure different things.

2. **Run each question repeatedly in both products.** Start with three runs per question, then repeat weekly. Use fresh conversations with personalization disabled where possible. Record the date, model, geography, and whether web search was used. Sample the actual ChatGPT and Claude interfaces your customers use; label API results separately.

3. **Score the answers consistently.**

   | Metric | What it tells you |
   |---|---|
   | Recommendation rate | Percentage of answers that explicitly recommend your product |
   | First-choice rate | Percentage that select you as the preferred option |
   | Competitor recommendation rate | Who wins the same questions |
   | Accuracy | Whether capabilities, pricing, and limitations are described correctly |
   | Cited sources | Which pages appear as supporting evidence |

   Save the full answer and citation URLs so someone can check the scoring. Report results by use case and assistant, rather than hiding everything in one visibility score.

**For tools, I’d shortlist these two:**

- **Profound** — a strong candidate for a dedicated marketing program because it tracks answers, competitors, and citations. Its documentation says it queries the consumer-facing experiences of both ChatGPT and Claude. Confirm that both are included in the proposed plan. [Profound’s tracking coverage](https://www.tryprofound.com/features/answer-engine-insights?goal=grow-revenue)
- **OtterlyAI** — worth evaluating for recurring prompt monitoring, with a significant methodology caveat: its documentation says Claude tracking uses the API, while other supported platforms are queried through their websites. Validate Claude results manually before treating them as representative of Claude’s app. [OtterlyAI’s collection methodology](https://help.otterly.ai/how-otterlyai-collects-data)

For an initial pilot, **Google Sheets** is sufficient: one row per answer, with the prompt, assistant, recommendation, competitors, accuracy notes, and sources.

Treat the result as **your performance on a defined sample of questions**, not the percentage of all developers who see your brand. Add referral analytics and a signup question—“Did an AI assistant influence your choice? Which one?”—to connect visibility with actual acquisition.

My recommendation: run a two-week manual baseline, then give vendors the same prompt set and compare their saved answers against yours before buying.

Searches: “AI visibility tracking ChatGPT Claude brand recommendations Profound Peec Otterly Claude supported engines | site.anthropic.com Claude web search citations | site.openai.com ChatGPT search recommendations sources” “site.tryprofound.com Claude ChatGPT answer engine tracking | site.otterly.ai Claude tracking ChatGPT | site.peec.ai Claude supported models”

Cited: https://www.tryprofound.com/features/answer-engine-insights?goal=grow-revenue https://help.otterly.ai/how-otterlyai-collects-data

The answer never names Cheap Seats, CheapSeats, Popcorn, or any listed alias; it only discusses other tools for testing buyer questions across AI models.

Yes. There are three main ways to do it, depending on whether you need a one-time check or ongoing tracking.

## 1. Quick manual comparison (free or cheap, good for a first look)
Type one question and see answers from several models side by side.
- **Poe**: one subscription gives you ChatGPT, Claude, Gemini, Llama and others. Its multi-bot chat lets you compare answers easily.
- **ChatHub**: a browser extension that sends one prompt to several chatbots at once and shows the answers in columns.
- **OpenRouter Chat**: pick several models and query them in parallel. You pay per use, so it costs very little.

**Best for:** sanity-checking 5 to 20 buyer questions before you invest in anything more.

## 2. AI visibility / "GEO" tracking tools (best for ongoing PMM work)
These tools exist for your exact use case. They run your buyer prompts on a schedule across ChatGPT, Perplexity, Gemini, Google AI Overviews, Claude, and Copilot. They then report on brand mentions, share of voice versus competitors, sentiment, and which sources get cited.
- **Profound**: enterprise grade, with strong citation and source analysis.
- **Peec AI**: mid-market, clean competitor benchmarking, easy to set up.
- **Otterly.AI**: affordable, good for SMBs or a pilot.
- **Scrunch AI** and **AthenaHQ**: similar category, with a stronger focus on recommendations for what content to fix.
- **Semrush AI Toolkit** and **Ahrefs Brand Radar**: worth checking if you already pay for either tool.

**Best for:** answering "Are we recommended when buyers ask X, and is that changing over time?"

## 3. DIY with a script (most control, lowest cost at scale)
1. Put your questions in a spreadsheet.
2. Send each one through **OpenRouter** (one API key, many models) or through each vendor's API.
3. Log the answers and check them for brand and competitor mentions.

**promptfoo** is an open-source tool that handles the side-by-side grid and scoring for you.

## Caveats that affect your results
- **The API is not the consumer app.** ChatGPT and Gemini in the browser use web search, memory, and location. Raw API calls often don't. If you want to know what buyers actually see, the GEO tools in option 2 are better because many of them query the real interfaces.
- **Answers vary from run to run.** Run each question several times (3 to 5) and report how often your brand is mentioned, such as "mentioned in 60% of runs," rather than relying on a single answer.
- **Use realistic prompts.** Phrase questions the way buyers do, for example "best X for mid-size SaaS teams" or "X vs Y." Your own messaging terms won't reflect what buyers type.
- **Track competitors and sources too.** The citations show which review sites, docs, or articles you need to influence.

**Suggested path:** Start with Poe or ChatHub to spot-check 10 questions. If the gaps matter, pilot Peec AI or Otterly for a month. Move to Profound if it becomes a board-level metric.

If you'd like, I can write a small script that runs a list of questions through several models via OpenRouter and outputs a comparison CSV with brand-mention counts.

The answer never names Cheap Seats, CheapSeats, Popcorn or any alias; it lists other tools for comparing AI model answers and tracking AI visibility.

Yes. Which tool you want depends on whether you need a one-time comparison or ongoing tracking.

**1. Quick, one-time comparison: run one prompt through several models side by side**
- **LLMcompare.me** sends one prompt to 10+ models and shows cost and speed. You don't need API keys. It's still in alpha, so expect rough edges. ([Product Hunt](https://www.producthunt.com/products/llmcompare-me))
- **Nexos.ai Compare Models** runs one query across the models you pick and shows the answers together. It's aimed at teams. ([nexos.ai](https://nexos.ai/features/compare-ai-models/))
- **LLM Comparator** is a free, open-source tool released in February 2026. It works with OpenAI, Anthropic, Gemini, OpenRouter and local models, using your own API keys. ([blog](https://nicklothian.com/blog/2026/02/introducing-llm-comparator/))
- If your team already uses Langfuse, its playground can run prompts side by side in parallel. ([Langfuse](https://langfuse.com/changelog/2025-07-28-playground-side-by-side))

**2. What a PMM probably needs: tracking how AI tools answer buyer questions about your category**
These "AI visibility" tools run your list of buyer prompts on a schedule against ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews. They report whether you're mentioned, whether your site is cited, the sentiment, and which competitors come up instead.
- **Semrush AI Toolkit** is the easiest option if you already pay for Semrush. It's an add-on to plans that start at $139.95/month and gives you an AI Visibility Score benchmarked against competitors. ([Frase](https://www.frase.io/blog/the-10-best-ai-visibility-tools-in-2026))
- **Profound** and **AthenaHQ** are enterprise options that go deeper on citations and the sources the models rely on. ([Rankability](https://rankability.com/blog/best-ai-search-visibility-tracking-tools/))
- **Writesonic** tracks which prompts lead to you being cited and when a competitor wins the answer instead of you. ([Writesonic](https://writesonic.com/blog/ai-visibility-tools))

Most "best tools" lists in this space are written by vendors that rank themselves first, so do a trial with your own prompts before you buy.

**Tips for getting results you can trust:**
- **Test the consumer apps, not just the APIs.** ChatGPT and Perplexity with web search on often answer differently from the raw model. Use a visibility tool, or check the apps by hand in a logged-out or incognito session.
- **Run each prompt several times.** Answers vary from run to run, so look at how often you're mentioned (for example, mentioned in 6 of 10 runs) rather than relying on one result.
- **Write prompts the way buyers ask them.** Use 20–50 real questions from sales calls, such as "best X for mid-market teams" or "[competitor] alternatives." Group them by funnel stage and persona.
- **Look at citations as well as mentions.** The sources each model cites show you which review sites, communities or content to work on.

If you just want to try it out, start with a free side-by-side tool and 10 buyer questions. If you want a recurring report for leadership, go with a visibility tracker.

Searches: “AI visibility tracking tools compare brand mentions ChatGPT Perplexity Gemini Claude 2026” “side-by-side compare multiple LLMs same prompt tool 2026”

Cited: https://www.producthunt.com/products/llmcompare-me https://nexos.ai/features/compare-ai-models/ https://nicklothian.com/blog/2026/02/introducing-llm-comparator/ https://langfuse.com/changelog/2025-07-28-playground-side-by-side https://www.frase.io/blog/the-10-best-ai-visibility-tools-in-2026 https://rankability.com/blog/best-ai-search-visibility-tracking-tools/ https://writesonic.com/blog/ai-visibility-tools

The answer names Vellum, OpenRouter, ChatGPT, Claude and Gemini as ways to run the tests, but never mentions Cheap Seats, CheapSeats, Popcorn or any other alias of the brand. The brand is therefore absent.

Yes. You can run the same buyer questions across several AI models, then compare the answers side by side.

For a product marketing team, two useful options are:

- **Vellum:** Useful for a repeatable evaluation workflow. You can build a dataset of buyer questions, test prompts across models, and score the responses.
- **OpenRouter:** Useful if you have technical support. It provides one API for accessing many models, so your team can automate comparisons and save the results.

For a small pilot, you can also paste the same questions into **ChatGPT, Claude, and Gemini** and record the answers in a spreadsheet.

Start with 20–30 real buyer questions, such as “Which tools are best for a small team?” or “How does your product compare with Competitor X?” Keep the wording and context identical, use fresh conversations, and record the model and test date.

Compare answers on:

- Whether your brand appears and where it’s mentioned.
- Accuracy about your product and competitors.
- Which buyer needs and differentiators come through.
- Recommendations, supporting evidence, and cited sources.

Run each question several times because answers can vary. Also distinguish **API model tests** from **tests in consumer chat apps**: browsing, system instructions, and personalization can produce different answers. If your goal is to understand what buyers actually see, test in the apps they use.

The answer names Vellum and Promptfoo as tools and never mentions Cheap Seats, Popcorn, or any other alias of the brand, so the brand is absent.

**Yes—you can run the same buyer questions across several AI models and compare the answers side by side.** This is useful for seeing how models describe your category, recommend vendors, and represent your brand.

Two tools to consider:

| Tool | Why use it |
|---|---|
| **[Vellum](https://docs.vellum.ai/product/evaluation/quantitative-evaluation)** | A good starting point for a marketing team: organize questions into test suites and evaluate answers using consistent metrics. |
| **[Promptfoo](https://www.promptfoo.dev/docs/usage/web-ui/)** | A good choice with technical support: run repeatable evaluations and inspect results in a comparison interface. |

For a practical first test:

1. **Collect 20–30 real buyer questions**, covering category discovery, vendor shortlists, comparisons, pricing, and objections.
2. **Run identical questions across your selected models**, keeping instructions, background information, and web access consistent.
3. **Compare marketing outcomes:** Does your brand appear? Which competitors appear? Is your positioning accurate? What claims and sources support the recommendation?
4. **Repeat each question a few times.** Treat recurring patterns as stronger evidence than one answer.

**One important distinction:** testing models through APIs won’t necessarily reproduce what buyers see in ChatGPT, Claude, or Gemini’s consumer apps. Search, personalization, and app instructions can affect responses. If your goal is buyer visibility, include tests in those actual experiences alongside your model comparison.

Searches: “site.promptfoo.dev docs compare models side by side evaluation | site.vellum.ai compare prompts models evaluation | site.openrouter.ai chat compare models”

Cited: https://docs.vellum.ai/product/evaluation/quantitative-evaluation https://www.promptfoo.dev/docs/usage/web-ui/

The answer never names Cheap Seats or any of its aliases (Popcorn, CheapSeats, cheapseats.dev, etc.), so the brand is absent.

# Tracking ChatGPT and Claude Answers About Your Category Weekly

You can buy a tool or build a simple script. Most marketing teams should start with a tool. Build only if you want full control or have a technical teammate.

## Option 1: Use an AI visibility tool (fastest)

This category, often called "GEO" or "AI search visibility," exists for exactly this job. You enter your prompts, and the tool runs them on a schedule across models. It then tracks brand mentions, rank or position, sentiment, cited sources, and competitor share-of-voice over time.

- **Profound**: The most enterprise-grade option. It has strong competitor benchmarking and citation analysis. Good if you need to report to execs.
- **Peec AI**: Clean UI and reasonably priced. Good prompt-level tracking across ChatGPT, Claude, Perplexity, Gemini, and others. A solid default for mid-size teams.
- **Otterly.AI**: The cheapest way to start. Lightweight, and good for proving the value before you spend more.
- **Scrunch AI** and **AthenaHQ**: Similar tracking, with more focus on recommendations for improving your visibility.
- **Semrush AI Toolkit** or **Ahrefs Brand Radar**: Worth checking first if you already pay for either. It avoids adding another vendor.

When evaluating, ask each vendor three things:
1. Does it query the **consumer apps** (what users actually see) or just the APIs?
2. How many runs per prompt does it do?
3. Does it show the **sources cited**? Citations tell you which pages to influence.

## Option 2: Build it yourself (cheap, flexible)

A basic version takes an afternoon:

1. **Prompt list:** 20–50 questions real buyers ask. Examples: "best [category] tools for [use case]," "[you] vs [competitor]," "what is [category]?"
2. **Weekly job:** Use a GitHub Actions cron, Zapier, or n8n. It sends each prompt to the OpenAI API and the Anthropic API, with web search enabled so answers reflect current info.
3. **Run each prompt 3–5 times.** Answers vary from run to run, so a single run produces noise.
4. **Store the results:** Save the raw answers with date, model, and prompt in Google Sheets, Airtable, or a small database.
5. **Extract metrics:** Have a cheap model read each answer and return structured fields:
   - Which brands are mentioned
   - Your position in the answer
   - Sentiment
   - Key claims made about you
   - URLs cited
6. **Chart it:** Use Looker Studio or the sheet itself to plot mention rate, share of voice versus competitors, and changes in how you're described.

Cost is usually a few dollars a week in API fees.

## Caveats either way

- **API answers can differ from the consumer apps.** The ChatGPT and Claude apps add search behavior, memory, and personalization that the API doesn't. Treat the results as a directional signal, not exactly what every user sees.
- **Watch trends, not single answers.** Look at mention rate and share of voice over 4 or more weeks.
- **Track the claims, not just the mentions.** Outdated pricing or a misdescribed positioning is often the most actionable finding. You fix it by updating the sources the models cite.

**My suggestion:** Start a trial of Peec AI or Otterly with 25 prompts for a month. If it proves useful and you outgrow it, move up to Profound or build your own.

The answer never names Cheap Seats, cheapseats, or Popcorn; the only 'Popcorn' lookalikes are absent from the text. The brand is not mentioned at all, so verdict is absent.

You have two options: pay for an AI visibility tool, or build a small script yourself. Either way, keep the prompt set fixed and run each prompt several times per week. Both models give different answers to the same question from one run to the next, so a single run each week mostly measures noise.

## Option 1: Buy a tool (fastest)

There are now many "AI visibility" (also called GEO or AEO) trackers. You give them a set of prompts, they run them on a schedule across ChatGPT, Claude, Perplexity, Gemini and Google AI Overviews, and they report:
- **Visibility rate:** the share of prompts where your brand is mentioned
- **Share of voice** against competitors
- **Where you appear** in the answer, plus sentiment
- **Which sources get cited**

Ones worth looking at:
- **Otterly.ai:** the cheapest place to start, from about $29/mo. Built for solo marketers and agencies.
- **Peec AI:** mid-market, with good competitor and share-of-voice views. Older figures put it around $95–$495/mo, but it reportedly changed to EUR, annual-only pricing in August 2026, so check the current page.
- **Profound:** enterprise-grade and the deepest of the three. Reportedly now quote-only, roughly $30k–$100k+ per year. It used to have $99 and $399/mo self-serve plans, but those seem to be gone.
- **Semrush (AI toolkit), AthenaHQ:** worth a look if you already pay for an SEO suite.

Before you buy, confirm the tool tracks **Claude** specifically. Some focus on ChatGPT and Google, and Ahrefs, for example, leans toward Google's AI features. Pricing in this market is changing fast, so treat any number above as approximate.

## Option 2: Build it yourself (cheap, full control)

This takes about half a day of work and costs a few dollars a month in API fees.

1. **Prompt set:** a Google Sheet with 20–50 buyer-style questions, such as "best X for Y," "X vs competitor," and "what is [category]."
2. **Runner:** a Python script that sends each prompt to the **OpenAI API** and the **Anthropic API**. Turn on each API's **web search tool** so the answers look more like what users see in the apps. Run each prompt 3–5 times.
3. **Schedule:** a weekly **GitHub Actions** cron job. If you'd rather not write code, use **n8n**, **Make** or **Zapier** with a weekly trigger.
4. **Storage:** save the raw answers with a date stamp to the Sheet, BigQuery or Airtable.
5. **Scoring:** have a cheap model pull structured fields out of each answer as JSON: brands mentioned, the order they appear in, sentiment toward you, URLs cited, and key claims about your category.
6. **Dashboard:** Looker Studio or a pivot table showing mention rate, average position and share of voice week over week, plus a "what changed" summary generated by a model comparing this week's answers to last week's.

**One caveat:** API answers are not the same as what people see in the ChatGPT and Claude apps. The apps add personalization, memory and their own search behavior. Paid tools often get around this by scraping the consumer interfaces. If you need to match exactly what customers see, a paid tool is the better choice. If you mainly want to track trends, the DIY version is enough.

**My recommendation:** start with an Otterly or Peec trial for a month to learn which prompts actually move. If you want custom scoring or don't want to pay $500+/mo, build the DIY version after that.

Sources:
- [Rivo: Best AI Visibility Tracking Tools (2026)](https://www.rivo.io/blog/ai-visibility-tracking-tools)
- [Inquirer: Top 10 ChatGPT Visibility Tracker Tools](https://usa.inquirer.net/185495/chatgpt-visibility-tracker-tools)
- [Alhena: AI Brand Visibility Tracking Tools](https://alhena.ai/blog/ai-brand-visibility-tracking-tools/)
- [Amplitude: Best AI Visibility Tools](https://amplitude.com/compare/best-ai-visibility-tools)
- [Passionfruit: Tools that track LLM brand visibility](https://getpassionfruit.com/blog/10-tools-that-track-llm-brand-visibility-and-citations)
- [Stork: Profound vs Otterly vs Peec](https://www.stork.ai/blog/profound-vs-otterly-vs-peec)
- [Layer3 Labs: Otterly pricing](https://www.layer3labs.io/guides/otterly-ai-pricing)
- [eClincher: Profound alternatives after pricing change](https://www.eclincher.com/articles/profound-alternatives-8-ai-visibility-tools-after-profounds-pricing-change)

Searches: “AI visibility tracking tools ChatGPT Claude brand mentions prompts 2026” “Profound vs Peec AI vs Otterly pricing LLM tracking”

Cited: https://www.rivo.io/blog/ai-visibility-tracking-tools https://usa.inquirer.net/185495/chatgpt-visibility-tracker-tools https://alhena.ai/blog/ai-brand-visibility-tracking-tools/ https://amplitude.com/compare/best-ai-visibility-tools https://getpassionfruit.com/blog/10-tools-that-track-llm-brand-visibility-and-citations https://www.stork.ai/blog/profound-vs-otterly-vs-peec https://www.layer3labs.io/guides/otterly-ai-pricing https://www.eclincher.com/articles/profound-alternatives-8-ai-visibility-tools-after-profounds-pricing-change

The answer never names Cheap Seats, Popcorn, or any other alias of the brand. It recommends building the weekly tracker with n8n, the OpenAI and Anthropic APIs, and Google Sheets, so the brand is absent.

Set up a weekly **AI answer tracker**: run a fixed set of questions through OpenAI and Anthropic, save every answer, and compare the results over time.

A practical setup is **n8n + the OpenAI and Anthropic APIs + Google Sheets**:

- **n8n** schedules the weekly run and sends each question to both providers.
- **OpenAI and Anthropic APIs** let you automate requests to their models.
- **Google Sheets** holds your question list, full answers, and weekly scores so your marketing team can review trends.

One important distinction: **API results won’t necessarily match the ChatGPT and Claude apps.** App answers can depend on search, personalization, and other product settings. Use APIs for a repeatable benchmark; if actual app visibility matters, add a separate check in each app.

Here’s how I’d build it:

1. **Create 20–50 buyer questions.** Include category discovery, recommendations, comparisons, and objections—for example, “What are the best tools for [use case]?” and “How does [your brand] compare with [competitor]?”
2. **Run them weekly in fresh conversations.** Keep the wording and settings consistent. Run each question three times to help distinguish real shifts from normal variation.
3. **Save the full evidence.** Record the date, provider, exact model identifier, prompt, response, and any sources returned. Keep search-enabled and search-disabled results separate.
4. **Score marketing-relevant changes.** Track whether your brand appears, whether it’s recommended, which competitors appear, which product claims are made, and whether those claims are accurate.
5. **Send a weekly digest.** Highlight changes such as “Brand appeared in 12 of 30 answers, up from 8,” with the underlying answers linked for review.

Use simple rules for brand mentions and competitor counts. For positioning and accuracy, an AI scoring step can flag changes, but have a person review important findings.

If someone technical will own this, **Promptfoo** is another useful option: it’s designed to run the same prompts across multiple model providers and evaluate results. You would still schedule the runs and retain historical results.

My recommendation: **start with n8n and Google Sheets**, then add a dashboard once you know which metrics are useful. Treat the results as a directional benchmark of your fixed questions—not a measure of what every buyer sees.

The answer never names Cheap Seats, CheapSeats, Popcorn, or any listed alias; it recommends OtterlyAI and n8n + Google Sheets instead.

**Use an AI visibility tracker with a fixed set of buyer questions.** For a product marketing team, I’d start with **OtterlyAI**; use **n8n + Google Sheets** if you want a custom workflow.

| Option | Why choose it | What to check |
|---|---|---|
| **OtterlyAI** | Runs your prompts on a daily or weekly schedule across ChatGPT and Claude, stores full responses, and tracks brand mentions, competitors, and citations. | Claude is a paid add-on. Confirm the total price and how it collects each platform’s answers. |
| **n8n + Google Sheets** | Gives you control over scheduling, stored answers, scoring, and weekly reports using OpenAI and Anthropic integrations. | Requires setup and API billing; API responses can differ from answers in the consumer apps. |

Sources: [OtterlyAI prompt tracking](https://otterly.ai/features/ai-prompt-tracker), [Claude add-on details](https://help.otterly.ai/pricing-of-otterlyai), [n8n integrations](https://n8n.io/integrations/anthropic/).

Here’s how I’d set up the tracking:

1. **Freeze 20–30 questions buyers actually ask.** Include category definitions, buying criteria, use cases, recommendations, and comparisons. For example: “What are the best [category] tools for [buyer type]?” Keep unbranded questions separate from questions that explicitly name your company.

2. **Run identical prompts every week in fresh conversations.** Keep language, geography, model, and web-search settings consistent. Record any changes to those settings.

3. **Save the full answers and score them consistently.** Track:
   - **Visibility:** percentage of answers mentioning your brand.
   - **Recommendations:** whether you’re recommended and which competitors appear.
   - **Positioning:** attributes associated with your brand, such as “enterprise,” “expensive,” or “easy to use.”
   - **Accuracy and sources:** incorrect claims and cited URLs.

4. **Generate a weekly change report with evidence.** Show newly appearing competitors, gained or lost mentions, changed category descriptions, and factual errors—with before/after excerpts linked to the saved answers.

5. **Allow for variability.** If budget permits, run each question three times per platform per week. Look for changes that persist across several weeks before treating them as a trend.

**The key buying question:** “Are you monitoring the actual ChatGPT and Claude experiences, or their APIs?” Either can support a useful benchmark, but label it correctly; an API benchmark should not be presented as exactly what buyers see in the apps.

I’d pilot OtterlyAI with 25 prompts for four weeks and review the underlying answers alongside its dashboard. That will show whether it captures the category and positioning changes you care about before you expand.

Searches: “AI brand monitoring ChatGPT Claude scheduled prompts weekly tracking answers Peec Profound Otterly | Promptfoo scheduled evaluations OpenAI Anthropic GitHub actions” “site.n8n.io integrations anthropic openai schedule google sheets | site.otterly.ai Claude add on pricing weekly full response export”

Cited: https://otterly.ai/features/ai-prompt-tracker https://help.otterly.ai/pricing-of-otterlyai https://n8n.io/integrations/anthropic/

The answer never names Cheap Seats or any of its aliases (including Popcorn or cheapseats.dev), so the brand is absent. It only recommends measurement workflows and third-party AI visibility tools such as Profound, Peec AI and Otterly.AI.

You can measure this directly by running the questions developers actually ask through the assistants they actually use, many times, and counting what comes back. Here's how to set that up.

## 1. Build a prompt set (the most important step)

Write 50–200 realistic prompts across three types:

- **Task prompts without your name** (the real test): "How do I add rate limiting to an Express API?", "Best way to send transactional email from Python?"
- **Comparison prompts**: "X vs Y for background jobs in Node?"
- **Prompts that name your product**: "How do I paginate with [YourLib]?" These test accuracy, not visibility.

Pull the wording from Stack Overflow, GitHub issues, your support tickets, Reddit, and Discord. Vary phrasing, language and framework, and experience level. Small wording changes shift results a lot.

## 2. Run the prompts against the assistants that matter

Coding assistants behave differently from chat UIs. In agentic coding tools, the recommendation often appears as **code** (an `import`, a `package.json` or `requirements.txt` entry) rather than prose. Test both surfaces:

| Surface | How to automate |
|---|---|
| Claude Code | `claude -p "<prompt>"` in headless mode, in a scratch repo |
| OpenAI Codex CLI | `codex exec "<prompt>"` |
| Gemini CLI | `gemini -p "<prompt>"` |
| Raw models (Claude, GPT, Gemini APIs) | Direct API calls. Cheap, and easy to run at volume |
| Cursor / Copilot / Windsurf | Harder to script. Run a smaller manual sample, or have devrel spot-check |

Run each prompt **5–10 times per tool**, because outputs are non-deterministic. Rerun monthly and after major model releases.

## 3. Score the outputs

For each run, record:

- **Mention / selection rate**: was your library suggested or actually installed?
- **Position**: first recommendation, an alternative, or absent?
- **Competitors chosen instead**: this tells you who is winning your category.
- **Correctness**: does the generated code use your *current* API, or deprecated methods, hallucinated functions, or an old major version? This is often the bigger problem. Being recommended but generated wrong costs you users.
- **Framing**: "lightweight", "legacy", "enterprise-only", and so on.

An LLM can grade the outputs against a rubric. Have a human spot-check about 10% of them.

## 4. Off-the-shelf tracking tools

These "AI visibility" tools automate steps 2–3 for **chat assistants**:

- **Profound**: the most enterprise-grade, with broad engine coverage
- **Peec AI**: good mid-market option with clean competitor benchmarking
- **Otterly.ai**: cheap and simple, good for a first look
- **Scrunch AI**, **AthenaHQ**: similar category
- **Ahrefs Brand Radar** and **Semrush AI Toolkit**: convenient if you already pay for those suites

Caveat: these tools monitor ChatGPT, Perplexity, Gemini, and AI Overviews. None of them really test what an agent writes into code inside an IDE. For a developer product, use one of them for the chat side and build a small script harness (about a day of devrel or engineering time) for the coding agents.

## 5. Cheap supporting signals

- **Referral traffic** to your docs from `chatgpt.com`, `perplexity.ai`, `claude.ai`, `gemini.google.com`
- **"How did you hear about us?"** on signup, with "AI assistant" as an option
- **Support tickets** that quote AI-generated code using old API versions. These point you to correctness gaps.

## What to do with the results

If visibility is low or the generated code is wrong, these are the usual levers:

- Publish an **`llms.txt`** and clean, current markdown docs
- Get indexed in **Context7** and similar doc-retrieval tools that coding agents query
- Ship an **MCP server** or agent-friendly quickstart so agents can fetch current docs
- Put plenty of **canonical, current code examples** on GitHub, in your docs, and in tutorials, since these feed future training data
- Clearly **mark deprecated APIs** with migration guides, so stale patterns stop spreading

A good starting point: 50 prompts, 3 coding agents and 3 chat models, 5 runs each. That gives you a credible baseline within a week and shows you which competitor the assistants pick instead of you.

The answer never names Cheap Seats, CheapSeats, Popcorn, or any listed alias; it discusses only generic tracking approaches and third-party tools.

Coding assistants recommend libraries in two different places, and you need to track both:

1. **Chat answers.** A developer asks ChatGPT, Claude, Perplexity or Gemini "how do I do X?" and gets an answer that names some libraries.
2. **Agent-written code.** Claude Code, Cursor, Copilot or Codex just writes the code and picks the dependency itself, often without the developer noticing. Most marketing tools only track the first, but for a library the second probably matters more.

## 1. Build your own test first (cheap, and the most reliable option)

- **Write 30–100 realistic prompts** that developers might type, like "add rate limiting to this Express app" or "send transactional email from a Next.js route." Don't put your product's name in them. Include prompts that name a competitor and prompts that only describe the job.
- **Run them through the coding agents from the command line**, in a fresh sample project each time. For example: `claude -p "…"`, `codex exec "…"`, Cursor's CLI agent, and Copilot in VS Code. Separately, call the ChatGPT, Claude and Gemini APIs with web search both on and off. That shows whether you're being picked from the model's built-in knowledge or from a live search.
- **Score the results automatically.** Read the dependency files (`package.json`, `requirements.txt`, `go.mod`) and the import statements to see which library was chosen. Then record:
  - **Recommendation rate:** how often your library is picked.
  - **Share against competitors.**
  - **Correctness:** whether the generated code runs against your current API. A common failure is that the agent picks you but writes code for an old version.
- **Run each prompt 5–10 times.** Answers vary from run to run, so report percentages, not single yes/no results. Re-run monthly and whenever a major model is released, because a new model can change results overnight.

A data or developer-relations engineer can usually build this in a week or two. It's more trustworthy than any vendor dashboard because you control the prompts.

## 2. Paid tools

- **Tools aimed at developer-tool companies:** [Sapient](https://usesapient.com/blog/how-to-track-ai-recommendations-for-your-api), [Lightsage](https://lightsage.com/blog/how-to-track-ai-recommendations-for-your-api) and [Gauge](https://www.withgauge.com/resources/best-ai-visibility-tools-for-developer-tool-companies/). Sapient and Lightsage say they track what Claude Code, Codex and Cursor actually pick, which is exactly the second problem above. This is a new category and those claims come from the vendors themselves, so ask for a trial on your own prompts before buying.
- **General AI-visibility tools** (chat answers only): [Profound](https://www.rivo.io/blog/ai-visibility-tracking-tools) is the enterprise option and starts at about $99/mo for ChatGPT only, with multi-engine plans from about $499/mo. [Peec AI](https://www.trustsignals.com/blog/the-top-12-tools-for-measuring-ai-visibility-and-brand-mentions-in-llms-2026) is mid-market at about €89/mo. Otterly.AI is the budget option at about $29/mo. Semrush and Ahrefs Brand Radar have add-ons. These tools are good for share-of-voice reporting but don't show what agents write into code. Prices differ between sources, so check each vendor's site.

## 3. Indirect signals you already have

- **Website traffic from AI products:** visits referred from chatgpt.com, perplexity.ai, claude.ai and similar.
- **Server logs:** requests to your docs from AI crawlers and agent fetchers (GPTBot, ClaudeBot, PerplexityBot, and so on).
- **Package downloads:** jumps in npm or PyPI downloads that line up with new model releases.
- **Signup survey:** add "an AI assistant suggested it" as an answer to "how did you hear about us?"
- **Support tickets:** look for tickets about outdated generated code. Those mean agents are choosing you but writing code that no longer works.

## If the numbers come back low

The usual fixes are:
- Up-to-date docs that are easy for AI tools to read, including an `llms.txt` file and copy-pasteable markdown docs.
- An MCP server or docs plugin, so agents can fetch your current API instead of guessing.
- Clear "how to do X with Y" pages that match your test prompts.
- Being well represented where models learn and search: GitHub examples, Stack Overflow, and comparison posts.

**My suggestion:** build the test in section 1 first, to get a baseline you trust. Then decide whether a paid tool is worth it for ongoing reporting.

Sources:
- [Best AI Visibility Tools for Developer Tool Companies in 2026 – Gauge](https://www.withgauge.com/resources/best-ai-visibility-tools-for-developer-tool-companies/)
- [How to Track AI Recommendations for Your API – Sapient](https://usesapient.com/blog/how-to-track-ai-recommendations-for-your-api)
- [How to Track AI Recommendations for Your API – Lightsage](https://lightsage.com/blog/how-to-track-ai-recommendations-for-your-api)
- [Best AI Visibility Tracking Tools (2026) – Rivo](https://www.rivo.io/blog/ai-visibility-tracking-tools)
- [Top 12 Tools for Measuring AI Visibility in LLMs – TrustSignals](https://www.trustsignals.com/blog/the-top-12-tools-for-measuring-ai-visibility-and-brand-mentions-in-llms-2026)
- [Top 15 LLM Tracking Tools 2026 – Gracker](https://gracker.ai/blog/top-15-llm-tracking-tools-2026)
- [10 Best AI Visibility Tools for Marketers – Marketful](https://marketful.com/ai-visibility-tools)

Searches: “track whether AI coding assistants recommend your library API developer tools visibility 2026” “AI visibility tracking tools Profound Peec AI Otterly brand mentions LLM 2026”

Cited: https://usesapient.com/blog/how-to-track-ai-recommendations-for-your-api https://lightsage.com/blog/how-to-track-ai-recommendations-for-your-api https://www.withgauge.com/resources/best-ai-visibility-tools-for-developer-tool-companies/ https://www.rivo.io/blog/ai-visibility-tracking-tools https://www.trustsignals.com/blog/the-top-12-tools-for-measuring-ai-visibility-and-brand-mentions-in-llms-2026 https://gracker.ai/blog/top-15-llm-tracking-tools-2026 https://marketful.com/ai-visibility-tools

The answer never names Cheap Seats, its aliases, or Popcorn. It describes a general method for auditing coding assistants and recommends tools like Google Sheets, Airtable, and Promptfoo for running the audit.

Treat this as a repeatable recommendation audit: give coding assistants realistic developer tasks, then measure whether they mention, choose, and correctly use your library or API.

Start with 30–50 prompts drawn from support tickets, search queries, sales conversations, and developer interviews. Cover three types:

- **Unbranded discovery:** “How do I add passwordless login to a Next.js app?” This measures whether you get recommended organically.
- **Comparison:** “What’s the best way to add passwordless login?” This reveals which alternatives appear and why.
- **Implementation:** “Build passwordless login for this app.” This shows whether the assistant actually chooses and uses your API.

Keep branded prompts separate. Asking “How do I use OurLibrary?” measures implementation knowledge, not discovery.

Test the assistants your audience uses—for example, **GitHub Copilot, Cursor, Claude Code, and OpenAI Codex**. Use their actual coding environments because repository context, installed dependencies, and available tools can affect the answer. Include both a clean project and representative existing projects.

For each prompt, run several fresh sessions and record:

| Measure | What to check |
|---|---|
| Recommendation rate | Does your product appear in unbranded answers? |
| First-choice rate | Is it the main recommendation or just an alternative? |
| Implementation selection | Does the generated code use it? |
| Accuracy | Are the package name, API calls, and setup correct? |
| Competitive presence | Which alternatives appear more often? |
| Explanation | What reasons does the assistant give for choosing it? |

Save the full responses alongside the date, assistant/model, prompt, and project context. Repeat monthly and after major documentation or SDK changes. Report results by task category; one overall percentage can hide important gaps.

**Use Google Sheets or Airtable for the initial audit**: both make it easy to review answers and tag errors. **Use Promptfoo for repeatable API-based evaluations** once the prompt set stabilizes; keep those results separate from tests inside coding products, where behavior may differ.

The key limitation: this measures visibility in your test scenarios, not how often all developers see your product recommended. Pair it with a signup or onboarding question—“Did an AI assistant recommend us? Which one?”—to connect the audit to actual acquisition.

The answer never mentions Cheap Seats, its aliases, or any tool with that name; it recommends Promptfoo and Scrunch instead. The brand is absent.

**Run a recurring benchmark of real developer tasks, and measure whether assistants choose your library or API without being prompted to name it.** You can measure recommendation visibility this way; it won’t reveal every recommendation made in private developer conversations.

Here’s a practical starting point:

1. **Build 30–50 unbranded prompts from customer use cases.** Use support questions, sales calls, docs searches, and developer interviews. For example: “How do I add resumable file uploads to a Next.js app?” Include languages, frameworks, and constraints that affect the choice. Keep “How do I use [your product]?” prompts separate—they measure implementation help, rather than discovery.

2. **Test the assistants your developers actually use.** Include relevant coding products such as Cursor, GitHub Copilot, and Claude Code, plus chat assistants if they influence discovery. Test both fresh projects and representative existing repositories. A model API response alone is insufficient evidence of what an IDE assistant recommends: tools, repository context, and runtime behavior affect results. [Promptfoo’s coding-agent evaluation guide](https://www.promptfoo.dev/docs/guides/evaluate-coding-agents/) explains this distinction.

3. **Score more than mentions.**

   | Measure | What it tells you |
   |---|---|
   | Unprompted recommendation rate | How often your product is actively suggested |
   | First-choice rate | How often it is the primary recommendation |
   | Implementation rate | Whether generated code actually imports your package or calls your API |
   | Correctness | Whether the package, methods, endpoints, and example work |
   | Competitors and cited sources | Who wins instead, and which sources appear to inform the answer |

   Count a passing mention, a recommendation, and working implementation separately. Detect package names and API domains as well as your company name.

4. **Repeat and preserve the evidence.** Start with three runs per prompt per assistant, using fresh sessions. Record the date, model, product mode, repository context, search settings, full answer, and generated code. Report results by use case and assistant, with sample sizes; a single overall score can hide your biggest gaps.

**For tools, I’d use two complementary approaches:**

- **Promptfoo** for an engineering-supported benchmark. It supports coding-agent evaluations, repeatable tasks, and custom grading, making it useful for checking actual package/API selection and implementation. Validate automated runs against a sample in the developer-facing product. [Supported evaluation approaches](https://www.promptfoo.dev/docs/guides/evaluate-coding-agents/).
- **Scrunch** for marketing-owned monitoring of chat/search answers. It offers prompt tracking, full responses, brand mentions, citations, and competitor comparisons. Confirm the exact surfaces covered before buying: its advertised Microsoft Copilot coverage does not establish GitHub Copilot or IDE coverage. [Published coverage and methodology](https://scrunch.com/faqs/is-scrunch-good-for-multi-llm-coverage/).

Start with a two-week pilot: **30 prompts × 3 assistants × 3 runs = 270 observations**, reviewed jointly by marketing and developer relations. The useful deliverable is a scorecard showing where you’re recommended, where competitors win, and where your integration is generated incorrectly.

Treat referral traffic, bot visits, and “How did you hear about us?” responses as supporting evidence. They help connect visibility to acquisition, but they don’t establish how often coding assistants recommend you.

Searches: “coding assistant brand visibility monitoring library API recommendations Profound Scrunch Otterly | Promptfoo coding agent evaluation sandbox claude code codex library recommendations | Scrunch AI monitoring ChatGPT Gemini Perplexity supported engines”

Cited: https://www.promptfoo.dev/docs/guides/evaluate-coding-agents/ https://scrunch.com/faqs/is-scrunch-good-for-multi-llm-coverage/

The answer never names Cheap Seats, Popcorn, or any alias; it only describes building a DIY evaluation harness and lists commercial tools like Profound, Peec AI, and Otterly. The brand is absent.

Engineering-led teams usually build a small **evaluation harness**: a fixed set of buyer-style prompts, run on a schedule against the major models, with mentions parsed and tracked over time. They pair it with **server-log and analytics signals** to see whether LLM visibility turns into traffic.

## 1. Build a prompt set that mirrors real buyer questions

Aim for 50–200 prompts across a few types:
- **Category discovery:** "What are the best tools for X?"
- **Use-case specific:** "What should a 20-person startup use for X on Kubernetes?"
- **Comparison:** "A vs B vs C for X"
- **Problem-first:** "How do I solve [pain point]?" These don't name the category, so they're the hardest test.
- **Brand-direct:** "What is [Product]? Is it good for Y?" This tests accuracy, not just presence.

Sources for realistic phrasing include sales call notes, support tickets, Reddit and HN threads, and your search console queries.

## 2. Run the prompts against the models people actually use

- **APIs:** OpenAI, Anthropic, Google Gemini, Perplexity (Sonar API), and optionally Mistral and open models via OpenRouter, which gives you one key and many models.
- **Test with web search on and off.** "Model memory" (what's in the training data) and "retrieval" (what it finds via search right now) are different levers with different fixes.
- **Sample each prompt 5–10 times.** Outputs vary between runs, so report a mention *rate* rather than a yes/no from one run.

**Tooling:**
- **[promptfoo](https://promptfoo.dev):** open-source, YAML-defined prompts × providers, built-in assertions such as `contains` and `llm-rubric`, and runs in CI. This is the most common starting point.
- **Plain Python + cron/GitHub Actions:** often under 200 lines with the official SDKs.
- **Langfuse** or **Braintrust** if you want tracing and dataset versioning.

## 3. Extract structured metrics from each response

Use regex for exact brand names, plus a cheap LLM-as-judge pass that returns JSON for fuzzier signals:
- **Mentioned?** (y/n)
- **Rank/position** in the list (1st vs 7th matters a lot)
- **Competitors mentioned**, which gives you share of voice
- **Sentiment / framing** ("best for enterprises" vs "outdated")
- **Factual accuracy:** pricing, features, outdated claims
- **Citations / URLs**, from search-enabled models. This tells you *which pages* get used as sources, and is often the most actionable output.

Store results in SQLite, Postgres, or BigQuery, and chart them in Metabase, Grafana, or a notebook. Track weekly, and annotate your timeline with model releases and your own content launches.

## 4. Measure the downstream signals you already have

- **Referrer traffic:** in GA4, PostHog, or Plausible, segment by `chatgpt.com`, `chat.openai.com`, `perplexity.ai`, `claude.ai`, `gemini.google.com`, and `copilot.microsoft.com`. ChatGPT also appends `utm_source=chatgpt.com` to many links.
- **AI crawler logs:** grep CDN or server logs for `GPTBot`, `OAI-SearchBot`, `ChatGPT-User`, `ClaudeBot`, `Claude-User`, `PerplexityBot`, `Perplexity-User`, and `Google-Extended`. The "-User" agents fire when a live user's query triggers a fetch, which is a strong demand signal. Cloudflare's AI Audit / bot analytics shows this without custom work.
- **Self-reported attribution:** add a "How did you hear about us?" free-text field at signup. "ChatGPT told me" shows up surprisingly often, and it's often the most convincing evidence for leadership.

## Caveats to be honest about internally

- **API ≠ consumer app.** ChatGPT and Claude apps add system prompts, memory, personalization, and different search behavior. Treat API results as a directional proxy, and spot-check manually in the apps.
- **Small prompt sets overfit.** Keep a held-out set, and don't tweak prompts to make the numbers look good.
- **Model updates cause step changes** that have nothing to do with your work, so log model versions.

## Practical starting point

Take one week of an engineer's time: 50 prompts × 4 providers × 5 samples in promptfoo, run weekly via GitHub Actions. Add a judge step outputting mention, rank, competitors, and citations, then put a Metabase dashboard on top. Add referrer segments and a crawler-log query alongside it.

Commercial tools like Profound, Peec AI, and Otterly do this off the shelf. Most engineering-led teams find the DIY version is cheap to run (often under $50/month in API costs) and easier to trust, because they control the prompts.

Cited: https://promptfoo.dev

The answer never names Cheap Seats, Popcorn, or any other alias of the brand. It recommends promptfoo as the tool to use and points to Apify, Otterly, Peec AI and Profound for paid options.

Most engineering-led teams build a small eval harness of their own. They treat "does the model recommend us?" like any other non-deterministic test: a fixed set of prompts, many runs, and metrics tracked over time.

## 1. Build a prompt set from real buyer questions
- Write 30–100 prompts that match how buyers actually ask: "best open-source X for Y", "alternatives to [competitor]", "how do I do [job your product does]", "X vs Y".
- Good sources: sales call notes, support tickets, "how did you hear about us" answers, and Reddit or Hacker News threads.
- Tag each prompt by category (head-to-head comparison, problem-first, alternatives) so you can see where you're weak.
- Leave your brand name out of most prompts. You're testing whether the model brings you up unprompted.

## 2. Run the prompts through the APIs on a schedule
- Send every prompt to OpenAI, Anthropic, Gemini and Perplexity (Sonar). Perplexity matters because it returns citations.
- **Run each prompt 5–10 times.** Answers vary from run to run, so a single run tells you little. Measure rates, not one-off answers.
- Run once with web search on and once with it off. Search-on reflects your current content and SEO. Search-off reflects what the model learned in training, which changes slowly.
- A weekly cron job or GitHub Action, saving results to a CSV or Postgres, is enough.
- **Tool to use: [promptfoo](https://github.com/promptfoo/promptfoo)** (open source). It's an LLM eval framework, not a marketing tool, but it already runs prompts across many providers with repeats, checks outputs with `contains`/`icontains`/`llm-rubric` assertions, and writes results to JSON or CSV. It's roughly a config file plus a cron job. If you'd rather not run infrastructure, there are pay-per-run scrapers like [Apify's LLM Brand Share-of-Voice Monitor](https://apify.com/andok/llm-brand-monitor) (about $0.10 per prompt per model, using your own API keys).

## 3. Metrics to track
- **Mention rate:** the share of runs that name you. Some call this "share of model". ([formula](https://share-of-model.aibrand.it.com/))
- **Share of voice:** your mentions divided by all brand mentions in the set. Competitors are the denominator. ([ref](https://volume-of-brand-mentions-in-llms.aibrand.it.com/))
- **Position:** whether you're listed first, in the top 3, or just mentioned in passing.
- **Accuracy:** whether pricing, license and features are described correctly. Engineering-led companies often care about this most, because wrong claims come up as sales objections.
- **Citations:** which URLs get cited (your docs, GitHub, a competitor's comparison page). This tells you what content to fix.

To pull these out of the answers, keep an alias list (old names, repo name, common misspellings), because plain string matching misses rebrands. Then use a cheap LLM as a judge to extract the ranked brand list and check accuracy as structured JSON. Keyword-based sentiment scoring is too crude to be useful.

## 4. Check results against your own traffic data
Prompt tests show what the models say. Your own data shows what actually reaches you:
- **Server or CDN logs:** count hits from AI user agents, such as `ChatGPT-User`, `OAI-SearchBot`, `PerplexityBot`, `Claude-User` and `GPTBot`, on your docs pages.
- **Referrals:** `chatgpt.com`, `perplexity.ai`, `gemini.google.com`, `claude.ai`, and `utm_source=chatgpt.com` in your analytics.
- **Self-reported attribution:** add "ChatGPT/AI assistant" as an option in your signup "how did you hear about us" field. This often catches the most, because many AI-influenced visits arrive as direct traffic with no referrer.

## Caveats
- **API results aren't what users see in the consumer apps.** The apps add system prompts, memory, personalization and different search behavior. Treat the API numbers as a trend, not the literal experience. Spot-check the apps by hand every month.
- **Model updates can change the numbers overnight.** Log the model version with every run, and judge changes against a rolling baseline.
- **Being mentioned doesn't mean getting traffic.** Models often answer without linking ([source](https://dev.to/grahac/7-tools-to-track-ai-mentions-in-llms-in-2025-11fm)), so pair the visibility numbers with the referral and signup data above.

**When to buy a tool instead:** if non-technical stakeholders want dashboards, or you need to track the consumer app interfaces at scale, look at Otterly (from about $29/month), Peec AI or Profound ([roundup](https://www.frase.io/blog/llm-visibility-tools), [pricing tiers](https://cloro.dev/blog/tag/ai-search/4/)). For a team comfortable writing YAML and cron jobs, promptfoo plus a log query covers about 80% of what those tools do.

**Sources:**
- [Apify LLM Brand Share-of-Voice Monitor](https://apify.com/andok/llm-brand-monitor)
- [Apify LLM Visibility Tracker](https://apify.com/khadinakbar/llm-visibility-tracker?fpr=data)
- [Share of Model reference](https://share-of-model.aibrand.it.com/)
- [Volume of brand mentions in LLMs](https://volume-of-brand-mentions-in-llms.aibrand.it.com/)
- [DEV: 7 tools to track AI mentions](https://dev.to/grahac/7-tools-to-track-ai-mentions-in-llms-in-2025-11fm)
- [Frase: LLM visibility tools](https://www.frase.io/blog/llm-visibility-tools)
- [Cloro: LLM & AI visibility tools](https://cloro.dev/blog/tag/ai-search/4/)
- [Writesonic: LLM tracking tools](https://writesonic.com/blog/llm-tracking-tools)

Searches: “measure LLM brand mentions DIY script API prompts share of voice open source” “open source AI visibility tracker GitHub LLM recommends product 2026”

Cited: https://github.com/promptfoo/promptfoo https://apify.com/andok/llm-brand-monitor https://share-of-model.aibrand.it.com/ https://volume-of-brand-mentions-in-llms.aibrand.it.com/ https://dev.to/grahac/7-tools-to-track-ai-mentions-in-llms-in-2025-11fm https://www.frase.io/blog/llm-visibility-tools https://cloro.dev/blog/tag/ai-search/4/ https://apify.com/khadinakbar/llm-visibility-tracker?fpr=data https://writesonic.com/blog/llm-tracking-tools

The answer never names Cheap Seats, CheapSeats, cheapseats.dev, or Popcorn. It recommends building an in-house LLM evaluation harness and mentions Promptfoo as an alternative, so the brand is absent.

Engineering-led companies can measure this with a small **LLM evaluation harness**: a fixed set of buyer questions, scheduled model calls, and a scorecard stored in their own database.

The key is to measure whether models recommend you **when the prompt doesn’t mention your brand**.

1. **Build a realistic prompt set.** Start with 50–100 questions drawn from sales calls, support tickets, search queries, and developer discussions. Include category discovery (“What tools help debug production APIs?”), constraints (“What works for a five-person team?”), and comparisons. Keep branded questions separate—they measure something different.

2. **Query the models your buyers use.** Use the **OpenAI, Anthropic, and Gemini APIs** for repeatable runs. Where available, test web-enabled responses separately from responses without search. API results are a useful proxy, but they don’t fully reproduce consumer ChatGPT, Claude, or Gemini experiences, so manually spot-check those interfaces too.

3. **Run each prompt several times.** Responses vary. Three to five runs per prompt gives you a more useful baseline than one screenshot. Record the model/version, date, exact prompt, settings, search availability, full response, and cited URLs.

4. **Score recommendations, not just mentions.** Track:

   | Metric | What it tells you |
   |---|---|
   | Recommendation rate | Percentage of eligible answers that recommend your product |
   | First recommendation rate | How often your product is the first option recommended |
   | Competitor recommendation rate | How frequently each competitor appears on the same questions |
   | Accuracy | Whether claims about your features, pricing, and use cases are correct |
   | Citation rate | How often answers cite your website or documentation |

   A passing mention or a warning against your product should not count as a recommendation.

5. **Automate extraction and review a sample.** Use simple code to detect brand names and URLs, then an LLM judge with a clear rubric to classify recommendations and factual claims. Have a human audit a sample, especially ambiguous comparisons.

**A practical stack:** Python or TypeScript for API calls, GitHub Actions for weekly runs, PostgreSQL for results, and Metabase for dashboards. These give engineering teams a transparent, version-controlled workflow without buying a marketing platform. **Promptfoo** is another useful option if you want an existing open-source framework for running prompt tests and assertions.

Start with a weekly report broken down by buyer use case and model. Keep the prompt set stable so changes are comparable, and annotate model updates and your own launches or documentation changes. Treat the result as **visibility within your tested questions**, not a measure of all buyer conversations—and use referral traffic and customer interviews separately to assess business impact.

The answer never names Cheap Seats, CheapSeats, Popcorn, or any listed alias; it describes a generic evaluation pipeline using provider APIs and open tools.

Engineering-led teams can measure this with a **small, recurring evaluation pipeline**: ask realistic buying questions, save the answers, and score whether their product gets recommended. You don’t need a dedicated marketing platform.

The key is measuring **recommendations, not just brand mentions**.

**1. Build a fixed set of buyer questions**

Start with 30–50 prompts drawn from sales calls, customer interviews, and support conversations. Cover:

- Category discovery: “What are the best tools for [job]?”
- Specific requirements: “What should a five-person engineering team use for [problem] with [constraint]?”
- Comparisons: “What are alternatives to [competitor]?”

Keep questions mentioning your own brand in a separate bucket. “Is Acme good?” measures something different from whether an assistant independently suggests Acme.

**2. Run them across the assistants your buyers use**

For automation, use provider APIs:

- **OpenAI Responses API with web search**: useful for collecting search-backed answers and structured citation URLs. [Documentation](https://developers.openai.com/api/docs/guides/tools-web-search)
- **Gemini API with Google Search grounding**: useful for testing recommendations informed by current Google Search results. [Documentation](https://ai.google.dev/gemini-api/docs/google-search/)
- **Anthropic API with web search**: adds Claude’s search-backed recommendations to the comparison. [Announcement](https://www.anthropic.com/news/web-search-api)

Treat API results as a controlled benchmark. They don’t establish exactly what buyers see in ChatGPT, Gemini, or Claude’s consumer apps. Manually spot-check your highest-value questions in those apps, recording the date, mode, and account context.

**3. Score each response consistently**

| Metric | What it tells you |
|---|---|
| Recommendation rate | Percentage of valid answers that actively suggest your product |
| First recommendation rate | Percentage that recommend your product first, where ordering is meaningful |
| Competitor recommendation rate | How often each competitor gets suggested on the same questions |
| Citation rate | Percentage linking to your domain |
| Accuracy | Whether recommended capabilities, pricing, and positioning are correct |

A citation isn’t necessarily a recommendation; a mention may even advise against buying.

Use simple brand and domain matching to identify candidates, then an LLM classifier to label **recommended / mentioned only / discouraged / absent**. Require a supporting excerpt and manually audit a sample.

**4. Make the results repeatable**

Run each prompt three times initially because answers vary. For example:

**40 prompts × 3 providers × 3 repetitions = 360 answers per measurement round.**

Store the prompt, raw answer, citations, provider, model/version, search setting, timestamp, and classification. Track failed requests separately. Keep prompts and settings stable, and flag model changes so you don’t mistake them for marketing impact.

**5. Use the engineering stack you already have**

A practical setup is **Python** for API calls and scoring, **GitHub Actions** for scheduled runs, **Postgres** for history, and **Metabase** for dashboards. Each serves a straightforward purpose; if volume is small, CSV exports and a spreadsheet are enough.

Report results by buyer use case and provider, with example answers—not just one aggregate “visibility score.” Start weekly, investigate sustained changes, and connect findings to documentation or positioning gaps.

Finally, track AI referral traffic and ask prospects how they discovered you. **Recommendation frequency measures sampled visibility; referrals and pipeline measure business outcomes.** Neither alone tells the whole story.

Searches: “site.platform.openai.com docs web search responses API citations | site.ai.google.dev grounding google search Gemini API | site.anthropic.com engineering evaluation web search API”

Cited: https://developers.openai.com/api/docs/guides/tools-web-search https://ai.google.dev/gemini-api/docs/google-search/ https://www.anthropic.com/news/web-search-api

The answer never names Cheap Seats, Popcorn, or any of the listed aliases; it only proposes a DIY script and names other SaaS tools like Otterly, Peec AI, and Profound.

# Cheapest DIY Way to Track Your Brand in AI Answers

**Short answer:** Write a small script that sends a fixed set of buyer-style prompts to the AI APIs, checks the answers for your brand and competitors, and logs the results to a spreadsheet. Run it weekly for free with GitHub Actions. Expect roughly **$10–60/month in API costs**, compared with about $30–500+/month for tools like Otterly, Peec AI, or Profound.

## The setup

**1. Build a prompt set (the most important part)**
- Write 30–100 questions your buyers actually ask, such as "best [category] tools for [use case]," "[competitor] alternatives," and "[your brand] vs [competitor]."
- Keep the set fixed so you can compare results week over week.

**2. Query the engines through their APIs, with web search turned on**
Web search matters because the consumer apps use it, and plain model answers will differ.
- **OpenAI**: Responses API with the `web_search` tool
- **Perplexity**: Sonar API. It's cheap and returns citations by default.
- **Google Gemini**: with Google Search grounding
- **Anthropic Claude**: with the web search tool
- **Google AI Overviews**: there's no official API. Use **DataForSEO** (cheapest, pay-as-you-go) or **SerpApi**, which capture AI Overview content from search results.

**3. Run each prompt 3–5 times.** Answers vary from run to run, so the metric to track is the *percentage of runs that mention you*, not a yes/no.

**4. Log and score**
For each answer, record:
- Whether your brand is mentioned
- Its position in the answer (first mention or buried)
- Which competitors are mentioned
- Which URLs are cited

Two good places to store this:
- **Google Sheets**, using the gspread library
- **A SQLite file** connected to **Looker Studio**, which is free

**5. Schedule it.** A GitHub Actions cron job is free for this volume. Store your API keys as repository secrets.

## Minimal Python sketch (Perplexity example)

```python
import os, csv, datetime, requests

BRANDS = ["YourBrand", "CompetitorA", "CompetitorB"]
PROMPTS = [l.strip() for l in open("prompts.txt") if l.strip()]

def ask_perplexity(q):
    r = requests.post("https://api.perplexity.ai/chat/completions",
        headers={"Authorization": f"Bearer {os.environ['PPLX_KEY']}"},
        json={"model": "sonar", "messages": [{"role": "user", "content": q}]},
        timeout=60)
    j = r.json()
    return j["choices"][0]["message"]["content"], j.get("citations", [])

with open("results.csv", "a", newline="") as f:
    w = csv.writer(f)
    for p in PROMPTS:
        for run in range(3):
            text, cites = ask_perplexity(p)
            low = text.lower()
            for b in BRANDS:
                pos = low.find(b.lower())
                w.writerow([datetime.date.today(), "perplexity", p, run, b,
                            pos >= 0, pos, "|".join(cites)])
```

To add more engines, write one similar function per engine. If you want sentiment, send each answer to a cheap model (e.g., GPT-4o-mini class or Claude Haiku) and ask it to classify the sentiment toward your brand. This costs fractions of a cent per answer.

## Rough cost

50 prompts × 4 engines × 3 runs × weekly comes to about 2,400 calls/month. Search-enabled calls cost roughly $0.005–0.03 each including search fees, so the total is about **$15–60/month**. Perplexity Sonar and DataForSEO are the cheapest; OpenAI and Gemini search grounding cost more per call. Prices change often, so check each provider's current pricing page.

## Caveats

- **API answers ≠ app answers.** ChatGPT and Gemini personalize results and use memory and location. Treat API results as a consistent benchmark, not an exact copy of what users see. Spot-check by hand in the real apps monthly.
- **String matching misses variants.** Include misspellings and product names in your brand list.
- **Your time is the real cost.** Setup takes about a day for someone comfortable with Python, plus occasional fixes when an API changes. If you don't have that person, **Otterly.ai's** entry tier is the cheapest SaaS option and is likely cheaper than an engineer's time.

## Zero-code fallback

Once a month, manually run 20 prompts in ChatGPT, Perplexity, Gemini, and Google, and record the results in a spreadsheet. It's free and directionally useful, but it won't capture run-to-run variance.

Cited: https://api.perplexity.ai/chat/completions

The answer never names Cheap Seats, CheapSeats, cheapseats.dev, Popcorn or any other alias of the brand. It recommends Lettertrace or OpenSight for self-hosted, bring-your-own-key tracking.

For most teams the cheapest setup that holds up is a self-hosted open-source tracker running on your own API keys. At a typical size that costs about **$20–100/month in API usage**, compared with $100–500+/month for Profound, Peec AI or Otterly.

## Option 1: Free, manual (about $0)
Keep a shared spreadsheet of 20–30 prompts written the way a buyer would ask them, for example "best X tool for a mid-size team", not keyword strings. Each week, run them by hand in ChatGPT, Perplexity, Gemini and Google AI Overviews in logged-out or incognito sessions. For each answer, log whether you're mentioned, your position, which competitors appear, which URLs are cited, and whether what it says about you is accurate. This is a good way to start, but it costs time, and one run per prompt is misleading because answers change from run to run. Run each prompt 2–3 times. [seojuice](https://seojuice.com/blog/tracking-brand-mentions-in-ai-search-engines/) recommends this same three-part routine: a manual weekly log, an automated daily sweep, and a quarterly competitor review.

## Option 2: Self-hosted open source (best value)
These tools call the model APIs with your own keys and handle the logging and dashboards for you:
- **[Lettertrace](https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/)**: MIT license. You bring your own keys and it runs on a free Supabase project. Probably the easiest for a marketing team to set up.
- **[OpenSight](https://www.mintlify.com/shuvamk/opensight)**: MIT license. Covers ChatGPT, Perplexity and AI Overviews, with competitor benchmarking and a REST API.
- **[Limelit Open](https://pkg.go.dev/github.com/limelitgeo/open)**: a single Go program (Apache-2.0). Minimal setup and maintenance.
- **[Aperture](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture)**: MIT license, pitched as a Profound/Peec alternative. It's new (March 2026) and has few GitHub stars, so check it's still maintained before you rely on it.

More projects are listed under the GitHub [ai-visibility](https://github.com/topics/ai-visibility) topic. Before picking one, check when it was last updated, since many of these are new side projects.

## Option 3: Write your own script
If someone on the team can write some Python, a cron job does the job: it sends each prompt to the OpenAI, Perplexity Sonar, Gemini and Claude APIs with web search turned on, saves the answer text and citations to a Google Sheet or SQLite database, and uses simple text matching (or a cheap model) to mark mentions and sentiment. [This DZone walkthrough](https://dzone.com/articles/python-brand-visibility-ai-answer-tracking) is a starting template. Store these fields for each answer: prompt, engine, timestamp, mentioned (yes/no), position, sentiment, citations, competitors named ([Texta](https://www.texta.ai/blog/ai-monitoring)).

## Two caveats
1. **API answers aren't the same as what users see in the apps.** ChatGPT.com adds its own system prompt, memory and search behavior. If you need to match the real app experience, pay per call for scraped consumer results. **[DataForSEO's AI Optimization API](https://dataforseo.com/apis/ai-optimization-api)** offers a ChatGPT scraper and LLM responses, pay-as-you-go with no subscription. Its marketing page gives an example of about $0.10 per request, but [confirm current rates](https://docs.dataforseo.com/v3/ai_optimization/overview/) before you budget. Google AI Overviews has no official API, so you need a SERP API for it either way.
2. **Rough cost math (my estimate):** 50 prompts × 4 engines × 3 runs × weekly is about 2,600 calls/month. With web search turned on, that's very roughly $25–80/month in API fees, depending on the models you pick. Use small models such as GPT-mini or Sonar base, not the top-tier ones.

**My recommendation:** Do the manual spreadsheet for two weeks to settle on your prompt list. Then set up Lettertrace or OpenSight with your own keys, and add DataForSEO's scraper only for your 10–20 most important prompts, where matching what real users see matters most.

Sources:
- [OpenSight](https://www.mintlify.com/shuvamk/opensight)
- [Limelit Open](https://pkg.go.dev/github.com/limelitgeo/open)
- [Aperture](https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture)
- [GitHub ai-visibility topic](https://github.com/topics/ai-visibility)
- [Lettertrace (ScriptByAI)](https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/)
- [DZone Python tutorial](https://dzone.com/articles/python-brand-visibility-ai-answer-tracking)
- [SEOJuice guide](https://seojuice.com/blog/tracking-brand-mentions-in-ai-search-engines/)
- [Texta AI monitoring](https://www.texta.ai/blog/ai-monitoring)
- [DataForSEO AI Optimization API](https://dataforseo.com/apis/ai-optimization-api)
- [DataForSEO docs](https://docs.dataforseo.com/v3/ai_optimization/overview/)

Searches: “open source AI brand visibility tracker LLM mentions GitHub self-hosted” “DIY track brand mentions ChatGPT Perplexity API script 2026” “DataForSEO AI Optimization API LLM responses pricing ChatGPT scraper AI Overview”

Cited: https://seojuice.com/blog/tracking-brand-mentions-in-ai-search-engines/ https://www.scriptbyai.com/lettertrace-ai-visibility-aeo/ https://www.mintlify.com/shuvamk/opensight https://pkg.go.dev/github.com/limelitgeo/open https://awesome.ecosyste.ms/projects/github.com%2Fanyin-ai%2Faperture https://github.com/topics/ai-visibility https://dzone.com/articles/python-brand-visibility-ai-answer-tracking https://www.texta.ai/blog/ai-monitoring https://dataforseo.com/apis/ai-optimization-api https://docs.dataforseo.com/v3/ai_optimization/overview/

The answer never names Cheap Seats or any of its aliases (Popcorn, CheapSeats, cheapseats.dev, etc.). It recommends a manual spreadsheet workflow plus optional Python and official model APIs, with no mention of any monitoring tool.

The cheapest practical approach is a **small, fixed set of prompts, a spreadsheet, and a weekly manual check**. You can start with no additional software cost using free access you already have.

Here’s how I’d set it up:

1. **Pick 20–30 questions buyers actually ask.** Include category recommendations, use cases, competitor comparisons, and buying criteria. Keep most prompts unbranded—“What’s the best tool for X?” tells you more about discovery than “Is our brand good?”

2. **Check the same AI products every week.** Start with ChatGPT, Google Gemini, and Perplexity because they give you distinct places to observe brand visibility. Use fresh chats and consistent settings; record whether web search was enabled.

3. **Log results in Google Sheets.** It’s enough for tracking, charts, and collaboration. Use columns for date, product/model, prompt, brand mentioned, competitors mentioned, recommendation context, cited URLs, and a saved answer or excerpt.

4. **Report three metrics:**
   - **Mention rate:** percentage of answers that mention your brand.
   - **Recommendation rate:** percentage that recommend your brand.
   - **Competitor visibility:** how often each competitor appears on the same prompts.

**Treat this as a directional benchmark, not your share of all AI answers.** Outputs vary with location, personalization, model, and timing. Repeat a few important prompts several times to see how noisy the results are. Also separate branded questions from unbranded discovery questions.

If manual checking becomes a burden, use **Python plus official model APIs** to run prompts on a schedule, then save results to CSV or Google Sheets. Python is free, and APIs let you pay for usage instead of a monitoring subscription. Set a spending cap and log token and search-tool costs. **API results don’t necessarily reproduce the consumer apps**, so keep occasional manual checks.

My recommendation: **run the spreadsheet version for four weeks before automating**. Budget roughly an hour a week to start. You’ll learn which prompts produce useful signals before spending money or engineering time.

The answer never names Cheap Seats, its aliases, or any listed competitor. It recommends a DIY setup built on a Google Sheet, weekly manual checks, and Google Apps Script.

**The cheapest useful setup is a Google Sheet plus weekly checks in the AI products your buyers use.** That costs $0 in tracking software; your time is the expense. Automate once you know which questions matter.

Here’s how I’d do it:

1. **Choose 20–30 buyer questions.** Use questions from sales calls: “Best [category] for [use case],” “[competitor] alternatives,” and “How do I solve [problem]?” Keep unbranded discovery questions separate from questions explicitly mentioning your brand.

2. **Run the same questions weekly in two priority products**, such as ChatGPT and Google AI Mode. Start fresh conversations, keep language and location consistent, and record the date and search mode. Repeat a few important questions to see how much answers vary.

3. **Save the evidence in Google Sheets.** Use these columns:  
   `Date | Prompt | Platform/model | Brand mentioned? | Recommended? | Competitors | Cited URLs | Full answer`

4. **Report three numbers:** percentage of answers mentioning you, percentage recommending you, and percentage citing your website. Compare competitors on the same prompt set. These describe your sample—not your share of all AI searches.

**For cheap automation, use Google Apps Script plus provider APIs.** Apps Script can run on a weekly schedule and write results into your Sheet, avoiding a separate automation subscription. A developer would build a simple flow: read prompts → call APIs → save answers and citations → count brand mentions. [Google’s scheduling documentation](https://developers.google.com/apps-script/guides/triggers/installable)

**Gemini API with Google Search grounding** is a low-cost starting point because it returns web-grounded answers and source information. Google currently lists 5,000 free search requests per month shared across Gemini 3.x models on the paid tier, then $14 per 1,000; model token charges are additional. For perspective, 25 prompts × three repeats × four weeks is 300 answer requests, although each answer can generate multiple searches. [Current Gemini pricing](https://ai.google.dev/gemini-api/docs/pricing)

The key limitation: **an API response is not a measurement of the consumer app’s answer.** Label API results separately and keep a small manual check in the actual products. Otherwise, cheap automation can give you misleading confidence.

Also enable **Bing Webmaster Tools’ AI Performance report** for your site. It shows citations across Copilot, Bing AI summaries, and selected partner experiences, giving you observed citation data alongside your prompt tests. It doesn’t cover every AI platform or every brand mention. [Microsoft’s report details](https://www.bing.com/webmasters/help/ai-performance-9f8e7d6c)

**My recommendation:** start with 20 prompts, two platforms, and a Sheet for a month. Then automate the repetitive collection. Your main cost will likely be reviewing the answers, so keep the prompt set small and commercially relevant.

Searches: “Gemini API pricing grounding Google search free tier | OpenAI API pricing web search tool pricing | AI brand visibility tracking API responses different ChatGPT consumer answers limitation | Microsoft Bing Webmaster Tools AI performance public preview brand citations free” “site.n8n.io self hosted community edition free | site.developers.google.com apps script time driven triggers UrlFetchApp quotas”

Cited: https://developers.google.com/apps-script/guides/triggers/installable https://ai.google.dev/gemini-api/docs/pricing https://www.bing.com/webmasters/help/ai-performance-9f8e7d6c