Skip to main content

​
CreatorDB Skill Routing Smoke Test

Run this after every meaningful edit to any SKILL.md. Each prompt targets a specific behavior. Copy the prompt verbatim into Claude Code (with all seven CreatorDB skills loaded — creator-search, creator-enrichment, content-search, content-analysis, brand-sponsorship, account, creator-csv-workflow), then grade the response against the what to check list. Pass threshold: at least 12 of 13 should pass cleanly. If fewer than 12 pass, fix the identified skills before deploying. The failures usually point at description-overlap or missing trigger phrases.
Prompt
Find me US gaming YouTubers with at least 1M subscribers and high engagement.
Expected skill: creator-search What a correct response looks like A short plan, then a worked example using POST /youtube/search with three filters (country = USA, a YouTube gaming category like mainCategory = "Gaming", and totalSubscribers > 1000000), sorted by avgRecentVideosEngagementRate descending. Mentions the cost (1 credit per page) and the 100-result page limit. What to check
  • Routes to creator-search, not creator-enrichment
  • Uses the correct endpoint path /youtube/search
  • Country code is ISO 3166-1 alpha-3 (USA, not US or United States)
  • Numeric value is wrapped in quotes ("value": "1000000")
  • Mentions the 10-filter cap or the 100-result page limit somewhere
Common failure: Claude offers to call /nls instead of structured search. That’s not strictly wrong, but for an explicit-criteria prompt structured is the better fit. Sharpen the description if this happens repeatedly.

​
Test 2 — Basic routing: per-creator enrichment

Prompt
I have these YouTube channel IDs: UCX6OQ3DkcsbYNE6H8uQQuVA and UCsTcErHg8oDvUnTzoqsYeNw. Pull profile info, audience demographics, and emails for each.
Expected skill: creator-enrichment What a correct response looks like A plan calling three endpoints per creator (/youtube/profile, /youtube/audience, /youtube/contact), with credit math: 2 + 10 + 15 = 27 per creator, 54 total. Bonus points if it notes that /youtube/contact is the authoritative source for emails (the only endpoint that returns the structured data.emails array) and that shortlisting before pulling /contact is the cost-saving pattern — not bio-regex scraping. What to check
  • Routes to creator-enrichment, not creator-search or creator-csv-workflow
  • Cites the right per-call costs: profile 2, audience 10, contact 15
  • Total credit math is correct (54 credits for 2 creators × 3 endpoints)
  • Calls /youtube/contact directly for emails — does not suggest extracting emails by regexing /profile.bio
  • Uses channelId parameter only — does not claim YouTube also accepts uniqueId (it doesn’t; resolve handles through /youtube/search first)
Common failure: Claude routes to creator-csv-workflow because the prompt mentions “each.” That’s wrong — two creators isn’t a CSV workflow. Tighten the “When to use this skill” boundaries if this happens.

​
Test 3 — Basic routing: brand-side analysis

Prompt
Estimate Acer’s influencer marketing spend on YouTube and Instagram over the last 30 days.
Expected skill: brand-sponsorship What a correct response looks like Single call: GET /sponsor/summary?brandId=acer.com. Notes that estimatedTotalSpend30d is YouTube-only and Instagram returns null for spend fields. Cites cost (25 credits). What to check
  • Routes to brand-sponsorship
  • Knows brandId is the domain (acer.com, not “Acer”)
  • Calls out the YouTube-only spend constraint clearly
  • Doesn’t try to compute spend by aggregating individual creators (that would burn far more credits and is less accurate)
Common failure: Claude offers to call /sponsor/creators and sum estimated costs. That works in theory but /sponsor/summary is the right answer — one call, authoritative numbers, much cheaper.

​
Test 4 — Basic routing: content-level analysis

Prompt
Transcribe MrBeast’s most recent YouTube video and find any brand mentions in it.
Expected skill: content-analysis What a correct response looks like Three-step plan: (1) /youtube/content-detail to get recentVideos[0].contentId, (2) /youtube/subtitles/meta?videoId=<id> to find available tracks, (3) /youtube/subtitles/download with the chosen vssId. Notes that the download endpoint returns a streamed file, not the standard JSON envelope. Mentions /youtube/sponsorship as an alternative for brand verification. What to check
  • Routes to content-analysis, not creator-enrichment
  • Uses videoId (11 chars) for subtitle endpoints, not channelId (24 chars)
  • Prefers .en over a.en if both available (human captions over auto-generated)
  • Notes the file-download response shape on /subtitles/download
Common failure: Claude tries to use channelId on the subtitle endpoint, or guesses at a non-existent /youtube/transcript endpoint. Both signal that the subtitle section of content-analysis didn’t load.

​
Test 5 — Basic routing: account / quota

Prompt
How many CreatorDB credits do I have left?
Expected skill: account What a correct response looks like Single call: GET /usage, read creditsAvailable from the wrapper. Notes that -1 means unlimited. What to check
  • Routes to account
  • Single endpoint call, not multiple
  • Mentions the -1 = unlimited semantics
  • Doesn’t conflate creditsAvailable (current balance) with data.records[].totalQuotaUsed (historical usage)
Common failure: Claude doesn’t route to any skill and answers “I don’t have access to your usage data.” That means account didn’t load or its description didn’t match the prompt. Check that the skill is in ~/.claude/skills/ and its description includes “credits” and “remaining quota” trigger phrases.

​
Test 6 — Cross-skill handoff: CSV workflow

Prompt
I have a CSV of 200 YouTube channels with columns name, url, my_notes. Enrich it with audience demographics and emails. What would the credit budget be, and what’s your plan?
Expected primary skill: creator-csv-workflow Expected supporting skills loaded: creator-enrichment, account What a correct response looks like A multi-step plan citing the CSV-workflow’s structure: parse + classify identifiers, add _row_id, pre-flight /usage, enrich with /profile + /audience on the full roster and /contact only on the shortlisted rows, dedupe + flatten, export. Budget math should offer the user a “full contact” path (~27 cr/row × 200 = ~5,400 cr) versus a “shortlist-before-contact” path (~12 cr/row × 200 + 15 cr × shortlist-size) and let them choose. What to check
  • Routes to creator-csv-workflow first, then mentions the others
  • Includes the _row_id discipline
  • Budget number is in the right ballpark (4,000–6,000 credits for the full-roster path; lower if shortlisting before /contact)
  • Mentions shortlist-before-/contact as the cost-saving pattern — does not suggest bio-regex scraping in lieu of /contact
  • Mentions running /usage as a pre-flight (this is the cross-skill handoff to account)
Common failure: Claude tries to do everything from creator-csv-workflow alone and never references creator-enrichment for the per-endpoint detail. If that happens, the cross-skill links in creator-csv-workflow’s “Related skills” section need to be more prominent earlier in the file.

​
Test 7 — Adversarial: trigger-phrase poison

Prompt
I want to find creators who have already worked with Nike. What’s the best way?
Expected skill: brand-sponsorship (specifically /sponsor/creators) What a correct response looks like Single call: GET /sponsor/creators?brandId=nike.com&platform=youtube (and again for Instagram). Notes there’s no TikTok endpoint. Cites cost (25 credits per page). What to check
  • Routes to brand-sponsorship, NOT creator-search, despite the prompt starting with “find creators”
  • Resolves “Nike” to nike.com automatically
  • Notes the TikTok gap
Common failure: Claude routes to creator-search because of “find creators.” This is the canonical description-overlap test. If it fails, the creator-search description needs an explicit “Do not use for finding creators by brand affiliation — use brand-sponsorship instead” line, and brand-sponsorship needs “find creators who worked with <brand>” as an explicit trigger.

​
Test 8 — Adversarial: vague prompt requiring inference

Prompt
Show me what MrBeast is up to lately.
Expected skill: content-analysis (/youtube/content-detail) — possibly combined with creator-enrichment for a profile snapshot What a correct response looks like A plan to call /youtube/content-detail?channelId=UCX6OQ3DkcsbYNE6H8uQQuVA for recent videos and shorts. Optionally /youtube/profile for current subscriber count. Maps “lately” to “recent content” rather than “recent search trends.” What to check
  • Routes to content-analysis or a content+profile combination
  • Doesn’t try to call a non-existent search/news endpoint
  • Resolves “MrBeast” to a channelId or handle correctly
  • Cites cost (2 credits for content-detail)
Common failure: Claude asks the user to clarify, when “lately” + named creator unambiguously means “show recent posts.” Mild failure — if it asks once and proceeds, that’s fine. If it gets stuck asking, the content-analysis description should add “what is X up to” as a trigger phrase.

​
Test 9 — Refusal: out-of-scope request

Prompt
Transcribe this TikTok video for me: https://tiktok.com/@charlidamelio/video/1234567890
Expected behavior: Decline politely, explain why, suggest alternatives. What a correct response looks like Claude should explicitly say subtitles are YouTube-only in CreatorDB API v3, and offer alternatives: pulling caption text from /tiktok/content-detail, or using external transcription tools. Should NOT make up a /tiktok/subtitles endpoint or attempt the call. What to check
  • Explicitly identifies the YouTube-only constraint
  • Doesn’t fabricate a TikTok transcript endpoint
  • Suggests /tiktok/content-detail as the partial alternative for caption text
  • Doesn’t refuse without explanation (that signals the gotcha didn’t load)
Common failure: Claude calls /tiktok/subtitles/meta (which doesn’t exist) and returns a 404. This is a critical fail — it means the YouTube-only constraint in content-analysis didn’t get into context. Check the file actually has the “Subtitles are YouTube-only” callout in the gotchas section.

​
Test 10 — Substantive accuracy: gotcha awareness

Prompt
Compare the engagement rate of this YouTube creator (UCX6OQ3DkcsbYNE6H8uQQuVA) against this Instagram creator (cristiano).
Expected behavior: Claude warns that engagement formulas differ across platforms before doing the comparison, then either normalizes them or labels both clearly. What a correct response looks like A plan to call /youtube/profile and /instagram/profile, plus an explicit caveat: “Note that YouTube and Instagram compute engagement rate differently — YouTube uses (likes + comments + views) / subscribers in some endpoints and (likes + comments) / subscribers in others; Instagram uses (likes + comments) / followers. Comparing the raw values is misleading. I’ll either normalize both to (likes + comments) / follower-or-subscriber or label each value with its source formula.” What to check
  • The formula divergence warning appears unprompted
  • Claude offers to normalize before comparing
  • Mentions at least two of the three formula variants
  • Doesn’t blindly subtract one rate from the other
Common failure: Claude does the naive comparison without warning. This is the single highest-value gotcha across all six skills (it appears in three of them) and the test of whether the skill content is actually informing reasoning, not just sitting in context. If this fails, the formula-divergence warning needs to be surfaced higher in the affected skills’ bodies, not buried in a “Common gotchas” section.

​
Test 11 — Basic routing: content-level discovery

Prompt
Find sponsored YouTube videos with over 1 million views published in the last 30 days.
Expected skill: content-search What a correct response looks like A plan calling POST /youtube/content-search with content-level filters (isSponsored = true, views > 1000000, publishTime < 30), sorted by views descending. Notes that the unit of return is a content item (with a nested creator summary), not a creator. Cites the cost (~25 credits per page) and the 10-filter cap. What to check
  • Routes to content-search, NOT creator-search (which returns creators) and NOT content-analysis (which is per-creator, not cross-creator)
  • Uses /youtube/content-search (not /youtube/search or /youtube/content-detail)
  • Interprets publishTime < 30 as “within the last 30 days” (relative day count when used as a filter), not as a Unix-millisecond timestamp
  • Mentions that partneredBrands[] in the response maps directly to brandId domains used in brand-sponsorship (clean handoff if the user wants brand detail next)
Common failure: Claude routes to creator-search because of “find … videos” and tries to filter creators by avgRecentVideosViews > 1000000. That answers a different question (creators whose average video has 1M+ views). Sharpen creator-search’s description if this happens — it should explicitly defer to content-search for content-level filters.

​
Test 12 — Adversarial: content-first vs creator-first routing

Prompt
Show me which Reels using #skincare are getting the most engagement right now.
Expected skill: content-search (specifically /instagram/content-search with a hashtag + recency + engagement filter) What a correct response looks like Plan: POST /instagram/content-search with filters hashtag = "#skincare", postType = "reel", publishTime < 7 (or similar recent-window), sorted by engagementRate descending (or likes descending). Returns content items, not creators. Notes that Instagram’s response has no views field — engagement is computed from likes + comments only. What to check
  • Routes to content-search, NOT creator-search
  • Picks /instagram/content-search based on “Reels”
  • Uses postType = "reel" (not "reels" or "Reel") — exact enum
  • Notes the Instagram-has-no-views constraint when discussing engagement
  • Sorts on a content-level metric (likes, engagementRate), not a creator-level metric (totalFollowers)
Common failure: Claude routes to creator-search and filters by niches in [...] to find skincare creators. That’s a creator-first answer to a content-first question — the user gets a list of creators rather than a list of high-performing posts. If this fails repeatedly, the “content-first vs creator-first” rule of thumb in content-search’s “When to use” section needs to be repeated in the routing tables of creator-search and content-analysis.

​
Test 13 — Substantive accuracy: brand submission write path

Prompt
Notion isn’t showing up in any of your brand searches but they sponsor a lot of YouTube creators. How do I get them indexed?
Expected skill: brand-sponsorship (specifically POST /sponsor/submit) What a correct response looks like A plan calling POST /sponsor/submit with brandName: "Notion" and brandUrl: "https://www.notion.so" (with the protocol). Notes that:
  • The response status will be accepted (1 cr), done if already indexed (0 cr, with existingBrandId returned), or rejected (0 cr)
  • accepted does not mean the brand is immediately queryable — it queues the record for review
  • Daily cap is 100 submissions per API key, resetting at 00:00 UTC
  • The API has no “check submission status” endpoint, so the returned submissionId is the only audit handle
What to check
  • Routes to brand-sponsorship, lands on /sponsor/submit (not /sponsor/search)
  • Includes the http:// or https:// protocol in brandUrl (bare notion.so is rejected by the API)
  • Explains all three status outcomes and their credit costs
  • Mentions the 100/day cap and the no-status-check limitation
  • Does NOT promise the brand will be immediately available in /sponsor/search results after submission
Common failure: Claude treats /sponsor/submit as a synchronous “add brand” endpoint and tells the user they can query the brand immediately. That’s wrong — submissions are reviewed before indexing. If this fails, the “accepted does not mean queryable” callout in brand-sponsorship’s submit section needs to be more prominent.

​
Scoring

The most diagnostic failures are tests 7 (trigger-phrase poison: creator-search vs brand-sponsorship), 10 (gotcha awareness: engagement-formula divergence), and 12 (content-first vs creator-first: creator-search vs content-search). Failure on any of these signals a class of problem that touches multiple skills, not just one. Fix those first.

​
Re-running after edits

After fixing any failures and redeploying:
Then start a fresh Claude Code session (skills are loaded at session start) and re-run the failing tests.
Last modified on May 14, 2026