Scorecard: how well did we do?
25 Oct 2026, Dubai time (UTC+4). Scores are my judgement from the evidence in 10-gully-audit.md, the two case files, and the live checks. Scale: 0 = absent, 5 = works but with known holes, 8 = good and proven, 10 = best practice and proven over time. Every score has a reason. Change a score only with new evidence.
Labels: [V] checked by me this run. [R] from a team report. [U] unverified.
1. Headline
| buydubai.io | benidorm.co.uk | Shared stack and process | |
|---|---|---|---|
| Overall | 5.6 / 10 | 6.6 / 10 | 6 / 10 |
| One line | Best data design we have, with the worst gap between design and what is live. | Better build discipline and clearer pages, but thinner proof of outcomes and no validation gate I could find. | The method is real and repeatable. The measurement is not yet there. |
Shared stack and process: process 7, stack 6.5, combined 6.
Both sites are two weeks old in search terms. The scores judge the system, not the traffic.
2. Scores by dimension
Scores by dimension (0 to 10)
Bars are drawn from the table below. Scale: 0 = absent, 5 = works but with known holes, 8 = good and proven, 10 = best practice and proven over time.
| Dimension | buydubai.io | benidorm.co.uk | Notes |
|---|---|---|---|
| 1. Source selection and rights | 7 | 8 | Both use official or open data and dated sources. Benidorm keeps a source register with a "checked" date on each figure and refuses scraped review text. buydubai uses DLD, Ejari, KHDA, OSM (strong) plus Property Finder and Google Places (rights question open; tiers decided 24 Oct but not implemented). [R/V] |
| 2. Ingestion and freshness | 5 | 7 | buydubai: nightly job failed 21 and 24 Oct and restored last good [R]; Town Square says "12 months to 22 October" on 25 Oct; llms.txt is 2–3 days behind the "updated every morning" claim; agent pages say "Checked 23 October". [V] Benidorm: 588 of 752 sitemap URLs carry lastmod 24 Oct; page dates come from git, not the build clock [V]. Caveat: a lastmod that moves for 78% of pages in one day may be a rebuild date, not a data change [U]. |
| 3. Identity and normalisation | 5 | 7 | buydubai: good registers (186 communities, 6,166 buildings) but in staging; link rates 87.9% / 75.5% / 30.8% (leases to building) [R]. Benidorm has a small closed set of entities (hotels, bars, restaurants) with OSM and venue IDs; I did not audit its matching rate [U]. |
| 4. Validation gates | 5 | 5 | buydubai: strong gates (lint, eval, spend) but they cover about a quarter of pages by manifest [R] and the eval gate went live only 25 Oct. 5,931 agent pages are outside. Benidorm: CI rebuild-and-diff is described in build_all.py and CLAUDE.md [R], but I did not find the workflow file in the Studio tree [U], and I found no number lint equivalent. 60 of 60 sampled pages had a canonical and a date [V]. |
| 5. Reproducibility | 6 | 8 | buydubai: staged cube rebuilt with 0 diff [R]; 23 Oct cube not reproducible (13,547 rows differ) [R]; live pages mix two cube hashes [V]. Benidorm: frozen clock, "HTML is output, never edit by hand", 1,351 commits [V/R]. |
| 6. Page quality and helpfulness | 5 | 7 | buydubai: gold pages (Business Bay, JVC, Town Square, Motor City) are deep (8,066–18,564 words) and candid about gaps [V]. But 5,931 thin agent pages and three community pages with placeholders are live [V]. Benidorm: median 1,878 words over 60 random pages, 0 noindex, 60 canonical, 60 dated [V]. 499 of 752 URLs (66%) are venue listings (bars 269, hotels 119, restaurants 111); that is a doorway-style pattern to watch even though the sampled pages are substantive [V]. |
| 7. Search and AI-search technical set-up | 8 | 9 | Both: JSON-LD, sitemaps, canonical, bots allowed. Benidorm adds a working MCP server (tools/list answered live), llms-full.txt (4.9 MB), and Hotel starRating only, no review markup [V]. Google says none of the AI files are required [V]; I score hygiene, not ranking power. |
| 8. Monitoring and measurement | 4 | 4 | GSC and GA4 are connected for both [V]. No dashboards on freshness or link rates seen. Studio's own monetisation note says no traffic data was found as of 23 Oct [R]. Internal and test traffic in GA4 is unconfirmed [U]. Benidorm A/B control domains: not registered as of the 14–17 Oct notes [R, re-verify]. |
| 9. Process and team method | 7 | 7 | Builder plus independent auditor, status ladder, SELF-CHECK rule, rollback ids [R]. Weakness: the gate came after the complaint, not before. Benidorm: documented rules, handoff files, bots writing to data/. |
| 10. Traffic evidence | 4 | 4 | See section 3. Real, early, tiny. |
| Average | 5.6 | 6.6 |
3. What the traffic says (read-only GSC and GA4, pulled 25 Oct about 13:00 Dubai time)
| buydubai.io | benidorm.co.uk | |
|---|---|---|
| GSC clicks / impressions | 69 / 6,249 (15–24 Oct) [V] | 70 / 6,421 (13–24 Oct) [V] |
| Best day | 23 Oct: 25 clicks, 2,004 impressions | 21 Oct: 18 clicks, 1,326 impressions |
| GA4 sessions 20 Sep–25 Oct, direct | 96 | 144 |
| GA4 chatgpt.com (ai-assistant) | 78 | 101 |
| GA4 Google organic | 77 | 84 |
| First ChatGPT session | 19 Oct | 19 Oct |
| First Google organic session | 17 Oct | 17 Oct |
Read: "rank and get AI traffic in days" is supported but small. Both sites show ChatGPT referrals on par with or above Google organic within days of first measurement. These are tens of clicks a day at most. "Direct" may include our own testing. Benidorm's GA4 tag was added on 15 Oct (DEPLOY.md [R]); buydubai's install date I have not confirmed. "In days" therefore means days after measurement began. Do not quote these as proof of scale.
4. Strengths
- A fact cube with IDs and hashes, and pages that carry a manifest of the facts they print. Few sites at any size do this. [V]
- Gates that block publishing: lint, eval, spend caps. [R]
- A second agent that audits and never writes code. [R]
- Honest page copy about what we could not confirm. [V]
- Benidorm: dated sources, deterministic build, MCP and full-text feed, no review text. [V]
- We write down our failures (gate rechecks, cube proof). That is rare and valuable.
5. Gaps
- Agent pages (72.7% of buydubai's main sitemap) and about 90,000 unit pages outside the gate; agent pages thin.
- Late wiring of the eval gate (25 Oct).
- Mixed cube hashes live; stale dates against freshness claims.
- Metric register covers 25 of 192 (metric, slice) pairs at the 23 Oct cube [R]; registers staging-only.
- No outcome measurement tied to releases; no baseline before launch for Benidorm.
- Benidorm: no number-lint equivalent found; A/B control not set up; affiliate crediting unchecked (an earlier "13 links without partner id" note did not reproduce; GYG links carry
partner_idon 4 pages checked 25 Oct [V]); CI workflow file not located. - No monitoring of the nightly job beyond someone noticing.
- Source-rights policy unimplemented (Property Finder, Google Places).
6. Risks (ranked by damage)
- Google scaled-content action on buydubai.io: 5,931 templated, thin agent pages, plus about 90,000 unit pages in two sitemaps that the index no longer lists but which still return 200 and
index, follow[V]. Google can exclude such pages from Search "however created". [V] A site-wide effect is possible. Seeresearch/x-posts.mdfor the National Today case. - A wrong public number under a named broker's name or in an AI answer, with no lineage to trace it.
- Stale-but-claimed-fresh pages (trust loss with AI systems that cite dates).
- Rights complaint over scraped listing or photo data.
- Benidorm doorway pattern: 66% of URLs are venue listings with the same template.
- Affiliate disclosure and tracking gaps on Benidorm (an earlier note of 13 links without a partner id did not reproduce on the four pages checked; GetYourGuide links carry
partner_id, and one generic city link per page does not [V]). - Single-operator dependency: MacPro deploys, Studio data, one human approver.
7. Top 10 fixes, ranked
| # | Fix | Why | Effort |
|---|---|---|---|
| 1 | Decide the agent pages and the ~90,000 unit pages: either noindex and remove from sitemaps until each has a manifest and real text, or prove each set earns its place (unique, sourced, gated). Remove or restore sitemap-units-*.xml deliberately; today they are orphaned. |
Biggest search risk. Cheap to reverse. | Hours |
| 2 | Gate every sitemap URL. No page in the sitemap without a facts-manifest, or an explicit exclusion list reviewed weekly. | Closes the "73% outside the gate" hole. | 1–2 days |
| 3 | One cube hash per release. Rebuild the five 23 Oct community pages from the new cube, or hold Town Square back to match. | Mixed lineage live. | 1 day |
| 4 | Fix or noindex the three placeholder community pages (discovery-gardens, jvt, dubai-silicon-oasis). |
Anthony's 24 Oct complaint; still live. | 1 day each |
| 5 | Freshness SLO plus alert. If the nightly job fails or data_as_of is older than 48 hours, page and llms.txt say so, and a message goes to the owner. |
Claim says "updated every morning". | 1 day |
| 6 | Register all 192 (metric, slice) pairs and make builders read the register, not hard-coded numbers. Kill hand-typed figures (JVC video section). | Single truth. | 1–2 weeks |
| 7 | Calibrate the eval against outcomes: pick 10 pages, compare scores with impressions, clicks and cited-by-AI after 4 weeks. | A score nobody has tested is a guess. | Ongoing |
| 8 | Benidorm: locate or add the CI rebuild-diff workflow, add a number lint, confirm affiliate credit in the partner dashboard, register the A/B control domains. | Proof and money. | 2–3 days |
| 9 | Source-rights register with a tier per source and an attribution field; resolve Property Finder and Google Places terms. | Legal risk. | 2 days |
| 10 | Weekly traffic report into one dashboard: GSC clicks, ChatGPT sessions, pages indexed, gate failures. Filter internal traffic in GA4. | Without it no claim survives scrutiny. | 1 day |
8. How well designed is the process and the stack?
- Process: 7/10. Plan, build, independent audit, gate, deploy with rollback. The sequence is right. It was run in the wrong order once (publish, complaint, gate).
- Stack: 6.5/10. Parquet + DuckDB cube, YAML register, Python builders, static HTML on Vercel, Supabase for the assistant. Simple, cheap, and inspectable. Weak points: deploys from one machine, no central scheduler with alerting, two stacks (buydubai cube vs Benidorm manifest build) that do the same job in different ways.
- Design vs delivery: the design deserves 8; the delivery 5. The average is what the table says.
9. Cross-references
- Gully audit:
10-gully-audit.md. - Case files:
30-case-buydubai.md,31-case-benidorm.md. - External cases and X posts:
research/x-posts.md. - Missing evidence:
MISSING.md. - Where the code, repos and Vercel projects live:
PROJECTS.md.