# As a condition of accessing this website, you agree to abide by the following
# content signals:

# (a)  If a Content-Signal = yes, you may collect content for the corresponding
#      use.
# (b)  If a Content-Signal = no, you may not collect content for the
#      corresponding use.
# (c)  If the website operator does not include a Content-Signal for a
#      corresponding use, the website operator neither grants nor restricts
#      permission via Content-Signal with respect to the corresponding use.

# The content signals and their meanings are:

# search:   building a search index and providing search results (e.g., returning
#           hyperlinks and short excerpts from your website's contents). Search does not
#           include providing AI-generated search summaries.
# ai-input: inputting content into one or more AI models (e.g., retrieval
#           augmented generation, grounding, or other real-time taking of content for
#           generative AI search answers).
# ai-train: training or fine-tuning AI models.
# use:      how AI systems may consume the content (immediate, reference, or full).

# ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF
# RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT
# AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET.

# BEGIN Cloudflare Managed content

# AI crawlers are allowed. This block previously carried `Disallow: /` for
# Amazonbot, Applebot-Extended, Bytespider, CCBot, ClaudeBot,
# CloudflareBrowserRenderingCrawler, Google-Extended, GPTBot and
# meta-externalagent, which shut the site out of exactly the answer engines it
# is trying to be cited in. Deliberately reversed: AI search is this property's
# strongest channel relative to classic organic.
#
# No effect on Google Search either way. Googlebot was never in that list, and
# Google-Extended governs Gemini grounding and training only -- Google documents
# that it does not affect Search ranking or inclusion, and AI Overviews are built
# from the Search index that Googlebot feeds. So this changes nothing about how
# the site ranks; it changes who may read it.
#
# The Content-Signal below is unchanged and still says ai-train=no. Access and
# training rights are separate: crawlers may read and cite this content, and the
# reservation of training rights stands. Content-Signal is a declared restriction
# under Article 4 of EU Directive 2019/790, not an enforcement mechanism.

User-agent: *
Content-Signal: search=yes,ai-train=no,use=reference
Allow: /

# END Cloudflare Managed Content

# A second bare `User-agent: *` group used to sit here carrying only `Allow: /`.
# Two groups for the same agent is ambiguous -- a crawler that takes the first
# match rather than merging would read the signal, one that takes the last would
# not -- and it granted nothing the group above does not already grant. Removed
# so there is exactly one `*` group and it carries the Content-Signal.

# Markdown twins (.md) of every page are published for AI answer engines --
# roughly 87x smaller than the HTML and far cheaper to parse. Search engines
# are told NOT to index them: they duplicate each page's content at a second
# URL, which would split signals for no gain. AI crawlers are welcome to them.
# Same split resources.getbluejay.ai already uses.
#
# The Content-Signal and Allow lines are repeated here on purpose: a crawler
# obeys only its most specific matching group, so once Googlebot has a group of
# its own it stops reading the `User-agent: *` group entirely and would
# otherwise lose the site's content signal.
User-agent: Googlebot
User-agent: Googlebot-Image
User-agent: Bingbot
Content-Signal: search=yes,ai-train=no,use=reference
Allow: /
Disallow: /*.md$

Sitemap: https://getbluejay.ai/sitemap.xml