not-ai.tools
robots.txt

Robots Exclusion Standard data for not-ai.tools

Resource Scan

Scan Details

Site Domain not-ai.tools
Base Domain not-ai.tools
Scan Status Ok
Last Scan2025-12-11T19:10:06+00:00
Next Scan 2025-12-18T19:10:06+00:00

Last Scan

Scanned2025-12-11T19:10:06+00:00
URL https://not-ai.tools/robots.txt
Redirect https://www.not-ai.tools/robots.txt
Redirect Domain www.not-ai.tools
Redirect Base not-ai.tools
Domain IPs 216.198.79.1
Redirect IPs 216.198.79.1, 64.29.17.1
Response IP 216.198.79.1
Found Yes
Hash f1560293637fa69588835be1e56441bb41d78a2485d5de6225cb48987c45c8fd
SimHash 47128a72e712

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /_next/
Disallow /private/

Other Records

Field Value
crawl-delay 1

gptbot
oai-searchbot

Rule Path
Allow /
Disallow /api/
Disallow /_next/

Other Records

Field Value
crawl-delay 2

anthropic-ai

Rule Path
Allow /
Disallow /api/
Disallow /_next/

google-extended

Rule Path
Allow /
Disallow /api/
Disallow /_next/

bingbot

Rule Path
Allow /
Disallow /api/
Disallow /_next/

Other Records

Field Value
crawl-delay 1

perplexitybot

Rule Path
Allow /
Disallow /api/
Disallow /_next/

Other Records

Field Value
sitemap https://www.not-ai.tools/sitemap.xml

Warnings

  • `host` is not a known field.