recipequality.com
robots.txt

Robots Exclusion Standard data for recipequality.com

Resource Scan

Scan Details

Site Domain recipequality.com
Base Domain recipequality.com
Scan Status Ok
Last Scan2026-02-06T10:28:36+00:00
Next Scan 2026-02-13T10:28:36+00:00

Last Scan

Scanned2026-02-06T10:28:36+00:00
URL https://recipequality.com/robots.txt
Domain IPs 104.21.40.153, 172.67.154.72, 2606:4700:3030::ac43:9a48, 2606:4700:3034::6815:2899
Response IP 172.67.154.72
Found Yes
Hash dd68798882ecbc52a39b01c6b6aa67dd7b9a5f74dc1b5837f569798acf477e62
SimHash 6900dc52a600

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /api/

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://recipequality.com/sitemap.xml

Comments

  • Sitemaps
  • Crawl delay (optional - helps prevent server overload)
  • Disallow admin and API paths