recentlybooked.com
robots.txt

Robots Exclusion Standard data for recentlybooked.com

Resource Scan

Scan Details

Site Domain recentlybooked.com
Base Domain recentlybooked.com
Scan Status Ok
Last Scan2024-11-15T01:32:31+00:00
Next Scan 2024-11-22T01:32:31+00:00

Last Scan

Scanned2024-11-15T01:32:31+00:00
URL https://recentlybooked.com/robots.txt
Domain IPs 69.167.187.92
Response IP 69.167.187.92
Found Yes
Hash 7183c322262e9e3a83f815e2ca2c68fb5287a1eadd86d8cb8fea99e7fc4686e1
SimHash 6515c281e790

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mediapartners-google

Rule Path
Disallow

googlebot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

bingbot

Rule Path
Disallow

facebookexternalhit

Rule Path
Disallow

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://recentlybooked.com/sitemap.xml

Comments

  • Group 1
  • User-agent: Amazonbot
  • Disallow: /
  • User-agent: PetalBot
  • Disallow: /
  • User-agent: Yahoobot
  • Disallow : /
  • User-agent: *
  • Allow: /
  • Allow only major search spiders
  • Block all other spiders