Log In Sign Up

bettybets.net
robots.txt

Robots Exclusion Standard data for bettybets.net

Resource Scan

Scan Details

Site Domain bettybets.net
Base Domain bettybets.net
Scan Status Ok
Last Scan2024-11-11T12:10:56+00:00
Next Scan 2024-11-25T12:10:56+00:00

Last Scan

Scanned2024-11-11T12:10:56+00:00
URL https://bettybets.net/robots.txt
Redirect https://www.bettybets.net/robots.txt
Redirect Domain www.bettybets.net
Redirect Base bettybets.net
Domain IPs 76.76.21.21
Redirect IPs 76.76.21.123, 76.76.21.61
Response IP 76.76.21.98
Found Yes
Hash 77070ecb62d3646689bb4bf5cc76f51d0062d623fac478114e1ea5f8f4b7d470
SimHash 2e3eb031c5f5

Groups

googlebot

Rule Path
Allow /?_escaped_fragment_
Allow /*
Allow /*/c/*

slurp

Rule Path
Allow /?_escaped_fragment_
Allow /*
Allow /*/c/*

yandex

Rule Path
Allow /?_escaped_fragment_
Allow /*
Allow /*/c/*

msnbot

Rule Path
Allow /?_escaped_fragment_
Allow /*
Allow /*/c/*

bingbot

Rule Path
Allow /?_escaped_fragment_
Allow /*
Allow /*/c/*

*

Rule Path
Allow /?_escaped_fragment_
Allow /*
Allow /*/c/*

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://www.bettybets.net/sitemap.xml
Back to top

Comments

  • Google Search Engine Robot
  • ==========================
  • Yahoo! Search Engine Robot
  • ==========================
  • Yandex Search Engine Robot
  • ==========================
  • Microsoft Search Engine Robot
  • =============================
  • Bing Search Engine Robot
  • ========================
  • Every bot that might possibly read and respect this file
  • ========================================================
  • Wait 1 second between successive requests. See ONBOARD-2698 for details.
  • Independent of user agent. Links in the sitemap are full URLs using https:// and need to match
  • the protocol of the sitemap.
Back to top