Log In Sign Up

bullsbikesusa.com
robots.txt

Robots Exclusion Standard data for bullsbikesusa.com

Resource Scan

Scan Details

Site Domain bullsbikesusa.com
Base Domain bullsbikesusa.com
Scan Status Ok
Last Scan2024-05-16T12:21:25+00:00
Next Scan 2024-05-30T12:21:25+00:00

Last Scan

Scanned2024-05-16T12:21:25+00:00
URL https://bullsbikesusa.com/robots.txt
Redirect https://www.bullsbikesusa.com/robots.txt/
Redirect Domain www.bullsbikesusa.com
Redirect Base bullsbikesusa.com
Domain IPs 52.85.151.3, 52.85.151.69, 52.85.151.96, 52.85.151.99
Redirect IPs 18.164.174.11, 18.164.174.4, 18.164.174.47, 18.164.174.63
Response IP 18.165.171.46
Found Yes
Hash 7c453d54ecf492d4b21303c58e4b967e7f106ceadae3ab52a50a4c3d96d87be0
SimHash 4b426f1744b6

Groups

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.bullsbikesusa.com/sitemap.xml
sitemap https://www.bullsbikesusa.com/pub/sitemap.xml
Back to top

Comments

  • robots.txt for Magento Community and Enterprise
  • GENERAL SETTINGS
  • Enable robots.txt rules for all crawlers
  • Crawl-delay parameter: number of seconds to wait between successive requests to the same server.
  • Set a custom crawl rate if you're experiencing traffic problems with your server.
  • Magento sitemap: uncomment and replace the URL to your Magento sitemap file
Back to top

Warnings

  • 1 invalid line.
Back to top