ophiropt.com
robots.txt

Robots Exclusion Standard data for ophiropt.com

Resource Scan

Scan Details

Site Domain ophiropt.com
Base Domain ophiropt.com
Scan Status Ok
Last Scan2024-09-06T20:35:28+00:00
Next Scan 2024-10-06T20:35:28+00:00

Last Scan

Scanned2024-09-06T20:35:28+00:00
URL https://ophiropt.com/robots.txt
Redirect https://www.ophiropt.com/en/robots.txt
Redirect Domain www.ophiropt.com
Redirect Base ophiropt.com
Domain IPs 192.124.249.56
Redirect IPs 132.147.114.72, 138.113.53.41
Response IP 138.113.53.41
Found Yes
Hash b0e522ef2103686eb7430c5b0d853ea22e11af8ac13400d5933dcd1a0e1bf9b6
SimHash 6c505716eff0

Groups

*

Rule Path
Disallow /en/cart
Disallow /en/checkout
Disallow /en/my-account

Other Records

Field Value Comment
crawl-delay 10 10 seconds between page requests

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.ophiropt.com/medias/sys_master/root/h7a/h06/9820861988894/Homepage-en-USD-7043518886418477855.xml
sitemap https://www.ophiropt.com/medias/sys_master/root/ha7/h71/9820862054430/Product-en-USD-2908955049986614820.xml
sitemap https://www.ophiropt.com/medias/sys_master/root/h78/hff/9820862119966/Category-en-USD-6082765942614746546.xml
sitemap https://www.ophiropt.com/medias/sys_master/root/h10/h20/9820862185502/CategoryLanding-en-USD-2764794771626703040.xml
sitemap https://www.ophiropt.com/medias/sys_master/root/h56/hbd/9820862316574/Content-en-USD-4630051711770956066.xml

Comments

  • For all robots
  • Block access to specific groups of pages
  • Allow search crawlers to discover the sitemap
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot