shop.opticsplanet.com
robots.txt

Robots Exclusion Standard data for shop.opticsplanet.com

Resource Scan

Scan Details

Site Domain shop.opticsplanet.com
Base Domain opticsplanet.com
Scan Status Ok
Last Scan2025-03-21T22:54:56+00:00
Next Scan 2025-04-20T22:54:56+00:00

Last Scan

Scanned2025-03-21T22:54:56+00:00
URL https://shop.opticsplanet.com/robots.txt
Domain IPs 23.46.230.140, 23.46.230.158
Response IP 23.49.60.50
Found Yes
Hash 4750d1db1570287b19915788960f8ffa135b52ffd34b44fdc2fd3c3b05455ec2
SimHash 6c386357ed50

Groups

*

Rule Path
Disallow /staging/
Disallow /cgi-bin/
Disallow /RT/
Disallow /TG/
Disallow /QS/
Disallow /wgl/
Disallow /P/
Disallow /ymix/
Disallow /OS/
Disallow /cs/
Disallow /en%3D
Disallow /p/s%3D
Disallow /bt%3D
Disallow /sid%3D
Disallow /io%3D
Disallow /ii%3D
Disallow /api/
Disallow /iv-api/
Disallow /checkout/
Disallow /email/manage
Disallow /email/unsubscribe
Disallow /email/hold
Disallow /email/resubscribe
Disallow /fed-experiments-7ae6f108-47ff-4853-ad45-c50e7f1fc5e9/
Disallow /utc-time
Disallow /nelmio-js-logger

roverbot

Rule Path
Disallow /

spotbot

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

halebot

Rule Path
Disallow /
Disallow /plugins/feedback.php

Other Records

Field Value
sitemap https://shop.opticsplanet.com/sitemap.xml

Comments

  • Sitemaps
  • Cap is html
  • Rover is a bad dog <http://www.roverbot.com>
  • Block Indix.com
  • Block TheFind.com
  • Block CommonCrawl
  • Block Halebot (biggeye.tacitknowledge.com)
  • Googlebot doesn't work well with Facebook Comments