Log In Sign Up

sidley.com
robots.txt

Robots Exclusion Standard data for sidley.com

Resource Scan

Scan Details

Site Domain sidley.com
Base Domain sidley.com
Scan Status Ok
Last Scan2025-09-26T15:06:52+00:00
Next Scan 2025-10-26T15:06:52+00:00

Last Scan

Scanned2025-09-26T15:06:52+00:00
URL https://sidley.com/robots.txt
Redirect https://www.sidley.com/robots.txt
Redirect Domain www.sidley.com
Redirect Base sidley.com
Domain IPs 23.100.43.208
Redirect IPs 104.18.32.2, 172.64.155.254
Response IP 172.64.155.254
Found Yes
Hash a96e07be0415522f134d36223c36e6be2d658b4d0ee1d5e907d2f961ba65a530
SimHash 6c5add08ede3

Groups

charlotte

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

converacrawler/0.9e

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow
Allow /-/media

*

Rule Path
Disallow /~/vcf/*
Disallow /*?exceptionLang*
Disallow /*?exceptionlang*
Disallow /*?regionchange*
Disallow /*?format=vcard*
Disallow /-/media
Allow /-/media/files
Back to top

Other Records

Field Value
sitemap https://www.sidley.com/GoogleSitemap
Back to top

Comments

  • Hi! I'm DynamicRobot 2.2.10.0. You'll be happy to know I'm installed properly.
  • I'm serving up robots-www.sidley.com.txt, because it matched the request's host "www.sidley.com".
Back to top