bhhsstein.com
robots.txt

Robots Exclusion Standard data for bhhsstein.com

Resource Scan

Scan Details

Site Domain bhhsstein.com
Base Domain bhhsstein.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-27T04:31:31+00:00
Next Scan 2025-11-25T04:31:31+00:00

Last Successful Scan

Scanned2025-04-22T10:26:35+00:00
URL https://bhhsstein.com/robots.txt
Redirect https://www.bhhsstein.com/robots.txt
Redirect Domain www.bhhsstein.com
Redirect Base bhhsstein.com
Domain IPs 162.159.141.129, 172.66.1.125, 2606:4700:7::179, 2a06:98c1:58::179
Redirect IPs 162.159.141.129, 172.66.1.125, 2606:4700:7::179, 2a06:98c1:58::179
Response IP 172.66.1.125
Found Yes
Hash cf2c8d1b97eb054515c90bf4fc428efcdd2cade2b3cc759c238f17b18e75d382
SimHash 6a455510c291

Groups

*

Rule Path
Disallow /emails_kvarea/
Disallow /mobilefirst/
Disallow /private_kvarea/
Disallow /index.php?similar=1
Disallow /language.php
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /admin/

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

googlebot

Rule Path
Disallow /index.php?advanced=1
Disallow /index.php?quick=1
Allow /sell
Allow /areas
Allow /pages
Allow /resources
Allow /index.php?showagency=1
Allow /index.php?showagent=1

adsbot-google

Rule Path
Allow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

googlebot-mobile

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 120

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 120

mediapartners-google

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

adsbot-google-mobile

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

adsbot-google-mobile-apps

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

powermapper

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

screaming frog seo spider

Rule Path
Allow /

semrushbot

Rule Path
Allow /

termlybot

Rule Path
Allow /

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.bhhsstein.com/sitemap-index.xml

Comments

  • Adsense