bobwards.com
robots.txt

Robots Exclusion Standard data for bobwards.com

Resource Scan

Scan Details

Site Domain bobwards.com
Base Domain bobwards.com
Scan Status Ok
Last Scan2024-05-21T14:43:40+00:00
Next Scan 2024-06-20T14:43:40+00:00

Last Scan

Scanned2024-05-21T14:43:40+00:00
URL https://bobwards.com/robots.txt
Redirect https://www.bobwards.com/robots.txt
Redirect Domain www.bobwards.com
Redirect Base bobwards.com
Domain IPs 89.106.200.1
Redirect IPs 18.172.64.26, 18.172.64.37, 18.172.64.55, 18.172.64.78, 2600:9000:2481:200:15:af44:2080:93a1, 2600:9000:2481:2800:15:af44:2080:93a1, 2600:9000:2481:5800:15:af44:2080:93a1, 2600:9000:2481:5c00:15:af44:2080:93a1, 2600:9000:2481:7000:15:af44:2080:93a1, 2600:9000:2481:7a00:15:af44:2080:93a1, 2600:9000:2481:8000:15:af44:2080:93a1, 2600:9000:2481:fa00:15:af44:2080:93a1
Response IP 18.155.68.64
Found Yes
Hash 1a8dfb483054a894cdafb539d67213cca1876a48109f317fab2e04c87d870110
SimHash b015e537cdc1

Groups

*

Rule Path
Disallow /beta.als.com
Disallow /https%3A//beta.als.com
Disallow /http%3A//beta.als.com
Disallow beta.als.com
Disallow /beta/

Other Records

Field Value
sitemap https://www.als.com/sitemap.xml

Comments

  • Disallow all crawlers access to certain pages.

Warnings

  • 2 invalid lines.