Log In Sign Up

custodib.us
robots.txt

Robots Exclusion Standard data for custodib.us

Resource Scan

Scan Details

Site Domain custodib.us
Base Domain custodib.us
Scan Status Ok
Last Scan2024-09-20T00:06:53+00:00
Next Scan 2024-10-04T00:06:53+00:00

Last Scan

Scanned2024-09-20T00:06:53+00:00
URL https://custodib.us/robots.txt
Redirect https://www.procustodibus.com/robots.txt
Redirect Domain www.procustodibus.com
Redirect Base procustodibus.com
Domain IPs 2600:9000:2175:2600:1b:b7f7:c9c0:93a1, 2600:9000:2175:3000:1b:b7f7:c9c0:93a1, 2600:9000:2175:4200:1b:b7f7:c9c0:93a1, 2600:9000:2175:5200:1b:b7f7:c9c0:93a1, 2600:9000:2175:6600:1b:b7f7:c9c0:93a1, 2600:9000:2175:7600:1b:b7f7:c9c0:93a1, 2600:9000:2175:bc00:1b:b7f7:c9c0:93a1, 2600:9000:2175:f800:1b:b7f7:c9c0:93a1, 52.84.45.42, 52.84.45.47, 52.84.45.69, 52.84.45.75
Redirect IPs 18.161.111.110, 18.161.111.40, 18.161.111.68, 18.161.111.90, 2600:9000:2127:0:b:efa7:2740:93a1, 2600:9000:2127:1200:b:efa7:2740:93a1, 2600:9000:2127:3600:b:efa7:2740:93a1, 2600:9000:2127:400:b:efa7:2740:93a1, 2600:9000:2127:4c00:b:efa7:2740:93a1, 2600:9000:2127:a00:b:efa7:2740:93a1, 2600:9000:2127:e000:b:efa7:2740:93a1, 2600:9000:2127:e600:b:efa7:2740:93a1
Response IP 18.165.140.119
Found Yes
Hash 0221ca33afda3959e81ac377e5aa591667a78ef3c314e1f29fbe53948c12e98a
SimHash 68304b226333

Groups

*

Rule Path
Disallow
Back to top

Comments

  • www.robotstxt.org/
  • Allow crawling of all content
Back to top