jacobcohen-rosenthal.me
robots.txt

Robots Exclusion Standard data for jacobcohen-rosenthal.me

Resource Scan

Scan Details

Site Domain jacobcohen-rosenthal.me
Base Domain jacobcohen-rosenthal.me
Scan Status Ok
Last Scan2024-10-28T08:39:23+00:00
Next Scan 2024-11-27T08:39:23+00:00

Last Scan

Scanned2024-10-28T08:39:23+00:00
URL https://jacobcohen-rosenthal.me/robots.txt
Redirect https://www.jacobcohen-rosenthal.me/robots.txt
Redirect Domain www.jacobcohen-rosenthal.me
Redirect Base jacobcohen-rosenthal.me
Domain IPs 18.139.194.139, 52.74.166.77
Redirect IPs 18.139.194.139, 52.74.166.77
Response IP 18.139.194.139
Found Yes
Hash 041bcb2415bf9cbb4b1f3a579ef8deb78fb592a9a949e74f45c49c28181322e7
SimHash 6510de62c633

Groups

*

Rule Path
Allow /
Disallow /search

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://jacobcohen-rosenthal.me/sitemap.xml

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.