cdli.mpiwg-berlin.mpg.de
robots.txt

Robots Exclusion Standard data for cdli.mpiwg-berlin.mpg.de

Resource Scan

Scan Details

Site Domain cdli.mpiwg-berlin.mpg.de
Base Domain mpg.de
Scan Status Ok
Last Scan2025-06-24T00:09:55+00:00
Next Scan 2025-07-24T00:09:55+00:00

Last Scan

Scanned2025-06-24T00:09:55+00:00
URL https://cdli.mpiwg-berlin.mpg.de/robots.txt
Redirect https://cdli.earth/robots.txt
Redirect Domain cdli.earth
Redirect Base cdli.earth
Domain IPs 141.14.250.111
Redirect IPs 141.5.123.37
Response IP 141.5.123.37
Found Yes
Hash ef57ec095efbfec0b525e37fb705e73216d6a8197aee35216b4a9a665ec1a4b9
SimHash 6558dcc0c1b5

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60