drunkcrab.net
robots.txt

Robots Exclusion Standard data for drunkcrab.net

Resource Scan

Scan Details

Site Domain drunkcrab.net
Base Domain drunkcrab.net
Scan Status Ok
Last Scan2024-09-14T15:34:37+00:00
Next Scan 2024-10-14T15:34:37+00:00

Last Scan

Scanned2024-09-14T15:34:37+00:00
URL https://www.drunkcrab.net/robots.txt
Domain IPs 34.213.52.225, 34.214.240.199, 34.215.105.120, 34.221.28.182, 52.88.189.47, 54.202.19.22, 54.218.201.246
Response IP 52.88.189.47
Found Yes
Hash 993bcf4fc6036937a2b78fac8f8d894e9266353dd169212d93b5c298381ccdd6
SimHash ca0d5c00a3f2

Groups

*

Rule Path
Disallow

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.drunkcrab.net/sitemap.xml

Comments

  • https://megaindex.com/crawler