meater.com
robots.txt

Robots Exclusion Standard data for meater.com

Resource Scan

Scan Details

Site Domain meater.com
Base Domain meater.com
Scan Status Ok
Last Scan2024-05-14T07:16:58+00:00
Next Scan 2024-05-28T07:16:58+00:00

Last Scan

Scanned2024-05-14T07:16:58+00:00
URL https://meater.com/robots.txt
Redirect https://www.meater.com/robots.txt
Redirect Domain www.meater.com
Redirect Base meater.com
Domain IPs 35.164.129.221
Redirect IPs 108.157.254.22, 108.157.254.30, 108.157.254.85, 108.157.254.9, 2600:9000:2753:1200:1b:8961:3280:93a1, 2600:9000:2753:4200:1b:8961:3280:93a1, 2600:9000:2753:4a00:1b:8961:3280:93a1, 2600:9000:2753:600:1b:8961:3280:93a1, 2600:9000:2753:6c00:1b:8961:3280:93a1, 2600:9000:2753:9200:1b:8961:3280:93a1, 2600:9000:2753:e00:1b:8961:3280:93a1, 2600:9000:2753:f800:1b:8961:3280:93a1
Response IP 108.157.254.85
Found Yes
Hash c3f75da21db9db38cabd93f46ea395e5fbfa5d9fac6a784f4adbf61b1d33a5b9
SimHash 4840cb004772

Groups

*

Rule Path
Disallow /accounts
Allow /

Comments

  • robots.txt
  • Block all crawlers for /accounts
  • Allow all other crawlers