Log In Sign Up

agri.kompas.com
robots.txt

Robots Exclusion Standard data for agri.kompas.com

Resource Scan

Scan Details

Site Domain agri.kompas.com
Base Domain kompas.com
Scan Status Ok
Last Scan2024-09-10T11:59:48+00:00
Next Scan 2024-10-10T11:59:48+00:00

Last Scan

Scanned2024-09-10T11:59:48+00:00
URL https://agri.kompas.com/robots.txt
Domain IPs 2600:9000:2003:1e00:c:b86e:4340:93a1, 2600:9000:2003:5400:c:b86e:4340:93a1, 2600:9000:2003:5a00:c:b86e:4340:93a1, 2600:9000:2003:6400:c:b86e:4340:93a1, 2600:9000:2003:a600:c:b86e:4340:93a1, 2600:9000:2003:ae00:c:b86e:4340:93a1, 2600:9000:2003:b000:c:b86e:4340:93a1, 2600:9000:2003:f200:c:b86e:4340:93a1, 52.84.229.128, 52.84.229.50, 52.84.229.7, 52.84.229.82
Response IP 52.84.229.82
Found Yes
Hash 3ca6b62599e80633ecea941abd80de193f504a749261976bc768967dfd1ae220
SimHash 51105971ec93

Groups

*

Rule Path
Allow /

gptbot

Rule Path
Disallow /
Disallow /
Disallow /
Disallow /
Disallow /

perplexitybot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://agri.kompas.com/sitemap.xml
Back to top

Warnings

  • `‍user-agent` is not a known field.
Back to top