Log In Sign Up

adrian.web.id
robots.txt

Robots Exclusion Standard data for adrian.web.id

Resource Scan

Scan Details

Site Domain adrian.web.id
Base Domain adrian.web.id
Scan Status Ok
Last Scan2025-10-16T19:40:07+00:00
Next Scan 2025-10-23T19:40:07+00:00

Last Scan

Scanned2025-10-16T19:40:07+00:00
URL https://adrian.web.id/robots.txt
Domain IPs 104.21.96.93, 172.67.176.71, 2606:4700:3031::ac43:b047, 2606:4700:3033::6815:605d
Response IP 172.67.176.71
Found Yes
Hash fe1007bf2967ef16b67d4fc89d5731f332615babfe5a45a9a8ffcd8f5612dfcb
SimHash 4155ae41e634

Groups

*

Rule Path
Disallow /refer
Disallow /cgi-bin
Disallow /browsers
Disallow /flags
Disallow /stats
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /wp-content/themes
Disallow /wp-content/downloads
Disallow /trackback
Disallow /comments
Disallow */trackback
Disallow */comments
Disallow /*?
Back to top

Other Records

Field Value
sitemap https://adrian.web.id/wp-sitemap.xml
Back to top

Comments

  • Google Image
  • User-agent: Googlebot-Image
  • Disallow:
  • Allow: /*
  • Google AdSense
  • User-agent: Mediapartners-Google
  • Disallow:
  • digg mirror
  • User-agent: duggmirror
  • Disallow: /
  • Disallow: /wp-content/uploads
  • Disallow: /category/*/*
Back to top