Log In Sign Up

aranustech.co.id
robots.txt

Robots Exclusion Standard data for aranustech.co.id

Resource Scan

Scan Details

Site Domain aranustech.co.id
Base Domain aranustech.co.id
Scan Status Ok
Last Scan2025-06-03T04:48:22+00:00
Next Scan 2025-07-03T04:48:22+00:00

Last Scan

Scanned2025-06-03T04:48:22+00:00
URL https://aranustech.co.id/robots.txt
Domain IPs 103.55.39.107
Response IP 103.55.39.107
Found Yes
Hash 0fb77c23e817c267ed22de7ee60a35ea2a55f47cdcf00b89c24f9ec55b301eab
SimHash 4011d9006390

Groups

googlebot

Rule Path
Disallow /api

googlebot
adsbot-google

Rule Path
Disallow /api

*

Rule Path
Disallow /api
Back to top

Comments

  • Example 1: Block only Googlebot
  • Example 2: Block Googlebot and Adsbot
  • Example 3: Block all crawlers except AdsBot (AdsBot crawlers must be named explicitly)
Back to top