mantap89oke.com
robots.txt

Robots Exclusion Standard data for mantap89oke.com

Resource Scan

Scan Details

Site Domain mantap89oke.com
Base Domain mantap89oke.com
Scan Status Ok
Last Scan2025-11-30T02:15:52+00:00
Next Scan 2025-12-07T02:15:52+00:00

Last Scan

Scanned2025-11-30T02:15:52+00:00
URL https://mantap89oke.com/robots.txt
Redirect https://rtpmantap89.co/robots.txt
Redirect Domain rtpmantap89.co
Redirect Base rtpmantap89.co
Domain IPs 104.21.65.169, 172.67.147.70, 2606:4700:3031::ac43:9346, 2606:4700:3033::6815:41a9
Redirect IPs 159.198.37.110
Response IP 159.198.37.110
Found Yes
Hash ad3ed1838fffc820ef399d367f8ad0ae2aa8d4114682b02b9584ea20d38a2409
SimHash ef175e337684

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /tmp/
Disallow /admin/
Disallow /wp-admin/
Disallow /private/
Allow /wp-admin/admin-ajax.php
Allow /public/
Allow /assets/
Allow /images/
Allow /videos/
Allow /js/
Allow /css/

Other Records

Field Value
crawl-delay 5

googlebot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 2

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 3

duckduckbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 3

yandex

Rule Path
Disallow

Other Records

Field Value
crawl-delay 3

baiduspider

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://rtpmantap89.co/sitemap.xml

Comments

  • Kecepatan crawl agar efisien
  • Sitemap utama
  • ================================
  • Bot whitelist (search engine utama)
  • ================================
  • ================================
  • Bot blacklist (pengamanan dasar)
  • ================================
  • ================================
  • Akhiran
  • ================================
  • File ini membantu Google, Bing, dan mesin pencari lain
  • menelusuri konten dengan cepat tanpa menabrak direktori sensitif.
  • Pastikan sitemap.xml selalu up-to-date.