klikpendidikan.com
robots.txt

Robots Exclusion Standard data for klikpendidikan.com

Resource Scan

Scan Details

Site Domain klikpendidikan.com
Base Domain klikpendidikan.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-03T21:13:53+00:00
Next Scan 2026-02-10T21:13:53+00:00

Last Successful Scan

Scanned2026-01-26T21:13:33+00:00
URL https://klikpendidikan.com/robots.txt
Redirect https://www.klikpendidikan.id/robots.txt
Redirect Domain www.klikpendidikan.id
Redirect Base klikpendidikan.id
Domain IPs 104.21.68.73, 172.67.191.171, 2606:4700:3036::ac43:bfab, 2606:4700:3037::6815:4449
Redirect IPs 104.18.6.204, 104.18.7.204, 2606:4700::6812:6cc, 2606:4700::6812:7cc
Response IP 104.18.6.204
Found Yes
Hash 57b640416e14f41026f5dcae40418bb272f2125efb508d02554552e2f98ffba4
SimHash 717cd470c030

Groups

*

Rule Path
Disallow /profeed/*
Disallow */fonts/*
Disallow /tv/detail/*
Disallow /pedoman-pemberitaan
Disallow /pedoman-konten-ai
Disallow /app-ads.txt
Disallow /wp-content/*

chatgpt-user

Rule Path
Disallow /

openai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

screaming frog seo

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

aliyunsecbot

Rule Path
Disallow /

httrack 3.0

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /profeed/*
Disallow */fonts/*

mediapartners-google

Rule Path
Disallow /profeed/*
Disallow */fonts/*

perplexitybot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

meta-webindexer

Rule Path
Disallow /

halobot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

gumgumbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /