pewartanusantara.com
robots.txt

Robots Exclusion Standard data for pewartanusantara.com

Resource Scan

Scan Details

Site Domain pewartanusantara.com
Base Domain pewartanusantara.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-27T15:23:48+00:00
Next Scan 2025-11-26T15:23:48+00:00

Last Successful Scan

Scanned2025-07-30T14:16:53+00:00
URL https://pewartanusantara.com/robots.txt
Domain IPs 103.251.44.218
Response IP 103.251.44.218
Found Yes
Hash e1bd59d04a7f965824504613ec0cf25fe88894ade4d6218f59abc554f9806471
SimHash 492ece62ea9a

Groups

googlebot-news

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /tmp/
Disallow /cgi-bin/
Disallow /private/
Disallow /backup/
Disallow /?s=

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

Other Records

Field Value
sitemap pewartanusantara.com/sitemap-index.xml
sitemap pewartanusantara.com/sitemap-category.xml
sitemap pewartanusantara.com/sitemap-page.xml
sitemap pewartanusantara.com/sitemap-tag.xml
sitemap pewartanusantara.com/sitemap-topic.xml
sitemap pewartanusantara.com/sitemap.xml