Log In Sign Up

seru.co.id
robots.txt

Robots Exclusion Standard data for seru.co.id

Resource Scan

Scan Details

Site Domain seru.co.id
Base Domain seru.co.id
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-03-30T08:35:44+00:00
Next Scan 2025-04-06T08:35:44+00:00

Last Successful Scan

Scanned2025-03-22T08:33:45+00:00
URL https://seru.co.id/robots.txt
Domain IPs 103.175.216.157
Response IP 103.175.216.157
Found Yes
Hash a820cdd0b18a672263ef97f298cbe0111d659b53f95981ae7f0a1a68a0a6b9bd
SimHash 4bea5140f933

Groups

googlebot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes
Disallow /wp-content
Disallow /page/
Allow /wp-admin/admin-ajax.php
Allow /wp-content/uploads/
Disallow /wp-content/plugins/
Disallow /readme.html
Disallow /wp-json/
Disallow /?rest_route=%2F
Disallow /*?comments
Disallow /wp-comments-post.php
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /go/
Disallow /tag/
Disallow /author/
Disallow /cdn-cgi/
Disallow /*/feed/
Disallow /*/comments/
Disallow /*?s=
Disallow /*?p=
Disallow /license.txt

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

xenuãƒâ¢ã¢â€šâ¬ã¢â€žâ¢s

Rule Path
Disallow /

xenuãƒâ¢ã¢â€šâ¬ã¢â€žâ¢s link sleuth 1.1c

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

archive.org bot

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

gigablast spider

Rule Path
Disallow /

picscout

Rule Path
Disallow /

blexbot crawler

Rule Path
Disallow /

tineye

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

ezooms robot

Rule Path
Disallow /

netestate ne crawler (+http://www.website-datenbank.de/)

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

turnitin robot

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

pi-monster

Rule Path
Disallow /

eccp/1.0 (search@eniro.com)

Rule Path
Disallow /

psbot

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /

zbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

simplepie

Rule Path
Disallow /

wget

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

quantify

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

cuam

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

openai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://seru.co.id/sitemap_index.xml
Back to top

Comments

  • Block NextGenSearchBot
  • Block ia-archiver from crawling site
  • Block archive.org_bot from crawling site
  • Block Archive.org Bot from crawling site
  • Block LinkWalker from crawling site
  • Block GigaBlast Spider from crawling site
  • Block PicScout Crawler from crawling site
  • Block BLEXBot Crawler from crawling site
  • Block TinEye from crawling site
  • Block SEOkicks
  • Block BlexBot
  • Block SISTRIX
  • Block Uptime robot
  • Block Ezooms Robot
  • Block netEstate NE Crawler (+http://www.website-datenbank.de/)
  • Block WiseGuys Robot
  • Block Turnitin Robot
  • Block Heritrix
  • Block pricepi
  • Block Eniro
  • Block Psbot
  • Block Youdao
  • BLEXBot
  • Block NaverBot
  • Block ZBot
  • Block Vagabondo
  • Block LinkWalker
  • Block SimplePie
  • Block Wget
  • Block Pixray-Seeker
  • Block BoardReader
  • Block Quantify
  • Block Plukkie
  • Block Cuam
  • https://megaindex.com/crawler
Back to top

Warnings

  • 2 invalid lines.
Back to top