Log In Sign Up

pes.neoseeker.com
robots.txt

Robots Exclusion Standard data for pes.neoseeker.com

Resource Scan

Scan Details

Site Domain pes.neoseeker.com
Base Domain neoseeker.com
Scan Status Ok
Last Scan2025-07-16T08:06:03+00:00
Next Scan 2025-08-15T08:06:03+00:00

Last Scan

Scanned2025-07-16T08:06:03+00:00
URL https://pes.neoseeker.com/robots.txt
Domain IPs 104.26.4.80, 104.26.5.80, 172.67.69.121, 2606:4700:20::681a:450, 2606:4700:20::681a:550, 2606:4700:20::ac43:4579
Response IP 104.26.5.80
Found Yes
Hash 9898e93bb53f60425dd565826dc28bec701dbf853a602d5c0388fab81eabdd97
SimHash 711cc100c580

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Allow /w/load.php?
Disallow /w/
Disallow /wiki/Image%3A
Disallow /wiki/File%3A
Disallow /w/skins/

ccbot

Rule Path
Disallow /

rogerbot
dotbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

semrushbot
semrushbot-sa

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

quetextbot
bspider
megaindex
exabot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

terrykyleseoagency.com

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

youbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://pes.neoseeker.com/sitemap.xml.php
Back to top

Comments

  • Moz
  • Search Engines
  • brandwatch
  • Majestic
  • trendiction.com
  • Naver (KR)
  • SoGou (CN)
  • Youdao (CN)
  • SEMRush
  • Other misc bots:
  • AI bots
  • Search Engines/AI
Back to top