Log In Sign Up

mobypicture.com
robots.txt

Robots Exclusion Standard data for mobypicture.com

Resource Scan

Scan Details

Site Domain mobypicture.com
Base Domain mobypicture.com
Scan Status Ok
Last Scan2024-06-13T21:40:30+00:00
Next Scan 2024-07-13T21:40:30+00:00

Last Scan

Scanned2024-06-13T21:40:30+00:00
URL https://www.mobypicture.com/robots.txt
Domain IPs 3.120.47.192
Response IP 3.120.47.192
Found Yes
Hash 66a1066a794c9f29562e8850da0054451bcb58e68c3a3112be8312408caa89f2
SimHash 8c5dde3012d3

Groups

googlebot

Rule Path
Disallow /*?
Disallow /*/with_friends

slurp

Rule Path
Disallow /*?
Disallow /*/with_friends

msnbot

Rule Path
Disallow /*?
Disallow /*/with_friends

uberbot

Rule Path
Disallow /

uberbot 1.0

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

topsy butterfly

Rule Path
Disallow /

topsy

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baidu

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /
Back to top

Comments

  • Google Search Engine Robot
  • Crawl-delay: 10 -- Googlebot ignores crawl-delay ftl
  • Yahoo! Search Engine Robot
  • Microsoft Search Engine Robot
Back to top