Log In Sign Up

grafphoto.com
robots.txt

Robots Exclusion Standard data for grafphoto.com

Resource Scan

Scan Details

Site Domain grafphoto.com
Base Domain grafphoto.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-14T15:18:54+00:00
Next Scan 2024-12-13T15:18:54+00:00

Last Successful Scan

Scanned2023-05-18T21:39:30+00:00
URL https://grafphoto.com/robots.txt
Domain IPs 104.21.44.44, 172.67.194.209, 2606:4700:3034::ac43:c2d1, 2606:4700:3036::6815:2c2c
Response IP 172.67.194.209
Found Yes
Hash 1259d39dcfc7b171d65d9e2ccff10e44803e7325aa25adbb8465bdb137f95bd0
SimHash 3a0c5105c1a0

Groups

ia_archiver

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

ecairn-grabber

Rule Path
Disallow /

changedetection

Rule Path
Disallow /

rssbandit

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

bingbot

Rule Path
Disallow /wordpress/
Disallow /gallery/

superfeedr

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

kraken

Rule Path
Disallow /

superfeedr bot

Rule Path
Disallow /

fever

Rule Path
Disallow /

trapitagent

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.grafphoto.com/sitemap_index.xml
Back to top

Comments

  • Block YandexBot
  • Block Baidu
  • Block SoGou
  • Block Youdao
  • Block MegaIndex.ru
  • Block MegaIndex.ru
  • Block eCairn-Grabber
  • Block ChangeDetection 20160527
  • Block RssBandit 20160527
  • Block AhrefsBot
  • Block Petal Search 20210225
  • Wait between successive requests
Back to top