Log In Sign Up

andrewsmithart.com
robots.txt

Robots Exclusion Standard data for andrewsmithart.com

Resource Scan

Scan Details

Site Domain andrewsmithart.com
Base Domain andrewsmithart.com
Scan Status Ok
Last Scan2026-02-05T00:33:52+00:00
Next Scan 2026-03-07T00:33:52+00:00

Last Scan

Scanned2026-02-05T00:33:52+00:00
URL https://andrewsmithart.com/robots.txt
Redirect https://www.andrewsmithart.com/robots.txt
Redirect Domain www.andrewsmithart.com
Redirect Base andrewsmithart.com
Domain IPs 104.21.78.200, 172.67.136.230, 2606:4700:3032::6815:4ec8, 2606:4700:3035::ac43:88e6
Redirect IPs 104.21.78.200, 172.67.136.230, 2606:4700:3032::6815:4ec8, 2606:4700:3035::ac43:88e6
Response IP 172.67.136.230
Found Yes
Hash cde7b0ba01f5bbecb0c503c8edfa7067f17449bbd2027df93d21311255253b55
SimHash 55147bf5f591

Groups

googlebot

Rule Path
Disallow

adsense

Rule Path
Disallow

inspection

Rule Path
Disallow

telegrambot

Rule Path
Disallow

yandexbot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

googlebot-video

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

bingbot

Rule Path
Disallow

adidxbot

Rule Path
Disallow

slurp

Rule Path
Disallow

duckduckbot

Rule Path
Disallow

baiduspider

Rule Path
Disallow

yandex

Rule Path
Disallow

sogou

Rule Path
Disallow

exabot

Rule Path
Disallow

facebot

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

applebot

Rule Path
Disallow

petalbot

Rule Path
Disallow

linkedinbot

Rule Path
Disallow

amazonbot

Rule Path
Disallow

amazonbot-image

Rule Path
Disallow

gptbot

Rule Path
Disallow

chatgpt-user

Rule Path
Disallow

ccbot

Rule Path
Disallow

claudebot

Rule Path
Disallow

claude-web

Rule Path
Disallow

perplexitybot

Rule Path
Disallow

oai-searchbot

Rule Path
Disallow

oai-search

Rule Path
Disallow

magpie-crawler

Rule Path
Disallow

cohere-ai

Rule Path
Disallow

youbot

Rule Path
Disallow

anthropic-ai

Rule Path
Disallow

bytedancespider

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow

semrushbot

Rule Path
Disallow

mj12bot

Rule Path
Disallow

dataforseobot

Rule Path
Disallow

diffbot

Rule Path
Disallow

scrapy

Rule Path
Disallow

serpstatbot

Rule Path
Disallow

dotbot

Rule Path
Disallow

archive.org_bot

Rule Path
Disallow

wayback-machine

Rule Path
Disallow

commoncrawl

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://www.andrewsmithart.com/sitemap.xml
Back to top

Comments

  • ======================================
  • Robots.txt - Allow All Listed Crawlers
  • ======================================
  • ======================================
  • END
  • ======================================
Back to top