adamthompsonphoto.com
robots.txt

Robots Exclusion Standard data for adamthompsonphoto.com

Resource Scan

Scan Details

Site Domain adamthompsonphoto.com
Base Domain adamthompsonphoto.com
Scan Status Ok
Last Scan2025-10-12T02:44:51+00:00
Next Scan 2025-10-19T02:44:51+00:00

Last Scan

Scanned2025-10-12T02:44:51+00:00
URL https://adamthompsonphoto.com/robots.txt
Redirect https://www.adamthompsonphoto.com/robots.txt
Redirect Domain www.adamthompsonphoto.com
Redirect Base adamthompsonphoto.com
Domain IPs 104.26.2.158, 104.26.3.158, 172.67.74.244, 2606:4700:20::681a:29e, 2606:4700:20::681a:39e, 2606:4700:20::ac43:4af4
Redirect IPs 34.224.182.222, 52.202.39.6, 72.44.41.128
Response IP 52.202.39.6
Found Yes
Hash be2c96777d199890c06d1f8e8e339d1ab70bf71d517eba52a6afad6bd95483e3
SimHash 29b0c3d2d475

Groups

adsbot-google
alexabot
bingpreview
cloudflareprefetch
friendfeedbot
funnelback
google
google favicon
google-site-verification
google-sitemaps
googlebot
googlebot-image
googlebot-mobile
googlebot-news
googlebot-video
mediapartners-google
pingdom
pinterest
scoutjet
slurp
spinn3r
teoma
twitterbot
yandex
yandeximages
yandexvideoparser
yeti
archive.org_bot
baiduspider
bingbot
facebookexternalhit
gsa-crawler
houzzbot
ia_archiver
msnbot
rogerbot

Rule Path
Disallow /404
Disallow /access-denied
Disallow /admin
Disallow /api
Disallow /cart
Disallow /checkout
Disallow /date
Disallow /downloads
Disallow /go
Disallow /hack
Disallow /keyword
Disallow /order
Disallow /password
Disallow /popular
Disallow /search
Disallow /services/api/php
Disallow /services/api/rest
Disallow /services/api/xmlrpc
Disallow /test
Allow /api/developer
Allow /api/doc
Allow /api/v2

google
twitterbot
facebookexternalhit
houzzbot

Rule Path
Allow /date
Allow /keyword
Allow /popular

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.adamthompsonphoto.com/sitemap-index.xml

Comments

  • See https://secure.smugmug.com/help/contact if you'd like to apply to be allowlisted for crawling SmugMug.