amethstreams.com
robots.txt

Robots Exclusion Standard data for amethstreams.com

Resource Scan

Scan Details

Site Domain amethstreams.com
Base Domain amethstreams.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan5/17/2025, 1:24:21 PM
Next Scan 6/16/2025, 1:24:21 PM

Last Successful Scan

Scanned3/26/2025, 11:09:00 AM
URL https://amethstreams.com/robots.txt
Redirect https://methstreams.ac/robots.txt
Redirect Domain methstreams.ac
Redirect Base methstreams.ac
Domain IPs 104.21.78.195, 172.67.136.164, 2606:4700:3033::ac43:88a4, 2606:4700:3035::6815:4ec3
Redirect IPs 104.21.17.25, 172.67.219.157, 2606:4700:3033::ac43:db9d, 2606:4700:3035::6815:1119
Response IP 104.21.17.25
Found Yes
Hash 1dbee46fdfb997bc414f06ea5a980eaabe97731dd754e0ed15ec5cb45b1ff2b2
SimHash 09175ba547d1

Groups

googlebot-news

Rule Path
Allow /

snapchatbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

bingbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

yandexbot

Rule Path
Allow /

sharethis

Rule Path
Allow /

applebot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebot

Rule Path
Allow /

slurp

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

googleother

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googleother

Rule Path
Allow /

baiduspider

Rule Path
Allow /

konqueror

Rule Path
Allow /

exabot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

iframely

Rule Path
Allow /

groupmebot

Rule Path
Allow /

snapchat

Rule Path
Allow /

telegrambot

Rule Path
Allow /

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://methstreams.ac/sitemap_index.xml