blog.seamusc.am
robots.txt

Robots Exclusion Standard data for blog.seamusc.am

Resource Scan

Scan Details

Site Domain blog.seamusc.am
Base Domain seamusc.am
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-10-06T09:57:21+00:00
Next Scan 2026-01-04T09:57:21+00:00

Last Successful Scan

Scanned2023-09-11T09:40:11+00:00
URL https://blog.seamusc.am/robots.txt
Domain IPs 74.114.154.18, 74.114.154.22
Response IP 74.114.154.18
Found Yes
Hash 92c474e01ad8d0722375a7f8600b78e8f85670db1563689b82176619cef69e67
SimHash 6b26d865cd23

Groups

*

Rule Path
Disallow /random
Disallow /day
Disallow /sticky-ad-iframe.html
Disallow /privacy/consent

Other Records

Field Value
crawl-delay 1

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://blog.seamusc.am/sitemap.xml

Comments

  • OpenAI's crawler
  • Common Crawl's crawler