Log In Sign Up

restructuredtext.net
robots.txt

Robots Exclusion Standard data for restructuredtext.net

Resource Scan

Scan Details

Site Domain restructuredtext.net
Base Domain restructuredtext.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-22T16:07:23+00:00
Next Scan 2025-10-06T16:07:23+00:00

Last Successful Scan

Scanned2025-09-07T15:30:47+00:00
URL https://restructuredtext.net/robots.txt
Redirect https://www.restructuredtext.net/robots.txt
Redirect Domain www.restructuredtext.net
Redirect Base restructuredtext.net
Domain IPs 104.26.4.11, 104.26.5.11, 172.67.71.219, 2606:4700:20::681a:40b, 2606:4700:20::681a:50b, 2606:4700:20::ac43:47db
Redirect IPs 104.26.4.11, 104.26.5.11, 172.67.71.219, 2606:4700:20::681a:40b, 2606:4700:20::681a:50b, 2606:4700:20::ac43:47db
Response IP 104.26.5.11
Found Yes
Hash 0f05b7b7a59282a08b9adc6bba140d4efedbd5497c9c29100ff4e6811e332ee4
SimHash aa0d2dad6440

Groups

semrushbot

Rule Path
Disallow /

blackwidow

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://restructuredtext.mystrikingly.com/sitemap.xml
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /
Back to top