Log In Sign Up

street-academy.com
robots.txt

Robots Exclusion Standard data for street-academy.com

Resource Scan

Scan Details

Site Domain street-academy.com
Base Domain street-academy.com
Scan Status Ok
Last Scan2024-09-25T02:28:36+00:00
Next Scan 2024-10-09T02:28:36+00:00

Last Scan

Scanned2024-09-25T02:28:36+00:00
URL https://street-academy.com/robots.txt
Redirect https://www.street-academy.com/robots.txt
Redirect Domain www.street-academy.com
Redirect Base street-academy.com
Domain IPs 18.178.85.191, 3.114.156.253
Redirect IPs 18.178.85.191, 3.114.156.253
Response IP 18.178.85.191
Found Yes
Hash dc3eb46c3904864e5ddc7ffa6ec9e362c31bf7e44395165c93629ce96c9d418a
SimHash 321c0900c460

Groups

*

Rule Path
Disallow /api/
Disallow /admin
Disallow /auth/*
Disallow /user_root
Disallow /d/users/*
Allow /d/users/sign_in
Disallow /conversations
Disallow /steachers/sales_detail*

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

pcore-http

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

yandex

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

awariobot
awariorssbot
awariosmartbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.street-academy.com/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /
Back to top