Log In Sign Up

vexacademy.net
robots.txt

Robots Exclusion Standard data for vexacademy.net

Resource Scan

Scan Details

Site Domain vexacademy.net
Base Domain vexacademy.net
Scan Status Ok
Last Scan2025-06-22T23:06:10+00:00
Next Scan 2025-06-29T23:06:10+00:00

Last Scan

Scanned2025-06-22T23:06:10+00:00
URL https://vexacademy.net/robots.txt
Domain IPs 2a02:4780:15:f8e0:e992:8f28:7f42:f098, 2a02:4780:38:a958:a7c8:a6ff:b238:c44, 84.32.84.117, 84.32.84.227
Response IP 77.37.66.221
Found Yes
Hash ca2a6fa34e7f4013caee40ba561a56f896bf338c65330fedf34a0ddaa404e526
SimHash 221cca436691

Groups

*

Rule Path
Disallow
Disallow /Administrator/Admin/Messages/
Disallow /Administrator/
Disallow /vex/Account/
Disallow /Media/CourseVideos/
Disallow /Media/EmailAttachments/
Disallow /Media/ProfilePhoto/
Disallow /Media/Resources/
Disallow /Students_Profile_pictures/
Disallow /ChrisKeyPhanox/
Disallow /vex/Message/
Disallow /vex/PaymentOAuth/
Disallow /vex/Styles/
Disallow /vex/Sessions/
Disallow /vex/Logout/
Allow /Administrator/Admin/Messages/Media/Thumbnails/
Allow /Administrator/Admin/Messages/Media/Trailer/

badbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://vexacademy.net/sitemaps/urllist.txt
sitemap https://vexacademy.net/sitemaps/sitemap_4455418.xml
sitemap https://vexacademy.net/sitemaps/sitemap_4455418.html
sitemap https://vexacademy.net/sitemap.xml
Back to top

Comments

  • robots.txt for www.vexacademy.net
  • Allow all search engines access
  • Block specific folders (use relative paths from the site's root)
  • Allow full access to certain directories
  • Block specific bots
  • Set crawl-delay (useful for controlling how often bots access the site)
  • Sitemap location
Back to top