Log In Sign Up

ntioman.com
robots.txt

Robots Exclusion Standard data for ntioman.com

Resource Scan

Scan Details

Site Domain ntioman.com
Base Domain ntioman.com
Scan Status Ok
Last Scan2025-11-26T18:45:40+00:00
Next Scan 2025-12-26T18:45:40+00:00

Last Scan

Scanned2025-11-26T18:45:40+00:00
URL https://ntioman.com/robots.txt
Redirect https://www.ntioman.com/robots.txt
Redirect Domain www.ntioman.com
Redirect Base ntioman.com
Domain IPs 173.254.88.118
Redirect IPs 173.254.88.118
Response IP 173.254.88.118
Found Yes
Hash 9edb852d885a397093a0073a9770a7a248039e82863113023d3f069d672fc2fe
SimHash 25bf3d71e3e6

Groups

*

Rule Path
Disallow /ajax-load/
Disallow /assets/
Disallow /color-switcher/
Disallow /css/
Disallow /downloads/
Disallow /errors/
Disallow /fonts/
Disallow /forms/
Disallow /images/
Disallow /includes/
Disallow /js/
Disallow /staff/
Disallow /customer/
Disallow /index.html

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://www.ntioman.com/sitemap.xml
Back to top

Comments

  • robots.txt with path relative to document root
  • allows all user agent to crawl
  • if below line is uncommented, it will request all agents not crawl at all
  • Disallow: /
  • delay added to reduce load on website due to search engine crawling activities
  • disallow specific folders under public/
  • sitemap of the website
Back to top