milesweb.in
robots.txt

Robots Exclusion Standard data for milesweb.in

Resource Scan

Scan Details

Site Domain milesweb.in
Base Domain milesweb.in
Scan Status Ok
Last Scan2025-11-01T09:44:28+00:00
Next Scan 2025-12-01T09:44:28+00:00

Last Scan

Scanned2025-11-01T09:44:28+00:00
URL https://milesweb.in/robots.txt
Redirect https://www.milesweb.in/robots.txt
Redirect Domain www.milesweb.in
Redirect Base milesweb.in
Domain IPs 172.66.40.128, 172.66.43.128, 2606:4700:3108::ac42:2880, 2606:4700:3108::ac42:2b80
Redirect IPs 172.66.40.128, 172.66.43.128, 2606:4700:3108::ac42:2880, 2606:4700:3108::ac42:2b80
Response IP 172.66.40.128
Found Yes
Hash 1de2adf567e7ac02397419bd4532d282f44277439ef06d9c347380fd9e44b20d
SimHash 6f519f357541

Groups

*

Rule Path
Disallow /inventory/*
Disallow /admin/*
Disallow /acceptable-usage-policy.php
Disallow /acceptable-usage-policy
Disallow /knowledgebase.php
Disallow /knowledgebase
Disallow /announcements.php
Disallow /announcements
Disallow /serverstatus.php
Disallow /serverstatus
Disallow /downloads.php
Disallow /downloads
Disallow /clientarea.php
Disallow /submitticket.php
Disallow /submitticket
Disallow /cart.php
Disallow /cart
Disallow /aup.php
Disallow /aup
Disallow /pwreset.php
Disallow /pwreset
Disallow /cgi-bin/*
Disallow /chat/*
Disallow /disclaimer.php
Disallow /disclaimer
Disallow /cancellation-refund-policy.php
Disallow /cancellation-refund-policy
Disallow /shipping-delivery-policy.php
Disallow /shipping-delivery-policy
Disallow /domain-offer.php
Disallow /domain-offer
Disallow /domain-promo.php
Disallow /domain-promo
Disallow /MilesWeb-Corporate-Profile.pdf
Disallow /newsletter/*
Disallow /free-domain-offer.php
Disallow /free-domain-offer
Disallow /checkout/*
Disallow /hosting/cloud-hosting/sla.php
Disallow /reseller.php
Disallow /shared-hosting.php
Disallow /old_website/*
Disallow /hindi-demo/*
Disallow /demo-pages/*
Disallow /hosting-demo/*
Disallow /hosting-offers/*
Disallow /popup/*

googlebot

Rule Path
Allow /*.js*
Allow /*.css*

gptbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

google-extended

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.milesweb.in/sitemap.xml
sitemap https://www.milesweb.in/blog/post-sitemap.xml
sitemap https://www.milesweb.in/hosting-faqs/post-sitemap1.xml
sitemap https://www.milesweb.in/hosting-faqs/post-sitemap2.xml

Comments

  • Allow ChatGPT / OpenAI crawlers
  • (Optional) allow Google's AI extension if you want