sundirect.in
robots.txt

Robots Exclusion Standard data for sundirect.in

Resource Scan

Scan Details

Site Domain sundirect.in
Base Domain sundirect.in
Scan Status Ok
Last Scan2025-02-27T10:18:08+00:00
Next Scan 2025-03-29T10:18:08+00:00

Last Scan

Scanned2025-02-27T10:18:08+00:00
URL https://sundirect.in/robots.txt
Redirect https://www.sundirect.in/robots.txt
Redirect Domain www.sundirect.in
Redirect Base sundirect.in
Domain IPs 103.13.178.148
Redirect IPs 103.13.178.148
Response IP 103.13.178.148
Found Yes
Hash 1cbf59dc75e35d550981905e2d0e010f314e0682038dcde8afdacf31f95cb992
SimHash 253d5bd7d401

Groups

*

Rule Path
Disallow /admin
Disallow oearel.sundirect.in
Disallow my.sundirect.in
Disallow rbuzz.sundirect.in
Disallow flexi.sundirect.in
Disallow reports.sundirect.in
Disallow gsb.sundirect.in
Disallow retailer.sundirect.in
Disallow po.sundirect.in
Disallow flexi.sundirect.in
Disallow flexi.sundirect.in
Disallow oea.sundirect.in
Disallow cs.sundirect.in
Disallow partners.sundirect.in
Disallow mapi.sundirect.in
Disallow api.sundirect.in
Disallow /admin/
Disallow /cart/
Disallow /checkout/
Disallow /user-profile/
Disallow /search/
Disallow /api/
Disallow /cgi-bin/
Disallow /private/

mj12bot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

spbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /*.pdf$

Other Records

Field Value
sitemap https://www.sundirect.in/sitemap.xml

Comments

  • Block specific bots
  • Disallow all bots from crawling PDF files
  • Allow all bots to crawl the sitemap