m.spicejet.com
robots.txt

Robots Exclusion Standard data for m.spicejet.com

Resource Scan

Scan Details

Site Domain m.spicejet.com
Base Domain spicejet.com
Scan Status Ok
Last Scan2024-05-17T06:11:28+00:00
Next Scan 2024-06-16T06:11:28+00:00

Last Scan

Scanned2024-05-17T06:11:28+00:00
URL https://m.spicejet.com/robots.txt
Redirect https://www.spicejet.com/robots.txt
Redirect Domain www.spicejet.com
Redirect Base spicejet.com
Domain IPs 104.26.12.170, 104.26.13.170, 172.67.71.155, 2606:4700:20::681a:caa, 2606:4700:20::681a:daa, 2606:4700:20::ac43:479b
Redirect IPs 104.26.12.170, 104.26.13.170, 172.67.71.155, 2606:4700:20::681a:caa, 2606:4700:20::681a:daa, 2606:4700:20::ac43:479b
Response IP 172.67.71.155
Found Yes
Hash f9bece01c38061a8f92eb25bc23f2827a40befc98af496983c638ac559851b67
SimHash 013155114592

Groups

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

teoma

Rule Path
Disallow

gigabot

Rule Path
Disallow

robozilla

Rule Path
Disallow

nutch

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

baiduspider

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

*

Rule Path
Disallow
Disallow /cgi-bin/

Other Records

Field Value
sitemap https://www.spicejet.com/sitemap.xml

Warnings

  • 4 invalid lines.