mba.com
robots.txt

Robots Exclusion Standard data for mba.com

Resource Scan

Scan Details

Site Domain mba.com
Base Domain mba.com
Scan Status Ok
Last Scan2024-10-05T06:42:00+00:00
Next Scan 2024-10-12T06:42:00+00:00

Last Scan

Scanned2024-10-05T06:42:00+00:00
URL https://mba.com/robots.txt
Redirect https://www.mba.com/robots.txt
Redirect Domain www.mba.com
Redirect Base mba.com
Domain IPs 107.154.102.70, 107.154.111.70
Redirect IPs 45.64.67.70
Response IP 45.64.67.70
Found Yes
Hash e799ba6b6a3acc88abd5f97cc679dcabc611bdd94488fc9e0bda4104cbdee442
SimHash 2a00bc740e92

Groups

rogerbot

Rule Path
Disallow /Search

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /sitecore
Disallow /Sitecore
Disallow /SITECORE
Disallow /sitecore_files/
Disallow /App_Browsers/
Disallow /App_config/
Disallow /App_Data/
Disallow /temp/
Disallow /upload/
Disallow /xsl/
Disallow /Search
Disallow /mba2/no-index

Other Records

Field Value
sitemap https://www.mba.com/sitemap.xml

Warnings

  • 1 invalid line.