indexmundi.com
robots.txt

Robots Exclusion Standard data for indexmundi.com

Resource Scan

Scan Details

Site Domain indexmundi.com
Base Domain indexmundi.com
Scan Status Ok
Last Scan2024-06-22T19:02:05+00:00
Next Scan 2024-06-29T19:02:05+00:00

Last Scan

Scanned2024-06-22T19:02:05+00:00
URL https://indexmundi.com/robots.txt
Redirect https://www.indexmundi.com/robots.txt
Redirect Domain www.indexmundi.com
Redirect Base indexmundi.com
Domain IPs 52.147.199.164
Redirect IPs 52.147.199.164
Response IP 52.147.199.164
Found Yes
Hash 6ba3652dffe2bc9f996951f9a93ed0c099207ddeafcf132ef2ae6b981d2e6517
SimHash c91d18716c13

Groups

*

Rule Path
Disallow /api/v2/

compatible; attributor

Rule Path
Disallow /

attributor

Rule Path
Disallow /

kalooga

Rule Path
Disallow /

mp3bot

Rule Path
Disallow /

webalta crawler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

gbplugin

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

trendkite-akashic-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.indexmundi.com/sitemap.xml