metaeducationindia.com
robots.txt

Robots Exclusion Standard data for metaeducationindia.com

Resource Scan

Scan Details

Site Domain metaeducationindia.com
Base Domain metaeducationindia.com
Scan Status Ok
Last Scan2024-06-11T04:59:20+00:00
Next Scan 2024-07-11T04:59:20+00:00

Last Scan

Scanned2024-06-11T04:59:20+00:00
URL https://metaeducationindia.com/robots.txt
Domain IPs 2a02:4780:84:3879:1292:e81c:632d:3862, 91.108.100.17
Response IP 77.37.75.68
Found Yes
Hash 19ac6f57af89b2da8fb3e87dc99d9e84cc1ac6bebe53141efca83fd6658646fa
SimHash 03555c014501

Groups

*

Rule Path
Disallow /wp-admin

googlebot

Rule Path
Allow /

msnbot

Rule Path
Allow /

gigabot

Rule Path
Allow /

robozilla

Rule Path
Allow /

nutch

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yahoo-mmcrawler

Rule Path
Allow /

psbot

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /cgi/

*

Rule Path
Disallow /database/

*

Rule Path
Disallow /cpanel/

unnecessarybot

Rule Path
Disallow /
Disallow /cgi-bin/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.metaeducationindia.com/sitemap_index.xml