Log In Sign Up

mtool.work
robots.txt

Robots Exclusion Standard data for mtool.work

Resource Scan

Scan Details

Site Domain mtool.work
Base Domain mtool.work
Scan Status Ok
Last Scan2024-11-17T20:46:25+00:00
Next Scan 2024-11-24T20:46:25+00:00

Last Scan

Scanned2024-11-17T20:46:25+00:00
URL https://mtool.work/robots.txt
Domain IPs 160.16.149.22
Response IP 160.16.149.22
Found Yes
Hash bf2b991d70e8c15fdd25f2f82f1f0e0c637941d86d21842189764c554d49888d
SimHash c22e5d54a6ba

Groups

*

Rule Path
Disallow /login/
Disallow /admin/
Disallow /private/
Disallow /sponsored/

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

baidumobaider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduimagespider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.5

psbot

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

teoma

Rule Path
Disallow /

ask jeeves

Rule Path
Disallow /

listran

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

daum

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

admantx-apacas

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

mappy

Rule Path
Disallow /

yak

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

pockey

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://mtool.work/sitemap-xml
Back to top

Warnings

  • 2 invalid lines.
Back to top