Log In Sign Up

sunlife.qa
robots.txt

Robots Exclusion Standard data for sunlife.qa

Resource Scan

Scan Details

Site Domain sunlife.qa
Base Domain sunlife.qa
Scan Status Ok
Last Scan2024-10-07T17:18:03+00:00
Next Scan 2024-11-06T17:18:03+00:00

Last Scan

Scanned2024-10-07T17:18:03+00:00
URL https://sunlife.qa/robots.txt
Domain IPs 107.154.147.223, 45.60.97.223
Response IP 45.60.97.223
Found Yes
Hash 4bbdff32198558f485a59a3d252f879535b7e5e874f19903d155f1b722a8d384
SimHash 525e4140a0a2

Groups

*

Rule Path
Disallow

facebookexternalhit/1.1

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

nbot/2.0

Rule Path
Disallow /

nbot

Rule Path
Disallow /

nbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

adsbot-amazon

Rule Path
Disallow /

amazonadbot

Rule Path
Disallow /

gptbot/1.2

Rule Path
Disallow /

claudebot/1.0

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bot\*

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

youbot

Rule Path
Disallow /
Back to top

Warnings

  • 2 invalid lines.
Back to top