Log In Sign Up

whatismyip.live
robots.txt

Robots Exclusion Standard data for whatismyip.live

Resource Scan

Scan Details

Site Domain whatismyip.live
Base Domain whatismyip.live
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-03-20T10:20:13+00:00
Next Scan 2024-06-18T10:20:13+00:00

Last Successful Scan

Scanned2023-08-22T21:06:34+00:00
URL https://whatismyip.live/robots.txt
Domain IPs 104.21.72.203, 172.67.187.81, 2606:4700:3031::6815:48cb, 2606:4700:3032::ac43:bb51
Response IP 172.67.187.81
Found Yes
Hash 00929cc264c9aeab8c9909117d7cf6beb7b0586316fdfbd46f35044015aa0959
SimHash 4a15c452d033

Groups

*

Rule Path
Allow /

ia_archiver

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

yandex

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://whatismyip.live/sitemap.xml
Back to top

Comments

  • All Others
  • A href
  • Moz
Back to top