thesisdefense.org
robots.txt

Robots Exclusion Standard data for thesisdefense.org

Resource Scan

Scan Details

Site Domain thesisdefense.org
Base Domain thesisdefense.org
Scan Status Ok
Last Scan2024-05-09T07:07:56+00:00
Next Scan 2024-06-08T07:07:56+00:00

Last Scan

Scanned2024-05-09T07:07:56+00:00
URL https://thesisdefense.org/robots.txt
Domain IPs 104.21.35.150, 172.67.176.252, 2606:4700:3031::6815:2396, 2606:4700:3036::ac43:b0fc
Response IP 172.67.176.252
Found Yes
Hash 9e933f61f029e80aa463db1508d15b9456bfbe5e657a2fecd98f1011d7ee6ec9
SimHash 003cc9eb6baa

Groups

*

Rule Path
Disallow /boy
Disallow /retired
Disallow /bench
Disallow /bomb/couple/face
Disallow /grand/physics/slap

ahrefsbot

Rule Path
Disallow /

ubersuggestbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mbcrawler/1.0.

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

se ranking gentle bot

Rule Path
Disallow /