aliensoup.com
robots.txt
Robots Exclusion Standard data for aliensoup.com
Resource Scan
Scan Details
Site Domain | aliensoup.com |
Base Domain | aliensoup.com |
Scan Status | Ok |
Last Scan | 2024-09-21T13:28:29+00:00 |
Next Scan | 2024-09-28T13:28:29+00:00 |
Last Scan
Scanned | 2024-09-21T13:28:29+00:00 |
URL | https://aliensoup.com/robots.txt |
Domain IPs | 104.21.55.43, 172.67.144.132, 2606:4700:3035::6815:372b, 2606:4700:3037::ac43:9084 |
Response IP | 172.67.144.132 |
Found | Yes |
Hash | d264b6c4ab07e7813bb8fc6cb3adf7010fcabc1f502344cca1bb3afaf4a0bb10 |
SimHash | 805663d2426a |
Groups
ahrefsbot
baidu
baiduspider
baiduspider-video
baiduspider-image
boardreader
boardtracker
bytespider
cliqzbot
diffbot
dotbot
easouspider
exabot
ichiro
linkdexbot
linkdexbot-mobile
magpie-crawler
meanpathbot
mj12bot
moget
naverbot
omgilibot
petalbot
proximic
rogerbot
sitebot
semrushbot
semrushbot-sa
spbot
trendictionbot
twiceler
urlappendbot
yeti
youdaobot
Rule | Path |
---|---|
Disallow | / |
*
Rule | Path |
---|---|
Disallow | /area51/ |
Disallow | /admin.php |
Disallow | /js/ |
Disallow | /vb3/ |
Disallow | /xftest/ |
Disallow | /search/ |
Disallow | /tmp/ |
Disallow | /temp/ |
Allow | / |
Other Records
Field | Value |
---|---|
sitemap | https://aliensoup.com/sitemap.php |
Comments