rabotat.top
robots.txt
Robots Exclusion Standard data for rabotat.top
Resource Scan
Scan Details
| Site Domain | rabotat.top |
| Base Domain | rabotat.top |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-07-31T08:08:23+00:00 |
| Next Scan | 2024-10-29T08:08:23+00:00 |
Last Successful Scan
| Scanned | 2024-04-03T07:38:19+00:00 |
| URL | https://rabotat.top/robots.txt |
| Domain IPs | 104.21.65.241, 172.67.152.250, 2606:4700:3031::ac43:98fa, 2606:4700:3034::6815:41f1 |
| Response IP | 172.67.152.250 |
| Found | Yes |
| Hash | 252900425dfccce7f424fdb28f5ed833c9ffc2c0675852f51f37274bb7a1d64a |
| SimHash | 7226cc408a33 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /go.php?id= |
| Disallow | /?query= |
| Disallow | /user_login?back= |
| Disallow | /user_login/?back= |
| Disallow | *your_company%3D |
moget
ichiro
naverbot
yeti
baiduspider
baiduspider-video
baiduspider-image
youdaobot
seokicks-robot
mj12bot
ahrefsbot
solomonobot
rogerbot
blekkobot
sistrix
proximic
turnitinbot
psbot
gigabot
irlbot
twiceler
cazoodlebot
webinject
spbot
grapeshot
shopwiki
twiceler
cazoodlebot
webinject
irlbot
gigabot
turnitinbot
psbot
riddler
lcc
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://rabotat.top/sitemap.php |
Warnings
- `host` is not a known field.