webcala.net
robots.txt
Robots Exclusion Standard data for webcala.net
Resource Scan
Scan Details
| Site Domain | webcala.net |
| Base Domain | webcala.net |
| Scan Status | Failed |
| Failure Reason | Scan timed out. |
| Last Scan | 2025-06-11T21:22:42+00:00 |
| Next Scan | 2025-08-10T21:22:42+00:00 |
Last Successful Scan
| Scanned | 2025-04-13T21:21:57+00:00 |
| URL | https://webcala.net/robots.txt |
| Domain IPs | 109.94.209.214 |
| Response IP | 109.94.209.214 |
| Found | Yes |
| Hash | 7e2127e5069568d826986df7056a38d7943023188ab1ad4d57aac4d3d484c9b9 |
| SimHash | 521e4d476e60 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /404/ |
| Disallow | /adm/ |
| Disallow | /adv/ |
| Disallow | /bones/ |
| Disallow | /bootstrap/ |
| Disallow | /cgi-bin/ |
| Disallow | /code/ |
| Disallow | /Connections/ |
| Disallow | /css/ |
| Disallow | /geo/ |
| Disallow | /js/ |
| Disallow | /skycom/ |
| Disallow | /timer/ |
| Disallow | /eurofood/ |
| Disallow | /forum/ |
| Disallow | /resume/ |
| Disallow | /rulya/ |
| Disallow | /sess/ |
| Disallow | /statya/add-statya/ |
| Disallow | /system/ |
| Disallow | /tatyana/ |
| Disallow | /test/ |
| Disallow | /timeline/ |
| Disallow | /topeny/ |
| Disallow | /vendor/ |
| Disallow | /polzovatelskoye-soglasheniye/ |
| Disallow | *shablon* |
| Disallow | *test* |
ahrefsbot
apptusbot
blp_bbot
businessdbbot
ccbot
covarioids
phantom
seznambot
surveybot
converacrawler
curl/
discobot
download ninja
email exractor
ezooms
fdm 3.x
flaxcrawler
grapeshot
grabber
gslfbot
bubing
heritrix
webindex
httrack
intelium_bot
istellabot
java/
prodvigatorbot
searchmetricsbot
genieo
wesee
wesee:ads/pagebot
wesee:ads/picturebot
lemurwebcrawler
libwww-perl
metamojicrawler
mj12bot
yyspider
aboundexbot
nutch
xovibot
openacoon
php/
webzip
plukkie
proximic
python-urllib
ruby
semrushbot
skimbot
seokicks
spbot
panscient
turnitinbot
wbsearchbot
weblexbot
voltron
wget
wire/0.
zyborg
shopwiki
sentibot
megaindex.ru
xovibot
advbot
memorybot
smtbot
easouspider
domainappender
baiduspider
baiduspider-video
baiduspider-image
sogou web spider
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://webcala.net/sitemap.xml |
Warnings
- 1 invalid line.
- `clean-param` is not a known field.
- `host` is not a known field.