freelancejob.ru
robots.txt

Robots Exclusion Standard data for freelancejob.ru

Resource Scan

Scan Details

Site Domain freelancejob.ru
Base Domain freelancejob.ru
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-15T04:51:25+00:00
Next Scan 2024-11-13T04:51:25+00:00

Last Successful Scan

Scanned2024-04-18T04:49:45+00:00
URL https://freelancejob.ru/robots.txt
Domain IPs 188.127.243.142
Response IP 188.127.243.142
Found Yes
Hash 086719ebecf0e38805a9d5161ff72215916bb96e70499ca7a025846159604abd
SimHash c0498c440511

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /edit.php
Disallow /big_search.php

Other Records

Field Value
sitemap https://www.freelancejob.ru/sitemap.xml

Warnings

  • `host` is not a known field.