extrastaff.com
robots.txt

Robots Exclusion Standard data for extrastaff.com

Resource Scan

Scan Details

Site Domain extrastaff.com
Base Domain extrastaff.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-28T11:13:46+00:00
Next Scan 2024-09-29T11:13:46+00:00

Last Successful Scan

Scanned2024-08-29T11:09:35+00:00
URL https://extrastaff.com/robots.txt
Redirect https://www.extrastaff.com/robots.txt
Redirect Domain www.extrastaff.com
Redirect Base extrastaff.com
Domain IPs 94.237.53.143
Redirect IPs 94.237.53.143
Response IP 94.237.53.143
Found Yes
Hash 6948419559dce2598e2ab41c7f058bbdb0bbbe3691b3a9fea2b835daf04ce768
SimHash 4b16d1e04833

Groups

*

Rule Path
Disallow /administrator/
Allow /cache/
Disallow /cli/
Allow /components/
Allow /images/
Disallow /includes/
Disallow /installation/
Disallow /language/
Allow /libraries/
Disallow /logs/
Allow /media/
Allow /modules/
Allow /plugins/
Allow /templates/
Disallow /tmp/

rogerbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexcatalog

Rule Path
Disallow /

yandexfavicons

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

jakarta commons

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

powermarks

Rule Path
Disallow /

offbyone

Rule Path
Disallow /

nutraspace

Rule Path
Disallow /

nutch

Rule Path
Disallow /

jobcrawlerbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sightupbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

wise-guys

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

java

Rule Path
Disallow /

www-mechanize

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

spbot

Rule Path
Disallow /

docomo

Rule Path
Disallow /

pingdom.com_bot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

obot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

innosense

Rule Path
Disallow /

aghaven

Rule Path
Disallow /

jobrapido

Rule Path
Disallow /

mozilla/5.0 (compatible; jobrapido/1.1; +http://www.jobrapido.com)

Rule Path
Disallow /

mozilla/5.0 (jobrapido webpump)

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Warnings

  • 2 invalid lines.