Log In Sign Up

pilotcareercenter.com
robots.txt

Robots Exclusion Standard data for pilotcareercenter.com

Resource Scan

Scan Details

Site Domain pilotcareercenter.com
Base Domain pilotcareercenter.com
Scan Status Ok
Last Scan2024-11-04T02:32:33+00:00
Next Scan 2024-12-04T02:32:33+00:00

Last Scan

Scanned2024-11-04T02:32:33+00:00
URL https://pilotcareercenter.com/robots.txt
Domain IPs 199.233.255.67
Response IP 199.233.255.67
Found Yes
Hash b14a3eda11e863c3f98714a723707f62ae9eb50a4ed0ab7535a29a294741ea4a
SimHash d11641429dd0

Groups

orbbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

repolookoutbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ezoicbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

daum

Rule Path
Disallow /

obot

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

webdatastats

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

bubing

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

electricmonk

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

spbot

Rule Path
Disallow /

googlebot-video/1.0

Rule Path
Disallow /

googlebot-image/1.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://pilotcareercenter.com/sitemap.xml
Back to top

Warnings

  • 4 invalid lines.
Back to top