Log In Sign Up

pilotcareercenter.com
robots.txt

Robots Exclusion Standard data for pilotcareercenter.com

Resource Scan

Scan Details

Site Domain pilotcareercenter.com
Base Domain pilotcareercenter.com
Scan Status Ok
Last Scan2024-09-04T19:46:24+00:00
Next Scan 2024-10-04T19:46:24+00:00

Last Scan

Scanned2024-09-04T19:46:24+00:00
URL https://pilotcareercenter.com/robots.txt
Domain IPs 199.233.255.67
Response IP 199.233.255.67
Found Yes
Hash de3fa0308f16d8beca9c1144b76f661f272c7c927decbc5e05a3e28435b8dc3e
SimHash d11641428db0

Groups

imagesiftbot

Rule Path
Disallow /

repolookoutbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ezoicbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

daum

Rule Path
Disallow /

obot

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

webdatastats

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

bubing

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

electricmonk

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

spbot

Rule Path
Disallow /

googlebot-video/1.0

Rule Path
Disallow /

googlebot-image/1.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://pilotcareercenter.com/sitemap.xml
Back to top

Warnings

  • 4 invalid lines.
Back to top