Log In Sign Up

lowongan-karir.com
robots.txt

Robots Exclusion Standard data for lowongan-karir.com

Resource Scan

Scan Details

Site Domain lowongan-karir.com
Base Domain lowongan-karir.com
Scan Status Ok
Last Scan2024-11-12T02:14:28+00:00
Next Scan 2024-12-12T02:14:28+00:00

Last Scan

Scanned2024-11-12T02:14:28+00:00
URL https://lowongan-karir.com/robots.txt
Domain IPs 51.161.91.172
Response IP 51.161.91.172
Found Yes
Hash 446af38c5b136ea1b5e6693141e22c9007c1d5ae31be59ca7e7f9e151f5a3f43
SimHash 62126f714392

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

majestic-12

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

bubing

Rule Path
Disallow /

slurp

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

obot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

james bot - webcrawler

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

enirobot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

ssearch

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yioopbot

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

stealthy siteopener

Rule Path
Disallow /

lipperhey spider

Rule Path
Disallow /

ranksonicbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

searchbot

Rule Path
Disallow /

safesearch microdata crawler

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

accoona-ai-agent

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

aphrospider

Rule Path
Disallow /

arachnode.net

Rule Path
Disallow /

backlink-check.de

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

browsershots

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

charlotte

Rule Path
Disallow /

cityreview

Rule Path
Disallow /

curious george

Rule Path
Disallow /

dataparksearch

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

docomo

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

dragonfly

Rule Path
Disallow /

dudley

Rule Path
Disallow /

ecxi

Rule Path
Disallow /

envolkits

Rule Path
Disallow /

expertsearchspider

Rule Path
Disallow /

factbot

Rule Path
Disallow /

fairshare

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

flipboardproxy

Rule Path
Disallow /

fyberspider

Rule Path
Disallow /

g2reader-bot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

glowscraper

Rule Path
Disallow /

gnam gnam spider

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

greenyogi

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

infohelfer

Rule Path
Disallow /

infohelfer

Rule Path
Disallow /

ingrid

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

iviacore

Rule Path
Disallow /

jam

Rule Path
Disallow /

justcrawl

Rule Path
Disallow /

kaspersky lab cfr link resolver

Rule Path
Disallow /

kmbot

Rule Path
Disallow /

kmccrew bot

Rule Path
Disallow /

kylinbot

Rule Path
Disallow /

larabot

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

livelapbot

Rule Path
Disallow /

loadtimebot

Rule Path
Disallow /

lycos_spider

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

meanpath bot

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

morning_paper

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

mxbot

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

netluchs

Rule Path
Disallow /

netseer

Rule Path
Disallow /

newsblur spider

Rule Path
Disallow /

nutanixclusterbot

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

openindexspider

Rule Path
Disallow /

pagesinventory

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

pebblecrawler

Rule Path
Disallow /

peepbot

Rule Path
Disallow /

pensacolahttp

Rule Path
Disallow /

phpcrawl

Rule Path
Disallow /

pollyapocalypse

Rule Path
Disallow /

postpost

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

sandcrawler

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

similarsitecheck

No rules defined. All paths allowed.

Back to top

Other Records

Field Value
sitemap https://lowongan-karir.com/sitemap.xml
sitemap https://lowongan-karir.com/sitemap.rss
Back to top

Warnings

  • 4 invalid lines.
Back to top