blackjobfindah.com
robots.txt

Robots Exclusion Standard data for blackjobfindah.com

Resource Scan

Scan Details

Site Domain blackjobfindah.com
Base Domain blackjobfindah.com
Scan Status Ok
Last Scan2024-10-25T07:12:38+00:00
Next Scan 2024-11-24T07:12:38+00:00

Last Scan

Scanned2024-10-25T07:12:38+00:00
URL https://blackjobfindah.com/robots.txt
Redirect http://www.blackjobfindah.com/robots.txt
Redirect Domain www.blackjobfindah.com
Redirect Base blackjobfindah.com
Domain IPs 8.40.7.35
Redirect IPs 8.40.7.35
Response IP 8.40.7.35
Found Yes
Hash ef8fefa6678bc633280758e81f23e8b871e41bc0f3526257a32ffafbaf6a006d
SimHash 5516dd732e03

Groups

*

Rule Path
Disallow /store/products
Disallow /store/basket.asp
Disallow /store/orders.asp
Disallow /post-jobs
Disallow /account.aspx
Disallow /applicants.asp
Disallow /applicants.aspx
Disallow /applicants_email_resumes.asp
Disallow /applicants_email_resumes.aspx
Disallow /applicants_history.asp
Disallow /applicants_history.aspx
Disallow /applicants_letters.asp
Disallow /applicants_letters.aspx
Disallow /applicants_notes.asp
Disallow /applicants_notes.aspx
Disallow /docviewer*
Disallow /webinar/register/
Disallow /employers/webinarregister
Disallow /job-network
Disallow /jobnetwork.asp
Disallow /jobnetwork.aspx
Disallow /jobs_manage.asp
Disallow /jobs_manage.aspx
Disallow /jobs_questionnaires.asp
Disallow /jobs_questionnaires.aspx
Disallow /legal.asp
Disallow /legal.aspx
Disallow /offline.asp
Disallow /offline.aspx
Disallow /offline.html
Disallow /privacy.asp
Disallow /privacy.aspx
Disallow /resumes.asp
Disallow /resumes.aspx
Disallow /resumes_folders.asp
Disallow /resumes_folders.aspx
Disallow /terms.asp
Disallow /terms.aspx
Disallow /terms
Disallow /contract-terms
Disallow /about/contractterms

anthropic-ai

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

discobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

jobrapido

Rule Path
Disallow /

linguee

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

mbcrawler/1.0

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

obot

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

oodlebot

Rule Path
Disallow /

openbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

similarpages

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

webalta

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

zibber

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

Warnings

  • 2 invalid lines.