cs.jobs-search.org
robots.txt

Robots Exclusion Standard data for cs.jobs-search.org

Resource Scan

Scan Details

Site Domain cs.jobs-search.org
Base Domain jobs-search.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-07-01T06:58:55+00:00
Next Scan 2025-09-29T06:58:55+00:00

Last Successful Scan

Scanned2025-02-25T06:45:45+00:00
URL https://cs.jobs-search.org/robots.txt
Domain IPs 104.21.36.244, 172.67.201.98, 2606:4700:3030::6815:24f4, 2606:4700:3036::ac43:c962
Response IP 172.67.201.98
Found Yes
Hash 11a304d4783fa8dfe994d554f97ca321f69745fa0323b45fe51cd73cee5a2615
SimHash 49441450c781

Groups

mediapartners-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile-apps

Rule Path
Allow /

adidxbot

Rule Path
Allow /

*

Rule Path
Disallow /oc-admin/
Disallow /adpointer/
Disallow /display/
Disallow *pattern%2C*
Disallow *contact?v=*
Disallow *sShowAs*
Disallow *sOrder%2Ci_price*
Disallow *iPage*
Disallow *job/mark*
Disallow *job/send-friend*
Disallow *item-prev*
Disallow *item-next*
Disallow *page%3Dajax*
Disallow *languages*

bingbot

Rule Path
Disallow *sFeed%2Crss*

Other Records

Field Value
crawl-delay 30

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://cs.jobs-search.org/map-cit
sitemap https://cs.jobs-search.org/map-img

Comments

  • DaumWebMasterTool:88daf99681e9d147db9e51dafd87e4cdb2b924ebb2c56c3c50f1764ca013e812:elCazkMJjYjxKQyXLeIlvw==