mcleanco.com
robots.txt

Robots Exclusion Standard data for mcleanco.com

Resource Scan

Scan Details

Site Domain mcleanco.com
Base Domain mcleanco.com
Scan Status Ok
Last Scan2024-09-30T18:53:31+00:00
Next Scan 2024-10-30T18:53:31+00:00

Last Scan

Scanned2024-09-30T18:53:31+00:00
URL https://mcleanco.com/robots.txt
Redirect https://hr.mcleanco.com:443/robots.txt
Redirect Domain hr.mcleanco.com
Redirect Base mcleanco.com
Domain IPs 52.9.32.178, 54.241.197.195
Redirect IPs 34.243.122.175, 52.19.238.88
Response IP 34.243.122.175
Found Yes
Hash 25730d57a697dddc916e5de07e800b96e32da16cf363046ab63d204f30914a30
SimHash 621c8964c383

Groups

*

Rule Path
Disallow /xml/
Disallow /search/
Disallow /research/search/
Disallow /auth/*
Disallow /sso/*
Disallow /tags/*

femtosearchbot

Rule Path
Disallow *

diffbot

Rule Path
Disallow *

mj12bot

Rule Path
Disallow *

ahrefsbot

Rule Path
Disallow *

gptbot

Rule Path
Disallow /research/*

google-extended

Rule Path
Disallow /

bingbot

Rule Path
Disallow /software-reviews/categories/*/async_quadrant_load?rel=nofollow
Disallow /software-reviews/categories/*/async_diamond_load?rel=nofollow
Disallow /software-reviews/categories/*/async_offerings_load?rel=nofollow

Other Records

Field Value
sitemap https://hr.mcleanco.com/hr_mcleanco_sitemap.xml.gz