crowdstaffing.com
robots.txt

Robots Exclusion Standard data for crowdstaffing.com

Resource Scan

Scan Details

Site Domain crowdstaffing.com
Base Domain crowdstaffing.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-05-28T09:57:45+00:00
Next Scan 2024-08-26T09:57:45+00:00

Last Successful Scan

Scanned2022-10-03T20:52:26+00:00
URL https://www.crowdstaffing.com/robots.txt
Response IP 199.60.103.31, 199.60.103.225
Found Yes
Hash 8d9319cfb1a867863dc3caecb72d870c406559d9c6862ef69e3618f54a8399c1
SimHash 4555d7c47790

Groups

*

Rule Path
Allow
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/

googlebot

Rule Path
Allow
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/

googlebot-image

Rule Path
Allow
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/

mediapartners-google

Rule Path
Allow
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/

adsbot-google

Rule Path
Allow
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/