media.newjobs.com
robots.txt

Robots Exclusion Standard data for media.newjobs.com

Resource Scan

Scan Details

Site Domain media.newjobs.com
Base Domain newjobs.com
Scan Status Ok
Last Scan2024-06-23T15:19:10+00:00
Next Scan 2024-07-23T15:19:10+00:00

Last Scan

Scanned2024-06-23T15:19:10+00:00
URL https://media.newjobs.com/robots.txt
Domain IPs 13.35.122.107, 13.35.122.62, 13.35.122.80, 13.35.122.98
Response IP 18.165.171.108
Found Yes
Hash 0ed08ba532728dcfe4b7ce32474089a7f5dcc599234ec7ce752f6c93fa4bd6bc
SimHash 084490505f71

Groups

twitterbot

Rule Path
Disallow *
Allow /cms
Allow /jeanne
Allow /niche

*

Rule Path
Disallow /
Disallow /marketing/2022/*