womeninretail.com
robots.txt

Robots Exclusion Standard data for womeninretail.com

Resource Scan

Scan Details

Site Domain womeninretail.com
Base Domain womeninretail.com
Scan Status Ok
Last Scan2025-03-06T16:24:39+00:00
Next Scan 2025-04-05T16:24:39+00:00

Last Scan

Scanned2025-03-06T16:24:39+00:00
URL https://www.womeninretail.com/robots.txt
Domain IPs 104.207.254.133
Response IP 104.207.254.133
Found Yes
Hash 5c5761b9c2811884f4751fd1866fdf434924bba95274792ab7e5bb3fad301e9e
SimHash 4948004475f8

Groups

*

Rule Path
Disallow

*

Rule Path
Disallow /job-listings/

googlebot

Rule Path
Allow /job-listings/

bingbot

Rule Path
Allow /job-listings/

lyticsbot

Rule Path
Allow /job-listings/

dotbot

Rule Path
Allow /job-listings/

googlebot

Rule Path
Allow /job-listings/

bingbot

Rule Path
Allow /job-listings/

*

Rule Path
Disallow /job/*

googlebot

Rule Path
Allow /job/*

bingbot

Rule Path
Allow /job/*

lyticsbot

Rule Path
Allow /job/*

dotbot

Rule Path
Allow /job/*

Other Records

Field Value
sitemap https://www.womeninretail.com/sitemap_index.xml

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK