Log In Sign Up

finder.com
robots.txt

Robots Exclusion Standard data for finder.com

Resource Scan

Scan Details

Site Domain finder.com
Base Domain finder.com
Scan Status Ok
Last Scan2024-05-22T18:10:05+00:00
Next Scan 2024-05-29T18:10:05+00:00

Last Scan

Scanned2024-05-22T18:10:05+00:00
URL https://finder.com/robots.txt
Redirect https://www.finder.com/robots.txt
Redirect Domain www.finder.com
Redirect Base finder.com
Domain IPs 13.224.163.116, 13.224.163.29, 13.224.163.68, 13.224.163.73
Redirect IPs 52.84.229.12, 52.84.229.128, 52.84.229.19, 52.84.229.53
Response IP 52.84.229.19
Found Yes
Hash 03b01dcbc309d4aa65823d4e9bc369d31c10c5e96e31b1a8d5208c6fe7b0ed55
SimHash bb15a231efd2

Groups

*

Rule Path
Disallow */wp-admin*
Disallow /wordpress/wp-admin/admin-ajax.php?action=get_geoip_country_modal_data_ajax
Disallow */wp-includes/*
Disallow */login
Disallow /redirect/*
Disallow */wp-login.php
Disallow /*?link
Disallow /search
Disallow /*?coupon
Disallow /*?pID
Disallow /*?pid
Disallow /*?link
Disallow /wp-api/1/user/me/notifications*
Disallow */feed/audit
Disallow /user/*
Disallow /search/*
Disallow *?search=*
Disallow /*/search/page/*
Disallow *?s=*
Allow /*?ver
Back to top

Other Records

Field Value
sitemap https://www.finder.com/sitemap_index.xml
sitemap https://www.finder.com/uk/sitemap_index.xml
sitemap https://www.finder.com/ca/sitemap_index.xml
Back to top

Comments

  • Prevent using redirect feature.
  • https://finder.atlassian.net/browse/ENG-481
  • Prevent crawling searches.
  • https://finder.atlassian.net/browse/CWS-452
  • Prevent crawling search url parameters
  • https://finder.atlassian.net/browse/CON-229
  • Prevent crawling additional searches.
  • https://finder.atlassian.net/browse/CWS-497
  • Prevent spam requests
  • https://finder.atlassian.net/browse/CC-1195
Back to top