Log In Sign Up

jdmbuysell.com
robots.txt

Robots Exclusion Standard data for jdmbuysell.com

Resource Scan

Scan Details

Site Domain jdmbuysell.com
Base Domain jdmbuysell.com
Scan Status Ok
Last Scan2024-11-13T16:33:14+00:00
Next Scan 2024-11-20T16:33:14+00:00

Last Scan

Scanned2024-11-13T16:33:14+00:00
URL https://jdmbuysell.com/robots.txt
Domain IPs 162.159.136.54, 162.159.137.54, 2606:4700:7::a29f:8836, 2606:4700:7::a29f:8936
Response IP 162.159.137.54
Found Yes
Hash 2d8269b0ad7f229d56416d21abd833d040f96f40770744f4bc5e2b42833c1022
SimHash 740178c10d12

Groups

*

Rule Path
Allow *.js
Allow *.css
Allow /wp-admin/admin-ajax.php
Disallow /?_search=*
Disallow /wp-admin/
Disallow /feed/
Disallow /embed/
Disallow /wp-login.php
Disallow /?post_type=*
Back to top

Other Records

Field Value
sitemap https://www.jdmbuysell.com/sitemap_index.xml
Back to top

Comments

  • Allow files critical for rendering
  • Allow AJAX - Do Not Remove
  • Prevent crawl-budget waste and indexing on search pages
  • Prevent private admin areas from being crawled and indexed
  • Prevent duplicate /feed/ pages from being crawled
  • Prevent login page crawls etc
  • Prevent crawling post type pages.
  • Add sitemap
Back to top