easycrm.me
robots.txt

Robots Exclusion Standard data for easycrm.me

Resource Scan

Scan Details

Site Domain easycrm.me
Base Domain easycrm.me
Scan Status Ok
Last Scan2024-09-09T13:07:32+00:00
Next Scan 2024-10-09T13:07:32+00:00

Last Scan

Scanned2024-09-09T13:07:32+00:00
URL https://easycrm.me/robots.txt
Domain IPs 35.214.108.70
Response IP 35.214.108.70
Found Yes
Hash a487c9fb70530b69dd08f008180104ba4ae9ec4b91a57b48e584f2925d858fbf
SimHash cd55f033b6f0

Groups

*

Rule Path
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /trackback
Disallow /feed
Disallow /comments
Disallow /category/*/*
Disallow */trackback
Disallow */feed
Disallow */feed/
Disallow */comments
Disallow /*.pdf$
Disallow /*?*
Disallow /*?
Disallow /?*
Disallow /?elementor_library=*
Disallow /wp-login.php
Disallow /?s=*
Disallow /?C=*
Disallow /search/*
Allow /wp-content/uploads

googlebot

Rule Path
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$
Disallow /*.pdf$
Disallow
Allow /*

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://easycrm.me/post-sitemap.xml
sitemap https://easycrm.me/page-sitemap.xml
sitemap https://easycrm.me/e-landing-page-sitemap.xml
sitemap https://easycrm.me/category-sitemap.xml
sitemap https://easycrm.me/video-sitemap.xml
sitemap https://easycrm.me/local-sitemap.xml

Comments

  • Global rules
  • -----------------
  • experimenting with blocking search results to prevent search result spam
  • Autoriser Google Image
  • Sitemap
  • -----------------
  • Ban bots that don't benefit us.
  • --------------------------------