2dehands.gratis
robots.txt

Robots Exclusion Standard data for 2dehands.gratis

Resource Scan

Scan Details

Site Domain 2dehands.gratis
Base Domain 2dehands.gratis
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-02-14T11:06:00+00:00
Next Scan 2025-05-15T11:06:00+00:00

Last Successful Scan

Scanned2024-01-22T08:16:26+00:00
URL https://2dehands.gratis/robots.txt
Domain IPs 104.21.40.176, 172.67.155.192, 2606:4700:3031::6815:28b0, 2606:4700:3037::ac43:9bc0
Response IP 172.67.155.192
Found Yes
Hash 6b52934943dce189f56b4155620b0f3a6e40d2e3bcd0d98ed86950a6a8dc87fb
SimHash 290769f4c761

Groups

*

Rule Path
Disallow /cache/
Disallow /config/
Disallow /library/
Disallow /module/
Disallow /SQL/
Disallow /cron.php
Disallow /install.php
Disallow /error_log
Disallow /*?SID=
Disallow /admin/
Disallow /administration/
Disallow /install/
Disallow /rss/
Disallow /advert/
Disallow /play-video/
Disallow /app/async*
Disallow /members-area/
Disallow /members/
Disallow /sign-in-modal/
Disallow /sign-up-modal/
Disallow /forgot-username/
Disallow /forgot-password/
Disallow /bid/
Disallow /buy/
Disallow /offer/
Disallow /cart/
Disallow /listings/cart/
Disallow /advanced-search/
Disallow /sell/

Comments

  • Crawl-delay: 10
  • Sitemap: %SITE_PATH%/sitemap.xml
  • Do not crawl application folders
  • Do not crawl root files
  • Do not crawl links with session ids
  • Do not crawl the admin module
  • Do not crawl the install module
  • Do not crawl selected app module pages
  • - rss pages
  • - advert related pages
  • - play video action
  • async controller
  • Do not crawl selected members module pages
  • Do not crawl selected listings module pages
  • - bid/purchase/cart related pages
  • - search pages
  • - sell pages
  • Uncomment if you do not wish to allow Google and Bing to index your images
  • User-agent: Googlebot-Image
  • Disallow: /
  • User-agent: msnbot-media
  • Disallow: /
  • Disallow: /*.php$