afremovart.com
robots.txt

Robots Exclusion Standard data for afremovart.com

Resource Scan

Scan Details

Site Domain afremovart.com
Base Domain afremovart.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-04-02T02:40:53+00:00
Next Scan 2025-05-02T02:40:53+00:00

Last Successful Scan

Scanned2025-02-09T02:34:19+00:00
URL https://afremovart.com/robots.txt
Domain IPs 116.203.225.75
Response IP 116.203.225.75
Found Yes
Hash 1b5bc744b6c465ac3c57f742894197f0af17566f7ddfb18b2482ed21a7d9b67f
SimHash 290769f4c761

Groups

*

Rule Path
Disallow /cache/
Disallow /config/
Disallow /library/
Disallow /module/
Disallow /SQL/
Disallow /cron.php
Disallow /install.php
Disallow /error_log
Disallow /*?SID=
Disallow /admin/
Disallow /administration/
Disallow /install/
Disallow /rss/
Disallow /advert/
Disallow /play-video/
Disallow /app/async*
Disallow /members-area/
Disallow /members/
Disallow /sign-in-modal/
Disallow /sign-up-modal/
Disallow /forgot-username/
Disallow /forgot-password/
Disallow /bid/
Disallow /buy/
Disallow /offer/
Disallow /cart/
Disallow /listings/cart/
Disallow /advanced-search/
Disallow /sell/

Comments

  • Crawl-delay: 10
  • Sitemap: %SITE_PATH%/sitemap.xml
  • Do not crawl application folders
  • Do not crawl root files
  • Do not crawl links with session ids
  • Do not crawl the admin module
  • Do not crawl the install module
  • Do not crawl selected app module pages
  • - rss pages
  • - advert related pages
  • - play video action
  • async controller
  • Do not crawl selected members module pages
  • Do not crawl selected listings module pages
  • - bid/purchase/cart related pages
  • - search pages
  • - sell pages
  • Uncomment if you do not wish to allow Google and Bing to index your images
  • User-agent: Googlebot-Image
  • Disallow: /
  • User-agent: msnbot-media
  • Disallow: /
  • Disallow: /*.php$