Log In Sign Up

agrlk.com
robots.txt

Robots Exclusion Standard data for agrlk.com

Resource Scan

Scan Details

Site Domain agrlk.com
Base Domain agrlk.com
Scan Status Ok
Last Scan2025-10-10T14:32:16+00:00
Next Scan 2025-11-09T14:32:16+00:00

Last Scan

Scanned2025-10-10T14:32:16+00:00
URL https://agrlk.com/robots.txt
Domain IPs 104.21.57.145, 172.67.146.162, 2606:4700:3030::6815:3991, 2606:4700:3034::ac43:92a2
Response IP 172.67.146.162
Found Yes
Hash 65f073510443e47ad315e2dbbe57397c21567fb83cb3c280d3f000c1cb5e410b
SimHash 8b154122c89a

Groups

facebookexternalhit

Rule Path
Allow /

*

Rule Path
Allow /
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pkginfo/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/
Disallow /pub/
Disallow /nginx.conf
Disallow /php.ini
Disallow /LICENSE.txt
Disallow /cron.php
Allow /pub/media/
Allow /pub/static/
Disallow /admin/
Disallow /checkout/
Disallow /customer/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /newsletter/
Disallow /review/
Disallow /sendfriend/
Disallow /wishlist/
Disallow /*?SID=
Disallow /*?p=
Disallow /*?dir=
Disallow /*?order=
Disallow /*?limit=
Disallow /*?mode=
Disallow /*?___from_store=
Disallow /*?___store=
Disallow /*?am_shopby_*=
Disallow /*?brand=
Disallow /*?cat=
Disallow /*?color=
Disallow /*?price=
Disallow /*?size=
Disallow /*?style=
Back to top

Other Records

Field Value
sitemap https://agrlk.com/sitemap.xml
Back to top

Comments

  • Directories to disallow
  • Disallow private user and system pages
  • Disallow URLs with session IDs and other query parameters
  • that create duplicate content
  • NOTE: The following rules block layered navigation filters.
  • This is generally recommended to conserve crawl budget.
  • However, if you have a specific SEO strategy for filtered pages,
  • you may need to adjust these.
Back to top