axcertro.com
robots.txt

Robots Exclusion Standard data for axcertro.com

Resource Scan

Scan Details

Site Domain axcertro.com
Base Domain axcertro.com
Scan Status Ok
Last Scan2025-07-17T19:04:49+00:00
Next Scan 2025-08-16T19:04:49+00:00

Last Scan

Scanned2025-07-17T19:04:49+00:00
URL https://axcertro.com/robots.txt
Domain IPs 216.198.79.1
Response IP 216.198.79.1
Found Yes
Hash fc4031823bbd7b57038d363a1f94321ac806ccf4ca0aa9c2af16a33333594e60
SimHash 6aa64a32e6a0

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /_next/
Disallow /admin/
Disallow /private/
Allow /assets/
Allow /images/
Allow /public/

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://axcertro.com/sitemap.xml

Comments

  • Optimize crawl budget
  • Allow important directories
  • Sitemap location
  • Crawl delay for respectful crawling
  • Specific bot configurations