Log In Sign Up

cyberajans.com
robots.txt

Robots Exclusion Standard data for cyberajans.com

Resource Scan

Scan Details

Site Domain cyberajans.com
Base Domain cyberajans.com
Scan Status Ok
Last Scan2026-01-22T00:17:09+00:00
Next Scan 2026-01-29T00:17:09+00:00

Last Scan

Scanned2026-01-22T00:17:09+00:00
URL https://cyberajans.com/robots.txt
Redirect https://www.cyberajans.com/robots.txt
Redirect Domain www.cyberajans.com
Redirect Base cyberajans.com
Domain IPs 104.21.39.105, 172.67.144.121, 2606:4700:3033::6815:2769, 2606:4700:3036::ac43:9079
Redirect IPs 104.21.39.105, 172.67.144.121, 2606:4700:3033::6815:2769, 2606:4700:3036::ac43:9079
Response IP 172.67.144.121
Found Yes
Hash acf96d98e269fef70c0e71af44a0edfdeef8df8a85c4b8eccc307693388ffc96
SimHash 441f4053a7d4

Groups

*

Rule Path
Allow /
Disallow /includes/
Disallow /api/
Disallow /js/
Disallow /admin.php
Disallow /*.sql$
Disallow /*.log$

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

yandex

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://cyberajans.com/sitemap.xml
Back to top

Comments

  • Cyber Ajans Robots.txt
  • https://cyberajans.com
  • Sitemap
  • Disallow - Güvenlik ve gereksiz sayfalar
  • Crawl-delay (optional - be nice to servers)
  • Google specific
  • Bing specific
  • Yandex specific
Back to top