Log In Sign Up

daffodilvarsity.edu.bd
robots.txt

Robots Exclusion Standard data for daffodilvarsity.edu.bd

Resource Scan

Scan Details

Site Domain daffodilvarsity.edu.bd
Base Domain daffodilvarsity.edu.bd
Scan Status Ok
Last Scan2026-02-21T03:17:00+00:00
Next Scan 2026-03-23T03:17:00+00:00

Last Scan

Scanned2026-02-21T03:17:00+00:00
URL https://daffodilvarsity.edu.bd/robots.txt
Domain IPs 13.204.201.17, 65.0.53.182
Response IP 13.204.201.17
Found Yes
Hash a7810272414ba44de88db71ff8a2efb7a06a26f836b63c321eb6c08d03209dd5
SimHash 2f3afc5266a7

Groups

*

Rule Path
Allow /
Allow /home
Allow /programs
Allow /faculties/
Allow /departments/
Allow /department/
Allow /admission/
Allow /tuition-fees
Allow /scholarship
Allow /events
Allow /noticeboard
Allow /gallery
Allow /teachers
Allow /staff/
Allow /administration
Allow /international
Allow /campus-life
Allow /research
Allow /lab-facilities
Allow /at-a-glance
Allow /profile
Allow /rankings
Disallow /admin/login
Disallow /private/
Disallow /_next/
Disallow /maintenance
Disallow /erp
Allow /api/consent
Allow /api/proxy

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /_next/
Disallow /maintenance

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://daffodilvarsity.edu.bd/sitemap.xml
sitemap https://daffodilvarsity.edu.bd/sitemap.xml
Back to top

Comments

  • Robots.txt for Daffodil International University
  • Sitemap location
  • Allow search engines to crawl all content
  • Disallow admin and private areas
  • Allow API endpoints that provide public data
  • Crawl-delay (optional - helps with server load)
  • Host preference
  • Allow Googlebot with more permissions
  • Sitemap location
  • Crawl delay
  • Host
Back to top

Warnings

  • `host` is not a known field.
Back to top