Log In Sign Up

celery94.github.io
robots.txt

Robots Exclusion Standard data for celery94.github.io

Resource Scan

Scan Details

Site Domain celery94.github.io
Base Domain celery94.github.io
Scan Status Ok
Last Scan2026-01-09T22:17:06+00:00
Next Scan 2026-01-16T22:17:06+00:00

Last Scan

Scanned2026-01-09T22:17:06+00:00
URL https://celery94.github.io/robots.txt
Redirect https://aidehub.top/robots.txt
Redirect Domain aidehub.top
Redirect Base aidehub.top
Domain IPs 185.199.108.153, 185.199.109.153, 185.199.110.153, 185.199.111.153, 2606:50c0:8000::153, 2606:50c0:8001::153, 2606:50c0:8002::153, 2606:50c0:8003::153
Redirect IPs 185.199.108.153, 185.199.109.153, 185.199.110.153, 185.199.111.153, 2606:50c0:8000::153, 2606:50c0:8001::153, 2606:50c0:8002::153, 2606:50c0:8003::153
Response IP 185.199.110.153
Found Yes
Hash dc1ad4cca6c2ff6e74a9a3da3961c05e6e6e09679b4210330cec2a9b7bf2e012
SimHash 4c13bc52e657

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /draft/
Disallow /api/
Disallow /*?

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2
Back to top

Other Records

Field Value
sitemap https://celery94.github.io/sitemap-index.xml
Back to top

Comments

  • robots.txt for Aide Hub
  • https://celery94.github.io/
  • Allow specific bots with custom rules
  • Sitemap
Back to top