Log In Sign Up

elastos.info
robots.txt

Robots Exclusion Standard data for elastos.info

Resource Scan

Scan Details

Site Domain elastos.info
Base Domain elastos.info
Scan Status Ok
Last Scan2025-07-18T23:07:35+00:00
Next Scan 2025-08-17T23:07:35+00:00

Last Scan

Scanned2025-07-18T23:07:35+00:00
URL https://elastos.info/robots.txt
Redirect https://elastos.net/robots.txt
Redirect Domain elastos.net
Redirect Base elastos.net
Domain IPs 162.159.135.42
Redirect IPs 104.21.62.36, 172.67.219.108, 2606:4700:3030::ac43:db6c, 2606:4700:3034::6815:3e24
Response IP 104.21.62.36
Found Yes
Hash 1a81d656d63bbe8c459f04c087027f35421ac3f75e6824e22d5323f588fbd376
SimHash 2d03c62466b1

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /
Allow /news/
Allow /announcements/
Allow /blog/
Allow /images/
Allow /videos/
Allow /docs/

Other Records

Field Value
crawl-delay 5

bingbot

Rule Path
Allow /
Allow /news/
Allow /announcements/
Allow /blog/
Allow /images/
Allow /videos/
Allow /docs/
Disallow /admin/
Disallow /private/
Disallow /api/
Disallow /internal/
Disallow /temp/
Disallow /draft/
Disallow /test/
Disallow /dev/
Disallow /staging/
Disallow /beta/
Disallow /debug/
Disallow /console/
Disallow /cgi-bin/
Disallow /*.env
Disallow /*.config
Disallow /*.ini
Disallow /*.conf
Disallow /*.htaccess
Disallow /*.htpasswd
Disallow /*.git/
Disallow /*.svn/

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://elastos.net/sitemap.xml
sitemap https://elastos.net/sitemap-image.xml
sitemap https://elastos.net/sitemap-video.xml
sitemap https://elastos.net/sitemap-news.xml
Back to top

Comments

  • Global rules for all crawlers
  • Google-specific rules
  • Bing-specific rules
  • Block sensitive areas only
  • Host directive
  • Sitemap locations
Back to top

Warnings

  • `host` is not a known field.
Back to top