Log In Sign Up

yetoot.com
robots.txt

Robots Exclusion Standard data for yetoot.com

Resource Scan

Scan Details

Site Domain yetoot.com
Base Domain yetoot.com
Scan Status Ok
Last Scan2026-03-02T15:05:41+00:00
Next Scan 2026-03-09T15:05:41+00:00

Last Scan

Scanned2026-03-02T15:05:41+00:00
URL https://yetoot.com/robots.txt
Redirect https://www.yetoot.com/robots.txt
Redirect Domain www.yetoot.com
Redirect Base yetoot.com
Domain IPs 104.21.63.6, 172.67.168.246, 2606:4700:3033::6815:3f06, 2606:4700:3035::ac43:a8f6
Redirect IPs 104.21.63.6, 172.67.168.246, 2606:4700:3033::6815:3f06, 2606:4700:3035::ac43:a8f6
Response IP 172.67.168.246
Found Yes
Hash 55c7cb6cff819a1ec4174b3e3992f39ae40d4ee0230daf20c2acf2a372221533
SimHash 6a089801ee22

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /.well-known/
Disallow /api/
Disallow /_astro/
Allow /blog/
Allow /about
Allow /contact
Allow /images/
Disallow /*?*
Allow /*?utm_*

Other Records

Field Value
crawl-delay 1

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.yetoot.com/sitemap.xml
Back to top

Comments

  • Sitemaps
  • Crawl-delay for polite crawling
  • Block crawling of private pages
  • Allow crawling of important pages
  • Block crawling of search and filter pages (if any)
  • Social media crawlers
Back to top