sosteeshirt.fr
robots.txt

Robots Exclusion Standard data for sosteeshirt.fr

Resource Scan

Scan Details

Site Domain sosteeshirt.fr
Base Domain sosteeshirt.fr
Scan Status Ok
Last Scan2025-10-21T10:29:55+00:00
Next Scan 2025-11-04T10:29:55+00:00

Last Scan

Scanned2025-10-21T10:29:55+00:00
URL https://sosteeshirt.fr/robots.txt
Domain IPs 104.21.10.183, 172.67.190.200, 2606:4700:3031::ac43:bec8, 2606:4700:3033::6815:ab7
Response IP 104.21.10.183
Found Yes
Hash 77d15bf97367d5c745df2133a284826495652f0b7f491220426694508b004229
SimHash 3884d613e5bb

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://sosteeshirt.fr/sitemap.xml

Comments

  • Sitemap
  • Disallow admin or sensitive areas (none for this site)
  • Allow all content for SEO indexing
  • Crawl-delay for respectful crawling
  • Allow Google Ads bot
  • Allow Google Images bot
  • Allow social media crawlers