Log In Sign Up

futbolsayfasi.net
robots.txt

Robots Exclusion Standard data for futbolsayfasi.net

Resource Scan

Scan Details

Site Domain futbolsayfasi.net
Base Domain futbolsayfasi.net
Scan Status Ok
Last Scan2025-04-02T07:31:53+00:00
Next Scan 2025-04-09T07:31:53+00:00

Last Scan

Scanned2025-04-02T07:31:53+00:00
URL https://futbolsayfasi.net/robots.txt
Domain IPs 104.21.25.52, 172.67.222.183, 2606:4700:3034::ac43:deb7, 2606:4700:3037::6815:1934
Response IP 172.67.222.183
Found Yes
Hash f4039117d88d87e723482ae854b69486bec817b302e2989103b699030e634e66
SimHash a9793d7144e5

Groups

*

Rule Path
Disallow /includes/

googlebot

Rule Path
Allow /includes/
Back to top

Other Records

Field Value
sitemap https://futbolsayfasi.net/sitemap.xml
sitemap https://futbolsayfasi.net/news-sitemap.xml
Back to top

Comments

  • This robots.txt file controls crawling of URLs under https://example.com.
  • All crawlers are disallowed to crawl files in the "includes" directory, such
  • as .css, .js, but Google needs them for rendering, so Googlebot is allowed
  • to crawl them.
Back to top