Log In Sign Up

espreso.tv
robots.txt

Robots Exclusion Standard data for espreso.tv

Resource Scan

Scan Details

Site Domain espreso.tv
Base Domain espreso.tv
Scan Status Ok
Last Scan2024-11-01T21:57:59+00:00
Next Scan 2024-11-08T21:57:59+00:00

Last Scan

Scanned2024-11-01T21:57:59+00:00
URL https://espreso.tv/robots.txt
Domain IPs 104.18.10.148, 104.18.11.148, 2606:4700::6812:a94, 2606:4700::6812:b94
Response IP 104.18.11.148
Found Yes
Hash 72cd2dec58e6985bc0dc3b04077e37b6498d104ece40bed735b74c4fb39eaa43
SimHash 28190f1241e9

Groups

*

Rule Path
Allow /
Disallow /*?q=
Disallow /search-results*
Disallow /streamonline*
Disallow /*?*
Disallow /*?
Allow /*?page*
Allow /*?amp$
Allow /uploads
Allow /uploads/photobank/
Allow /uploads/*.png
Allow /uploads/*.jpg
Allow /uploads/*.jpeg
Allow /uploads/*.gif
Allow /uploads/*.svg
Allow /uploads/*.pdf

googlebot

Rule Path
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$

googlebot-image

Rule Path
Allow /*
Back to top

Other Records

Field Value
sitemap https://espreso.tv/sitemap.xml
Back to top

Comments

  • Disalow search :
  • Disallow indexation of URLs having duplicate content parameters
  • Allow to index images
  • Allow images in plugins, cache (check your path!!)
  • Disallow indexation of sensitive files (check yours)
  • Allow Google Image Bot
  • Show to spiders sitemap (create a sitemap with fresh news only - last week, and place on this link)
Back to top