Log In Sign Up

webtrek.it
robots.txt

Robots Exclusion Standard data for webtrek.it

Resource Scan

Scan Details

Site Domain webtrek.it
Base Domain webtrek.it
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2026-02-05T03:33:15+00:00
Next Scan 2026-02-12T03:33:15+00:00

Last Successful Scan

Scanned2026-01-28T03:32:36+00:00
URL https://webtrek.it/robots.txt
Domain IPs 151.11.49.13
Response IP 151.11.49.13
Found Yes
Hash 05ce946322a368cb7d75a6e94d3d9d5de379d418b6e0bc1dda5591e5e4dbf42c
SimHash 4d395f6252d1

Groups

*

Rule Path
Allow /
Allow /wp-content/uploads/
Disallow /wp-admin/
Disallow /wp-content/

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.webtrek.it/sitemap_index.xml
Back to top

Comments

  • User-agent: *
  • Disallow: /wp-
  • Disallow: /cgi-bin/
  • Disallow: /wp-admin/
  • Disallow: /wp-content/
  • Disallow: /trackback/
  • Disallow: /feed/
  • Disallow: */trackback/
  • Disallow: */feed/
  • Disallow: /svil/
  • Disallow: /hashtag/
  • Disallow: /members/
Back to top