Log In Sign Up

fanpuglia.it
robots.txt

Robots Exclusion Standard data for fanpuglia.it

Resource Scan

Scan Details

Site Domain fanpuglia.it
Base Domain fanpuglia.it
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-30T20:12:24+00:00
Next Scan 2024-10-29T20:12:24+00:00

Last Successful Scan

Scanned2024-07-02T20:11:25+00:00
URL https://fanpuglia.it/robots.txt
Domain IPs 135.181.171.228
Response IP 135.181.171.228
Found Yes
Hash 834127f3249ed637f8d1ac27d718b892e69c0614f0df9a694cc0c88206cd408c
SimHash ea6ad94a4c35

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-login.php
Disallow /wp-includes/
Disallow /?filter*
Disallow /cache*
Disallow /*/page

mediapartners-google

Rule Path
Allow /
Disallow /readme.html
Disallow /license.txt
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Allow /wp-content/uploads/
Allow /wp-content/cache/
Disallow /feed/
Disallow /comments/feed/
Disallow /*?s=*
Disallow /*?replytocom=*
Back to top

Other Records

Field Value
sitemap https://www.fanpuglia.it/sitemap-news.xml
sitemap https://www.fanpuglia.it/sitemap.xml
Back to top

Comments

  • General directives for all user agents
  • Allow Google Ads bot to access all pages
  • Block access to specific WordPress files
  • Block access to themes and plugins directories
  • Allow specific important directories
  • Optional: block duplicate content generated by feeds
  • Optional: block parameters used for tracking or sorting that create duplicate content
  • Specify the location of the sitemap
Back to top