autorita-trasporti.it
robots.txt

Robots Exclusion Standard data for autorita-trasporti.it

Resource Scan

Scan Details

Site Domain autorita-trasporti.it
Base Domain autorita-trasporti.it
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-07T21:37:57+00:00
Next Scan 2024-12-06T21:37:57+00:00

Last Successful Scan

Scanned2023-10-21T07:15:05+00:00
URL https://autorita-trasporti.it/robots.txt
Redirect https://www.autorita-trasporti.it/robots.txt
Redirect Domain www.autorita-trasporti.it
Redirect Base autorita-trasporti.it
Domain IPs 2600:1413:b000:6::17d5:2bc7, 2600:1413:b000:6::17d5:2bce, 96.17.96.16, 96.17.96.7
Redirect IPs 2600:1413:b000:6::17d5:2bc7, 2600:1413:b000:6::17d5:2bce, 96.17.96.16, 96.17.96.7
Response IP 104.88.70.154
Found Yes
Hash aac314bd5a896730058b0db2b8192695835e90eaaeb76dffe85948fd8a81f138
SimHash 6a415c928140

Groups

*

Rule Path
Allow /wp-content/uploads/
Disallow /wp-content/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Allow /feed/$
Disallow /feed
Disallow /comments/feed
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$
Disallow /?s=
Disallow /search
Disallow /pdf

googlebot

Rule Path
Disallow /*.php$
Disallow /*.js$
Disallow /*.inc$
Disallow /*.css$
Disallow /*.gz$
Disallow /*.wmv$
Disallow /*.cgi$
Disallow /*.xhtml$
Disallow /*?*
Disallow /*.txt$

mediapartners-google*

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.autorita-trasporti.it/sitemap.xml

Comments

  • SITEMAP.XML - MUST BE FILLED, OR DELETE IT
  • USER-AGENT
  • DO NOT INDEX DUPLICATE POSTS, COMMENTS AND TRACKBACKS
  • DO NOT INDEX FILES WITH THIS EXTENSIONS
  • ALLOW ADSENSE BOT (OPTIONAL)
  • ALLOW GOOGLE IMAGES (OPTIONAL)
  • LIMIT Yahoo, MSN AND Noxtrum BOTS (OPTIONAL)