mobility.smartworld.it
robots.txt

Robots Exclusion Standard data for mobility.smartworld.it

Resource Scan

Scan Details

Site Domain mobility.smartworld.it
Base Domain smartworld.it
Scan Status Ok
Last Scan2024-10-28T21:35:17+00:00
Next Scan 2024-11-27T21:35:17+00:00

Last Scan

Scanned2024-10-28T21:35:17+00:00
URL https://mobility.smartworld.it/robots.txt
Domain IPs 23.59.168.121, 2600:1413:b000:6::17d5:2bc9, 2600:1413:b000:6::17d5:2bce
Response IP 104.88.70.120
Found Yes
Hash 4a857e8ee20f5dd4c4d5f2356b609e785e4a34db307df302d1ca92ee9cf2d2ec
SimHash 5154eb45c3a3

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /wp-admin/
Disallow /cgi-bin/
Disallow /trackback/
Disallow */trackback/
Disallow /2009/
Disallow /2010/
Disallow /2011/
Disallow /2012/
Allow /info-utili/wordpress/wp-admin/admin-ajax.php
Disallow /info-utili/wordpress/wp-admin/
Disallow info-utili/*/feed$

googlebot-image

Rule Path
Allow /*

gptbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

seekr

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

Other Records

Field Value
sitemap https://mobility.smartworld.it/sitemap_index.xml
sitemap https://mobility.smartworld.it/news-sitemap.xml
sitemap https://mobility.smartworld.it/video-sitemap.xml
sitemap https://mobility.smartworld.it/contributors-sitemap.xml
sitemap https://mobility.smartworld.it/info-utili/sitemap_index.xml

Comments

  • abilita adsense
  • disallow all files in these directories
  • Ex-Wheels
  • allow google image bot to search all images