comingsoon.it
robots.txt

Robots Exclusion Standard data for comingsoon.it

Resource Scan

Scan Details

Site Domain comingsoon.it
Base Domain comingsoon.it
Scan Status Ok
Last Scan2024-11-09T06:52:59+00:00
Next Scan 2024-11-16T06:52:59+00:00

Last Scan

Scanned2024-11-09T06:52:59+00:00
URL https://comingsoon.it/robots.txt
Redirect https://www.comingsoon.it/robots.txt
Redirect Domain www.comingsoon.it
Redirect Base comingsoon.it
Domain IPs 104.22.70.145, 104.22.71.145, 172.67.23.189, 2606:4700:10::6816:4691, 2606:4700:10::6816:4791, 2606:4700:10::ac43:17bd
Redirect IPs 104.22.70.145, 104.22.71.145, 172.67.23.189, 2606:4700:10::6816:4691, 2606:4700:10::6816:4791, 2606:4700:10::ac43:17bd
Response IP 104.22.70.145
Found Yes
Hash 77f0807452e54f5f941d9c043bb6609cfabaee879da6737e54b9d5b851e62baf
SimHash 736d5054a213

Groups

acunetix
anthropicbot
awariorssbot
awariosmartbot
bomborabot
bubing
chinaclaw
claude
claudebot
dotbot
diibot
fhscan
mj12bot
mauibot
megaindex.ru
serpstatbot
trendictionbot
sogou spider
sogou web spider
screaming frog seo
veooz
veoozbot
petalbot
ahrefsbot*
blexbot
criteobot*
npbot*
ttd-content
clickagy intelligence*
grapeshot

Rule Path
Disallow /

slurp
yandex
proximic
webnews-arianna
dataforseobot
shopwiki
*
googlebot
adsbot-google
mediapartners-google

Rule Path
Disallow /backup/
Disallow /banners/
Disallow /img/
Disallow /imgdb/banner/
Disallow /inc/
Disallow /InterStitial/
Disallow /link.asp
Disallow /MDB/
Disallow /radiodwnl/
Disallow /rating/
Disallow /rss/
Disallow /sharedcount/
Disallow /news/sharedcount/
Disallow /videoplayer/player/embed/
Disallow /imgdb/primopiano/impaginate/gallery/
Disallow /anteprima/
Disallow /testarea/
Disallow /POLL/
Disallow /content/uci/undefined/
Disallow /%28X%281%29S%28*%29%29/
Disallow /%28x%281%29s%28*
Disallow *AspxAutoDetectCookieSupport%3D1

Comments

  • Bad Bots
  • User-agent: Yandex
  • User-agent: Baiduspider
  • User-agent: Cliqzbot
  • User-agent: SemrushBot*
  • User-agent: GPTBot
  • User-agent: ChatGPT-User
  • Slow Bots
  • Crawl-Delay: 4
  • Good Bots