Log In Sign Up

nospoiler.it
robots.txt

Robots Exclusion Standard data for nospoiler.it

Resource Scan

Scan Details

Site Domain nospoiler.it
Base Domain nospoiler.it
Scan Status Ok
Last Scan2024-06-25T23:37:14+00:00
Next Scan 2024-07-02T23:37:14+00:00

Last Scan

Scanned2024-06-25T23:37:14+00:00
URL https://nospoiler.it/robots.txt
Redirect https://cpop.it/robots.txt
Redirect Domain cpop.it
Redirect Base cpop.it
Domain IPs 104.21.30.217, 172.67.173.243, 2606:4700:3032::ac43:adf3, 2606:4700:3037::6815:1ed9
Redirect IPs 104.26.10.122, 104.26.11.122, 172.67.70.51, 2606:4700:20::681a:a7a, 2606:4700:20::681a:b7a, 2606:4700:20::ac43:4633
Response IP 172.67.70.51
Found Yes
Hash 58552ff7c437c8761049a6821db32ca942e45b7d756d242a88e4aec6ad368214
SimHash 254dd0666cd1

Groups

acunetix
chinaclaw
dotbot
fhscan
mj12bot
mauibot
npbot*
npbot-1/2.0
teleport
magpie-crawler
piplbot
exabot

Rule Path
Disallow /

*
googlebot
adsbot-google
mediapartners-google

Rule Path
Disallow /cms
Disallow /tags
Disallow /search
Disallow /video
Disallow /video/*
Disallow /*?*query=
Disallow /guida-tv
Disallow /movies
Disallow /tvs
Disallow /movies/*
Disallow /tvs/*
Back to top

Other Records

Field Value
sitemap https://cpop.it/sitemap.xml
sitemap https://cpop.it/google-news-feed.xml
Back to top

Comments

  • Bad Bots
  • Good Bots
  • Areas to disallow
  • No-content pages
  • Movies and TV Shows
Back to top