Log In Sign Up

masstamilan.so
robots.txt

Robots Exclusion Standard data for masstamilan.so

Resource Scan

Scan Details

Site Domain masstamilan.so
Base Domain masstamilan.so
Scan Status Ok
Last Scan2024-06-21T13:48:45+00:00
Next Scan 2024-06-28T13:48:45+00:00

Last Scan

Scanned2024-06-21T13:48:45+00:00
URL https://masstamilan.so/robots.txt
Redirect https://masstamilan.dev/robots.txt
Redirect Domain masstamilan.dev
Redirect Base masstamilan.dev
Domain IPs 104.21.88.33, 172.67.150.78, 2606:4700:3034::ac43:964e, 2606:4700:3037::6815:5821
Redirect IPs 104.21.92.69, 172.67.187.163, 2606:4700:3031::ac43:bba3, 2606:4700:3032::6815:5c45
Response IP 172.67.187.163
Found Yes
Hash fa33ba8d37e9dbe755b058dce0642c9f0d5d37aea09d731eea284d281aa74b99
SimHash 3a8c8d8de6f0

Groups

*

Rule Path
Disallow /vote/
Disallow /dload/
Disallow /download/
Disallow /dl/
Disallow /downloader/

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://masstamilan.dev/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
Back to top