Log In Sign Up

vpnjantit.com
robots.txt

Robots Exclusion Standard data for vpnjantit.com

Resource Scan

Scan Details

Site Domain vpnjantit.com
Base Domain vpnjantit.com
Scan Status Ok
Last Scan2024-11-14T10:50:49+00:00
Next Scan 2024-11-21T10:50:49+00:00

Last Scan

Scanned2024-11-14T10:50:49+00:00
URL https://vpnjantit.com/robots.txt
Redirect https://www.vpnjantit.com/robots.txt
Redirect Domain www.vpnjantit.com
Redirect Base vpnjantit.com
Domain IPs 165.22.38.47
Redirect IPs 104.21.31.136, 172.67.176.210, 2606:4700:3035::6815:1f88, 2606:4700:3037::ac43:b0d2
Response IP 172.67.176.210
Found Yes
Hash 60d50f76db6a7bddc882a2184315dcfe525d1259d5702ced8e4ca4556d6f104a
SimHash f81477106409

Groups

googlebot
googlebot-image
applebot

Rule Path
Disallow /images/

baiduspider

Rule Path
Disallow /images/

bingbot

Rule Path
Disallow /images/

ia_archiver

Rule Path
Disallow /images/

msnbot

Rule Path
Disallow /images/

naverbot

Rule Path
Disallow /images/

seznambot

Rule Path
Disallow /images/

slurp

Rule Path
Disallow /images/

teoma

Rule Path
Disallow /images/

twitterbot

Rule Path
Disallow /images/

yandex

Rule Path
Disallow /images/

yeti

Rule Path
Disallow /images/

applebot

Rule Path
Disallow /images/

baiduspider

Rule Path
Disallow /images/

bingbot

Rule Path
Disallow /images/

googlebot

Rule Path
Disallow /images/

ia_archiver

Rule Path
Disallow /images/

msnbot

Rule Path
Disallow /images/

naverbot

Rule Path
Disallow /images/

seznambot

Rule Path
Disallow /images/

slurp

Rule Path
Disallow /images/

teoma

Rule Path
Disallow /images/

twitterbot

Rule Path
Disallow /images/

yandex

Rule Path
Disallow /images/

yeti

Rule Path
Disallow /images/
Back to top

Comments

  • Notice: Crawling Facebook is prohibited unless you have express written
  • permission. See: http://www.facebook.com/apps/site_scraping_tos_terms.php
Back to top

Warnings

  • 1 invalid line.
Back to top