industrystock.com
robots.txt

Robots Exclusion Standard data for industrystock.com

Resource Scan

Scan Details

Site Domain industrystock.com
Base Domain industrystock.com
Scan Status Ok
Last Scan2024-06-13T01:54:52+00:00
Next Scan 2024-07-13T01:54:52+00:00

Last Scan

Scanned2024-06-13T01:54:52+00:00
URL https://industrystock.com/robots.txt
Redirect https://www.industrystock.com/robots.txt
Redirect Domain www.industrystock.com
Redirect Base industrystock.com
Domain IPs 104.21.10.179, 172.67.131.174, 2606:4700:3035::6815:ab3, 2606:4700:3037::ac43:83ae
Redirect IPs 104.21.10.179, 172.67.131.174, 2606:4700:3035::6815:ab3, 2606:4700:3037::ac43:83ae
Response IP 172.67.131.174
Found Yes
Hash 321033ff5f115dda57cade5dfa34c6fc9eed1b2a912752b6a07f333ea79354a5
SimHash 123d0b4c35db

Groups

amazonbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

vegi bot (we follow your robots.txt settings before crawling, you can slow down the bot by change the crawl-delay parameter in the settings.if you have an enquiry, please email to: abuse-report@terrykyleseoagency.com)

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

del.icio.us

Rule Path
Disallow /

del.icio.us-thumbnails/1.0

Rule Path
Disallow /

riddler

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

*

Rule Path
Disallow /extern/outbound.php
Disallow /outbound/
Disallow /produkt_db/
Disallow /ajax/
Disallow /get/
Disallow /goto/
Disallow /uploads/
Disallow /vcard/
Disallow /pt/videos/
Disallow /ru/videos/
Disallow /zh/videos/
Disallow /ja/videos/
Disallow /ko/videos/
Disallow /fr/videos/
Disallow /ar/videos/
Disallow /fa/videos/
Disallow /nl/videos/
Disallow /en/videos/
Disallow /de/videos/
Disallow /es/videos/
Disallow /hu/videok/
Disallow /cs/videa/
Disallow /it/video/
Disallow /pl/filmy/
Disallow /tr/videolar/
Disallow /pt/empregos/
Disallow /ru/jobs/
Disallow /zh/jobs/
Disallow /ja/jobs/
Disallow /ko/jobs/
Disallow /fr/emplois/
Disallow /ar/jobs/
Disallow /fa/jobs/
Disallow /nl/jobs/
Disallow /en/jobs/
Disallow /de/jobs/
Disallow /es/empleo/
Disallow /hu/allasok/
Disallow /cs/nabidka-prace/
Disallow /it/posti-di-lavoro/
Disallow /pl/praca/
Disallow /tr/isler/
Disallow /pt/universities/
Disallow /ru/universities/
Disallow /zh/universities/
Disallow /ja/universities/
Disallow /ko/universities/
Disallow /fr/universities/
Disallow /ar/universities/
Disallow /fa/universities/
Disallow /nl/universities/
Disallow /en/universities/
Disallow /de/universit%C3%83%C2%A4ten/
Disallow /es/universities/
Disallow /hu/universities/
Disallow /cs/universities/
Disallow /it/universities/
Disallow /pl/universities/
Disallow /tr/universities/
Disallow /pt/catalogos/
Disallow /ru/catalogues/
Disallow /zh/catalogues/
Disallow /ja/catalogues/
Disallow /ko/catalogues/
Disallow /fr/catalogues/
Disallow /ar/catalogues/
Disallow /fa/catalogues/
Disallow /nl/catalogues/
Disallow /en/catalogues/
Disallow /de/kataloge/
Disallow /es/catalogos/
Disallow /hu/katalogus/
Disallow /cs/katalog/
Disallow /it/cataloghi/
Disallow /pl/katalogi/
Disallow /tr/kataloglar/
Disallow /pt/produtos/detalhe/
Disallow /ru/product/detail/
Disallow /zh/product/detail/
Disallow /ja/product/detail/
Disallow /ko/product/detail/
Disallow /fr/produit/details/
Disallow /ar/product/detail/
Disallow /fa/product/detail/
Disallow /nl/product/detail/
Disallow /en/product/detail/
Disallow /de/produkt/anzeigen/
Disallow /es/productos/detalle/
Disallow /hu/termek/megjelenites/
Disallow /cs/produkt/detail/
Disallow /it/prodotti/dettagli/
Disallow /pl/produkty/szczegoly/
Disallow /tr/urun/goster/
Disallow /pt/solicitar/acesso/empresa/
Disallow /ru/request/company/access/
Disallow /zh/request/company/access/
Disallow /ja/request/company/access/
Disallow /ko/request/company/access/
Disallow /fr/demander/acces/entreprise/
Disallow /ar/request/company/access/
Disallow /fa/request/company/access/
Disallow /nl/request/company/access/
Disallow /en/request/company/access/
Disallow /de/unternehmenszugang/einrichten/
Disallow /es/solicitar/acceso/empresa/
Disallow /hu/vallalati-hozzaferes/beallitas/
Disallow /cs/pozadavek/firma/pristup/
Disallow /it/richiesta/accesso-azienda/
Disallow /pl/zapytanie/firma/dostep/
Disallow /tr/sirketerisimi/olustur/
Disallow /de/suche/nach/