Log In Sign Up

digchip.com
robots.txt

Robots Exclusion Standard data for digchip.com

Resource Scan

Scan Details

Site Domain digchip.com
Base Domain digchip.com
Scan Status Ok
Last Scan2024-10-03T17:55:28+00:00
Next Scan 2024-10-10T17:55:28+00:00

Last Scan

Scanned2024-10-03T17:55:28+00:00
URL https://digchip.com/robots.txt
Domain IPs 57.128.52.5
Response IP 57.128.52.5
Found Yes
Hash 1eadb87d6e388b5aca9cde2232d253c6adb12e853b49fe0d5d482a3f3147de06
SimHash 933c5d227e18

Groups

a6-indexer

Rule Path
Disallow /

a6-indexer/1.0

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /
Disallow /

ntentbot-fetch

Rule Path
Disallow /

ntentbot-news

Rule Path
Disallow /

proximic

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

wesee

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

*

Rule Path
Disallow /members/search_categories_files/
Disallow /datasheets/redirect_company.php
Disallow /datasheets/suppliers_find.php
Disallow /datasheets/suppliers_xhr.php
Disallow /datasheets/suppliers_stock.php
Disallow /datasheets/suppliers_stock.php?*
Disallow /datasheets/suppliers_offer.php
Disallow /datasheets/suppliers_offer.php?*
Disallow /math-formula
Back to top

Warnings

  • `user-agnet` is not a known field.
Back to top