idtechex.com
robots.txt

Robots Exclusion Standard data for idtechex.com

Resource Scan

Scan Details

Site Domain idtechex.com
Base Domain idtechex.com
Scan Status Ok
Last Scan2024-10-28T04:42:36+00:00
Next Scan 2024-11-27T04:42:36+00:00

Last Scan

Scanned2024-10-28T04:42:36+00:00
URL https://idtechex.com/robots.txt
Redirect https://www.idtechex.com/robots.txt
Redirect Domain www.idtechex.com
Redirect Base idtechex.com
Domain IPs 52.4.254.103
Redirect IPs 52.4.254.103
Response IP 52.4.254.103
Found Yes
Hash fe6e2107ae1c2fe990c203f2bfaf791748e2b4bdd88b130dc93a464a14a8e49d
SimHash 511c3452e000

Groups

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

bytedance

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dow jones searchbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

eyemonit

Rule Path
Disallow /

goldfire server

Rule Path
Disallow /

goldfire server

Rule Path
Disallow /

goldfire+server

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

ev-crawler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

hawaiibot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

googlebot

Rule Path
Disallow /en/research/blog?after=*
Disallow /de/research/blog?after=*
Disallow /fr/research/blog?after=*
Disallow /es/research/blog?after=*
Disallow /it/research/blog?after=*
Disallow /th/research/blog?after=*
Disallow /ko/research/blog?after=*
Disallow /ja/research/blog?after=*
Disallow /zh/research/blog?after=*

bingbot

Rule Path
Disallow /en/research/blog?after=*
Disallow /de/research/blog?after=*
Disallow /fr/research/blog?after=*
Disallow /es/research/blog?after=*
Disallow /it/research/blog?after=*
Disallow /th/research/blog?after=*
Disallow /ko/research/blog?after=*
Disallow /ja/research/blog?after=*
Disallow /zh/research/blog?after=*

*

Rule Path
Disallow /emails/click.asp?*
Disallow /users/fwdtofriend.asp?*
Disallow /fwd?*
Disallow /journal/print-articles.asp?*