ent.com
robots.txt

Robots Exclusion Standard data for ent.com

Resource Scan

Scan Details

Site Domain ent.com
Base Domain ent.com
Scan Status Ok
Last Scan2024-09-07T11:16:45+00:00
Next Scan 2024-10-07T11:16:45+00:00

Last Scan

Scanned2024-09-07T11:16:45+00:00
URL https://ent.com/robots.txt
Redirect https://www.ent.com/robots.txt
Redirect Domain www.ent.com
Redirect Base ent.com
Domain IPs 217.114.85.70
Redirect IPs 104.18.39.101, 172.64.148.155, 2606:4700:4400::6812:2765, 2606:4700:4400::ac40:949b
Response IP 104.18.39.101
Found Yes
Hash 14120dce8185d9b15ebbe8dd5403efb28a2a2e859bded6b46a4721ea80ad65d4
SimHash 3805d840cb92

Groups

*

Rule Path
Disallow /EPiServer/CMS
Disallow /Util
Disallow /z_archived_pages
Disallow /recycle-bin/

mj12bot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

mauibot (crawler.feedback+dc@gmail.com)

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

ahc/1.0

Rule Path
Disallow /

ahc/2.0

Rule Path
Disallow /

ahc/2.1

Rule Path
Disallow /

axios 0.21.1

Rule Path
Disallow /

axios 1.2.4

Rule Path
Disallow /

axios 1

Rule Path
Disallow /

got 8.3.1

Rule Path
Disallow /

got 8

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.ent.com/sitemap.xml

Comments

  • blocking bad bots