ernestech.com
robots.txt

Robots Exclusion Standard data for ernestech.com

Resource Scan

Scan Details

Site Domain ernestech.com
Base Domain ernestech.com
Scan Status Ok
Last Scan2024-10-01T08:56:38+00:00
Next Scan 2024-10-08T08:56:38+00:00

Last Scan

Scanned2024-10-01T08:56:38+00:00
URL https://www.ernestech.com/robots.txt
Domain IPs 104.21.14.253, 172.67.160.216, 2606:4700:3034::6815:efd, 2606:4700:3034::ac43:a0d8
Response IP 104.21.14.253
Found Yes
Hash ed0a63d975634fa5fde56e98884c70d9f9a62baf4c6504aae64f6fca7f97a79b
SimHash 101c9d436619

Groups

semrushbot

Rule Path
Disallow *

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bingbot

Rule Path
Allow *

*

Rule Path
Allow *
Disallow /progressBar
Disallow /userFeed
Disallow /commentsHub

googlebot

Rule Path
Allow *
Disallow /progressBar
Disallow /userFeed
Disallow /commentsHub

googlebot-image

Rule Path
Allow *
Disallow /progressBar
Disallow /userFeed
Disallow /commentsHub

libwww-perl

Rule Path
Allow *
Disallow /progressBar
Disallow /userFeed
Disallow /commentsHub

yahoo-slurp

Rule Path
Allow *
Disallow /progressBar
Disallow /userFeed
Disallow /commentsHub

msnbot

Rule Path
Allow *
Disallow /progressBar
Disallow /userFeed
Disallow /commentsHub

Other Records

Field Value
sitemap https://www.ernestech.com/sitemap.xml