goskills.com
robots.txt

Robots Exclusion Standard data for goskills.com

Resource Scan

Scan Details

Site Domain goskills.com
Base Domain goskills.com
Scan Status Ok
Last Scan2024-06-13T01:15:42+00:00
Next Scan 2024-06-27T01:15:42+00:00

Last Scan

Scanned2024-06-13T01:15:42+00:00
URL https://goskills.com/robots.txt
Redirect https://www.goskills.com/robots.txt
Redirect Domain www.goskills.com
Redirect Base goskills.com
Domain IPs 40.112.243.49
Redirect IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash 9ed6f77564b7088e460c3eacd080d52b735cb0dc55cc36060d0b471a9fa643fb
SimHash edd658947790

Groups

mozilla/4.0 (compatible; synapse)

Rule Path
Disallow /

domain re-animator bot (http://domainreanimator.com) - support@domainreanimator.com

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

riddler (http://riddler.io/about)

Rule Path
Disallow /

curb

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

acahibot

Rule Path
Disallow /

ccahibot (ccahi.feedback@gmail.com)

Rule Path
Disallow /

bcahibot

Rule Path
Disallow /

sitesucker/2.4.6

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

test-bot

Rule Path
Disallow /

my-tiny-bot

Rule Path
Disallow /

thesis-research-bot

Rule Path
Disallow /

*

Rule Path
Disallow /Account/ExternalLogin
Disallow /Account/ExternalLoginCallback
Disallow /Reset
Disallow /Go/
Disallow /Share/
Disallow /Email/
Disallow /Cart/Add/
Disallow /Cron/
Disallow /External/

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://www.goskills.com/sitemap.xml