unitedlanguagegroup.com
robots.txt

Robots Exclusion Standard data for unitedlanguagegroup.com

Resource Scan

Scan Details

Site Domain unitedlanguagegroup.com
Base Domain unitedlanguagegroup.com
Scan Status Ok
Last Scan2024-10-25T04:00:11+00:00
Next Scan 2024-11-24T04:00:11+00:00

Last Scan

Scanned2024-10-25T04:00:11+00:00
URL https://unitedlanguagegroup.com/robots.txt
Redirect https://www.unitedlanguagegroup.com/robots.txt
Redirect Domain www.unitedlanguagegroup.com
Redirect Base unitedlanguagegroup.com
Domain IPs 199.60.103.226, 199.60.103.30, 2606:2c40::c73c:671e, 2606:2c40::c73c:67e2
Redirect IPs 199.60.103.226, 199.60.103.30, 2606:2c40::c73c:671e, 2606:2c40::c73c:67e2
Response IP 199.60.103.226
Found Yes
Hash 3a384ceb1c45780a2de3ffc5c3ae991483e551d5026108cfcfe5508813e4e6b1
SimHash 3a29ac6056a0

Groups

*

Rule Path
Disallow /topic/*
Disallow /tag/*
Disallow /author/*
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

semrushbot

Rule Path
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

Other Records

Field Value
crawl-delay 10

semrushbot-sa

Rule Path
Disallow /_hcms/preview/
Disallow /hs/manage-preferences/
Disallow /hs/preferences-center/
Disallow /*?*hs_preview=*
Disallow /*?*hsCacheBuster=*

Other Records

Field Value
crawl-delay 10