deskeng.com
robots.txt

Robots Exclusion Standard data for deskeng.com

Resource Scan

Scan Details

Site Domain deskeng.com
Base Domain deskeng.com
Scan Status Ok
Last Scan2025-02-16T10:02:33+00:00
Next Scan 2025-03-18T10:02:33+00:00

Last Scan

Scanned2025-02-16T10:02:33+00:00
URL http://deskeng.com/robots.txt
Redirect https://www.digitalengineering247.com/robots.txt
Redirect Domain www.digitalengineering247.com
Redirect Base digitalengineering247.com
Domain IPs 64.62.152.92
Redirect IPs 104.21.66.109, 172.67.159.113, 2606:4700:3034::6815:426d, 2606:4700:3037::ac43:9f71
Response IP 104.21.66.109
Found Yes
Hash 8e1bb8a9efb5fb22bdedb3c86fb1f5d9ad87dad33f9ff4dc7e9b41b10a187dae
SimHash 6a5c70c4529c

Groups

df bot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

bomborabot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

blexbot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

sentibot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

turnitin

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

artiwisebot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

ahrefsbot

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

dataforseobot

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

bytespider

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

wordpress

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

slackbot

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

bingbot

Rule Path
Disallow /cp/
Disallow /phoenix.php
Disallow /site/login/
Disallow /protected/
Disallow /article/wp-admin/
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 20

googlebot

Rule Path
Disallow /cp/
Disallow /site/login/
Disallow /protected/
Disallow /21832593685/
Disallow /21832593685
Disallow /phoenix.php?%2F
Disallow /phoenix.php
Disallow /article/wp-admin/
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 20

*

Rule Path
Disallow /cp/
Disallow /phoenix.php/
Disallow /site/login/
Disallow /protected/
Disallow /21832593685/
Disallow /article/wp-admin/
Disallow /wp-admin/
Disallow /*page
Disallow /*feed
Disallow /topic/tag/video/

Other Records

Field Value
crawl-delay 20