materialhandling247.com
robots.txt

Robots Exclusion Standard data for materialhandling247.com

Resource Scan

Scan Details

Site Domain materialhandling247.com
Base Domain materialhandling247.com
Scan Status Ok
Last Scan2025-08-23T20:48:20+00:00
Next Scan 2025-08-30T20:48:20+00:00

Last Scan

Scanned2025-08-23T20:48:20+00:00
URL https://materialhandling247.com/robots.txt
Redirect https://www.materialhandling247.com/robots.txt
Redirect Domain www.materialhandling247.com
Redirect Base materialhandling247.com
Domain IPs 104.21.27.181, 172.67.143.112, 2606:4700:3036::ac43:8f70, 2606:4700:3037::6815:1bb5
Redirect IPs 104.21.27.181, 172.67.143.112, 2606:4700:3036::ac43:8f70, 2606:4700:3037::6815:1bb5
Response IP 104.21.27.181
Found Yes
Hash 76711fe7f0149ed7b4d4dd4fc4745459d8287eef71f5ee8d9f57a1b142f08a97
SimHash 6a7e18425098

Groups

bingbot

Rule Path
Disallow /cp/
Disallow /global.php/
Disallow /cgi-bin/
Disallow /poll/
Disallow /scripts/
Disallow /member/
Disallow /forums/member/
Disallow /site/register/
Disallow /member/register/
Disallow /forums/member/register/
Disallow /site/login/
Disallow /protected/

Other Records

Field Value
crawl-delay 20

df bot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

awariobot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

blexbot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

sentibot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

turnitin

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

artiwisebot

Rule Path
Disallow /cp/
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

ahrefsbot

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

dataforseobot

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

bytespider

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

wordpress

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

slackbot

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

seekportbot

Rule Path
Disallow /
Disallow /global.php/

Other Records

Field Value
crawl-delay 30

*

Rule Path
Disallow /cp/
Disallow /global.php/
Disallow /criticaltopic/
Disallow /criticaltopics/
Disallow /demo/
Disallow /rss/opml
Disallow /rss/opml/
Disallow /cgi-bin/
Disallow /poll/
Disallow /scripts/
Disallow /member/
Disallow /forums/member/
Disallow /site/register/
Disallow /member/register/
Disallow /forums/member/register/
Disallow /site/login/
Disallow /protected/
Disallow /21832593685/

Other Records

Field Value
crawl-delay 60