globalpenfriends.com
robots.txt

Robots Exclusion Standard data for globalpenfriends.com

Resource Scan

Scan Details

Site Domain globalpenfriends.com
Base Domain globalpenfriends.com
Scan Status Ok
Last Scan2024-05-29T06:04:20+00:00
Next Scan 2024-06-05T06:04:20+00:00

Last Scan

Scanned2024-05-29T06:04:20+00:00
URL https://globalpenfriends.com/robots.txt
Redirect https://www.globalpenfriends.com/robots.txt
Redirect Domain www.globalpenfriends.com
Redirect Base globalpenfriends.com
Domain IPs 104.26.14.10, 104.26.15.10, 172.67.70.115, 2606:4700:20::681a:e0a, 2606:4700:20::681a:f0a, 2606:4700:20::ac43:4673
Redirect IPs 104.26.14.10, 104.26.15.10, 172.67.70.115, 2606:4700:20::681a:e0a, 2606:4700:20::681a:f0a, 2606:4700:20::ac43:4673
Response IP 104.26.15.10
Found Yes
Hash a947a7b3d560e2d70e44e59182f19aa5dcc120de4aca33469ff5c318883f0725
SimHash ae1c82c0f371

Groups

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

bingpreview

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /webdate_bak/
Disallow /webdate3/
Disallow /photos/
Disallow /index.php?page=send_vkiss*
Disallow /index.php?page=contact*
Disallow /index.php?page=tell_friend*
Disallow /index.php?action*
Disallow /smartway/
Disallow /index.php?page=adv_search*
Disallow /index.php?page=search*