manchesterstudentsunion.com
robots.txt

Robots Exclusion Standard data for manchesterstudentsunion.com

Resource Scan

Scan Details

Site Domain manchesterstudentsunion.com
Base Domain manchesterstudentsunion.com
Scan Status Ok
Last Scan2025-09-25T12:54:08+00:00
Next Scan 2025-10-25T12:54:08+00:00

Last Scan

Scanned2025-09-25T12:54:08+00:00
URL https://manchesterstudentsunion.com/robots.txt
Domain IPs 104.21.25.230, 172.67.168.46, 2606:4700:3035::6815:19e6, 2606:4700:3035::ac43:a82e
Response IP 104.21.25.230
Found Yes
Hash f9865b8b7cd0601ea0b7d35994156bf16b966f388af9a3b91662c4ba577dff8a
SimHash 4016d3f54557

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Allow /
Disallow /shop/*
Allow /shop/product/*

bingbot

Rule Path
Allow /
Disallow /shop/*
Allow /shop/product/*

facebookexternalhit

Rule Path
Allow /
Disallow /shop/*
Allow /shop/product/*

slurp

Rule Path
Allow /
Disallow /shop/*
Allow /shop/product/*

baiduspider

Rule Path
Allow /
Disallow /shop/*
Allow /shop/product/*

applebot

Rule Path
Allow /
Disallow /shop/*
Allow /shop/product/*

ibm-appconnect/v1

Rule Path
Allow /
Disallow /shop/*
Allow /shop/product/*