swgfl.org.uk
robots.txt

Robots Exclusion Standard data for swgfl.org.uk

Resource Scan

Scan Details

Site Domain swgfl.org.uk
Base Domain swgfl.org.uk
Scan Status Ok
Last Scan2025-12-07T10:49:38+00:00
Next Scan 2026-01-06T10:49:38+00:00

Last Scan

Scanned2025-12-07T10:49:38+00:00
URL https://swgfl.org.uk/robots.txt
Domain IPs 194.164.92.131
Response IP 194.164.92.131
Found Yes
Hash 491efaa381d233b52dccd5d091e9f324dc7780ff72945339eac3a0a188734a56
SimHash 8804d01fe1ad

Groups

*

Rule Path
Allow /
Disallow /products/projectevolve-safeguarding/
Disallow /products/wellbeing/
Disallow /training/online-safety-live/resources/england/
Disallow /helplines/stopncii/api/
Disallow /resources/international-schools-online-safety-policy-templates/
Disallow /resources/safe-remote-learning/dfe-laptops-for-disadvantaged-children-primary/
Disallow /resources/safe-remote-learning/dfe-laptops-for-disadvantaged-children-programme-secondary/
Disallow /resources/digital-wellbeing/
Disallow /projects/scotland/webinars/j455qo43bc116qa2w0e01u3z6z965jmv/
Disallow /projects/scotland/webinars/j455qo43bc11235640e01u3z6z965jmv/safe-and-empowered-workbook/
Disallow /projects/project-ember-support-page/
Disallow /projects/project-ember-research-into-children-and-young-people-s-online-experiences/
Disallow /magazine/cymru/
Disallow /login/
Disallow /admin/
Disallow /shopify-callback/
Disallow /training-and-events/
Disallow /account/
Disallow /whisper/
Disallow /contact/
Disallow /wellbeing/
Disallow /logout/
Disallow /search/
Disallow /page-not-found/
Disallow /page-not-available/
Disallow /unauthorised/
Disallow /500-internal-server-error/
Disallow /bots/

cyotekhttp

Rule Path
Disallow /

cyotekwebcopy

Rule Path
Disallow /

powershell

Rule Path
Disallow /

github

Rule Path
Disallow /

grub

Rule Path
Disallow /

amazon-kendra-customer-id

Rule Path
Disallow /

client

Rule Path
Disallow /

deno

Rule Path
Disallow /

dart

Rule Path
Disallow /

cpp-httplib

Rule Path
Disallow /

cakephp

Rule Path
Disallow /

bun

Rule Path
Disallow /

ruby

Rule Path
Disallow /

rest

Rule Path
Disallow /

wget

Rule Path
Disallow /

curl

Rule Path
Disallow /

java

Rule Path
Disallow /

axios

Rule Path
Disallow /

postmanruntime

Rule Path
Disallow /

grabber

Rule Path
Disallow /

jsdom

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

okhttp

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

node-fetch

Rule Path
Disallow /

headlesschrome

Rule Path
Disallow /

Other Records

Field Value
sitemap https://swgfl.org.uk/sitemap.xml