Log In Sign Up

prodigits.co.uk
robots.txt

Robots Exclusion Standard data for prodigits.co.uk

Resource Scan

Scan Details

Site Domain prodigits.co.uk
Base Domain prodigits.co.uk
Scan Status Ok
Last Scan2024-11-14T05:04:48+00:00
Next Scan 2024-11-21T05:04:48+00:00

Last Scan

Scanned2024-11-14T05:04:48+00:00
URL https://prodigits.co.uk/robots.txt
Domain IPs 198.16.70.234, 198.16.70.235, 198.16.70.237
Response IP 198.16.70.237
Found Yes
Hash 590479426f50a7dd7b9373b346f69fc9a0182f65cbe58d585a0069295e81c8d1
SimHash 133c53756e4a

Groups

boardreader

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

exabot

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

linkdexbot/2.1

Rule Path
Disallow /

linkdexbot/2.0

Rule Path
Disallow /

linkdexbot/2.2

Rule Path
Disallow /

linkdex.com/v2.2

Rule Path
Disallow /

linkdex.com/v2.1

Rule Path
Disallow /

linkdex.com/v2.0

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /
Back to top

Warnings

  • 2 invalid lines.
Back to top