atfc.org.uk
robots.txt

Robots Exclusion Standard data for atfc.org.uk

Resource Scan

Scan Details

Site Domain atfc.org.uk
Base Domain atfc.org.uk
Scan Status Ok
Last Scan2024-06-01T03:05:42+00:00
Next Scan 2024-07-01T03:05:42+00:00

Last Scan

Scanned2024-06-01T03:05:42+00:00
URL https://atfc.org.uk/robots.txt
Redirect https://www.atfc.org.uk/robots.txt
Redirect Domain www.atfc.org.uk
Redirect Base atfc.org.uk
Domain IPs 199.34.228.72
Redirect IPs 199.34.228.72
Response IP 199.34.228.72
Found Yes
Hash deb3113abf20824e03a54e633c12f2aea800dba3c3d07e7af7ab823d491fc799
SimHash 881cd8d042b2

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /http%3A//www.betbind.com
Disallow /dynion--men.html
Disallow /menywod--women.html
Disallow /holl-dicircmau--all-teams.html
Disallow /http%3A//www.vaughans.co.uk/store_aberystwyth
Disallow /sefydliad-cpdta--atfc-foundation.html
Disallow /cae-3g--3g-pitch.html
Disallow /d19-ac-academi--u19-and-academy.html
Disallow /rhaglennu--programmes.html
Disallow /19092022css.html
Disallow /cyfleusterau--facilities.html
Disallow /gwybodaeth--information.html
Disallow /siarterau--charters.html
Disallow /dolennau--links.html
Disallow /https%3A//www.atfc.org.uk/home/category/aber-town-women

Other Records

Field Value
sitemap https://www.atfc.org.uk/sitemap.xml