givefreely.com
robots.txt

Robots Exclusion Standard data for givefreely.com

Resource Scan

Scan Details

Site Domain givefreely.com
Base Domain givefreely.com
Scan Status Ok
Last Scan2024-11-11T06:51:34+00:00
Next Scan 2024-12-11T06:51:34+00:00

Last Scan

Scanned2024-11-11T06:51:34+00:00
URL https://givefreely.com/robots.txt
Domain IPs 76.76.21.21
Response IP 76.76.21.21
Found Yes
Hash 9db0f5941c88dbbc78cecd1bd899acac6db1d0faa9188d1b61b1b8436d7937a2
SimHash 4954d540e531

Groups

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://givefreely.com/sitemap.xml
sitemap https://givefreely.com/charity-directory/sitemap-0.xml
sitemap https://givefreely.com/charity-directory/sitemap-1.xml

Comments

  • Allow all crawlers