Log In Sign Up

fec.gov
robots.txt

Robots Exclusion Standard data for fec.gov

Resource Scan

Scan Details

Site Domain fec.gov
Base Domain fec.gov
Scan Status Ok
Last Scan2024-06-08T06:13:32+00:00
Next Scan 2024-07-08T06:13:32+00:00

Last Scan

Scanned2024-06-08T06:13:32+00:00
URL https://fec.gov/robots.txt
Redirect https://www.fec.gov/robots.txt
Redirect Domain www.fec.gov
Redirect Base fec.gov
Domain IPs 65.132.39.254
Redirect IPs 18.164.154.117, 18.164.154.20, 18.164.154.24, 18.164.154.30
Response IP 3.160.246.108
Found Yes
Hash 4987e5db77131442ce9830a75f9b55dad3bc23e26d9028f3852739d3f45269a4
SimHash c1383f03a313

Groups

usasearch

Rule Path
Allow /
Disallow /search/?*
Disallow /data/legal/search/?*
Disallow /data/search/?*

Other Records

Field Value
crawl-delay 2

*

Rule Path
Disallow /data/allocated-federal-nonfederal-disbursements/?*
Disallow /data/candidates/?*
Disallow /data/candidates/house/?*
Disallow /data/candidates/president/?*
Disallow /data/candidates/senate/?*
Disallow /data/committees/?*
Disallow /data/committees/pac-party/?*
Disallow /data/communication-costs/?*
Disallow /data/debts/?*
Disallow /data/disbursements/?*
Disallow /data/electioneering-communications/?*
Disallow /data/filings/?*
Disallow /data/independent-expenditures/?*
Disallow /data/legal/search/?*
Disallow /data/legal/search/advisory-opinions/?*
Disallow /data/legal/search/murs/?*
Disallow /data/loans/?*
Disallow /data/party-coordinated-expenditures/?*
Disallow /data/receipts/?*
Disallow /data/receipts/individual-contributions/?*
Disallow /data/reports/house-senate/?*
Disallow /data/reports/presidential/?*
Disallow /data/search/?*
Disallow /search/?*

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.fec.gov/sitemap-wagtail.xml
sitemap https://www.fec.gov/resources/cms-content/documents/sitemap_pdf.xml
sitemap https://www.fec.gov/resources/cms-content/documents/sitemap_html.xml
Back to top