sec.gov
robots.txt
Robots Exclusion Standard data for sec.gov
Resource Scan
Scan Details
Site Domain | sec.gov |
Base Domain | sec.gov |
Scan Status | Ok |
Last Scan | 2024-04-25T15:52:47+00:00 |
Next Scan | 2024-05-25T15:52:47+00:00 |
Last Scan
Scanned | 2024-04-25T15:52:47+00:00 |
URL | https://sec.gov/robots.txt |
Redirect | https://www.sec.gov/robots.txt |
Redirect Domain | www.sec.gov |
Redirect Base | sec.gov |
Domain IPs | 184.25.221.70, 2600:1413:5000:693::2374, 2600:1413:5000:699::2374 |
Redirect IPs | 104.83.197.188, 2600:1413:b000:683::2374, 2600:1413:b000:69b::2374 |
Response IP | 184.25.221.70 |
Found | Yes |
Hash | ac42fff23a8c49c6385b4aba8c2561388c823515683b79ec51b982a987d7a597 |
SimHash | ac92bd1bc768 |
Groups
*
Rule | Path |
---|---|
Allow | /core/*.css$ |
Allow | /core/*.css? |
Allow | /core/*.js$ |
Allow | /core/*.js? |
Allow | /core/*.gif |
Allow | /core/*.jpg |
Allow | /core/*.jpeg |
Allow | /core/*.png |
Allow | /core/*.svg |
Allow | /profiles/*.css$ |
Allow | /profiles/*.css? |
Allow | /profiles/*.js$ |
Allow | /profiles/*.js? |
Allow | /profiles/*.gif |
Allow | /profiles/*.jpg |
Allow | /profiles/*.jpeg |
Allow | /profiles/*.png |
Allow | /profiles/*.svg |
Disallow | /core/ |
Disallow | /profiles/ |
Disallow | /README.txt |
Disallow | /web.config |
Disallow | /admin/ |
Disallow | /comment/reply/ |
Disallow | /filter/tips |
Disallow | /node/add/ |
Disallow | /search/ |
Disallow | /user/register |
Disallow | /user/password |
Disallow | /user/login |
Disallow | /user/logout |
Disallow | /media/oembed |
Disallow | /*/media/oembed |
Disallow | /index.php/admin/ |
Disallow | /index.php/comment/reply/ |
Disallow | /index.php/filter/tips |
Disallow | /index.php/node/add/ |
Disallow | /index.php/search/ |
Disallow | /index.php/user/password |
Disallow | /index.php/user/register |
Disallow | /index.php/user/login |
Disallow | /index.php/user/logout |
Disallow | /index.php/media/oembed |
Disallow | /index.php/*/media/oembed |
Allow | /Archives/edgar/data |
Disallow | /Archives/bin |
Disallow | /Archives/etc |
Disallow | /Archives/usr |
Disallow | /cgi-bin |
Disallow | /bin |
Disallow | /Archives/edgar/vprr/XXXX |
Disallow | /Archives/edgar/vprr/vprr_removal |
Disallow | /Archives/edgar/vprr/bin |
Disallow | /nb |
Disallow | /video/live |
Disallow | /Archives/edgar/data/1473971/000109181814000042/ex101002.gif |
Disallow | /node |
Disallow | /spotlight/equity-market-structure-advisory-committee.shtml |
Disallow | /additional-resources/news-alerts/press-releases |
Disallow | /additional-resources/news-alerts/press-releases/* |
Disallow | /additional-resources/news-alerts/selected-press-releases |
Disallow | /additional-resources/news-alerts/alerts-bulletins/updated-investor-alert-be-lookout-advance-fee |
Disallow | /additional-resources/news-alerts/alerts-bulletins/updated-investor-bulletin-accredited-investors |
Disallow | /additional-resources/news-alerts/investor-alerts/investor-alert-beware-claims-sec-has-approved |
Disallow | /additional-resources/news-alerts/investor-bulletins/investor-bulletin-escheatment-process |
Disallow | /additional-resources/news-alerts/press-releases/elon-musk-charged-securities-fraud-misleading-tweets |
Other Records
Field | Value |
---|---|
sitemap | https://www.sec.gov/sec-sitemap.xml |
sitemap | https://www.sec.gov/sitemap/sitemap-index.xml |
sitemap | https://www.investor.gov/sitemap.xml |
Comments