epa.gov
robots.txt
Robots Exclusion Standard data for epa.gov
Resource Scan
Scan Details
Site Domain | epa.gov |
Base Domain | epa.gov |
Scan Status | Ok |
Last Scan | 2024-04-15T15:04:09+00:00 |
Next Scan | 2024-05-15T15:04:09+00:00 |
Last Scan
Scanned | 2024-04-15T15:04:09+00:00 |
URL | https://epa.gov/robots.txt |
Redirect | https://www.epa.gov/robots.txt |
Redirect Domain | www.epa.gov |
Redirect Base | epa.gov |
Domain IPs | 134.67.21.34, 2620:117:506f:15::f022 |
Redirect IPs | 108.138.189.15, 108.138.189.50, 108.138.189.56, 108.138.189.87, 2600:9000:269a:2800:10:f3db:af40:93a1, 2600:9000:269a:3000:10:f3db:af40:93a1, 2600:9000:269a:3200:10:f3db:af40:93a1, 2600:9000:269a:5000:10:f3db:af40:93a1, 2600:9000:269a:5e00:10:f3db:af40:93a1, 2600:9000:269a:9e00:10:f3db:af40:93a1, 2600:9000:269a:a000:10:f3db:af40:93a1, 2600:9000:269a:de00:10:f3db:af40:93a1 |
Response IP | 18.165.171.121 |
Found | Yes |
Hash | 4f8709bc9234453e778ab19cfba611bca6102173d63d0bdaca3adb3a595590d6 |
SimHash | 3d94bd5bc760 |
Groups
*
Rule | Path |
---|---|
Allow | /core/*.css$ |
Allow | /core/*.css? |
Allow | /core/*.js$ |
Allow | /core/*.js? |
Allow | /core/*.gif |
Allow | /core/*.jpg |
Allow | /core/*.jpeg |
Allow | /core/*.png |
Allow | /core/*.svg |
Allow | /profiles/*.css$ |
Allow | /profiles/*.css? |
Allow | /profiles/*.js$ |
Allow | /profiles/*.js? |
Allow | /profiles/*.gif |
Allow | /profiles/*.jpg |
Allow | /profiles/*.jpeg |
Allow | /profiles/*.png |
Allow | /profiles/*.svg |
Disallow | /core/ |
Disallow | /profiles/ |
Disallow | /README.txt |
Disallow | /web.config |
Disallow | /admin/ |
Disallow | /comment/reply/ |
Disallow | /filter/tips |
Disallow | /node/add/ |
Disallow | /search/ |
Disallow | /user/register/ |
Disallow | /user/password/ |
Disallow | /user/login/ |
Disallow | /user/logout/ |
Disallow | /index.php/admin/ |
Disallow | /index.php/comment/reply/ |
Disallow | /index.php/filter/tips |
Disallow | /index.php/node/add/ |
Disallow | /index.php/search/ |
Disallow | /index.php/user/password/ |
Disallow | /index.php/user/register/ |
Disallow | /index.php/user/login/ |
Disallow | /index.php/user/logout/ |
Disallow | /themes/ |
Allow | /themes/*.css$ |
Allow | /themes/*.css? |
Allow | /themes/*.js$ |
Allow | /themes/*.js? |
Allow | /themes/*.gif |
Allow | /themes/*.jpg |
Allow | /themes/*.jpeg |
Allow | /themes/*.png |
Allow | /themes/*.svg |
Comments