pct.edu
robots.txt
Robots Exclusion Standard data for pct.edu
Resource Scan
Scan Details
| Site Domain | pct.edu |
| Base Domain | pct.edu |
| Scan Status | Ok |
| Last Scan | 2024-10-28T04:10:50+00:00 |
| Next Scan | 2024-11-27T04:10:50+00:00 |
Last Scan
| Scanned | 2024-10-28T04:10:50+00:00 |
| URL | https://pct.edu/robots.txt |
| Domain IPs | 23.185.0.3, 2620:12a:8000::3, 2620:12a:8001::3 |
| Response IP | 23.185.0.3 |
| Found | Yes |
| Hash | ab5dbbc2e024eb616156b3ad86eda52c293d5f3da288a72777ba7443f60481e4 |
| SimHash | 2d12fd4bc76c |
Groups
*
| Rule | Path |
|---|---|
| Allow | /core/*.css$ |
| Allow | /core/*.css? |
| Allow | /core/*.js$ |
| Allow | /core/*.js? |
| Allow | /core/*.gif |
| Allow | /core/*.jpg |
| Allow | /core/*.jpeg |
| Allow | /core/*.png |
| Allow | /core/*.svg |
| Allow | /profiles/*.css$ |
| Allow | /profiles/*.css? |
| Allow | /profiles/*.js$ |
| Allow | /profiles/*.js? |
| Allow | /profiles/*.gif |
| Allow | /profiles/*.jpg |
| Allow | /profiles/*.jpeg |
| Allow | /profiles/*.png |
| Allow | /profiles/*.svg |
| Disallow | /core/ |
| Disallow | /profiles/ |
| Disallow | /README.md |
| Disallow | /composer/Metapackage/README.txt |
| Disallow | /composer/Plugin/ProjectMessage/README.md |
| Disallow | /composer/Plugin/Scaffold/README.md |
| Disallow | /composer/Plugin/VendorHardening/README.txt |
| Disallow | /composer/Template/README.txt |
| Disallow | /modules/README.txt |
| Disallow | /sites/README.txt |
| Disallow | /themes/README.txt |
| Disallow | /web.config |
| Disallow | /admin/ |
| Disallow | /comment/reply/ |
| Disallow | /filter/tips |
| Disallow | /node/add/ |
| Disallow | /search/ |
| Disallow | /user/register |
| Disallow | /user/password |
| Disallow | /user/login |
| Disallow | /user/logout |
| Disallow | /media/oembed |
| Disallow | /*/media/oembed |
| Disallow | /index.php/admin/ |
| Disallow | /index.php/comment/reply/ |
| Disallow | /index.php/filter/tips |
| Disallow | /index.php/node/add/ |
| Disallow | /index.php/search/ |
| Disallow | /index.php/user/password |
| Disallow | /index.php/user/register |
| Disallow | /index.php/user/login |
| Disallow | /index.php/user/logout |
| Disallow | /index.php/media/oembed |
| Disallow | /index.php/*/media/oembed |
| Disallow | /student-life/orientation/welcome-weekend/ |
| Disallow | /admissions/scholarships/penn-college-scholarship |
| Disallow | /node/ |
| Disallow | /r/ |
| Disallow | /catalog/ |
| Disallow | /email/ |
| Disallow | /academics/career-services/career-fair/ |
| Disallow | /academics/career-services/career-fair/directory |
| Disallow | /academics/career-services/career-fair/register |
| Disallow | /search |
| Disallow | /search/ |
| Disallow | /webform/ |
| Disallow | /academicaffairs/ |
| Disallow | /academicsuccess/ |
| Disallow | /admissions_email/ |
| Disallow | /admin/ |
| Disallow | /advertising/ |
| Disallow | /advidement/ |
| Disallow | /armyROTC/ |
| Disallow | /ARP/ |
| Disallow | /artcamp/ |
| Disallow | /askpenncolleg/ |
| Disallow | /ASPIE2016/ |
| Disallow | /assessment/ |
| Disallow | /athletetraining/ |
| Disallow | /athletics/ |
| Disallow | /bcards/ |
| Disallow | /bursar/ |
| Disallow | /busind/ |
| Disallow | /businesscamp/ |
| Disallow | /careerservices/ |
| Disallow | /catalog/majors/GSP.shtml |
| Disallow | /childcare/ |
| Disallow | /closetohome/ |
| Disallow | /collegehealth/ |
| Disallow | /commencement/ |
| Disallow | /facilitiesandevents/ |
| Disallow | /consumerinfo/ |
| Disallow | /contact/ |
| Disallow | /counseling/ |
| Disallow | /ctm-employee-discount-program |
| Disallow | /currentstudents/ |
| Disallow | /degrees/ |
| Disallow | /degreesthatwork/ |
| Disallow | /digitalcamp/ |
| Disallow | /dining/ |
| Disallow | /disabilityservices/ |
| Disallow | /distance/ |
| Disallow | /ecards/ |
| Disallow | /emergencyResponse/ |
| Disallow | /esl/ |
| Disallow | /etec/ |
| Disallow | /explore/ |
| Disallow | /finaid/ |
| Disallow | /firstyear/ |
| Disallow | /fitness/ |
| Disallow | /foodcamp/ |
| Disallow | /futurestudents/ |
| Disallow | /giving/ |
| Disallow | /glbt/ |
| Disallow | /governance/ |
| Disallow | /green/ |
| Disallow | /healthcamp/ |
| Disallow | /holidaycard/ |
| Disallow | /homecoming/ |
| Disallow | /homeimages/ |
| Disallow | /hospitalityservices/ |
| Disallow | /humanresources/ |
| Disallow | /ia/ |
| Disallow | /international/ |
| Disallow | /intramurals/ |
| Disallow | /its/ |
| Disallow | /iwanttobe/ |
| Disallow | /jobsatpenncollege/ |
| Disallow | /k12/admin/ |
| Disallow | /k12educators/ |
| Disallow | /library/ |
| Disallow | /nacat/ |
| Disallow | /naturalgas/ |
| Disallow | /newHire/ |
| Disallow | /nssc/ |
| Disallow | /och/ |
| Disallow | /offcampus/ |
| Disallow | /oh2015/ |
| Disallow | /oit/ |
| Disallow | /openhouse/ |
| Disallow | /orientation/ |
| Disallow | /pctoday/ |
| Disallow | /photos/ |
| Disallow | /picnic/ |
| Disallow | /pirc/ |
| Disallow | /police/ |
| Disallow | /president/ |
| Disallow | /privacy/ |
| Disallow | /prm/ |
| Disallow | /profile/ |
| Disallow | /psn/ |
| Disallow | /r/ |
| Disallow | /registrar/ |
| Disallow | /requestinfo/ |
| Disallow | /reslife/ |
| Disallow | /rmce/ |
| Disallow | /scholarships/ |
| Disallow | /schools/ |
| Disallow | /scripts/ |
| Disallow | /share/ |
| Disallow | /sis/ |
| Disallow | /smartgirls/ |
| Disallow | /sonsofitaly/ |
| Disallow | /spring/ |
| Disallow | /studentactivities/ |
| Disallow | /studentaffairs/ |
| Disallow | /studentblog/ |
| Disallow | /studentpolicy/ |
| Disallow | /styles/ |
| Disallow | /sunnyportal/ |
| Disallow | /surplus/ |
| Disallow | /testcenter/ |
| Disallow | /tparticulation/ |
| Disallow | /trail/ |
| Disallow | /transferstudents/ |
| Disallow | /victorianhouse/ |
| Disallow | /videos/ |
| Disallow | /visit/ |
| Disallow | /wdce/ |
| Disallow | /wtc/ |
| Disallow | /files/imported/ |
| Disallow | /admissions/financial-aid/scholarships/penn-college-scholarships |
| Disallow | /chef |
| Disallow | /cgi-bin/ |
| Disallow | /files/cgi-bin/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Comments