magna.utah.gov
robots.txt
Robots Exclusion Standard data for magna.utah.gov
Resource Scan
Scan Details
| Site Domain | magna.utah.gov |
| Base Domain | utah.gov |
| Scan Status | Ok |
| Last Scan | 2024-06-12T08:11:05+00:00 |
| Next Scan | 2024-07-12T08:11:05+00:00 |
Last Scan
| Scanned | 2024-06-12T08:11:05+00:00 |
| URL | https://magna.utah.gov/robots.txt |
| Domain IPs | 162.221.183.17 |
| Response IP | 162.221.183.17 |
| Found | Yes |
| Hash | 5131f0cbb0d48f15d73b2f376d1197268c7762e2c53f6f0f839098797e90de46 |
| SimHash | 569a335948f0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /includes/ |
| Disallow | /misc/ |
| Disallow | /modules/ |
| Disallow | /profiles/ |
| Disallow | /scripts/ |
| Disallow | /sites/all/modules/ |
| Disallow | /sites/all/libraries/ |
| Disallow | /sites/all/include/ |
| Disallow | /themes/ |
| Disallow | /CHANGELOG.txt |
| Disallow | /cron.php |
| Disallow | /INSTALL.mysql.txt |
| Disallow | /INSTALL.pgsql.txt |
| Disallow | /INSTALL.sqlite.txt |
| Disallow | /install.php |
| Disallow | /INSTALL.txt |
| Disallow | /LICENSE.txt |
| Disallow | /MAINTAINERS.txt |
| Disallow | /update.php |
| Disallow | /UPGRADE.txt |
| Disallow | /xmlrpc.php |
| Disallow | /admin/ |
| Disallow | /comment/reply/ |
| Disallow | /filter/tips/ |
| Disallow | /node/add/ |
| Disallow | /search/ |
| Disallow | /user/register/ |
| Disallow | /user/password/ |
| Disallow | /user/login/ |
| Disallow | /user/logout/ |
| Disallow | /login |
| Disallow | /?q=admin%2F |
| Disallow | /?q=comment%2Freply%2F |
| Disallow | /?q=filter%2Ftips%2F |
| Disallow | /?q=node%2Fadd%2F |
| Disallow | /?q=search%2F |
| Disallow | /?q=user%2Fpassword%2F |
| Disallow | /?q=user%2Fregister%2F |
| Disallow | /?q=user%2Flogin%2F |
| Disallow | /?q=user%2Flogout%2F |
| Disallow | /?q=login%2F |
| Disallow | /?q=sites%2Fall%2Fmodules%2F |
| Disallow | /?q=sites%2Fall%2Flibraries%2F |
| Disallow | /?q=sites%2Fall%2Finclude%2F |
| Disallow | /printmail/ |
| Disallow | /?q=printmail%2F |
| Disallow | /printpdf/ |
| Disallow | /?q=printpdf%2F |
| Disallow | /print/ |
| Disallow | /?q=print%2F |
| Disallow | /citizenrequest/ |
| Disallow | /?q=citizenrequest%2F |
| Disallow | /calendar.ics |
| Disallow | /?q=calendar.ics |
| Disallow | /banner/ |
| Disallow | /?q=banner%2F |
| Disallow | /simplenews/ |
| Disallow | /?q=simplenews%2F |
| Disallow | /faq/ |
| Disallow | /?q=faq%2F |
| Disallow | /standard-location/ |
| Disallow | /?q=standard-location%2F |
| Disallow | /intranet/ |
| Disallow | /?q=intranet%2F |
| Disallow | /banner/ |
| Disallow | /?q=banner%2F |
| Disallow | /rss/ |
| Disallow | /?q=rss%2F |
| Disallow | /data/ |
| Disallow | /?q=data%2F |
| Disallow | /dept/ |
| Disallow | /?q=dept%2F |
| Disallow | /sites/all/modules/ |
| Disallow | /?q=sites%2Fall%2Fmodules%2F |
| Disallow | /sites/all/libraries/ |
| Disallow | /?q=sites%2Fall%2Flibraries%2F |
| Disallow | /user/login |
| Disallow | /ticket-administration |
| Disallow | /?q=%2Fticket-administration |
| Disallow | /ticket_status |
| Disallow | /?q=%2Fticket_status |
| Disallow | /directory/xls |
| Disallow | /?q=%2Fdirectory%2Fxls |
| Disallow | /calendar/day/ |
| Disallow | /q%3Dcalendar/day/ |
| Disallow | /calendar/week/ |
| Disallow | /q%3Dcalendar/week/ |
| Disallow | /calendar/year/ |
| Disallow | /q%3Dcalendar/year/ |
| Disallow | /dashboard |
| Disallow | /?q=dashboard%2F |
| Disallow | /my-business-licenses |
| Disallow | /?q=my-business-licenses%2F |
| Disallow | /my-business-license-orders |
| Disallow | /?q=my-business-license-orders%2F |
| Disallow | /bsns-license-renewal-search |
| Disallow | /?q=bsns-license-renewal-search%2F |
| Disallow | /bsns-license/ |
| Disallow | /?q=bsns-license%2F |
| Disallow | /mybusiness |
| Disallow | /?q=mybusiness%2F |
| Disallow | /business-registration-* |
| Disallow | /?q=business-registration-* |
| Disallow | /participantprofile/ |
| Disallow | /?q=participantprofile%2F |
| Disallow | /taxonomy/term/ |
| Disallow | /?q=taxonomy%2Fterm%2F |
| Disallow | /sites/default/files/webform/ |
| Disallow | /?q=sites%2Fdefault%2Ffiles%2Fwebform%2F |
| Disallow | /interested-parties/ |
| Disallow | /?q=interested-parties%2F |
| Allow | /core/*.css$ |
| Allow | /core/*.js$ |
| Disallow | /core/ |
| Disallow | /profiles/ |
| Disallow | /README.txt |
| Disallow | /web.config |
| Disallow | /admin/ |
| Disallow | /comment/reply/ |
| Disallow | /filter/tips/ |
| Disallow | /node/add/ |
| Disallow | /search/ |
| Disallow | /user/register/ |
| Disallow | /user/password/ |
| Disallow | /user/login/ |
| Disallow | /user/logout/ |
| Disallow | /index.php/admin/ |
| Disallow | /index.php/comment/reply/ |
| Disallow | /index.php/filter/tips/ |
| Disallow | /index.php/node/add/ |
| Disallow | /index.php/search/ |
| Disallow | /index.php/user/password/ |
| Disallow | /index.php/user/register/ |
| Disallow | /index.php/user/login/ |
| Disallow | /index.php/user/logout/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |
Comments