dgaauto.com
robots.txt

Robots Exclusion Standard data for dgaauto.com

Resource Scan

Scan Details

Site Domain dgaauto.com
Base Domain dgaauto.com
Scan Status Ok
Last Scan2024-05-26T04:45:12+00:00
Next Scan 2024-06-25T04:45:12+00:00

Last Scan

Scanned2024-05-26T04:45:12+00:00
URL https://dgaauto.com/robots.txt
Redirect https://www.dgaauto.com/robots.txt
Redirect Domain www.dgaauto.com
Redirect Base dgaauto.com
Domain IPs 199.34.228.77
Redirect IPs 199.34.228.77
Response IP 199.34.228.77
Found Yes
Hash dda0a8a391c2f722c1ee04a379b8033a275e62a6dfdf0ad6ee629247b6445f04
SimHash 0710d2e60abb

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /mydga.html
Disallow /lucy.html
Disallow /app.html
Disallow /availability.html
Disallow /exit.html
Disallow /gc.html
Disallow /legal.html
Disallow /https%3A//apps.dgaauto.com/login
Disallow /terms.html
Disallow /morganauto.html
Disallow /techassessment.html
Disallow /setup.html
Disallow /feedback.html
Disallow /upsell.html
Disallow /alliedtechassessment.html
Disallow /confirmation.html
Disallow /democonfirmation.html
Disallow /wfhspecs.html
Disallow /application-form.html

Other Records

Field Value
sitemap https://www.dgaauto.com/sitemap.xml