pg-azure-to-azure-dev.azurewebsites.net
robots.txt

Robots Exclusion Standard data for pg-azure-to-azure-dev.azurewebsites.net

Resource Scan

Scan Details

Site Domain pg-azure-to-azure-dev.azurewebsites.net
Base Domain pg-azure-to-azure-dev.azurewebsites.net
Scan Status Ok
Last Scan2025-10-09T12:26:04+00:00
Next Scan 2025-10-16T12:26:04+00:00

Last Scan

Scanned2025-10-09T12:26:04+00:00
URL https://pg-azure-to-azure-dev.azurewebsites.net/robots.txt
Domain IPs 20.119.8.37
Response IP 20.119.8.37
Found Yes
Hash ec1865e9c85bbd9b89f47f42e9b10cd1917bf6965a5b4dbcba59da81605f6ceb
SimHash fd211516fb89

Groups

*

Rule Path
Disallow /nonexistent-directory/
Disallow /old-page/
Disallow /javascript/
Disallow /images/
Disallow /Pages/
Disallow /MyPG/
Disallow /admin/
Disallow /Admin/
Disallow /Tournaments/Admin/
Disallow /css/
Disallow /Error/
Disallow /requestinvite/Admin/
Disallow /Trackman/
Disallow /MyPg/
Disallow /PGBAAdmin/
Disallow /events/tournaments/rosterentry/team.aspx
Disallow /DiamondKast/GameSandBox.aspx
Disallow /CRM/
Disallow /Players/PlayerProfile_Articles.aspx
Disallow /Players/PlayerProfile_Bio.aspx
Disallow /Players/PlayerProfile_CollegeReports.aspx
Disallow /Players/PlayerProfile_ContactInfo.aspx
Disallow /Players/PlayerProfile_Events.aspx
Disallow /Players/PlayerProfile_Stats.aspx
Disallow /Players/PlayerProfile_TrophyRoom.aspx
Disallow /Players/PlayerProfile_Videos.aspx
Disallow /PGBA/Manage/Teams/Team.aspx
Disallow /PGBA/Manage/Teams/Teams.aspx
Disallow /PGBA/Manage/Orgs/Org.aspx
Disallow /PGBA/Manage/Orgs/Orgs.aspx

Other Records

Field Value
crawl-delay 5