jleague.co
robots.txt
Robots Exclusion Standard data for jleague.co
Resource Scan
Scan Details
Site Domain | jleague.co |
Base Domain | jleague.co |
Scan Status | Ok |
Last Scan | 2024-05-30T21:02:47+00:00 |
Next Scan | 2024-06-29T21:02:47+00:00 |
Last Scan
Scanned | 2024-05-30T21:02:47+00:00 |
URL | https://jleague.co/robots.txt |
Redirect | https://www.jleague.co/robots.txt |
Redirect Domain | www.jleague.co |
Redirect Base | jleague.co |
Domain IPs | 172.66.40.175, 172.66.43.81, 2606:4700:3108::ac42:28af, 2606:4700:3108::ac42:2b51 |
Redirect IPs | 172.66.40.175, 172.66.43.81, 2606:4700:3108::ac42:28af, 2606:4700:3108::ac42:2b51 |
Response IP | 172.66.40.175 |
Found | Yes |
Hash | 20e0fa08747c57c765f16bc521d0cd68cc502557332f41a8c3257d2d6b2268b6 |
SimHash | 297f8e40a38b |
Groups
*
Rule | Path |
---|---|
Disallow | /documents/ |
Disallow | /widgets/ |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
Other Records
Field | Value |
---|---|
sitemap | https://www.jleague.co/sitemap.xml |
sitemap | https://www.jleague.co/clubs/sitemap.xml |
sitemap | https://www.jleague.co/players/sitemap.xml |
sitemap | https://www.jleague.co/match/sitemap.xml |
sitemap | https://www.jleague.co/standings/sitemap.xml |