turbo.pt
robots.txt

Robots Exclusion Standard data for turbo.pt

Resource Scan

Scan Details

Site Domain turbo.pt
Base Domain turbo.pt
Scan Status Ok
Last Scan2024-09-20T11:59:53+00:00
Next Scan 2024-09-27T11:59:53+00:00

Last Scan

Scanned2024-09-20T11:59:53+00:00
URL https://turbo.pt/robots.txt
Redirect https://www.turbo.pt/robots.txt
Redirect Domain www.turbo.pt
Redirect Base turbo.pt
Domain IPs 104.21.57.219, 172.67.192.115, 2606:4700:3031::6815:39db, 2606:4700:3035::ac43:c073
Redirect IPs 104.21.57.219, 172.67.192.115, 2606:4700:3031::6815:39db, 2606:4700:3035::ac43:c073
Response IP 104.21.57.219
Found Yes
Hash 9cb37f03702b1c05dde4126274a27e12da9e0983805137de772ce9cb60854bc3
SimHash 6804da124791

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.turbo.pt/sitemap_index.xml