vertical-magazine.com
robots.txt

Robots Exclusion Standard data for vertical-magazine.com

Resource Scan

Scan Details

Site Domain vertical-magazine.com
Base Domain vertical-magazine.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-26T11:48:12+00:00
Next Scan 2024-06-25T11:48:12+00:00

Last Successful Scan

Scanned2024-04-27T11:46:57+00:00
URL https://vertical-magazine.com/robots.txt
Redirect https://www.vertical-magazine.com/robots.txt
Redirect Domain www.vertical-magazine.com
Redirect Base vertical-magazine.com
Domain IPs 104.21.89.190, 172.67.191.8, 2606:4700:3031::ac43:bf08, 2606:4700:3035::6815:59be
Redirect IPs 104.21.89.190, 172.67.191.8, 2606:4700:3031::ac43:bf08, 2606:4700:3035::6815:59be
Response IP 104.21.89.190
Found Yes
Hash 85886df9fe23b22b5f2f8d860e35a8eb52fea50dd5a6eb1ab63d362237cdd289
SimHash 2e72d8408751

Groups

*

Rule Path
Disallow /admin/
Disallow /widget/
Disallow /xwidget/
Disallow *sort%3D*
Disallow *resultat-recherche*
Disallow *resultats-recherche*
Disallow /app-store

nutch

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.vertical-magazine.com/sitemap.xml

Comments

  • User rules