lab.fm
robots.txt

Robots Exclusion Standard data for lab.fm

Resource Scan

Scan Details

Site Domain lab.fm
Base Domain lab.fm
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-14T00:37:50+00:00
Next Scan 2024-08-12T00:37:50+00:00

Last Successful Scan

Scanned2023-04-20T13:53:46+00:00
URL https://lab.fm/robots.txt
Redirect https://www.nme.com/robots.txt
Redirect Domain www.nme.com
Redirect Base nme.com
Domain IPs 18.64.37.116, 18.64.37.46, 18.64.37.89, 18.64.37.95
Redirect IPs 13.35.8.10, 13.35.8.109, 13.35.8.33, 13.35.8.56
Response IP 13.35.8.109
Found Yes
Hash 8d00f156033cf21532db6ae2d926b7b2ab45a99908923f85bc1cb1605ca3f5b4
SimHash 180ac860c11a

Groups

weborama-fetcher

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /discount-codes/discount-codes/visit/*
Disallow /**.html$
Disallow /search/*
Disallow /*search/
Disallow *?s=*
Disallow */utils/*
Disallow */utils/definitions
Disallow */ChildVisitor
Disallow */ProxyVisitor
Disallow */encode
Disallow */decode
Disallow */definitions
Disallow */Message
Disallow */consent-string
Disallow */PlaceholderVisitor
Disallow */0/*
Disallow /nme-video/*
Disallow /musicvideos/*
Disallow /tickets/artist/*
Disallow /tickets/cities/*
Disallow /tickets/event/*
Disallow /tickets/venues/*

Other Records

Field Value
sitemap https://www.nme.com/news-sitemap.xml
sitemap https://www.nme.com/sitemap_index.xml
sitemap https://www.nme.com/discount-codes/sitemap.xml

Comments

  • Block links within GTM JavaScript
  • Others