spotifyadblocker.live
robots.txt

Robots Exclusion Standard data for spotifyadblocker.live

Resource Scan

Scan Details

Site Domain spotifyadblocker.live
Base Domain spotifyadblocker.live
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-10-03T19:48:51+00:00
Next Scan 2025-12-02T19:48:51+00:00

Last Successful Scan

Scanned2025-07-13T01:24:57+00:00
URL https://spotifyadblocker.live/robots.txt
Redirect https://www.spotifyadblocker.live/robots.txt
Redirect Domain www.spotifyadblocker.live
Redirect Base spotifyadblocker.live
Domain IPs 104.21.19.14, 172.67.184.113, 2606:4700:3036::ac43:b871, 2606:4700:3037::6815:130e
Redirect IPs 104.21.19.14, 172.67.184.113, 2606:4700:3036::ac43:b871, 2606:4700:3037::6815:130e
Response IP 104.21.19.14
Found Yes
Hash 49fd2c5968f3ac598729157608474267be59e87961650955518ffa99dad010a4
SimHash 4c749d7a3747

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /googlebot/

Comments

  • Allow all user agents.
  • User agent "Googlebot" should not crawl the `http://example.com/nogooglebot/` folder or any of its subdirectories.