Log In Sign Up

songkick.com
robots.txt

Robots Exclusion Standard data for songkick.com

Resource Scan

Scan Details

Site Domain songkick.com
Base Domain songkick.com
Scan Status Ok
Last Scan2024-11-13T10:11:13+00:00
Next Scan 2024-11-20T10:11:13+00:00

Last Scan

Scanned2024-11-13T10:11:13+00:00
URL https://songkick.com/robots.txt
Redirect https://www.songkick.com/robots.txt
Redirect Domain www.songkick.com
Redirect Base songkick.com
Domain IPs 34.142.20.234
Redirect IPs 151.101.130.217, 151.101.194.217, 151.101.2.217, 151.101.66.217
Response IP 199.232.46.217
Found Yes
Hash 926ee5be8d028a093fae39bf7277857858f279c073221a0831c7a71e0ead1f8f
SimHash 6a52f1484dc3

Groups

blexbot

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

import.io

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

netseer

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

botify

Rule Path
Disallow /

blekkobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seznambot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /44522520/*
Disallow /announcements/seen
Disallow /artists/*/images/new
Disallow /artists/*/videos/new
Disallow /behaviour
Disallow /concerts/*/edit
Disallow /concerts/*/images/new
Disallow /concerts/*/posters/new
Disallow /concerts/*/setlists/*
Disallow /concerts/*/videos/new
Disallow /concerts/new
Disallow /festival_instances/new
Disallow /festivals/*/edit
Disallow /festivals/*/id/*/images/new
Disallow /festivals/*/id/*/posters/new
Disallow /festivals/*/id/*/setlists/*
Disallow /festivals/*/id/*/videos/new
Disallow /images/new
Disallow /images/api/
Disallow /iphone/
Disallow /light_signups/new
Disallow /password_reset_requests
Disallow /posters/new
Disallow /problems/new
Disallow /record-share
Disallow /rx
Disallow /search
Disallow /session/create
Disallow /session/filter_metro_area/edit
Disallow /session/new
Disallow /store/events/*/unavailable_allocations
Disallow /store/reservation_orders*
Disallow /store/payments*
Disallow /store/confirmation
Disallow /tickets/
Disallow /tracker
Disallow /trackings
Disallow /venues/*/edit
Disallow /venues/*/images/new
Allow /users/*/calendars.ics
Disallow /venues/*/videos/new
Disallow /venues/new
Disallow /videos/new
Disallow /harming/humans
Disallow /ignoring/human/orders
Disallow /harm/to/self
Disallow /artists/9115824
Disallow /artists/9115824-songkickloadtesting1
Disallow /events/30168404
Disallow /events/30168429
Disallow /events/30168489
Disallow /events/30168499
Disallow /events/30168504
Disallow /events/30168539
Disallow /events/30168549
Disallow /events/30168554
Disallow /events/30168559
Disallow /events/30168564
Disallow /events/30181959
Disallow /events/30181969
Disallow /events/30181984
Disallow /events/30181994
Disallow /events/30182004
Disallow /events/30182014
Disallow /events/30182029
Disallow /events/30182039
Disallow /events/30182044
Disallow /events/30182049
Disallow /events/30181949
Disallow /events/30181964
Disallow /events/30181989
Disallow /events/30181999
Disallow /events/30182009
Disallow /events/30182034
Disallow /events/30182249
Disallow /events/30182254
Disallow /events/30182264
Disallow /events/30182274
Disallow /events/30182284
Disallow /events/30182289
Disallow /events/30182294
Disallow /events/30182559
Disallow /events/30182584
Disallow /events/30182594
Disallow /events/30182644
Disallow /events/30182559
Disallow /events/30182694
Disallow /events/30182774
Disallow /page-view.png
Disallow /*/utils/callbackRegistryFactory
Disallow /*/id/utils
Disallow /*/utils/utils
Disallow /*/id/child/Message
Disallow /*/id/Message
Disallow /*/id/utils/constants
Disallow /*/utils/constants
Disallow /*/child/ChildVisitor
Disallow /*/child/makeChildMessageListener
Disallow /*/child/Message
Disallow /*/countries/child/makeChildMessageListener
Disallow /*/countries/crossDomain
Disallow /*/countries/Message
Disallow /*/countries/utils
Disallow /*/crossDomain
Disallow /*/id/child/ChildVisitor
Disallow /*/id/child/makeChildMessageListener
Disallow /*/id/crossDomain
Disallow /*/id/strategies/ProxyVisitor
Disallow /*/id/units/crossDomain
Disallow /*/id/units/version
Disallow /*/Message
Disallow /*/strategies/LocalVisitor
Disallow /*/strategies/PlaceholderVisitor
Disallow /*/strategies/ProxyVisitor
Disallow /*/units/crossDomain
Disallow /*/units/makeDestinationPublishing
Disallow /*/units/version
Disallow /*/utils
Disallow /*/utils/asyncParallelApply
Disallow /*/utils/enums
Disallow /*/utils/getDomain
Disallow /*/utils/polyfills
Disallow /*/utils/asyncParallelApply
Disallow /*/units/makeCorsRequest
Disallow /*/id/units/makeCorsRequest
Disallow /*/id/units/makeDestinationPublishing
Disallow /*/calendar/countries/units/makeCorsRequest
Back to top

Other Records

Field Value
sitemap https://www.songkick.com/sitemap.xml
Back to top

Comments

  • If you add a new exclusion to this file please ensure that route is removed
  • from the sitemap.xml.
  • The songkick.com sitemaps are generated by this project through cron
  • regularly, and stored in S3 via media-service whence they can be served
  • directly by apache. see the README for more
  • These rules are specifically to prevent google from crawling our adobe launch script
Back to top

Warnings

  • 2 invalid lines.
Back to top