app.sensortower.com
robots.txt

Robots Exclusion Standard data for app.sensortower.com

Resource Scan

Scan Details

Site Domain app.sensortower.com
Base Domain sensortower.com
Scan Status Ok
Last Scan2024-06-16T10:01:56+00:00
Next Scan 2024-06-30T10:01:56+00:00

Last Scan

Scanned2024-06-16T10:01:56+00:00
URL https://app.sensortower.com/robots.txt
Domain IPs 52.0.168.14, 54.165.121.51
Response IP 54.165.121.51
Found Yes
Hash 233a088610d5ca53068720ad150161510599e7ce7a4381b52c2570c6848926f3
SimHash 205f153bcc53

Groups

*

Rule Path
Disallow /
Allow /$
Allow /overview/
Allow /top-charts
Allow /news-feed
Allow /ios/publisher/
Allow /android/publisher/
Allow /images/
Allow /favicon
Allow /packs/
Allow /assets/
Allow /images/
Allow /sitemap.xml
Allow /robots.txt
Allow *api/ios/apps/
Allow *api/android/apps/
Allow *api/ios/category/
Allow *api/ios/category_rankings*
Allow *api/android/category_rankings*
Allow *api/ios/publisher/publisher_apps*
Allow *api/android/publisher/publisher_apps*
Allow *api/global_alerts*

ahrefsbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrush

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

majestic12

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

digext

Rule Path
Disallow /

rogerbot

Rule Path
Disallow

ccbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

coccoc

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

zumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://app.sensortower.com/sitemap.xml

Comments

  • Do not remove the sections below (Assets & API Endpoints)
  • Search engines will not be able to crawl your *web app*
  • since all page resources are filtered though this file.
  • Assets
  • API Endpoints