Log In Sign Up

epicpxls.com
robots.txt

Robots Exclusion Standard data for epicpxls.com

Resource Scan

Scan Details

Site Domain epicpxls.com
Base Domain epicpxls.com
Scan Status Ok
Last Scan2026-03-20T08:47:04+00:00
Next Scan 2026-03-27T08:47:04+00:00

Last Scan

Scanned2026-03-20T08:47:04+00:00
URL https://epicpxls.com/robots.txt
Redirect https://www.epicpxls.com/robots.txt
Redirect Domain www.epicpxls.com
Redirect Base epicpxls.com
Domain IPs 104.21.63.75, 172.67.170.69, 2606:4700:3033::ac43:aa45, 2606:4700:3034::6815:3f4b
Redirect IPs 104.21.63.75, 172.67.170.69, 2606:4700:3033::ac43:aa45, 2606:4700:3034::6815:3f4b
Response IP 104.21.63.75
Found Yes
Hash 09be088962d333e28be11e416e095626b494e52113e712ffeb3f43f3249fe128
SimHash 6940e80bc071

Groups

gptbot

Rule Path
Allow /items
Allow /free
Allow /alternatives
Disallow /admin
Disallow /users
Disallow */download
Disallow */carousel

Other Records

Field Value
crawl-delay 1

anthropic-web

Rule Path
Allow /
Disallow /admin
Disallow /users
Disallow */download
Disallow */carousel

Other Records

Field Value
crawl-delay 1

perplexitybot

Rule Path
Allow /
Disallow /admin
Disallow /users
Disallow */download

Other Records

Field Value
crawl-delay 1

google-extended

Rule Path
Allow /
Disallow /admin
Disallow /users

Other Records

Field Value
crawl-delay 1

ccbot

Rule Path
Allow /
Disallow /admin
Disallow /users

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /admin
Disallow */carousel
Disallow */download
Disallow /items/*/page/*
Disallow */page/*
Disallow */sort/*
Disallow /*?*format=dialog
Disallow /*?*format=js
Disallow /*?*preview_size=
Disallow /*?*preview_text=
Disallow /*?*style=
Disallow /*?*PageSpeed=
Disallow /*?*ref=
Disallow /*?*utm_source=
Disallow /*?*utm_medium=
Disallow /*?*utm_campaign=
Disallow /*?*q=
Back to top

Other Records

Field Value
sitemap https://www.epicpxls.com/sitemap.xml
Back to top

Comments

  • EpicPxls robots.txt
  • AI Crawlers - Welcome with guidelines
  • See also: /llms.txt for detailed AI crawler guidance
  • Default rules for all other crawlers
  • Admin area
  • Don't index carousel links
  • Don't index download links
  • Block pagination pages (crawl budget optimization)
  • Page 2+ are noindexed but blocking prevents wasted crawls
  • Block sort variations (same content, different order)
  • Filter pages (category, platform, type) are legitimate and indexable
  • Canonical tags handle deduplication; sort variations are blocked above
  • Block dialog format URLs (bare modal fragments, no SEO value)
  • These are internal AJAX endpoints that render without <head>/<meta> tags
  • Block junk query parameters (crawl budget optimization)
  • Font preview state — creates 1000s of duplicate URL variations
  • Font style filter with preview creates spam-injected variations
  • PageSpeed noscript artifacts
  • Referral tracking params
  • UTM tracking params (campaign tracking, not content)
  • Internal search results (thin content, infinite variations)
Back to top