podnews.net
robots.txt

Robots Exclusion Standard data for podnews.net

Resource Scan

Scan Details

Site Domain podnews.net
Base Domain podnews.net
Scan Status Ok
Last Scan2024-09-21T13:37:50+00:00
Next Scan 2024-10-05T13:37:50+00:00

Last Scan

Scanned2024-09-21T13:37:50+00:00
URL https://podnews.net/robots.txt
Domain IPs 108.156.133.111, 108.156.133.39, 108.156.133.69, 108.156.133.72, 2600:9000:2755:1000:1e:d028:1500:93a1, 2600:9000:2755:1600:1e:d028:1500:93a1, 2600:9000:2755:4000:1e:d028:1500:93a1, 2600:9000:2755:6600:1e:d028:1500:93a1, 2600:9000:2755:9800:1e:d028:1500:93a1, 2600:9000:2755:9a00:1e:d028:1500:93a1, 2600:9000:2755:ac00:1e:d028:1500:93a1, 2600:9000:2755:b400:1e:d028:1500:93a1
Response IP 108.156.133.39
Found Yes
Hash eca028344a138242d527e9c3dfc0b7089dc5eba15ab36a66a0748b6eb8292ecb
SimHash cc44cb736557

Groups

*

Rule Path
Disallow */listen/episode/
Disallow /imgbuild/*
Disallow /authorize_interact*
Disallow /api/fetchrss/*
Allow /

Other Records

Field Value
sitemap https://podnews.net/sitemap.xml
sitemap https://podnews.net/sitemap-podcasts.xml
sitemap https://podnews.net/sitemap-directory.xml
sitemap https://podnews.net/sitemap-events.xml

Comments

  • individual episode links
  • cache building
  • Stop with your random JavaScript crawling, Googlebot
  • rss proxies