phillytalk.com
robots.txt

Robots Exclusion Standard data for phillytalk.com

Resource Scan

Scan Details

Site Domain phillytalk.com
Base Domain phillytalk.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-10T03:37:32+00:00
Next Scan 2025-11-17T03:37:32+00:00

Last Successful Scan

Scanned2025-10-31T10:26:29+00:00
URL https://phillytalk.com/robots.txt
Domain IPs 2600:4c00:400:7::130, 67.222.31.238
Response IP 67.222.31.238
Found Yes
Hash 5f034f6872f46186bd30dc4b55775a16b4bd32154f3efa65f9b1a46db06daeb5
SimHash 4a3c5778c672

Groups

blexbot

Rule Path
Disallow /

izsearch.com

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

*

Rule Path
Allow /*.js*
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*
Allow /components/*.js
Allow /components/*.css
Allow /modules/*.js
Allow /modules/*.css
Allow /plugins/*.js
Allow /plugins/*.css
Allow /templates/*.js
Allow /templates/*.css
Allow /component/osmap/?view=xml&id=1&format=xml
Disallow /*?
Disallow /*%3D
Disallow /administrator/
Disallow /wp-admin/
Disallow /api/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://phillytalk.com/site-info/site-map?view=html&id=1

Comments

  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • User-agent: AhrefsSiteAudit
  • Allow: /
  • User-agent: AhrefsBot
  • Allow: /