Log In Sign Up

pottstownmercury.com
robots.txt

Robots Exclusion Standard data for pottstownmercury.com

Resource Scan

Scan Details

Site Domain pottstownmercury.com
Base Domain pottstownmercury.com
Scan Status Ok
Last Scan2025-11-04T16:25:17+00:00
Next Scan 2025-12-04T16:25:17+00:00

Last Scan

Scanned2025-11-04T16:25:17+00:00
URL http://pottstownmercury.com/robots.txt
Redirect https://www.pottsmerc.com/robots.txt
Redirect Domain www.pottsmerc.com
Redirect Base pottsmerc.com
Domain IPs 96.45.82.238, 96.45.82.64, 96.45.83.150, 96.45.83.7
Redirect IPs 192.0.66.136
Response IP 192.0.66.136
Found Yes
Hash f2eb3c59011dbac19508145fa4a6e70113c2ff1c572b0f7a1bb3113cfdfea292
SimHash fb13915b4ef7

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /cgi-bin/
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /trackback/
Disallow /comments/

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow

oai-searchbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

mozilla/5.0 (compatible; newsroom.bi/0.1; +http://www.newsroom.bi/bot.html)

Rule Path
Allow /

mozilla/5.0 (linux; android 6.0; nexus 5 build/mra51n) applewebkit/537.36 (khtml, like gecko) chrome/87.0.4280.67 mobile safari/537.36 (compatible; mrfcompass-booldog/1.0)

Rule Path
Allow /

mozilla/5.0 (macintosh; intel mac os x 10_9_2) applewebkit/537.36(khtml, like gecko) chrome/35.0.1916.153 safari/537.36 (compatible; mrfcompass-booldog/1.0)

Rule Path
Allow /

mozilla/5.0 (linux; android 6.0.1; nexus 5x build/mmb29p) applewebkit/537.36 (khtml, like gecko) chrome/41.0.2272.96 mobile safari/537.36 (compatible; mrfcompass-marshall/1.0)

Rule Path
Allow /

mozilla/5.0 (linux; android 6.0; nexus 5 build/mra51n) applewebkit/537.36 (khtml, like gecko) chrome/87.0.4280.67 mobile safari/537.36 (compatible; mrfcompass-jukebox/1.0)

Rule Path
Allow /

perplexitybot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

docspotbot

Rule Path
Disallow /

mantisbot

Rule Path
Disallow /

thinkbot

Rule Path
Disallow /

grok-crawler

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.pottsmerc.com/sitemap.xml
Back to top

Comments

  • Our Content is made available for your personal, non-commercial use subject to our Terms of Service here: https://www.medianewsgroup.com/terms-of-use/
  • Any other uses are prohibited, including but not limited to:
  • (1) text and data mining activities under Art. 4 of the EU Directive on Copyright in the Digital Single Market;
  • (2) use of any Content or information available on the Site for purposes of retrieval augmented generation, grounding, training, or development of machine learning models, algorithms, or artificial intelligence (AI) systems, or to generate substitute content or develop any products, services, or technology;
  • (3) caching or archiving the Content; and/or
  • (4) any commercial purposes.
  • Contact us at https://www.pottsmerc.com/contact-us/ for assistance.
  • Sitemap archive
Back to top