memoirmug.com
robots.txt

Robots Exclusion Standard data for memoirmug.com

Resource Scan

Scan Details

Site Domain memoirmug.com
Base Domain memoirmug.com
Scan Status Ok
Last Scan2025-06-06T18:29:55+00:00
Next Scan 2025-06-13T18:29:55+00:00

Last Scan

Scanned2025-06-06T18:29:55+00:00
URL https://memoirmug.com/robots.txt
Domain IPs 162.214.80.167
Response IP 162.214.80.167
Found Yes
Hash dfd48c4f2a564a1b1887e071d5a4ee5d85e765a6cc69ad75adb54b4e3112eb3d
SimHash 8b30f903c59a

Groups

yumbot/1.0

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

*

Rule Path
Disallow */tag/

*

Rule Path
Disallow *?s=

*

Rule Path
Disallow *?cat=

*

Rule Path
Disallow *?p

*

Rule Path
Disallow */search
Disallow */feed
Disallow /search/
Disallow /?s=
Disallow /tag/
Disallow *?utm_source=
Disallow */*/feed
Disallow */tag/
Disallow */search/
Disallow *rss

Other Records

Field Value
sitemap https://memoirmug.com/sitemap.xml
sitemap https://memoirmug.com/news-sitemap.xml

Comments

  • removes tags from crawling
  • removes from crawling anything that starts with a question mark sign
  • new changes to restrict crawling