Log In Sign Up

meetyu.app
robots.txt

Robots Exclusion Standard data for meetyu.app

Resource Scan

Scan Details

Site Domain meetyu.app
Base Domain meetyu.app
Scan Status Ok
Last Scan2025-12-15T20:10:10+00:00
Next Scan 2026-01-14T20:10:10+00:00

Last Scan

Scanned2025-12-15T20:10:10+00:00
URL https://meetyu.app/robots.txt
Redirect https://www.meetyu.app/robots.txt
Redirect Domain www.meetyu.app
Redirect Base meetyu.app
Domain IPs 104.21.69.176, 172.67.210.253, 2606:4700:3035::6815:45b0, 2606:4700:3035::ac43:d2fd
Redirect IPs 104.21.69.176, 172.67.210.253, 2606:4700:3035::6815:45b0, 2606:4700:3035::ac43:d2fd
Response IP 172.67.210.253
Found Yes
Hash 1d1dcf873ab578b61af4ca2b884943ac852bd6d7bee61ca8e1a481613c43ea21
SimHash 21455cc2a7db

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot
bingbot
slurp
duckduckbot
baiduspider
yandexbot
facebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-content/cache/
Disallow /wp-content/upgrade/
Disallow /wp-content/uploads/wpo-plugins-tables-list.json
Disallow /xmlrpc.php
Disallow /wp-*.php
Disallow /readme.html
Disallow /license.txt
Disallow /wp-config.php
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /wp-signup.php
Disallow /?s=
Disallow /search/
Disallow /*?s=
Disallow /*?p=
Disallow /*?attachment_id=
Disallow /*?replytocom=
Disallow /*?preview=
Disallow /*%26
Disallow /wp-json/
Disallow /feed/
Disallow /*/feed/
Disallow /comments/feed/
Disallow /trackback/
Disallow /*/trackback/
Disallow /wp-comments-post.php

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mb2345browser

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

httrack

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

webzip

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

wget

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.meetyu.app/sitemaps.xml
Back to top

Comments

  • Robots.txt for MeetYu
  • https://www.meetyu.app/robots.txt
  • Last updated: 2025-10-22
  • ============================================
  • GOOD BOTS (Search Engines)
  • ============================================
  • Allow major search engines full access
  • ============================================
  • WORDPRESS ADMIN & CORE
  • ============================================
  • ============================================
  • WORDPRESS SYSTEM FILES
  • ============================================
  • ============================================
  • SEARCH & QUERY PARAMETERS
  • ============================================
  • ============================================
  • FEEDS, API & TRACKBACKS
  • ============================================
  • ============================================
  • SPECIFIC PATHS (Ajusted based on our needs)
  • ============================================
  • Disallow: /site-url
  • ============================================
  • BLOCK ARCHIVE.ORG
  • ============================================
  • ============================================
  • BLOCK BAD BOTS & SCRAPERS
  • ============================================
  • SEO Tool Crawlers
  • Malicious/Aggressive Crawlers
  • ============================================
  • SITEMAPS
  • ============================================
Back to top