wpgetapi.com
robots.txt

Robots Exclusion Standard data for wpgetapi.com

Resource Scan

Scan Details

Site Domain wpgetapi.com
Base Domain wpgetapi.com
Scan Status Ok
Last Scan5/9/2025, 3:27:03 AM
Next Scan 6/8/2025, 3:27:03 AM

Last Scan

Scanned5/9/2025, 3:27:03 AM
URL https://wpgetapi.com/robots.txt
Domain IPs 104.21.2.121, 172.67.129.40, 2606:4700:3035::ac43:8128, 2606:4700:3037::6815:279
Response IP 172.67.129.40
Found Yes
Hash 214f4a2145fa0a5b5224230cfea36a3f6673dd9e54cabb31be7f8ac4c857748e
SimHash 8beccdc25eb3

Groups

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

companybook-crawler

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot/7~bl

Rule Path
Disallow /

linguee

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

carsnipbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seekport

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

*

Rule Path
Disallow /*?wc_country_preselect
Disallow /forums/ticket/
Disallow /forums/reply/

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /wp-content/uploads/wpo/wpo-plugins-tables-list.json

*

Rule Path
Disallow /?s=
Disallow /page/*/?s=
Disallow /search/
Disallow /wp-json/
Disallow /?rest_route=

Other Records

Field Value
sitemap https://wpgetapi.com/sitemap_index.xml

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK

Warnings

  • 2 invalid lines.