emergeapp.net
robots.txt

Robots Exclusion Standard data for emergeapp.net

Resource Scan

Scan Details

Site Domain emergeapp.net
Base Domain emergeapp.net
Scan Status Ok
Last Scan2026-01-24T13:16:31+00:00
Next Scan 2026-01-31T13:16:31+00:00

Last Scan

Scanned2026-01-24T13:16:31+00:00
URL https://emergeapp.net/robots.txt
Domain IPs 104.26.2.36, 104.26.3.36, 172.67.72.18, 2606:4700:20::681a:224, 2606:4700:20::681a:324, 2606:4700:20::ac43:4812
Response IP 104.26.2.36
Found Yes
Hash 719393a0568ab55b26d8d7a027e7112328d4babc3086fb7a5fa87e4318e9448a
SimHash ec017bc49c93

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-admin/
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

googlebot-mobile

Rule Path
Allow /
Disallow /wp-admin
Disallow *feed/
Disallow /wp-login.php

googlebot

Rule Path
Allow /

Other Records

Field Value
sitemap http://emergeapp.net/page-sitemap.xml
sitemap https://emergeapp.net/post-sitemap.xml
sitemap https://emergeapp.net/category-sitemap.xml
sitemap http://emergeapp.net/sitemap_index.xml

Comments

  • Prevent private admin areas from being crawled
  • Prevent duplicate /feed/ pages from being crawled
  • Prevent login page crawls etc