mamapedia.com
robots.txt

Robots Exclusion Standard data for mamapedia.com

Resource Scan

Scan Details

Site Domain mamapedia.com
Base Domain mamapedia.com
Scan Status Ok
Last Scan2024-11-15T09:50:50+00:00
Next Scan 2024-11-16T09:50:50+00:00

Last Scan

Scanned2024-11-15T09:50:50+00:00
URL https://www.mamapedia.com/robots.txt
Domain IPs 104.21.65.89, 172.67.161.130, 2606:4700:3037::6815:4159, 2606:4700:3037::ac43:a182
Response IP 104.21.65.89
Found Yes
Hash d90f2c01ac4a834d7945baf614652ef73e260af2dcf6cd0a5d752ee3e3bd836a
SimHash cc2c99834b54

Groups

mediapartners-google

Rule Path
Disallow

googlebot

Rule Path
Disallow /register
Disallow /profile
Disallow /home
Disallow /static
Disallow /mamapedia/item_rate
Disallow /mmp_join
Disallow /np_score/submit_rating/
Disallow /timeline/memory
Disallow /outbound_link/
Disallow /message/
Disallow /questions/

*

Rule Path
Disallow /register
Disallow /profile
Disallow /home
Disallow /static
Disallow /mamapedia/item_rate
Disallow /mmp_join
Disallow /np_score/submit_rating/
Disallow /timeline/memory
Disallow /outbound_link/
Disallow /message/
Disallow /questions/

Comments

  • We will allow G to crawl /join since we add the canonical link element to them.

Warnings

  • `noindex` is not a known field.