themonopolitan.com
robots.txt

Robots Exclusion Standard data for themonopolitan.com

Resource Scan

Scan Details

Site Domain themonopolitan.com
Base Domain themonopolitan.com
Scan Status Ok
Last Scan2024-09-21T15:54:31+00:00
Next Scan 2024-09-28T15:54:31+00:00

Last Scan

Scanned2024-09-21T15:54:31+00:00
URL https://themonopolitan.com/robots.txt
Domain IPs 98.142.96.2
Response IP 98.142.96.2
Found Yes
Hash 54ae9018fe66d1c8416ac1887d78d88df8be2cad313d8e54694bdf45ba3a3e9e
SimHash 80041c244ed0

Groups

*

Rule Path
Disallow /dating-training-london/
Disallow /baggage-dating-show-full-episodes/
Disallow /best-dating-advice-ever/
Disallow /dating-eko-ranger/
Disallow /lego-wolf-speed-dating/
Disallow /best-online-dating-for-relationships/
Disallow /dating-site-bahamas/
Disallow /aarp-dating-reviews/
Disallow /online-dating-reply-time/
Disallow /dating-a-girl-10-years-younger/
Disallow /dating-a-guy-with-social-anxiety-disorder/
Disallow /mcallen-dating/
Disallow /pisces-man-dating-leo-woman/
Disallow /fat-dating-app/
Disallow /ottawa-dating-scene/
Disallow /new-zealand-dating-sites/
Disallow /free-dating-numbers/
Disallow /speed-dating-brisbane/
Disallow /category/diseno/page/14/
Disallow /category/retail/page/2/
Disallow /monitor/negocio/
Disallow /what-are-the-limitations-of-relative-dating/

semrushbot
yandeximages
ahrefsbot
mj12bot
yandexbot
baidu
dalvik
fetcher
grapeshotcrawler

Rule Path
Disallow /

grapeshot

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.themonopolitan.com/sitemap.xml
sitemap http://themonopolitan.com/sitemap.xml

Comments

  • Rule 1
  • Inhabiliar los Bots que no son importantes