Log In Sign Up

mx-m.ca
robots.txt

Robots Exclusion Standard data for mx-m.ca

Resource Scan

Scan Details

Site Domain mx-m.ca
Base Domain mx-m.ca
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-11-13T10:43:34+00:00
Next Scan 2026-02-11T10:43:34+00:00

Last Successful Scan

Scanned2024-10-19T10:29:53+00:00
URL https://mx-m.ca/robots.txt
Domain IPs 199.16.172.1, 199.16.173.140
Response IP 199.16.173.140
Found Yes
Hash cf74f51f39341fc6139cc094d6e2216934cfd84d9b245b840897a48b50753afc
SimHash e8485f6dac77

Groups

*

Rule Path
Allow /wp-content/uploads/*
Allow /wp-content/*.js
Allow /wp-content/*.css
Allow /wp-includes/*.js
Allow /wp-includes/*.css
Disallow /cgi-bin
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-includes/
Disallow /*/attachment/
Disallow /tag/*/page/
Disallow /tag/*/feed/
Disallow /page/
Disallow /comments/
Disallow /xmlrpc.php
Disallow /?attachment_id*
Disallow /*?

Other Records

Field Value
crawl-delay 120

*

Rule Path
Disallow /?s=
Disallow /search

*

Rule Path
Disallow /trackback
Disallow /*trackback
Disallow /*trackback*
Disallow /*/trackback

*

Rule Path
Allow /feed/$
Disallow /feed/
Disallow /comments/feed/
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

Rule Path
Allow *.css$
Allow *.js$
Allow *.html$
Allow *.htm$
Allow *.doc$
Allow *.docx$
Allow *.xls$
Allow *.xlt$
Allow *.xlsx$
Allow *.pp$
Allow *.sldx$
Allow *.pub$
Allow *.inc$
Allow *.wmv$
Allow *.gif$
Allow *.jpg$
Allow *.jpeg$
Allow *.pdf$
Allow *.png$
Allow *.wmv$
Allow *.tar$
Allow *.gz$
Allow *.tar$
Allow *.gz.tar$
Allow *.zip$
Allow *.xml$
Disallow /readme.html
Disallow /license.txt
Disallow .php$
Disallow *.*.php$

Other Records

Field Value
crawl-delay 20

*

Rule Path
Disallow /search
Allow /search/about
Allow /search/static
Allow /search/howsearchworks
Disallow /sdch
Disallow /groups
Disallow /index.html?
Disallow /?
Allow /?hl=
Disallow /?hl=*&
Allow /?hl=*&gws_rd=ssl$
Disallow /?hl=*&*&gws_rd=ssl
Allow /?gws_rd=ssl$
Allow /?pt1=true$
Disallow /imgres
Disallow /u/
Disallow /preferences
Disallow /setprefs
Disallow /default
Disallow /m?
Disallow /m/
Allow /m/finance
Disallow /wml?
Disallow /wml/?
Disallow /wml/search?
Disallow /xhtml?
Disallow /xhtml/?
Disallow /xhtml/search?
Disallow /xml?
Disallow /imode?
Disallow /imode/?
Disallow /imode/search?
Disallow /jsky?
Disallow /jsky/?
Disallow /jsky/search?
Disallow /pda?
Disallow /pda/?
Disallow /pda/search?
Disallow /sprint_xhtml
Disallow /sprint_wml
Disallow /pqa
Disallow /palm
Disallow /gwt/
Disallow /purchases
Disallow /local?
Disallow /local_url
Disallow /shihui?
Disallow /shihui/
Disallow /products?
Disallow /product_
Disallow /products_
Disallow /products%3B
Disallow /print
Disallow /books/
Disallow /bkshp?*q=*
Disallow /books?*q=*
Disallow /books?*output=*
Disallow /books?*pg=*
Disallow /books?*jtp=*
Disallow /books?*jscmd=*
Disallow /books?*buy=*
Disallow /books?*zoom=*
Allow /books?*q=related%3A*
Allow /books?*q=editions%3A*
Allow /books?*q=subject%3A*
Allow /books/about
Allow /booksrightsholders
Allow /books?*zoom=1*
Allow /books?*zoom=5*
Allow /books/content?*zoom=1*
Allow /books/content?*zoom=5*
Disallow /ebooks/
Disallow /ebooks?*q=*
Disallow /ebooks?*output=*
Disallow /ebooks?*pg=*
Disallow /ebooks?*jscmd=*
Disallow /ebooks?*buy=*
Disallow /ebooks?*zoom=*
Allow /ebooks?*q=related%3A*
Allow /ebooks?*q=editions%3A*
Allow /ebooks?*q=subject%3A*
Allow /ebooks?*zoom=1*
Allow /ebooks?*zoom=5*
Disallow /patents?
Disallow /patents/download/
Disallow /patents/pdf/
Disallow /patents/related/
Disallow /scholar
Disallow /citations?
Allow /citations?user=
Disallow /citations?*cstart=
Allow /citations?view_op=new_profile
Allow /citations?view_op=top_venues
Allow /scholar_share
Disallow /s?
Allow /maps?*output=classic*
Allow /maps?*file=
Allow /maps/d/
Disallow /maps?
Disallow /mapstt?
Disallow /mapslt?
Disallow /maps/stk/
Disallow /maps/br?
Disallow /mapabcpoi?
Disallow /maphp?
Disallow /mapprint?
Disallow /maps/api/js/
Allow /maps/api/js
Disallow /maps/api/place/js/
Disallow /maps/api/staticmap
Disallow /maps/api/streetview
Disallow /maps/_/sw/manifest.json
Disallow /mld?
Disallow /staticmap?
Disallow /maps/preview
Disallow /maps/place
Disallow /maps/timeline/
Disallow /help/maps/streetview/partners/welcome/
Disallow /help/maps/indoormaps/partners/
Disallow /lochp?
Disallow /center
Disallow /ie?
Disallow /blogsearch/
Disallow /blogsearch_feeds
Disallow /advanced_blog_search
Disallow /uds/
Disallow /chart?
Disallow /transit?
Allow /calendar$
Allow /calendar/about/
Disallow /calendar/
Disallow /cl2/feeds/
Disallow /cl2/ical/
Disallow /coop/directory
Disallow /coop/manage
Disallow /trends?
Disallow /trends/music?
Disallow /trends/hottrends?
Disallow /trends/viz?
Disallow /trends/embed.js?
Disallow /trends/fetchComponent?
Disallow /trends/beta
Disallow /trends/topics
Disallow /musica
Disallow /musicad
Disallow /musicas
Disallow /musicl
Disallow /musics
Disallow /musicsearch
Disallow /musicsp
Disallow /musiclp
Disallow /urchin_test/
Disallow /movies?
Disallow /wapsearch?
Allow /safebrowsing/diagnostic
Allow /safebrowsing/report_badware/
Allow /safebrowsing/report_error/
Allow /safebrowsing/report_phish/
Disallow /reviews/search?
Disallow /orkut/albums
Disallow /cbk
Disallow /recharge/dashboard/car
Disallow /recharge/dashboard/static/
Disallow /profiles/me
Allow /profiles
Disallow /s2/profiles/me
Allow /s2/profiles
Allow /s2/oz
Allow /s2/photos
Allow /s2/search/social
Allow /s2/static
Disallow /s2
Disallow /transconsole/portal/
Disallow /gcc/
Disallow /aclk
Disallow /cse?
Disallow /cse/home
Disallow /cse/panel
Disallow /cse/manage
Disallow /tbproxy/
Disallow /imesync/
Disallow /shenghuo/search?
Disallow /support/forum/search?
Disallow /reviews/polls/
Disallow /hosted/images/
Disallow /ppob/?
Disallow /ppob?
Disallow /accounts/ClientLogin
Disallow /accounts/ClientAuth
Disallow /accounts/o8
Allow /accounts/o8/id
Disallow /topicsearch?q=
Disallow /xfx7/
Disallow /squared/api
Disallow /squared/search
Disallow /squared/table
Disallow /qnasearch?
Disallow /app/updates
Disallow /sidewiki/entry/
Disallow /quality_form?
Disallow /labs/popgadget/search
Disallow /buzz/post
Disallow /compressiontest/
Disallow /analytics/feeds/
Disallow /analytics/partners/comments/
Disallow /analytics/portal/
Disallow /analytics/uploads/
Allow /alerts/manage
Allow /alerts/remove
Disallow /alerts/
Allow /alerts/$
Disallow /ads/search?
Disallow /ads/plan/action_plan?
Disallow /ads/plan/api/
Disallow /ads/hotels/partners
Disallow /phone/compare/?
Disallow /travel/clk
Disallow /hotelfinder/rpc
Disallow /hotels/rpc
Disallow /commercesearch/services/
Disallow /evaluation/
Disallow /chrome/browser/mobile/tour
Disallow /compare/*/apply*
Disallow /forms/perks/
Disallow /shopping/suppliers/search
Disallow /ct/
Disallow /edu/cs4hs/
Disallow /trustedstores/s/
Disallow /trustedstores/tm2
Disallow /trustedstores/verify
Disallow /adwords/proposal
Disallow /shopping?*
Disallow /shopping/product/
Disallow /shopping/seller
Disallow /shopping/ratings/account/metrics
Disallow /shopping/ratings/merchant/immersivedetails
Disallow /shopping/reviewer
Disallow /about/careers/applications/
Disallow /about/careers/applications-a/
Disallow /landing/signout.html
Disallow /webmasters/sitemaps/ping?
Disallow /ping?
Disallow /gallery/
Disallow /landing/now/ontap/
Allow /searchhistory/
Allow /maps/reserve
Allow /maps/reserve/partners
Disallow /maps/reserve/api/
Disallow /maps/reserve/search
Disallow /maps/reserve/bookings
Disallow /maps/reserve/settings
Disallow /maps/reserve/manage
Disallow /maps/reserve/payment
Disallow /maps/reserve/receipt
Disallow /maps/reserve/sellersignup
Disallow /maps/reserve/payments
Disallow /maps/reserve/feedback
Disallow /maps/reserve/terms
Disallow /maps/reserve/m/
Disallow /maps/reserve/b/
Disallow /maps/reserve/partner-dashboard
Disallow /about/views/
Disallow /intl/*/about/views/
Disallow /local/cars
Disallow /local/cars/
Disallow /local/dealership/
Disallow /local/dining/
Disallow /local/place/products/
Disallow /local/place/reviews/
Disallow /local/place/rap/
Disallow /local/tab/
Disallow /localservices/*
Allow /finance
Allow /js/
Disallow /nonprofits/account/
Disallow /fbx
Disallow /uviewer

adsbot-google

Rule Path
Disallow /maps/api/js/
Allow /maps/api/js
Disallow /maps/api/place/js/
Disallow /maps/api/staticmap
Disallow /maps/api/streetview

twitterbot

Rule Path
Allow /imgres

facebookexternalhit

Rule Path
Allow /imgres

googlebot

Rule Path
Allow /*.css$
Allow /*.js$
Allow /wp-cron.php
Allow /wp-feed.php
Allow /wp-login.php
Allow /xmlrpc.php
Allow /index.php
Allow /wp-admin/$

twitterbot

Rule Path
Allow /images

facebookexternalhit

Rule Path
Allow /images

mediapartners-google*

Rule Path
Disallow
Allow /*

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /*
Disallow /cgi-bin
Disallow /wp-admin
Disallow /wp-includes
Disallow /trackback
Disallow /comments

googlebot-mobile

Rule Path
Allow /*

twitterbot

Rule Path
Allow /*

ia_archiver-web.archive.org

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://dq.saviezvousque.net/sitemap.xml
Back to top

Comments

  • Sample Robots.txt
  • Ramón Vila Ferreres - <ramon@vilafer.tech>
  • -------------------------------------------
  • Bloqueo basico para todos los bots y crawlers
  • puede dar problemas por bloqueo de recursos en GWT
  • Bloqueo de las URL dinamicas
  • Bloqueo de busquedas
  • Bloqueo de trackbacks
  • Bloqueo de feeds para crawlers
  • Ralentizamos algunos bots que se suelen volver locos
  • should already be disallowed
  • Disallow: *.cgi
  • Disallow: *.exe
  • Disallow: *.xds
  • Disallow: *.xls
  • Disallow: *.bmp
  • Disallow: *.xhtml
  • needed to protect .php in the folders above, but has to be here to stop hacks such as jpg.php
  • Bloqueo de bots y crawlers poco utiles
  • DISALOW BOTS (not really needed but helps)
  • AdsBot
  • Crawlers of certain social media sites are allowed to access page markup when google.com/imgres* links are shared. To learn more, please contact images-robots-allowlist@google.com.
  • Previene problemas de recursos bloqueados en Google Webmaster Tools
  • ALLOW STUFF THROUGH
  • Allow: /wp-admin/admin.php
  • Allow: /wp-admin/admin-ajax.php
  • Allow: /wp-admin/load-styles.php
  • Allow: /wp-admin/load-scripts.php
  • Allow: /wp-admin/about.php
  • Allow: /wp-admin/media-new.php
  • Allow: /wp-admin/post-new.php
  • Allow: /wp-admin/page.php
  • Allow: /wp-admin/themes.php
  • Allow: /wp-admin/profile.php
  • Allow: /wp-admin/update-core.php
  • Certain social media sites are whitelisted to allow crawlers to access page markup when links to /images are shared.
  • Google AdSense
  • Google Image
  • En condiciones normales este es el sitemap
  • Si utilizas Yoast SEO estos son los sitemaps principales
  • Sitemap: https://dq.saviezvousque.net/sitemap_index.xml
  • Sitemap: https://dq.saviezvousque.net/category-sitemap.xml
  • Sitemap: https://dq.saviezvousque.net/page-sitemap.xml
  • Sitemap: https://dq.saviezvousque.net/post-sitemap.xml
Back to top