Log In Sign Up

rsm-wmv.nl
robots.txt

Robots Exclusion Standard data for rsm-wmv.nl

Resource Scan

Scan Details

Site Domain rsm-wmv.nl
Base Domain rsm-wmv.nl
Scan Status Ok
Last Scan2024-10-29T09:41:55+00:00
Next Scan 2024-11-28T09:41:55+00:00

Last Scan

Scanned2024-10-29T09:41:55+00:00
URL https://rsm-wmv.nl/robots.txt
Redirect https://www.rsm.global/netherlands/nl/robots.txt
Redirect Domain www.rsm.global
Redirect Base rsm.global
Domain IPs 108.157.254.15, 108.157.254.33, 108.157.254.35, 108.157.254.8, 2600:9000:2753:2e00:5:6383:cd40:93a1, 2600:9000:2753:4a00:5:6383:cd40:93a1, 2600:9000:2753:8600:5:6383:cd40:93a1, 2600:9000:2753:9e00:5:6383:cd40:93a1, 2600:9000:2753:d400:5:6383:cd40:93a1, 2600:9000:2753:d600:5:6383:cd40:93a1, 2600:9000:2753:ee00:5:6383:cd40:93a1, 2600:9000:2753:f800:5:6383:cd40:93a1
Redirect IPs 104.22.50.157, 104.22.51.157, 172.67.15.112, 2606:4700:10::6816:329d, 2606:4700:10::6816:339d, 2606:4700:10::ac43:f70
Response IP 104.22.51.157
Found Yes
Hash 1bbcc3daf11632e60c968b729d6cac8997b5b8e405e5e4c75182734e77d3c95d
SimHash 34009d0ac974

Groups

*

Rule Path
Allow */core/*.css$
Allow */core/*.css?
Allow */core/*.js$
Allow */core/*.js?
Allow */core/*.gif
Allow */core/*.jpg
Allow */core/*.jpeg
Allow */core/*.png
Allow */core/*.svg
Allow */profiles/*.css$
Allow */profiles/*.css?
Allow */profiles/*.js$
Allow */profiles/*.js?
Allow */profiles/*.gif
Allow */profiles/*.jpg
Allow */profiles/*.jpeg
Allow */profiles/*.png
Allow */profiles/*.svg
Disallow */core/
Disallow */profiles/
Disallow */README.txt
Disallow */web.config
Disallow */admin/
Disallow */comment/reply/
Disallow */filter/tips
Disallow */node/add/
Disallow */search/
Disallow */user/register
Disallow */user/password
Disallow */user/login
Disallow */user/logout
Disallow */index.php/admin/
Disallow */index.php/comment/reply/
Disallow */index.php/filter/tips
Disallow */index.php/node/add/
Disallow */index.php/search/
Disallow */index.php/user/password
Disallow */index.php/user/register
Disallow */index.php/user/login
Disallow */index.php/user/logout
Disallow *?*utm_term
Disallow *?*utm_campaign
Disallow *?*utm_medium
Disallow *?*rsm_medium
Disallow *?*rsm_campaign
Disallow *?*elqTrackId
Disallow *?*page
Disallow *?*rsm_term
Disallow *?*rsm_source
Disallow *?*utm_source
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
  • CSS, JS, Images
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Stop certain parameters from being crawled
Back to top