Log In Sign Up

unrealfacts.com
robots.txt

Robots Exclusion Standard data for unrealfacts.com

Resource Scan

Scan Details

Site Domain unrealfacts.com
Base Domain unrealfacts.com
Scan Status Ok
Last Scan2025-05-25T18:07:55+00:00
Next Scan 2025-06-01T18:07:55+00:00

Last Scan

Scanned2025-05-25T18:07:55+00:00
URL https://unrealfacts.com/robots.txt
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.32.1
Found Yes
Hash 936ced337077a1d6964a0e81ffcc540c2965e1eaa8eb6a5ca1439d1a26d7c07b
SimHash a8008a420370

Groups

*

Rule Path
Allow /
Disallow /trackback/
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /wp-
Disallow /cgi-bin
Disallow /readme.html
Disallow /license.txt
Disallow /*?*
Disallow /*.php$
Disallow /*.js$
Disallow /*.inc$
Disallow /*.css$
Disallow /*.gz$
Disallow /*.wmv$
Disallow /*.cgi$
Disallow /*.xhtml$
Disallow /*/wp-*
Disallow /*/feed/*
Disallow /*/*?s=*
Disallow /*/*.js$
Disallow /*/*.inc$
Disallow /post-arrays/
Disallow /post-arrays/*
Allow /wp-content/uploads/

ia_archiver*

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://unrealfacts.com/sitemap_index.xml
Back to top

Comments

  • Robots Rule! - Sometimes...
  • Disallow these directories, url types & file-types
Back to top