Log In Sign Up

placelibertine.com
robots.txt

Robots Exclusion Standard data for placelibertine.com

Resource Scan

Scan Details

Site Domain placelibertine.com
Base Domain placelibertine.com
Scan Status Ok
Last Scan2024-09-14T06:39:50+00:00
Next Scan 2024-10-14T06:39:50+00:00

Last Scan

Scanned2024-09-14T06:39:50+00:00
URL https://placelibertine.com/robots.txt
Redirect https://www.placelibertine.com/robots.txt
Redirect Domain www.placelibertine.com
Redirect Base placelibertine.com
Domain IPs 176.31.63.35
Redirect IPs 176.31.63.35
Response IP 176.31.63.35
Found Yes
Hash 76ef121a0639fb538047e832ed0ed6ff964f543c22b02312ac7806ebe6b88eb7
SimHash ec63ce27d6b0

Groups

*

Rule Path
Allow /blog/*/uploads
Allow /blog/*.js
Allow /blog/*.css
Allow /blog/*.png
Allow /blog/*.jpg
Allow /blog/*.jpeg
Allow /blog/*.gif
Allow /blog/*.svg
Allow /blog/*.webp
Allow /blog/*.pdf
Disallow /blog/wp-admin
Disallow /blog/wp-includes
Disallow /blog/wp-login.php
Disallow /blog/wp-content/plugins
Disallow /blog/wp-content/cache
Disallow /blog/wp-content/themes
Disallow /blog/trackback
Disallow /blog/*/trackback
Disallow /blog/*/*/trackback
Disallow /blog/*/*/feed/*/
Disallow /blog/*/feed
Disallow /blog/*?*
Disallow /blog/cgi-bin
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.placelibertine.com/index-sitemap.xml
sitemap https://www.placelibertine.com/blog/sitemaps.xml
Back to top

Comments

  • January 2024
  • Regular allow of necessary style & image files. Do not remove /blog from the instructions
  • ------------------------------------------------------------------------------------------
  • Regular Wordpress instructions for /blog
  • --------------------------------
  • Ban bots that don't benefit us
  • --------------------------------
  • Sitemaps
  • ----------
Back to top