emmastoneweb.com
robots.txt

Robots Exclusion Standard data for emmastoneweb.com

Resource Scan

Scan Details

Site Domain emmastoneweb.com
Base Domain emmastoneweb.com
Scan Status Ok
Last Scan2024-06-16T11:38:57+00:00
Next Scan 2024-06-23T11:38:57+00:00

Last Scan

Scanned2024-06-16T11:38:57+00:00
URL https://emmastoneweb.com/robots.txt
Domain IPs 170.39.77.82
Response IP 170.39.77.82
Found Yes
Hash d7c37203a72548378dc645c3671b16e36762fd105589626c1f4241f16bb4cad5
SimHash e8805b14fbd0

Groups

scrapy

Rule Path
Allow /

twitterbot

Rule Path
Disallow *
Allow /wp-content/uploads/*

*

Rule Path
Disallow /wp-admin*
Disallow /wp-login.php
Disallow /wp-content*
Disallow /*/feed
Disallow /*/trackback
Disallow /*?
Disallow /*.js$
Disallow /*.inc$
Disallow /*.css$
Disallow /category/
Disallow /page/
Disallow /pages/
Disallow /feed/
Disallow /feed
Disallow /tag/
Disallow /archives/