bastillepost.com
robots.txt

Robots Exclusion Standard data for bastillepost.com

Resource Scan

Scan Details

Site Domain bastillepost.com
Base Domain bastillepost.com
Scan Status Ok
Last Scan2024-11-15T11:57:27+00:00
Next Scan 2024-11-22T11:57:27+00:00

Last Scan

Scanned2024-11-15T11:57:27+00:00
URL https://bastillepost.com/robots.txt
Redirect https://www.bastillepost.com/robots.txt
Redirect Domain www.bastillepost.com
Redirect Base bastillepost.com
Domain IPs 18.136.61.79, 18.143.27.128
Redirect IPs 52.220.246.187, 52.220.46.99, 54.251.48.49
Response IP 52.220.46.99
Found Yes
Hash 9ccaed1c18e9844aa0ad63b703a05b721bb4c0c7f225d01764fcdfec4733f00a
SimHash e87811536cd5

Groups

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yahoobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /hongkong/Category/CateFirst
Disallow /macau/Category/CateFirst
Disallow /china/Category/CateFirst
Disallow /global/Category/CateFirst
Disallow /hongkong/Output
Disallow /macau/Output
Disallow /china/Output
Disallow /global/Output
Disallow /wp-content/
Disallow /hongkong/wp-content/
Disallow /macau/wp-content/
Disallow /global/wp-content/
Disallow /hongkong/hongkong/post.php
Disallow /macau/macau/post.php
Disallow /hongkong/category/157276
Disallow /australia/article/2105436
Disallow /australia/wp-content/themes/BastillePost2/includes/
Disallow /hongkong/wp-content/themes/BastillePost2/includes/
Disallow /macau/wp-content/themes/BastillePost2/includes/
Disallow /vote/
Disallow /australia/australia/post.php
Disallow /hongkong/search-result

googlebot

Rule Path
Disallow /hongkong/Category/CateFirst
Disallow /macau/Category/CateFirst
Disallow /china/Category/CateFirst
Disallow /global/Category/CateFirst
Disallow /hongkong/Output
Disallow /macau/Output
Disallow /china/Output
Disallow /global/Output
Disallow /wp-content/
Disallow /hongkong/wp-content/
Disallow /macau/wp-content/
Disallow /global/wp-content/
Disallow /hongkong/hongkong/post.php
Disallow /macau/macau/post.php
Disallow /hongkong/category/157276
Disallow /australia/article/2105436
Disallow /australia/wp-content/themes/BastillePost2/includes/
Disallow /hongkong/wp-content/themes/BastillePost2/includes/
Disallow /macau/wp-content/themes/BastillePost2/includes/
Disallow /vote/
Disallow /hongkong/author/*/feed
Disallow /hongkong/search-result

trendkite-akashic-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.bastillepost.com/hongkong/sitemap/generate/index