bigxp.weebly.com
robots.txt

Robots Exclusion Standard data for bigxp.weebly.com

Resource Scan

Scan Details

Site Domain bigxp.weebly.com
Base Domain weebly.com
Scan Status Ok
Last Scan2024-09-28T05:05:33+00:00
Next Scan 2024-10-28T05:05:33+00:00

Last Scan

Scanned2024-09-28T05:05:33+00:00
URL https://bigxp.weebly.com/robots.txt
Domain IPs 74.115.51.8, 74.115.51.9
Response IP 74.115.51.9
Found Yes
Hash ba9dfa914c045c224c2d01a78bd51fbb434849175781031002661851abb0501f
SimHash 68a6d860eff1

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /the-bean-tales-data.html
Disallow /get-this-game-stealth.html
Disallow /get-this-game--the-bean-tales.html
Disallow /get-information.html
Disallow /latest-games.html
Disallow /download-the-bean-tales.html
Disallow /download-stealth.html
Disallow /notifier.html
Disallow /download-bigxp-toolbar-and-notifier.html
Disallow /feedback.html
Disallow /toolbar.html
Disallow /watch-dogs.html
Disallow /sleeping-dogs-limited-edition-free-download.html
Disallow /blog.html
Disallow /1/
Disallow /blog/
Disallow /blog
Disallow /ghost-recon-phantom.html
Disallow /download-pc-games.html
Disallow /check.html

Other Records

Field Value
sitemap https://bigxp.weebly.com/sitemap.xml