wanderlustwomen.discussion.community
robots.txt
Robots Exclusion Standard data for wanderlustwomen.discussion.community
Resource Scan
Scan Details
Site Domain | wanderlustwomen.discussion.community |
Base Domain | discussion.community |
Scan Status | Ok |
Last Scan | 2024-07-04T16:14:28+00:00 |
Next Scan | 2024-07-11T16:14:28+00:00 |
Last Scan
Scanned | 2024-07-04T16:14:28+00:00 |
URL | https://wanderlustwomen.discussion.community/robots.txt |
Domain IPs | 107.21.35.214, 18.213.166.18 |
Response IP | 107.21.35.214 |
Found | Yes |
Hash | 3219c80e360361ceec9dde27cf7a72f951950a586fb67148452d49eca3f1e0ec |
SimHash | 68429b72fdc0 |
Groups
bubing
alphaseobot
ltx71
companybook-crawler
bdcbot
spbot
semrushbot
siteauditbot
ahrefsbot
mj12bot
dotbot
omgili
blexbot
magpie-crawler
extlinksbot
netseer
weborama-fetcher
linkfluence
sentibot
seokicks
barkrowler
ccbot
trendictionbot
amazonbot
serpstatbot
petalbot
dataforseobot
censysinspect
awariorssbot
awariosmartbot
awariobot
phxbot
bytespider
bl.uk_lddc_bot
ecoresearchcrawler
turnitinbot
zoominfobot
dataprovider
velenpublicwebcrawler
domainstatsbot
hypestat
panscient.com
yak
lcc
makemerrybot
ioncrawl
googleother
anderspinkbot
friendlycrawler
checkmarknetwork/1.0 (+https://www.checkmarknetwork.com/spider.html)
Rule | Path |
---|---|
Disallow | / |
mediapartners-google
Rule | Path |
---|---|
Disallow | /images/ |
Disallow | /pm |
Disallow | /album |
Disallow | /mbactions |
Disallow | /register |
Disallow | |
Disallow | /search |
Disallow | /profile |
Disallow | /file |
Disallow | /thumb |
Disallow | /upload |
Disallow | /printthread |
Disallow | /post/show_single_post |
Disallow | /post/printadd |
Disallow | /post/upost |
Disallow | /post/hpt |
Disallow | /subscribe |
Disallow | /calendar |
Disallow | /calendar/newevent |
Disallow | /calendar/daydetail?*&nav= |
Disallow | /calendar/display*view%3Dweekly |
Disallow | /calendar/display*view%3Dmonthly |
Disallow | /calendar/showbirthday |
Disallow | /external |
Disallow | /tool/view/gb/private |
Disallow | /tool/view/gb/email |
Disallow | /tool/pm |
Disallow | /tool/members/ |
Disallow | /tool/ticket/ |
Disallow | /cgi/view/poll.cgi |
Disallow | /cgi/view/topsites.cgi |
Disallow | /cgi/view/member.cgi |
Disallow | /cgi/view/out.cgi |
Disallow | /?authtoken= |
Other Records
Field | Value |
---|---|
crawl-delay | 15 |
*
Rule | Path |
---|---|
Disallow | /images/ |
Disallow | /pm |
Disallow | /album |
Disallow | /mbactions |
Disallow | /register |
Disallow | |
Disallow | /search |
Disallow | /profile |
Disallow | /file |
Disallow | /thumb |
Disallow | /upload |
Disallow | /printthread |
Disallow | /post/show_single_post |
Disallow | /post/printadd |
Disallow | /post/upost |
Disallow | /post/hpt |
Disallow | /subscribe |
Disallow | /calendar |
Disallow | /calendar/newevent |
Disallow | /calendar/daydetail?*&nav= |
Disallow | /calendar/display*view%3Dweekly |
Disallow | /calendar/display*view%3Dmonthly |
Disallow | /calendar/showbirthday |
Disallow | /external |
Disallow | /tags |
Disallow | /tool/view/gb/private |
Disallow | /tool/view/gb/email |
Disallow | /tool/pm |
Disallow | /tool/members/ |
Disallow | /tool/ticket/ |
Disallow | /cgi/view/poll.cgi |
Disallow | /cgi/view/topsites.cgi |
Disallow | /cgi/view/member.cgi |
Disallow | /cgi/view/out.cgi |
Disallow | /?authtoken= |
Disallow | /contact?*subject= |
Disallow | /post*?goto= |
Disallow | /post*%26goto%3D |
Disallow | /post*?id= |
Disallow | *?*&sort= |
Disallow | *?sort= |
Disallow | *?full_version= |
Disallow | *?*full_version= |
Disallow | /?s= |
Disallow | /?t= |
Disallow | /?p= |
Disallow | /?id= |
Disallow | /?profile= |
Disallow | /*.jpg$ |
Disallow | /*.jpeg$ |
Disallow | /*.gif$ |
Disallow | /*.png$ |
Disallow | /new$ |
Disallow | /top$ |
Disallow | /top$ |
Disallow | /top?period= |
Allow | /site/images/support/ |
Allow | /tool/members/login |
Allow | /images/favicon/ |
Disallow | /contact/ |
Disallow | /support/ |
Other Records
Field | Value |
---|---|
crawl-delay | 15 |
Other Records
Field | Value |
---|---|
sitemap | https://wanderlustwomen.discussion.community/sitemap.xml |
Warnings
- 1 invalid line.
Comments