Log In Sign Up

belgianfoodtruckfederation.org
robots.txt

Robots Exclusion Standard data for belgianfoodtruckfederation.org

Resource Scan

Scan Details

Site Domain belgianfoodtruckfederation.org
Base Domain belgianfoodtruckfederation.org
Scan Status Ok
Last Scan2024-09-28T04:03:53+00:00
Next Scan 2024-10-05T04:03:53+00:00

Last Scan

Scanned2024-09-28T04:03:53+00:00
URL https://belgianfoodtruckfederation.org/robots.txt
Domain IPs 178.32.64.47
Response IP 178.32.64.47
Found Yes
Hash d3ea2273b19be3fae929c17eb841520c7e45ec0abea6a483f8231d2ce84be950
SimHash 8b42639260a6

Groups

adsbot
ahrefsbot
aihitbot
awariosmartbot
baiduspider
barkrowler
blexbot
boost.beast
bspider
catexplorador
ccbot
checkmarknetwork
crawler4j
crawlson
curious george
daumoa
domainsbot
domaincrawler
domains project
domainsproject.org
domainstatsbot
dotbot
exabot
expanseinc
findxbot
foregenix
foregenix web scan
go.mail.ru
grover
ia_archiver
inoopa
intelx.io_bot
jakarta
lightspeedsystemscrawler
linguee
looid.com
ltx71
mail.ru_bot
mediatoolkitbot
megaindex
megaindex.ru
memorybot
miniredir
mixrankbot
mj12
mj12bot
mtrobot
nimbostratus
netcraftsurveyagent
netestate
netsystemsresearch
obot
opensiteexplorer
panscient.com
petalbot
photon
proximic
qwantify
semanticbot
semanticscholarbot
semrushbot
seokicks
seolyt
sirdatabot
smtbot
smrjbot
spbot
steeler
seznambot
tombapublicwebcrawler
ultraseek
uptimebot
vagabondo
vuhuv
wbsrch
yak
yandex
zgrab

Rule Path
Disallow /

*

Rule Path
Disallow /*.cfc$
Disallow /*.cgi$
Disallow /*.csv$
Disallow /*.doc$
Disallow /*.gz$
Disallow /*.inc$
Disallow /*.ini$
Disallow /*.pdf$
Disallow /*.ppt$
Disallow /*.svg$
Disallow /*.tar$
Disallow /*.xls$
Disallow /*.zip$
Disallow /c_*
Disallow /m_*
Disallow /x_*
Disallow /view_*
Disallow /demo.files*

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://belgianfoodtruckfederation.org/sitemap.xml
Back to top

Comments

  • __ ___ __
  • |__) |__ | / _` | /\ |\ |
  • |__) |___ |___ \__> | /--\ | \|
  • ___ __ __ __ ___ __ __
  • |__ / \/ \| \ | |__)| |/ `|__/
  • | \__/\__/|__/ | | \\__/\__,| \
  • ___ ___ __ ___ __ ___ __
  • |__ |__ | \|__ |__) /\ | | / \|\ |
  • | |___|__/|___| \/--\ | | \__/| \|
  • ) .--,--.
  • ( ( ( `. ',.'
  • ) ) |___|
  • ____FOODTRUCK__ __FOODTRUCK___
  • /__|| | | | | | | ||__\
  • | _-_| `======´ | | `======´ |_-_ |
  • `-(o)-------(o)-´ `-(o)-------(o)-´
Back to top

Warnings

  • 1 invalid line.
Back to top