edge-star-mini-shv-02-hkt1.facebook.com
robots.txt
Robots Exclusion Standard data for edge-star-mini-shv-02-hkt1.facebook.com
Resource Scan
Scan Details
Site Domain | edge-star-mini-shv-02-hkt1.facebook.com |
Base Domain | facebook.com |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Couldn't connect to server. |
Last Scan | 2024-04-06T05:37:46+00:00 |
Next Scan | 2024-07-05T05:37:46+00:00 |
Last Successful Scan
Scanned | 2023-12-02T05:36:11+00:00 |
URL | https://edge-star-mini-shv-02-hkt1.facebook.com/robots.txt |
Domain IPs | 31.13.75.35 |
Response IP | 31.13.75.35 |
Found | Yes |
Hash | a247a411e83a281b0dbb572dcbfcb9b5eb290853de48555c13cfdd0767eda380 |
SimHash | f85147144739 |
Groups
applebot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
baiduspider
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
bingbot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
discordbot
Rule | Path |
---|---|
Disallow | / |
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
facebookexternalhit
Rule | Path |
---|---|
Disallow | / |
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
googlebot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
googlebot-image
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /map_tile.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /static_map.php |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
ia_archiver
Rule | Path |
---|---|
Disallow | / |
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
linkedinbot
Rule | Path |
---|---|
Disallow | / |
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
msnbot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
naverbot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
pinterestbot
Rule | Path |
---|---|
Disallow | / |
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
screaming frog seo spider
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
seznambot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
slurp
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
teoma
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
telegrambot
Rule | Path |
---|---|
Disallow | / |
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
twitterbot
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
yandex
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
yeti
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /album.php |
Disallow | /checkpoint/ |
Disallow | /contact_importer/ |
Disallow | /dialog/ |
Disallow | /fbml/ajax/dialog/ |
Disallow | /feeds/ |
Disallow | /file_download.php |
Disallow | /job_application/ |
Disallow | /l.php |
Disallow | /moments_app/ |
Disallow | /p.php |
Disallow | /photos.php |
Disallow | /plugins/ |
Disallow | /share.php |
Disallow | /share/ |
Disallow | /sharer.php |
Disallow | /sharer/ |
Disallow | /tr/ |
Disallow | /tr? |
Disallow | /x/oauth/ |
applebot
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
baiduspider
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
bingbot
Rule | Path |
---|---|
Allow | /*/videos/ |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
Allow | /watch |
discordbot
Rule | Path |
---|---|
Allow | /*/videos/ |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
Allow | /watch/?v=* |
facebookexternalhit
Rule | Path |
---|---|
Allow | /*/videos/ |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
Allow | /watch/?v=* |
googlebot
Rule | Path |
---|---|
Allow | /*/videos/ |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
Allow | /watch |
ia_archiver
Rule | Path |
---|---|
Allow | /about/privacy |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /full_data_use_policy |
Allow | /legal/terms |
Allow | /policy.php |
Allow | /safetycheck/ |
linkedinbot
Rule | Path |
---|---|
Allow | /*/videos/ |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
Allow | /watch/?v=* |
msnbot
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
naverbot
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
pinterestbot
Rule | Path |
---|---|
Allow | /*/videos/ |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
Allow | /watch/?v=* |
screaming frog seo spider
Rule | Path |
---|---|
Allow | /*/videos/ |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
Allow | /watch |
seznambot
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
slurp
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
teoma
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
telegrambot
Rule | Path |
---|---|
Allow | /*/videos/ |
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
Allow | /watch/?v=* |
twitterbot
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
yandex
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
yeti
Rule | Path |
---|---|
Allow | /ajax/bootloader-endpoint/ |
Allow | /ajax/pagelet/generic.php/PagePostsSectionPagelet |
Allow | /careers/ |
Allow | /safetycheck/ |
*
Rule | Path |
---|---|
Disallow | / |
Comments