edge-star-mini-shv-01-any2.facebook.com
robots.txt

Robots Exclusion Standard data for edge-star-mini-shv-01-any2.facebook.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	edge-star-mini-shv-01-any2.facebook.com
Base Domain	facebook.com
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Couldn't connect to server.
Last Scan	2024-04-08T12:59:05+00:00
Next Scan	2024-07-07T12:59:05+00:00

Last Successful Scan

Scanned	2023-03-09T12:01:37+00:00
URL	https://edge-star-mini-shv-01-any2.facebook.com/robots.txt
Domain IPs	69.171.250.35
Response IP	69.171.250.35
Found	Yes
Hash	f42b8c10899049c3d68d3108c8fd9acf70a57b8ba77849fc4226ecedaa5dd7e4
SimHash	f85157144739

Groups

applebot

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

baiduspider

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

bingbot

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

discordbot

Rule	Path
Disallow	/
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

facebookexternalhit

Rule	Path
Disallow	/
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

googlebot

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

googlebot-image

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/map_tile.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/static_map.php
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/map_tile.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/static_map.php

Disallow

/tr/

Disallow

/tr?

ia_archiver

Rule	Path
Disallow	/
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

linkedinbot

Rule	Path
Disallow	/
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

msnbot

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

naverbot

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

pinterestbot

Rule	Path
Disallow	/
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

screaming frog seo spider

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

seznambot

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

slurp

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

teoma

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

telegrambot

Rule	Path
Disallow	/
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

twitterbot

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

yandex

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

yeti

Rule	Path
Disallow	/ajax/
Disallow	/album.php
Disallow	/checkpoint/
Disallow	/contact_importer/
Disallow	/dialog/
Disallow	/fbml/ajax/dialog/
Disallow	/feeds/
Disallow	/file_download.php
Disallow	/job_application/
Disallow	/l.php
Disallow	/moments_app/
Disallow	/p.php
Disallow	/photo.php
Disallow	/photos.php
Disallow	/plugins/
Disallow	/share.php
Disallow	/share/
Disallow	/sharer.php
Disallow	/sharer/
Disallow	/tr/
Disallow	/tr?

Rule

Path

Disallow

/ajax/

Disallow

/album.php

Disallow

/checkpoint/

Disallow

/contact_importer/

Disallow

/dialog/

Disallow

/fbml/ajax/dialog/

Disallow

/feeds/

Disallow

/file_download.php

Disallow

/job_application/

Disallow

/l.php

Disallow

/moments_app/

Disallow

/p.php

Disallow

/photo.php

Disallow

/photos.php

Disallow

/plugins/

Disallow

/share.php

Disallow

/share/

Disallow

/sharer.php

Disallow

/sharer/

Disallow

/tr/

Disallow

/tr?

applebot

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

baiduspider

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

bingbot

Rule	Path
Allow	/*/videos/
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/
Allow	/watch

Rule

Path

Allow

/*/videos/

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

Allow

/watch

discordbot

Rule	Path
Allow	/*/videos/
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/
Allow	/watch/?v=*

Rule

Path

Allow

/*/videos/

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

Allow

/watch/?v=*

facebookexternalhit

Rule	Path
Allow	/*/videos/
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/
Allow	/watch/?v=*

Rule

Path

Allow

/*/videos/

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

Allow

/watch/?v=*

googlebot

Rule	Path
Allow	/*/videos/
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/
Allow	/watch

Rule

Path

Allow

/*/videos/

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

Allow

/watch

ia_archiver

Rule	Path
Allow	/about/privacy
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/full_data_use_policy
Allow	/legal/terms
Allow	/policy.php
Allow	/safetycheck/

Rule

Path

Allow

/about/privacy

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/full_data_use_policy

Allow

/legal/terms

Allow

/policy.php

Allow

/safetycheck/

linkedinbot

Rule	Path
Allow	/*/videos/
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/
Allow	/watch/?v=*

Rule

Path

Allow

/*/videos/

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

Allow

/watch/?v=*

msnbot

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

naverbot

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

pinterestbot

Rule	Path
Allow	/*/videos/
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/
Allow	/watch/?v=*

Rule

Path

Allow

/*/videos/

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

Allow

/watch/?v=*

screaming frog seo spider

Rule	Path
Allow	/*/videos/
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/
Allow	/watch

Rule

Path

Allow

/*/videos/

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

Allow

/watch

seznambot

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

slurp

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

teoma

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

telegrambot

Rule	Path
Allow	/*/videos/
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/
Allow	/watch/?v=*

Rule

Path

Allow

/*/videos/

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

Allow

/watch/?v=*

twitterbot

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

yandex

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

yeti

Rule	Path
Allow	/ajax/bootloader-endpoint/
Allow	/ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow	/careers/
Allow	/safetycheck/

Rule

Path

Allow

/ajax/bootloader-endpoint/

Allow

/ajax/pagelet/generic.php/PagePostsSectionPagelet

Allow

/careers/

Allow

/safetycheck/

*

Rule	Path
Disallow	/

Rule

Path

Disallow

Comments

Notice: Collection of data on Facebook through automated means is
prohibited unless you have express written permission from Facebook
and may only be conducted for the limited purpose contained in said
permission.
See: http://www.facebook.com/apps/site_scraping_tos_terms.php

edge-star-mini-shv-01-any2.facebook.comrobots.txt

Resource Scan

Scan Details

Last Successful Scan

Groups

applebot

baiduspider

bingbot

discordbot

facebookexternalhit

googlebot

googlebot-image

ia_archiver

linkedinbot

msnbot

naverbot

pinterestbot

screaming frog seo spider

seznambot

slurp

teoma

telegrambot

twitterbot

yandex

yeti

applebot

baiduspider

bingbot

discordbot

facebookexternalhit

googlebot

ia_archiver

linkedinbot

msnbot

naverbot

pinterestbot

screaming frog seo spider

seznambot

slurp

teoma

telegrambot

twitterbot

yandex

yeti

*

Comments

edge-star-mini-shv-01-any2.facebook.com
robots.txt