billiongraves.com
robots.txt

Robots Exclusion Standard data for billiongraves.com

Resource Scan

Scan Details

Site Domain billiongraves.com
Base Domain billiongraves.com
Scan Status Ok
Last Scan2024-05-18T18:16:04+00:00
Next Scan 2024-05-25T18:16:04+00:00

Last Scan

Scanned2024-05-18T18:16:04+00:00
URL https://billiongraves.com/robots.txt
Domain IPs 69.167.60.71
Response IP 69.167.60.71
Found Yes
Hash 542c2c51560c42e9762fba0fcee867ebf6529dbc3827fa8eaee52ca0064bc1e8
SimHash f716894cbbd3

Groups

ahrefsbot
ezooms
mj12bot
xovibot
baiduspider
baidu
yandex
yandexbot
proximic
claudebot
semrushbot
blexbot
petalbot
neckerbot
criteobot
bytespider
imagesiftbot
brightbot
censysinspect
foregenix

Rule Path
Disallow /

*

Rule Path
Disallow /backend/
Disallow /android/
Disallow /maintainance/
Disallow /js/
Disallow /includes/
Disallow *?*returl=*
Disallow /add-batch
Disallow /add-cemetery
Disallow /add-photo-request
Disallow /add-record
Disallow /bgplus-manage-subscription
Disallow /cemeteries
Disallow /contact-us
Disallow /create-event
Disallow /familysearch-link
Disallow /favorites
Disallow /how-to-transcribe
Disallow /leader-board
Disallow /messages
Disallow /my-*
Disallow /pages/*
Disallow /photo-requests
Disallow /profile
Disallow /search/results
Disallow /settings
Disallow /site-map
Disallow /site-index
Disallow /translate
Disallow /transcribe
Disallow /tree
Disallow /cemetery/*/*/edit
Disallow /cemetery/*/*/image
Disallow /cemetery/*/*/boundary
Disallow /grave/*/*/edit
Disallow /grave/*/*/image
Disallow /grave/*/*/family
Disallow /grave/*/*/memory
Disallow /grave/*/*/military
Disallow /grave/*/*/timeline
Disallow /grave/*/*/editLocation

Other Records

Field Value
sitemap https://billiongraves.com/sitemap.xml
sitemap https://billiongraves.com/cemetery-siteindex.xml
sitemap https://billiongraves.com/record-siteindex.xml
sitemap https://billiongraves.com/supp-siteindex.xml

Comments

  • Domain:[billiongraves.com]
  • LAST CHANGED: Wed Oct 17 16:37:19 UTC 2018
  • A few user agents that we don't want to index the site
  • Don't index stuff from these folders

Warnings

  • `noindex` is not a known field.