vtools.ieee.org
robots.txt

Robots Exclusion Standard data for vtools.ieee.org

Resource Scan

Scan Details

Site Domain vtools.ieee.org
Base Domain ieee.org
Scan Status Ok
Last Scan2024-09-12T07:35:50+00:00
Next Scan 2024-10-12T07:35:50+00:00

Last Scan

Scanned2024-09-12T07:35:50+00:00
URL https://vtools.ieee.org/robots.txt
Domain IPs 35.231.9.82
Response IP 35.231.9.82
Found Yes
Hash 17e13b339721e03ad8a388db125022b78ace944b37b6156e8f059a13b76e8eb4
SimHash 784077916149

Groups

*

Rule Path
Disallow *post_type%3Dtribe_events*
Disallow *hide_subsequent_recurrences%3D*
Disallow *tribe-bar-date%3D*
Disallow *tribe-venue%3D*
Disallow *eventDisplay%3D*
Disallow *eventDate%3D*
Disallow *paged%3D*
Disallow *pagename%3D*
Disallow *shortcode%3D*
Disallow *ical%3D*
Disallow *outlook-ical%3D*
Disallow *related_series%3D*
Disallow *tribe_geofence%3D*
Disallow /calendar/action~posterboard/
Disallow /calendar/action~agenda/
Disallow /calendar/action~oneday/
Disallow /calendar/action~month/
Disallow /calendar/action~week/
Disallow /calendar/action~stream/
Disallow /calendar/action~undefined/
Disallow /calendar/action~http%3A/
Disallow /calendar/action~default/
Disallow /calendar/action~poster/
Disallow /calendar/action~*/
Disallow /*controller%3Dai1ec_exporter_controller*
Disallow /*/action~*/
Disallow /program-2/action~posterboard/
Disallow /program-2/action~agenda/
Disallow /program-2/action~oneday/
Disallow /program-2/action~month/
Disallow /program-2/action~week/
Disallow /program-2/action~stream/
Disallow /program-2/action~undefined/
Disallow /program-2/action~http%3A/
Disallow /program-2/action~default/
Disallow /program-2/action~poster/
Disallow /program-2/action~*/

Other Records

Field Value
crawl-delay 20

applebot

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

baiduspider

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

bingbot

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

googlebot

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

googlebot-image

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /map_tile.php
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/
Disallow /static_map.php

ia_archiver

Rule Path
Disallow /
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

msnbot

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

naverbot

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

seznambot

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

slurp

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

teoma

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

twitterbot

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

yandex

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

yeti

Rule Path
Disallow /ajax/
Disallow /album.php
Disallow /checkpoint/
Disallow /contact_importer/
Disallow /dialog/
Disallow /fbml/ajax/dialog/
Disallow /feeds/
Disallow /file_download.php
Disallow /hashtag/
Disallow /l.php
Disallow /live/
Disallow /moments_app/
Disallow /p.php
Disallow /photo.php
Disallow /photos.php
Disallow /share.php
Disallow /share/
Disallow /sharer.php
Disallow /sharer/

applebot

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

baiduspider

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

bingbot

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

googlebot

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

ia_archiver

Rule Path
Allow /about/privacy
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /full_data_use_policy
Allow /legal/terms
Allow /policy.php
Allow /safetycheck/

msnbot

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

naverbot

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

seznambot

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

slurp

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

teoma

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

twitterbot

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

yandex

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

yeti

Rule Path
Allow /ajax/bootloader-endpoint/
Allow /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow /safetycheck/

*

Rule Path
Disallow /

*

Rule Path
Disallow /?blackhole

Comments

  • Notice: Crawling Facebook is prohibited unless you have express written
  • permission. See: http://www.facebook.com/apps/site_scraping_tos_terms.php