fod.infobase.com
robots.txt

Robots Exclusion Standard data for fod.infobase.com

Resource Scan

Scan Details

Site Domain fod.infobase.com
Base Domain infobase.com
Scan Status Ok
Last Scan2025-11-03T15:45:07+00:00
Next Scan 2025-11-17T15:45:07+00:00

Last Scan

Scanned2025-11-03T15:45:07+00:00
URL https://fod.infobase.com/robots.txt
Domain IPs 54.196.229.154
Response IP 54.196.229.154
Found Yes
Hash 99faa349d1be372c62417535df25d2adaa3acae98ad3617b9b328e73720f3f3b
SimHash 411fe8538c37

Groups

amazonbot

Rule Path
Disallow /

*

Rule Path
Disallow /bin/
Disallow /LandingPages/Commcenter_NOCRAWL/
Disallow /StateStandards/NO_CRAWL/
Disallow /TempFiles_NOCRAWL/
Disallow /play/
Disallow Admin.aspx
Disallow Cart.aspx
Disallow CheckOut.aspx
Disallow ConfirmOrder.aspx
Disallow EmailItems.aspx
Disallow ErrorPage400.aspx
Disallow ErrorPage500.aspx
Disallow MyAccount.aspx
Disallow OrderSummary.aspx
Disallow PlaceOrder.aspx
Disallow PlayMedia.aspx
Disallow PopContactUs.aspx
Disallow PortalAdmin.aspx
Disallow PortalAssignments.aspx
Disallow PortalChatRoom.aspx
Disallow PortalFreeTrial.aspx
Disallow PortalFreeTrialWelcome.aspx
Disallow PortalPreferences.aspx
Disallow PortalProxyLogin.aspx
Disallow PortalViewVideo.aspx
Disallow PortalViewAssignment.aspx
Disallow PortalSignin.aspx
Disallow PreviewClip.aspx
Disallow PreviewClipFramed.aspx
Disallow PrintItems.aspx
Disallow Registration.aspx
Disallow RequestPwd.aspx
Disallow Search.aspx
Disallow SessionRedirector.aspx
Disallow Signin.aspx
Disallow SpecialOffers.aspx
Disallow StandardsByTitle.aspx
Disallow StandardsResults.aspx
Disallow Survey.aspx
Disallow SurveyCoupon.aspx
Disallow SurveyPromoNewProgramming.aspx
Disallow TrackingCode.aspx
Disallow WishList.aspx
Disallow PortalPlaylists.aspx
Disallow p_ViewVideo.aspx
Disallow p_Search.aspx
Disallow p_ViewPlaylist.aspx
Disallow OnDemandEmbed.aspx
Disallow OnDemandContentWidget.aspx
Disallow SocialLogin.aspx
Disallow Integrations.aspx
Disallow p_Login.aspx
Disallow KalturaStats.ashx
Disallow KalturaUpload.ashx
Disallow GoogleSearch.ashx
Disallow EventLogAjax.ashx
Disallow DownloadHandler.ashx
Disallow ChromeExtensionOAuth.ashx
Disallow AutoCompleteData.ashx
Disallow Autocomplete.asmx
Disallow ReadAloud.ashx

Other Records

Field Value
crawl-delay 2

yandexbot
serpstatbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semanticscholarbot
bubing
gluten free crawler
ahrefsbot
mj12bot
mauibot
semrushbot
quora-bot
googlebot

Rule Path
Disallow /