jayegardiner.com
robots.txt

Robots Exclusion Standard data for jayegardiner.com

Resource Scan

Scan Details

Site Domain jayegardiner.com
Base Domain jayegardiner.com
Scan Status Ok
Last Scan2025-04-24T02:40:31+00:00
Next Scan 2025-05-24T02:40:31+00:00

Last Scan

Scanned2025-04-24T02:40:31+00:00
URL https://jayegardiner.com/robots.txt
Redirect https://www.jayegardiner.com/robots.txt
Redirect Domain www.jayegardiner.com
Redirect Base jayegardiner.com
Domain IPs 199.34.228.49
Redirect IPs 199.34.228.49
Response IP 199.34.228.49
Found Yes
Hash 517d67ac4b3d925570c8150058a0f0d82be0592063ed2fe686b65b7628d5f6ae
SimHash 4934f8c46393

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /digital-illustrations.html
Disallow /uniquescientists.html
Disallow /tradingcards.html
Disallow /womennb-in-stem.html
Disallow /portraits.html
Disallow /ifthen-ambassadors.html
Disallow /lgbtqia-in-stem.html
Disallow /illustrations.html
Disallow /http%3A//www.jkxcomics.com
Disallow /latinx-in-stem.html
Disallow /history-month-highlights.html
Disallow /commissions.html
Disallow /black-in-stem.html
Disallow /outreach.html
Disallow /comics--illustrations.html
Disallow /first-gen-in-stem.html
Disallow /disabled-in-stem.html

Other Records

Field Value
sitemap https://www.jayegardiner.com/sitemap.xml