fcsurplus.com
robots.txt
Robots Exclusion Standard data for fcsurplus.com
Resource Scan
Scan Details
Site Domain | fcsurplus.com |
Base Domain | fcsurplus.com |
Scan Status | Ok |
Last Scan | 2024-05-19T12:51:54+00:00 |
Next Scan | 2024-05-26T12:51:54+00:00 |
Last Scan
Scanned | 2024-05-19T12:51:54+00:00 |
URL | https://fcsurplus.com/robots.txt |
Domain IPs | 104.26.8.174, 104.26.9.174, 172.67.71.120, 2606:4700:20::681a:8ae, 2606:4700:20::681a:9ae, 2606:4700:20::ac43:4778 |
Response IP | 104.26.9.174 |
Found | Yes |
Hash | 15f9f06face2077aff1bcecdef8361b61749341a71a0de322a1122779d2f31d5 |
SimHash | e2b77b2349e3 |
Groups
*
googlebot
googlebot-image
bingbot
mediapartners-google
msnbot
msnbot-media
slurp
yahoo-blogs
yahoo-mmcrawler
ia_archiver
duckduckbot
exabot
facebot
Rule | Path |
---|---|
Disallow | /cgi-bin/ |
Disallow | /main_images/ |
Disallow | /stats/ |
Disallow | /forum/ |
Disallow | /webalizer/ |
Disallow | /clients/ |
Disallow | /awstats/ |
Disallow | /DB/ |
Disallow | /data/ |
Disallow | /aspnet_client/ |
Disallow | /vote/ |
Disallow | /test/ |
Disallow | /mxq/ |
Disallow | /cgi-bin |
Disallow | /main_images |
Disallow | /stats |
Disallow | /forum |
Disallow | /subscribe |
Disallow | /webalizer |
Disallow | /clients |
Disallow | /awstats |
Disallow | /DB |
Disallow | /data |
Disallow | /aspnet_client |
Disallow | /emailstuff |
Disallow | /sspace |
Disallow | /_vti_cnf |
Disallow | /_vti_log |
Disallow | /_vti_pvt |
Disallow | /vote |
Disallow | /test |
Disallow | /mxq |
Disallow | /searchenginebuttons |
Disallow | /shopping/main_images |
Other Records
Field | Value |
---|---|
sitemap | https://www.fcsurplus.com/sitemap.xml |
Comments