appisfree.com
robots.txt

Robots Exclusion Standard data for appisfree.com

Resource Scan

Scan Details

Site Domain appisfree.com
Base Domain appisfree.com
Scan Status Ok
Last Scan2024-06-12T14:45:09+00:00
Next Scan 2024-06-19T14:45:09+00:00

Last Scan

Scanned2024-06-12T14:45:09+00:00
URL https://appisfree.com/robots.txt
Redirect https://www.appisfree.com/robots.txt
Redirect Domain www.appisfree.com
Redirect Base appisfree.com
Domain IPs 104.18.14.17, 104.18.15.17, 2606:4700::6812:e11, 2606:4700::6812:f11
Redirect IPs 104.18.14.17, 104.18.15.17, 2606:4700::6812:e11, 2606:4700::6812:f11
Response IP 104.18.15.17
Found Yes
Hash cee4a77fd95e14a3755a8132104827fb24b19a2e555dc79f39e69539ade0df8c
SimHash 4874c8788f10

Groups

*

Rule Path
Disallow /*about.html$
Disallow /*contact.html$
Disallow /*guidance.html$
Disallow /*disclaimer.html$
Disallow /*dmca.html$
Disallow /*privacyPolicy.html$
Disallow /*rules.html$
Disallow /search
Disallow /*/search
Disallow /developer/*
Disallow /version/*

baiduspider*
sogou*

Rule Path
Disallow /

scrapy
ahrefsbot
blexbot
ccbot
cliqzbot
dotbot
ia_archiver
mbcrawler
mj12bot
photon
linguee
amazonbot
barkrowler
bytespider
zoominfobot
seznambot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.appisfree.com/sitemap_index.xml