appurse.com
robots.txt

Robots Exclusion Standard data for appurse.com

Resource Scan

Scan Details

Site Domain appurse.com
Base Domain appurse.com
Scan Status Ok
Last Scan2024-05-18T03:34:15+00:00
Next Scan 2024-06-17T03:34:15+00:00

Last Scan

Scanned2024-05-18T03:34:15+00:00
URL https://appurse.com/robots.txt
Redirect https://www.appurse.com/robots.txt
Redirect Domain www.appurse.com
Redirect Base appurse.com
Domain IPs 104.20.30.121, 104.20.31.121, 2606:4700:10::6814:1e79, 2606:4700:10::6814:1f79
Redirect IPs 104.20.30.121, 104.20.31.121, 2606:4700:10::6814:1e79, 2606:4700:10::6814:1f79
Response IP 104.20.31.121
Found Yes
Hash 7406b4f4f9382794c40608688abef35f0af1e35dbd5ae145731374e20305af11
SimHash 581cc0727f50

Groups

*

Rule Path
Disallow /articles.html
Disallow /articles/*.html
Disallow /search
Disallow /show_more.html
Disallow /viewmore.html
Disallow /down/*
Disallow /APPURSE/
Disallow /each_country.html
Disallow /APPURSE-admin/
Disallow /text/APPURSE/
Disallow /eyeblaster/
Disallow /addineyeV2.html

baiduspider*
sogou*

Rule Path
Disallow /

ahrefsbot
blexbot
ccbot
cliqzbot
dotbot
ia_archiver
mbcrawler
mj12bot
photon
scrapy
semrushbot
linguee

Rule Path
Disallow /