allofapps.com
robots.txt

Robots Exclusion Standard data for allofapps.com

Resource Scan

Scan Details

Site Domain allofapps.com
Base Domain allofapps.com
Scan Status Ok
Last Scan2024-11-12T17:04:47+00:00
Next Scan 2024-11-19T17:04:47+00:00

Last Scan

Scanned2024-11-12T17:04:47+00:00
URL https://allofapps.com/robots.txt
Redirect https://www.allofapps.com/robots.txt
Redirect Domain www.allofapps.com
Redirect Base allofapps.com
Domain IPs 104.18.20.108, 104.18.21.108, 2606:4700::6812:146c, 2606:4700::6812:156c
Redirect IPs 104.18.20.108, 104.18.21.108, 2606:4700::6812:146c, 2606:4700::6812:156c
Response IP 104.18.20.108
Found Yes
Hash 574e1908113bad49fd848c82bdb6b19525daf3a3f07216f0b81c62ee9a108ff5
SimHash 1814c83a4e10

Groups

*

Rule Path
Disallow /search
Disallow */developer/*
Disallow /version/
Disallow /*/search

baiduspider*
sogou*

Rule Path
Disallow /

scrapy
ahrefsbot
blexbot
ccbot
cliqzbot
dotbot
ia_archiver
mbcrawler
mj12bot
photon
linguee

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.allofapps.com/sitemap_index.xml