asianboxing.info
robots.txt

Robots Exclusion Standard data for asianboxing.info

Resource Scan

Scan Details

Site Domain asianboxing.info
Base Domain asianboxing.info
Scan Status Ok
Last Scan2024-11-08T22:34:08+00:00
Next Scan 2024-11-15T22:34:08+00:00

Last Scan

Scanned2024-11-08T22:34:08+00:00
URL https://asianboxing.info/robots.txt
Redirect https://www.asianboxing.info/robots.txt
Redirect Domain www.asianboxing.info
Redirect Base asianboxing.info
Domain IPs 199.34.228.41
Redirect IPs 199.34.228.41
Response IP 199.34.228.41
Found Yes
Hash 789cf6640f7178cfdbc68eb85dfe6806d9699b6fddedd71a0b8a2407dd4df9ee
SimHash 7b17c87e2ebb

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /http%3A//www.tbrb.org/all-rankings/
Disallow /http%3A//assets.ibf-usba-boxing.com/ratings/IBF102m.pdf
Disallow /http%3A//www.wbanews.com/wp-content/uploads/rankings/2014-may.pdf
Disallow /http%3A//wbcboxing.com/ratings/ratings_Junio2014.pdf
Disallow /http%3A//www.wboboxing.com/wp-content/uploads/2014/06/WBO-Ranking-as-of-Jun.-20142.pdf
Disallow /http%3A//www.opbf.jp/data/ratings/2014/06.pdf
Disallow /http%3A//www.jbc.or.jp/ranking/rank/2014/e/201406e.pdf
Disallow /http%3A//ringtv.craveonline.com/ratings
Disallow /http%3A//www.pabaonline.com/index2.php
Disallow /http%3A//www.iboboxing.com/top_100_computerized_rankings.html
Disallow /http%3A//boxrec.com/ratings.php
Disallow /http%3A//asianboxing.proboards.com
Disallow /https%3A//docs.google.com/spreadsheets/d/1kbOW-0gyftmDh87nrIZ8cWpu9L9GWygjPnksRMgx-eg/edit?usp=sharing

Other Records

Field Value
sitemap https://www.asianboxing.info/sitemap.xml