pbnation.com
robots.txt
Robots Exclusion Standard data for pbnation.com
Resource Scan
Scan Details
| Site Domain | pbnation.com |
| Base Domain | pbnation.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-04-19T00:04:08+00:00 |
| Next Scan | 2024-07-18T00:04:08+00:00 |
Last Successful Scan
| Scanned | 2023-09-22T03:45:46+00:00 |
| URL | https://pbnation.com/robots.txt |
| Redirect | https://www.pbnation.com/robots.txt |
| Redirect Domain | www.pbnation.com |
| Redirect Base | pbnation.com |
| Domain IPs | 104.16.248.51, 104.16.249.51 |
| Redirect IPs | 104.17.97.5, 104.17.98.5, 2606:4700::6811:6105, 2606:4700::6811:6205 |
| Response IP | 104.17.97.5 |
| Found | Yes |
| Hash | cfc246e9f30a987a7320fc9c41ff59284f2aedbeb1d8c7612611ed32ddab8f73 |
| SimHash | 668a534d4e7d |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /archive/ |
| Disallow | /ajax.php |
| Disallow | /attachment.php |
| Disallow | /bump.php |
| Disallow | /calendar.php |
| Disallow | /cron.php |
| Disallow | /editpost.php |
| Disallow | /forumrunner/ |
| Disallow | /global.php |
| Disallow | /image.php |
| Disallow | /inlinemod.php |
| Disallow | /joinrequests.php |
| Disallow | /login.php |
| Disallow | /memberlist.php |
| Disallow | /misc.php |
| Disallow | /moderator.php |
| Disallow | /newattachment.php |
| Disallow | /newreply.php |
| Disallow | /newthread.php |
| Disallow | /online.php |
| Disallow | /out.php |
| Disallow | /poll.php |
| Disallow | /postings.php |
| Disallow | /printthread.php |
| Disallow | /private.php |
| Disallow | /profile.php |
| Disallow | /register.php |
| Disallow | /report.php |
| Disallow | /reputation.php |
| Disallow | /search.php |
| Disallow | /sendmessage.php |
| Disallow | /showgroups.php |
| Disallow | /subscription.php |
| Disallow | /threadrate.php |
| Disallow | /usercp.php |
| Disallow | /usernote.php |
| Disallow | /fielddiscuss.php |
bingbot
| Rule | Path |
|---|---|
| Disallow | /install/ |
| Disallow | /customavatars/ |
| Disallow | /admincp/ |
| Disallow | /clientscript/ |
| Disallow | /cpstyles/ |
| Disallow | /customprofilepics/ |
| Disallow | /modcp/ |
| Disallow | /ajax.php |
| Disallow | /attachment.php |
| Disallow | /calendar.php |
| Disallow | /cron.php |
| Disallow | /editpost.php |
| Disallow | /global.php |
| Disallow | /image.php |
| Disallow | /inlinemod.php |
| Disallow | /joinrequests.php |
| Disallow | /login.php |
| Disallow | /members/ |
| Disallow | /members.php |
| Disallow | /misc.php |
| Disallow | /moderator.php |
| Disallow | /newattachment.php |
| Disallow | /newreply.php |
| Disallow | /newthread.php |
| Disallow | /online.php |
| Disallow | /postings.php |
| Disallow | /printthread.php |
| Disallow | /private.php |
| Disallow | /register.php |
| Disallow | /report.php |
| Disallow | /reputation.php |
| Disallow | /search.php |
| Disallow | /sendmessage.php |
| Disallow | /showgroups.php |
| Disallow | /showpost.php |
| Disallow | /subscription.php |
| Disallow | /threadrate.php |
| Disallow | /usercp.php |
| Disallow | /usernote.php |
| Disallow | /customgroupicons/ |
| Disallow | /signaturepics/ |
| Disallow | /showthread.php |
| Disallow | /member.php |
| Disallow | /forumdisplay.php |
| Disallow | /blog.php |
| Disallow | /activity.php |
| Disallow | /index.php |
| Disallow | /reviewsubmit.php |
| Disallow | /feedbackdetail.php |
| Disallow | /external.php |
| Disallow | /fieldshow.php |
| Disallow | /review.php |
| Disallow | /fieldrevise.php |
| Disallow | /announcement.php |
| Disallow | /asm.php |
| Disallow | /leaderboard.php |
| Disallow | /faq.php |
| Disallow | /forum.php |
| Disallow | /bsttrust.php |
| Disallow | /up.php |
| Disallow | /feedback.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
slurp
yahoo! slurp
msnbot
msnbot-media
twitterbot
tweetedtimes bot
tweetmemebot
paperlibot
tineye-bot
qwantify
exabot
coccoc
blexbot
umbot-ln
queryseekerspider
yandex
yandexbot
yandexmobilebot
yandeximageresizer
yandeximages
sogou spider
sogou web spider
baiduspider
youdaobot
mj12bot
spbot
seznambot
ahrefsbot
linkdex
linkdexbot
semrushbot
semrushbot-sa
cfnetwork
sputnikbot
turnitinbot
proximic
grapeshot
cliqzbot
dotbot
exabot
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Warnings
- 1 invalid line.