Log In Sign Up

awardwallet.com
robots.txt

Robots Exclusion Standard data for awardwallet.com

Resource Scan

Scan Details

Site Domain awardwallet.com
Base Domain awardwallet.com
Scan Status Ok
Last Scan2024-10-30T21:51:07+00:00
Next Scan 2024-11-06T21:51:07+00:00

Last Scan

Scanned2024-10-30T21:51:07+00:00
URL https://awardwallet.com/robots.txt
Domain IPs 2600:1f18:230b:f801:ef72:d591:335c:e9a1, 2600:1f18:230b:f802:3f60:e91d:b108:7a5a, 52.6.236.159, 54.174.11.107
Response IP 52.6.236.159
Found Yes
Hash 50486975a0a6426151e7ffef569e6a96cbe9446cef880ca28964fc1026697ef6
SimHash 4816d3278b83

Groups

facebookexternalhit

Rule Path
Allow /

*

Rule Path
Disallow /admin/
Disallow /trips/
Disallow /blog/wp-admin/
Disallow /share/
Disallow /*/set_locale$
Disallow /r/
Disallow /restore
Disallow /blog/cards/
Disallow /timeline/summary/blog/
Disallow /*?
Allow /*?extract

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://awardwallet.com/sitemap.xml
sitemap https://awardwallet.com/sitemap-blog-posts-0.xml
sitemap https://awardwallet.com/sitemap-blog-posts-1.xml
sitemap https://awardwallet.com/sitemap-blog-posts-2.xml
sitemap https://awardwallet.com/sitemap-blog-posts-3.xml
sitemap https://awardwallet.com/sitemap-blog-posts-4.xml
sitemap https://awardwallet.com/sitemap-blog-posts-5.xml
sitemap https://awardwallet.com/sitemap-blog-posts-6.xml
sitemap https://awardwallet.com/sitemap-blog-posts-7.xml
sitemap https://awardwallet.com/sitemap-blog-posts-8.xml
sitemap https://awardwallet.com/sitemap-blog-posts-9.xml
sitemap https://awardwallet.com/sitemap-blog-posts-10.xml
sitemap https://awardwallet.com/sitemap-blog-posts-11.xml
sitemap https://awardwallet.com/sitemap-blog-posts-12.xml
sitemap https://awardwallet.com/sitemap-blog-posts-13.xml
sitemap https://awardwallet.com/sitemap-blog-posts-14.xml
sitemap https://awardwallet.com/sitemap-site.xml
sitemap https://awardwallet.com/sitemap-blog-pages.xml
sitemap https://awardwallet.com/sitemap-blog-authors.xml
sitemap https://awardwallet.com/sitemap-blog-tags.xml
sitemap https://awardwallet.com/sitemap-blog-categories.xml
sitemap https://awardwallet.com/sitemap-blog-news.xml
Back to top

Warnings

  • 2 invalid lines.
Back to top