thefridaytimes.com
robots.txt

Robots Exclusion Standard data for thefridaytimes.com

Resource Scan

Scan Details

Site Domain thefridaytimes.com
Base Domain thefridaytimes.com
Scan Status Ok
Last Scan2026-01-25T15:13:13+00:00
Next Scan 2026-02-01T15:13:13+00:00

Last Scan

Scanned2026-01-25T15:13:13+00:00
URL https://thefridaytimes.com/robots.txt
Domain IPs 65.21.10.187
Response IP 65.21.10.187
Found Yes
Hash c06a8023af7f66cbd878b1e4fa5699b54bd9bf81d0a0becc26ce35b6dd1151d4
SimHash 59b558c1b594

Groups

*

Rule Path
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

googlebot

Rule Path
Allow /

google-extended

Rule Path
Allow /

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

claudebot

Rule Path
Allow /

ccbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

youbot

Rule Path
Allow /

neevabot

Rule Path
Allow /

amazonbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

applebot

Rule Path
Allow /

yandex

Rule Path
Allow /
Disallow /admin/
Disallow /test-ad-code/
Disallow /xajax/
Disallow /ajax_post_update
Disallow /ajax_post_pagination
Disallow /ajax_author_type_pagination
Disallow /ajax_block_pagination
Disallow /ajax_poll_pagination
Disallow /poll_entry_submission
Disallow /parse_oembed_links
Disallow /oembed
Disallow /oembed_videos

Other Records

Field Value
sitemap https://thefridaytimes.com/sitemap_news.xml
sitemap https://thefridaytimes.com/sitemap_news_google.xml
sitemap https://thefridaytimes.com/sitemap.xml