airdate.cc
robots.txt

Robots Exclusion Standard data for airdate.cc

Resource Scan

Scan Details

Site Domain airdate.cc
Base Domain airdate.cc
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2025-12-21T00:35:09+00:00
Next Scan 2026-02-19T00:35:09+00:00

Last Successful Scan

Scanned2025-10-20T22:20:56+00:00
URL https://airdate.cc/robots.txt
Domain IPs 168.119.89.50, 2a01:4f8:251:520d::1
Response IP 168.119.89.50
Found Yes
Hash fb549a443c41a3ae242eadfb048ed75604d57839742a5294366a2fb17eaa4d18
SimHash a01cab5316b0

Groups

yanga

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

msnbot/

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider+(

Rule Path
Disallow /

mj12bot/

Rule Path
Disallow /

gigabot/

Rule Path
Disallow /

msnbot-media/

Rule Path
Disallow /

scooter/

Rule Path
Disallow /

ask jeeves

Rule Path
Disallow /

msnbot-newsblogs/

Rule Path
Disallow /

seo search crawler/

Rule Path
Disallow /

seosearch/

Rule Path
Disallow /

voyager/1.0

Rule Path
Disallow /

w3 sitesearch crawler

Rule Path
Disallow /

w3c-checklink/

Rule Path
Disallow /

w3c_*validator

Rule Path
Disallow /

yahoo-mmcrawler/

Rule Path
Disallow /

yahoo! de slurp

Rule Path
Disallow /

yahooseeker/

Rule Path
Disallow /

*

Rule Path
Allow /uploads/
Disallow */dg-admin*
Disallow */dg-core*
Disallow /bookmarks*
Disallow /reviews*
Disallow /users*
Disallow /help*
Disallow /myshows*
Disallow /myschedule*
Disallow /feedback*
Disallow /search*
Disallow /personal*
Disallow /enter*
Disallow /forgotpass*
Disallow /registration*
Disallow /ua/bookmarks*
Disallow /ua/reviews*
Disallow /ua/users*
Disallow /ua/help*
Disallow /ua/myshows*
Disallow /ua/myschedule*
Disallow /ua/feedback*
Disallow /ua/search*
Disallow /ua/personal*
Disallow /ua/enter*
Disallow /ua/forgotpass*
Disallow /ua/registration*
Disallow /en/bookmarks*
Disallow /en/reviews*
Disallow /en/users*
Disallow /en/help*
Disallow /en/myshows*
Disallow /en/myschedule*
Disallow /en/feedback*
Disallow /en/search*
Disallow /en/personal*
Disallow /en/enter*
Disallow /en/forgotpass*
Disallow /en/registration*
Disallow *zyorder%3D*
Disallow *?zsort=*
Disallow /out
Allow /*.js*
Allow /*.css*
Allow /*.jpg*
Allow /*.gif*
Allow /*.png*

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://airdate.cc/sitemap_index.xml

Comments

  • Disallow: *?p=*

Warnings

  • `host` is not a known field.