timeanddate.com
robots.txt

Robots Exclusion Standard data for timeanddate.com

Resource Scan

Scan Details

Site Domain timeanddate.com
Base Domain timeanddate.com
Scan Status Ok
Last Scan2024-05-03T14:13:32+00:00
Next Scan 2024-05-10T14:13:32+00:00

Last Scan

Scanned2024-05-03T14:13:32+00:00
URL https://timeanddate.com/robots.txt
Redirect https://www.timeanddate.com/robots.txt
Redirect Domain www.timeanddate.com
Redirect Base timeanddate.com
Domain IPs 108.59.14.4, 162.210.194.115
Redirect IPs 104.18.20.57, 104.18.21.57
Response IP 104.18.21.57
Found Yes
Hash f29cbbf6360a75e79e01882c5df6f6e139d98fd6deb47162863ac4e178b04c01
SimHash fc370eb2f357

Groups

msiecrawler

Rule Path
Disallow /

portalbspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

mediapartners-google*

Rule Path
Disallow

yahoo pipes 1.0

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /*?
Disallow /information/privacy.html
Disallow /information/terms-conditions.html

dotbot

Rule Path
Disallow /*?
Disallow /information/privacy.html
Disallow /information/terms-conditions.html

tadcrawler

Rule Path
Disallow /calendar/*?year*&
Disallow /eclipse/globe/*?
Disallow /eclipse/map/*?
Allow /information/privacy.html
Allow /information/terms-conditions.html
Allow /eclipse/globe/202
Allow /eclipse/map/202
Allow /calendar/*?year=2020
Allow /calendar/*?year=2021
Allow /calendar/*?year=2022
Allow /calendar/*?year=2023
Allow /calendar/*?year=2024
Allow /calendar/*?year=2025
Allow /astronomy/moon/distance.html?year=2022
Allow /astronomy/moon/distance.html?year=2023
Allow /astronomy/moon/distance.html?year=2024
Allow /worldclock/clockchange.html?*year=2022
Allow /worldclock/clockchange.html?*year=2023
Allow /worldclock/clockchange.html?*year=2024
Disallow /*y1%3D
Disallow /*nojs%3D
Disallow /worldclock/fixedform.html?
Disallow /worldclock/fixedtime.html?
Disallow /worldclock/sunearth.html?
Disallow /worldclock/sunearth.html?iso
Disallow /worldclock/sunearth.html?earth
Disallow /astronomy/moon/light.html?
Disallow /worldclock/timezone.html?*syear
Disallow /scripts/goweather.php
Disallow /scripts/ics.php
Disallow /worldclock/dialing.html?
Disallow /worldclock/custom.html?nurlid=
Disallow /worldclock/custom.html?cities=
Disallow /calendar/*?year
Disallow /calendar/seasons.html?
Disallow /clocks/freecountdown.html?n=
Disallow /clocks/countdown.html?n=
Disallow /countdown/create?
Disallow /countdown/eclipse?
Disallow /countdown/generic?
Disallow /countdown/to?
Disallow /astronomy/moon/distance.html?year
Disallow /eclipse/globe/
Disallow /eclipse/map/
Disallow /moon/*?
Disallow /sun/*?
Disallow /time/change/%40*
Disallow /time/change/*?
Disallow /time/zone/%40*
Disallow /time/zone/*?
Disallow /worldclock/astronomy.html?*year=
Disallow /worldclock/clockchange.html?
Disallow /worldclock/converter.html?
Disallow /worldclock/results.html
Disallow /scripts/tzq.php

scoutjet

Rule Path
Disallow /worldclock/distanceresult.html
Disallow /worldclock/distanceresult.html*
Disallow /information/privacy.html
Disallow /information/terms-conditions.html

Other Records

Field Value
crawl-delay 1

*

Rule Path
Allow /calendar/create.html?*year=2022
Allow /calendar/create.html?*year=2023
Allow /calendar/create.html?*year=2024
Disallow /adverts/
Disallow /advgifs/
Disallow /astronomy/moon/light.html?iso
Disallow /calendar/basic.html?year
Disallow /calendar/create.html?*df=
Disallow /calendar/create.html?*year
Disallow /calendar/custom.html?year
Disallow /calendar/custommenu.html?year
Disallow /calendar//index.html
Disallow /calendar/monthly.html?year=1
Disallow /calendar/monthly.html?year=21
Disallow /calendar/moonphases.html?year=&
Disallow /calendar/moonphases.html?year=10
Disallow /calendar/moonphases.html?year=11
Disallow /calendar/moonphases.html?year=12
Disallow /calendar/moonphases.html?year=13
Disallow /calendar/moonphases.html?year=14
Disallow /calendar/moonphases.html?year=15
Disallow /calendar/moonphases.html?year=16
Disallow /calendar/moonphases.html?year=17
Disallow /calendar/moonphases.html?year=18
Disallow /calendar/moonphases.html?year=2006&
Disallow /calendar/moonphases.html?year=2007&
Disallow /calendar/moonphases.html?year=2008&
Disallow /calendar/moonphases.html?year=21
Disallow /calendar/moonphases.html?year=22
Disallow /calendar/moonphases.html?year=23
Disallow /calendar/moonphases.html?year=24
Disallow /calendar/moonphases.html?year=25
Disallow /calendar/moonphases.html?year=26
Disallow /calendar/moonphases.html?year=27
Disallow /calendar/moonphases.html?year=28
Disallow /calendar/moonphases.html?year=29
Disallow /calendar/moonphases.html?year=3
Disallow /calendar/moonphases.html?year=4
Disallow /calendar/moonphases.html?year=5
Disallow /calendar/moonphases.html?year=6
Disallow /calendar/moonphases.html?year=7
Disallow /calendar/moonphases.html?year=8
Disallow /calendar/moonphases.html?year=9
Disallow /calendar/print.html
Disallow /calendar/repeating.html?year=1
Disallow /calendar/repeating.html?year=21
Disallow /calendar/repeating.html?year=22
Disallow /calendar/repeating.html?year=23
Disallow /calendar/repeating.html?year=23
Disallow /calendar/repeating.html?year=24
Disallow /calendar/repeating.html?year=25
Disallow /calendar/repeating.html?year=3
Disallow /calendar/?year=10
Disallow /calendar/?year=11
Disallow /calendar/?year=12
Disallow /calendar/?year=13
Disallow /calendar/?year=14
Disallow /calendar/?year=15
Disallow /calendar/?year=16
Disallow /calendar/?year=17
Disallow /calendar/?year=18
Disallow /calendar/?year=21
Disallow /calendar/?year=22
Disallow /calendar/?year=23
Disallow /calendar/?year=24
Disallow /calendar/?year=25
Disallow /calendar/?year=26
Disallow /calendar/?year=27
Disallow /calendar/?year=28
Disallow /calendar/?year=29
Disallow /calendar/?year=3
Disallow /clocks/free.html?n
Disallow /clocks/onlyforusebyconfiguration.php
Disallow /counters/customcounter.html?day
Disallow /counters/fullscreen.html?
Disallow /counters/millennium.html?p0
Disallow /counters/year2000a.html?p0
Disallow /counters/year2000.html?p0
Disallow /createshort.html
Disallow /custom/created.html
Disallow /date/dateadded.html?
Disallow /date/dateadd.html?
Disallow /date/durationed.html?
Disallow /date/durationresult.html?
Disallow /date/timeadd.html?
Disallow /date/timeduration.html?
Disallow /date/weekdayadd.html?
Disallow /date/workdays.html?
Disallow /eclipse/in/*?starty=
Disallow /eclipse/in/*?iso
Allow /eclipse/in/*?iso=202
Disallow /gfx/stock/
Disallow /information/feedback.html?
Disallow /information/privacy.html
Disallow /information/terms-conditions.html
Disallow /moon/*year%3D
Disallow /scripts/accountextlogin.php
Disallow /scripts/distanceserver.php
Disallow /scripts/moon.php
Disallow /scripts/savecustom.php
Disallow /scripts/savepersonal.php
Disallow /scripts/wquery.php
Disallow /search/results.html
Disallow /sun/*year%3D
Disallow /time/map/*_escaped_fragment_
Disallow /weather/*?hd=*
Disallow /weather/*/historic?hd=*
Disallow /weather/*/hourly?hd=*
Disallow /weather/*sort*
Disallow /worldclock/astronomy.html?*year=
Disallow /worldclock/custom.html?*n=
Disallow /worldclock/custom.html?*sort*
Disallow /worldclock/custom.html?n=
Disallow /worldclock/custom.html?query=
Disallow /worldclock/difference.html?*continent
Disallow /worldclock/difference.html?*sort*
Disallow /worldclock/distance.html?p1
Disallow /worldclock/distanceresult.html?
Disallow /worldclock/distances.html?
Disallow /worldclock/fixedform.html?p1
Disallow /worldclock/fixedtime.html?*continent*
Disallow /worldclock/fixedtime.html?*sort*
Disallow /worldclock/fixedtime.html?day
Disallow /worldclock/fixedtime.html?iso=200
Disallow /worldclock/fixedtime.html?iso=201
Disallow /worldclock/fixedtime.html?month
Disallow /worldclock/full.html/city.html
Disallow /worldclock/fullscreen.html?
Disallow /worldclock/meeting.html?p1
Disallow /worldclock/meetingdetails.html?
Disallow /worldclock/meetingtime.html?
Disallow /worldclock/personal.html?*continent*
Disallow /worldclock/personal.html?*sort*
Disallow /worldclock/setpersonal.cgi
Disallow /worldclock/sunearth.html?
Disallow /worldclock/sunearth.html?iso

Comments

  • http://web.nexor.co.uk/mak/doc/robots/norobots.html
  • internal note, this file is in git now!
  • disallow any urls with ? in