harpercollins.it
robots.txt

Robots Exclusion Standard data for harpercollins.it

Resource Scan

Scan Details

Site Domain harpercollins.it
Base Domain harpercollins.it
Scan Status Ok
Last Scan2025-11-11T18:34:56+00:00
Next Scan 2025-12-11T18:34:56+00:00

Last Scan

Scanned2025-11-11T18:34:56+00:00
URL https://harpercollins.it/robots.txt
Domain IPs 162.159.134.42
Response IP 162.159.134.42
Found Yes
Hash 977ce88de451bd653892e29b7150edff56aabd1febc63660048f094c420cc636
SimHash 6804d000c1b7

Groups

*

Rule Path
Disallow /

applebot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

bingbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

discordbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

duckduckbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

facebookexternalhit

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

googlebot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

googlebot-image

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

ia_archiver

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

linkedinbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

msnbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

pinterestbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

slurp

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

teoma

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

twitterbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

yandex

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.harpercollins.it/sitemap_index.xml