thecitypulsenews.com
robots.txt

Robots Exclusion Standard data for thecitypulsenews.com

Resource Scan

Scan Details

Site Domain thecitypulsenews.com
Base Domain thecitypulsenews.com
Scan Status Ok
Last Scan2024-09-22T07:20:54+00:00
Next Scan 2024-10-22T07:20:54+00:00

Last Scan

Scanned2024-09-22T07:20:54+00:00
URL https://thecitypulsenews.com/robots.txt
Domain IPs 68.65.122.73
Response IP 68.65.122.73
Found Yes
Hash 07a344165a06ba1247be9020453a9b430a7ec72a993adc69184393b4f39a65cc
SimHash 40044941849b

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

ninjabot

Rule Path
Allow /

mediapartners-google*

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

No rules defined. All paths allowed.

googlebot

Rule Path
Disallow /?*

baiduspider

Rule Path
Disallow /?*

yandexbot

Rule Path
Disallow /?*

ichiro

Rule Path
Disallow /?*

sogou spider

Rule Path
Disallow /?*

sosospider

Rule Path
Disallow /?*

youdaobot

Rule Path
Disallow /?*

yetibot

Rule Path
Disallow /?*

rdfbot

Rule Path
Disallow /?*

ia_archiver

Rule Path
Disallow

*

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.societypulsenews.com/sitemap.xml
sitemap http://www.societypulsenews.com/news-sitemap.xml

Warnings

  • 1 invalid line.