sosfanta.com
robots.txt

Robots Exclusion Standard data for sosfanta.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	sosfanta.com
Base Domain	sosfanta.com
Scan Status	Ok
Last Scan	2024-05-05T22:01:11+00:00
Next Scan	2024-05-12T22:01:11+00:00

Last Scan

Scanned	2024-05-05T22:01:11+00:00
URL	https://www.sosfanta.com/robots.txt
Domain IPs	2600:9000:2024:0:8:63d:1800:93a1, 2600:9000:2024:2800:8:63d:1800:93a1, 2600:9000:2024:4600:8:63d:1800:93a1, 2600:9000:2024:5400:8:63d:1800:93a1, 2600:9000:2024:6600:8:63d:1800:93a1, 2600:9000:2024:8400:8:63d:1800:93a1, 2600:9000:2024:9600:8:63d:1800:93a1, 2600:9000:2024:ee00:8:63d:1800:93a1, 3.160.188.103, 3.160.188.113, 3.160.188.30, 3.160.188.57
Response IP	18.165.171.97
Found	Yes
Hash	87561ea88a9d456d6275e15a310e7b44de032b1fd974d2979c0e79443e140fe5
SimHash	211f52604bf1

Groups

turnitinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

npbot-1/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

npbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

sogou

Rule	Path
Disallow	/

Rule

Path

Disallow

spinn3r

Rule	Path
Disallow	/

Rule

Path

Disallow

coccoc

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

ezooms

Rule	Path
Disallow	/

Rule

Path

Disallow

fatbot

Rule	Path
Disallow	/

Rule

Path

Disallow

jikespider

Rule	Path
Disallow	/

Rule

Path

Disallow

psbot

Rule	Path
Disallow	/

Rule

Path

Disallow

sitebot

Rule	Path
Disallow	/

Rule

Path

Disallow

sosospider

Rule	Path
Disallow	/

Rule

Path

Disallow

voilabot

Rule	Path
Disallow	/

Rule

Path

Disallow

willybot

Rule	Path
Disallow	/

Rule

Path

Disallow

yodaobot

Rule	Path
Disallow	/

Rule

Path

Disallow

germcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

huaweisymantecspider

Rule	Path
Disallow	/

Rule

Path

Disallow

offline explorer

Rule	Path
Disallow	/

Rule

Path

Disallow

sitesnagger

Rule	Path
Disallow	/

Rule

Path

Disallow

teleportpro

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier

Rule	Path
Disallow	/

Rule

Path

Disallow

webreaper

Rule	Path
Disallow	/

Rule

Path

Disallow

webstripper

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip

Rule	Path
Disallow	/

Rule

Path

Disallow

xaldon_webspider

Rule	Path
Disallow	/

Rule

Path

Disallow

bingbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	5

Field

Value

crawl-delay

*

Rule	Path
Disallow	*/commenti/$
Disallow	/rcs-community-comments-rest-api/
Disallow	/archivio/pagina-*/pagina-
Disallow	/archivio/page/
Disallow	/archivio/categoria/
Disallow	/archivio/gallery/
Disallow	/archivio/video/
Disallow	/*commenti/
Disallow	/*?app_v2
Disallow	/*?app_v1

Rule

Path

Disallow

*/commenti/$

Disallow

/rcs-community-comments-rest-api/

Disallow

/archivio/pagina-*/pagina-

Disallow

/archivio/page/

Disallow

/archivio/categoria/

Disallow

/archivio/gallery/

Disallow

/archivio/video/

Disallow

/*commenti/

Disallow

/*?app_v2

Disallow

/*?app_v1

Other Records

Field	Value
sitemap	https://www.sosfanta.com/sitemaps/sitemap.xml
sitemap	https://www.sosfanta.com/sitemaps/sitemap-news.xml

Field

Value

sitemap

https://www.sosfanta.com/sitemaps/sitemap.xml

sitemap

https://www.sosfanta.com/sitemaps/sitemap-news.xml

sosfanta.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

turnitinbot

npbot-1/2.0

npbot

mj12bot

seznambot

sogou

spinn3r

coccoc

exabot

ezooms

fatbot

jikespider

psbot

sitebot

sosospider

voilabot

willybot

yodaobot

germcrawler

huaweisymantecspider

offline explorer

sitesnagger

teleportpro

webcopier

webreaper

webstripper

webzip

xaldon_webspider

bingbot

Other Records

*

Other Records

sosfanta.com
robots.txt