avsoccer.com
robots.txt

Robots Exclusion Standard data for avsoccer.com

Resource Scan

Scan Details

Site Domain avsoccer.com
Base Domain avsoccer.com
Scan Status Ok
Last Scan2024-06-11T03:37:00+00:00
Next Scan 2024-06-18T03:37:00+00:00

Last Scan

Scanned2024-06-11T03:37:00+00:00
URL https://avsoccer.com/robots.txt
Domain IPs 35.183.73.218
Response IP 35.183.73.218
Found Yes
Hash ac27f9ace3195320938917aef8c30f6990e379b260f28aecc878f50f77385a24
SimHash d4f88cc26bb9

Groups

*

Rule Path
Disallow /admin/
Disallow /leagues/admin/
Disallow /leagues/adminESD/
Disallow /leagues/adminV2/
Disallow /adm1n2x4/
Disallow /aawtg3EMAILBLASTERu4v54j8vm/
Disallow /feeds/
Disallow /leagues2/
Disallow /leaguesgr/
Disallow /leagues/league_rssfeed.cfm
Disallow /rsmreg/registration/teamReg1.cfm
Disallow /rsmreg/registration/indReg1.cfm
Disallow /rsmreg/registration2/
Disallow /rsmreg/registration3/
Disallow /rsmreg/registration/
Disallow /leagues/registration2/
Disallow /leagues/registration3/
Disallow /leagues/registration/
Disallow /leagues/MemberPortal/
Disallow /scheduledTasks/
Disallow /datafeed/
Disallow /leagues/MemberPortal/
Disallow /leagues/cfcomponents/
Disallow /leagues/mobileapp/
Disallow /searchwest/
Disallow /artists/
Disallow /artists2/
Disallow /PayPalPro/
Disallow /leagues/*clientID%3D5539
Disallow /leagues/*clientid%3D5539
Disallow /media/
Disallow /team/
Disallow /wp-content/
Disallow /leagues/cantour*
Disallow /leagues/Cantour*

mediapartners-google

Rule Path
Disallow /admin/
Disallow /leagues/admin/
Disallow /leagues/adminESD/
Disallow /leagues/adminV2/
Disallow /adm1n2x4/
Disallow /aawtg3EMAILBLASTERu4v54j8vm/
Disallow /feeds/
Disallow /leagues2/
Disallow /leaguesgr/
Disallow /leagues/league_rssfeed.cfm
Disallow /rsmreg/registration/teamReg1.cfm
Disallow /rsmreg/registration/indReg1.cfm
Disallow /rsmreg/registration2/
Disallow /rsmreg/registration3/
Disallow /rsmreg/registration/
Disallow /leagues/registration2/
Disallow /leagues/registration3/
Disallow /leagues/registration/
Disallow /leagues/MemberPortal/
Disallow /scheduledTasks/
Disallow /datafeed/
Disallow /leagues/MemberPortal/
Disallow /leagues/cfcomponents/
Disallow /leagues/mobileapp/
Disallow /searchwest/
Disallow /artists/
Disallow /artists2/
Disallow /PayPalPro/
Disallow /media/
Disallow /team/
Disallow /wp-content/
Disallow /leagues/cantour*
Disallow /leagues/Cantour*

twitterbot

Rule Path
Disallow *
Allow /media/images/

gptbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

contextad bot 1.0

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

queryseekerspider ( http://queryseeker.com/bot.html )

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Comments

  • Disallow: /leagues/mobile/