Active TopicsActive Topics  Display List of Forum MembersMemberlist  HelpHelp   RegisterRegister  LoginLogin
HotlinkBlocker support forum
 Helicon Tech : HotlinkBlocker support forum
Subject Topic: User Agent Whitelist Post ReplyPost New Topic
Author
Message << Prev Topic | Next Topic >>
jmu2216
Newbie
Newbie


Joined: 04 January 2006
Location: United States
Online Status: Offline
Posts: 4
Posted: 18 January 2006 at 2:42pm | IP Logged Quote jmu2216

What is the correct usage for User Agent Whitelisting?? 

For example to allow all Internet Explorer would I add IE or Internet Explorer to my User Agent Whitelist.

Thanks
Back to Top View jmu2216's Profile Search for other posts by jmu2216
 
Yaroslav
Moderator Group
Moderator Group


Joined: 15 August 2002
Online Status: Offline
Posts: 6467
Posted: 19 January 2006 at 3:16am | IP Logged Quote Yaroslav
To allow all Internet Explorer add:
*MSIE

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top View Yaroslav's Profile Search for other posts by Yaroslav Visit Yaroslav's Homepage
 
ShawnO
Newbie
Newbie


Joined: 17 July 2006
Online Status: Offline
Posts: 7
Posted: 26 July 2006 at 6:07pm | IP Logged Quote ShawnO
I am having a problem with UA Whitelisting for the search engine bots.

I want them to be able tos earch our site, but I can't seem to get it right.

For google, I have tried:
*google
*Google*
@^*/Google/*$

I have tried the same thing for Yahoo and it doesn't work. What am I doing wrong?

Shawn
Back to Top View ShawnO's Profile Search for other posts by ShawnO Visit ShawnO's Homepage
 
Yaroslav
Moderator Group
Moderator Group


Joined: 15 August 2002
Online Status: Offline
Posts: 6467
Posted: 27 July 2006 at 8:12am | IP Logged Quote Yaroslav
Please show your configuration file.

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top View Yaroslav's Profile Search for other posts by Yaroslav Visit Yaroslav's Homepage
 
ShawnO
Newbie
Newbie


Joined: 17 July 2006
Online Status: Offline
Posts: 7
Posted: 27 July 2006 at 8:13am | IP Logged Quote ShawnO
Here is my current INI:

#################################################
# HotlinkBlocker Configuration file


Signature=cc16bb3a-ecaf-4e9b-ac83-baf252f0b22b
LinkExpires=3600
NotifyOrder=MEDIUM


[Protect]
LINK        /city_hall/departments/clerks/supply_and_services/tender_att ach   


[ReferersBlackList]
@[^?]*bid.*
[ReferersWhiteList]
@^$
@.*
*google*
*yahoo*
*msn*
*aolsearch*
*excite*
[UserAgentsBlackList]
[UserAgentsWhiteList]
*Yahoo
*Google
Back to Top View ShawnO's Profile Search for other posts by ShawnO Visit ShawnO's Homepage
 
Yaroslav
Moderator Group
Moderator Group


Joined: 15 August 2002
Online Status: Offline
Posts: 6467
Posted: 27 July 2006 at 8:14am | IP Logged Quote Yaroslav
I guess you are looking for this:

@.*Googlebot.*

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top View Yaroslav's Profile Search for other posts by Yaroslav Visit Yaroslav's Homepage
 
ShawnO
Newbie
Newbie


Joined: 17 July 2006
Online Status: Offline
Posts: 7
Posted: 27 July 2006 at 8:22am | IP Logged Quote ShawnO
I put the change into the system and added Yahoo and MSN but they are still being blocked.
Back to Top View ShawnO's Profile Search for other posts by ShawnO Visit ShawnO's Homepage
 
Yaroslav
Moderator Group
Moderator Group


Joined: 15 August 2002
Online Status: Offline
Posts: 6467
Posted: 27 July 2006 at 8:49am | IP Logged Quote Yaroslav
Show your current configuration. And how do you know they are blocked?

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top View Yaroslav's Profile Search for other posts by Yaroslav Visit Yaroslav's Homepage
 
ShawnO
Newbie
Newbie


Joined: 17 July 2006
Online Status: Offline
Posts: 7
Posted: 27 July 2006 at 8:56am | IP Logged Quote ShawnO
#################################################
# HotlinkBlocker Configuration file


Signature=cc16bb3a-ecaf-4e9b-ac83-baf252f0b22b
LinkExpires=3600
NotifyOrder=MEDIUM


[Protect]
LINK         &n bsp;/city_hall/departments/clerks/supply_and_services/tender _attach     


[ReferersBlackList]
@[^?]*bid.*
[ReferersWhiteList]
@^$
@.*
*google*
*yahoo*
*msn*
*aolsearch*
*excite*
[UserAgentsBlackList]
[UserAgentsWhiteList]
*yahoo
*google
@.*googlebot.*
@.*yahoo.*
@.*msnbot.*
@msnbot.*


I know they are being blocked because I use a custom 404 script that e-mails me when an item is blocked.
Back to Top View ShawnO's Profile Search for other posts by ShawnO Visit ShawnO's Homepage
 
Yaroslav
Moderator Group
Moderator Group


Joined: 15 August 2002
Online Status: Offline
Posts: 6467
Posted: 28 July 2006 at 4:58am | IP Logged Quote Yaroslav
Does your script send you a User-Agent value of the blocked request and URL originally requested? I need to see this information.

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top View Yaroslav's Profile Search for other posts by Yaroslav Visit Yaroslav's Homepage
 
ShawnO
Newbie
Newbie


Joined: 17 July 2006
Online Status: Offline
Posts: 7
Posted: 28 July 2006 at 2:17pm | IP Logged Quote ShawnO
Yaroslav,

My other posts to this thread don't seem to be here - I am not sure what happened. But, I noticed that the Search Egnines were attempting to retrieve files with the Hotlink Blocker string in the path. Most likely, the string was old and that's why it was being blocked.

When I test it using the Yahoo/MSN/Google user agents, without the blocker string in the path, it works. I am going to change my code to sniff for those user agents and have it not input the string for those agents.

Shawn
Back to Top View ShawnO's Profile Search for other posts by ShawnO Visit ShawnO's Homepage
 
Yaroslav
Moderator Group
Moderator Group


Joined: 15 August 2002
Online Status: Offline
Posts: 6467
Posted: 31 July 2006 at 9:07am | IP Logged Quote Yaroslav
Surely search engines should never get a signed links.
Have you contacted support by email before? I remen=mber very similar problem with PDF files and I suggested to use REDIRECT protection method instead of LINK protection. Leave white lists as is but change protection method.
Or you may change the code to not sign the links when request is coming from SE bot.

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top View Yaroslav's Profile Search for other posts by Yaroslav Visit Yaroslav's Homepage
 
ShawnO
Newbie
Newbie


Joined: 17 July 2006
Online Status: Offline
Posts: 7
Posted: 31 July 2006 at 9:15am | IP Logged Quote ShawnO
At this point, I have set it so that the SE Bot's don't get signed links.  This should work fine for now.  I will continue to monitor it.

I did contact support, but that wasn't suggested.  I would prefer to stay with the LINK because it is supposed to offer the best protection.

The only other item I have contacted support for was the errors in my event log and the IIS Worker Process errors that come up frequently.

Unhandled exception caught. Last error is: (0) No description Exception code is: (c0000005)
Back to Top View ShawnO's Profile Search for other posts by ShawnO Visit ShawnO's Homepage
 

If you wish to post a reply to this topic you must first login
If you are not already registered you must first register

  Post ReplyPost New Topic
Printable version Printable version

Forum Jump
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot delete your posts in this forum
You cannot edit your posts in this forum
You cannot create polls in this forum
You cannot vote in polls in this forum