This forum has been moved here:
Helicon Tech Community Forum

  Active TopicsActive Topics  Display List of Forum MembersMemberlist  HelpHelp   RegisterRegister  LoginLogin
ISAPI_Rewrite 2.x
 Helicon Tech : ISAPI_Rewrite 2.x
Subject Topic: Using multiple RewriteCond’s (Topic Closed Topic Closed) Post ReplyPost New Topic
Author
Message << Prev Topic | Next Topic >>
ac3creative
Newbie
Newbie


Joined: 09 June 2006
Location: United Kingdom
Online Status: Offline
Posts: 14
Posted: 05 July 2006 at 2:29am | IP Logged  

>>>If google will try to index this link then it will receive 403

Thanks, that what i needed to know, I was just slightly concerned that i was being shown a 403 when requesting the folder, but not when i was requesting the files within.

As long /blocked-folder/file.html reurns a 403, i'm happy

Back to Top View ac3creative's Profile Search for other posts by ac3creative
 
ac3creative
Newbie
Newbie


Joined: 09 June 2006
Location: United Kingdom
Online Status: Offline
Posts: 14
Posted: 14 July 2006 at 11:43am | IP Logged  

Hi Guys,

I have just had files indexed by Google, even though the googlebot spider is blocked from the folder they reside in.

As i link from pages i want spidered to the pages i don't want spidered, do i have to block each page that i don't want crawled, and if so, what do i need to do to modify the code. (code found above)

Thanks in advance - Iain

Back to Top View ac3creative's Profile Search for other posts by ac3creative
 
Lexey
Moderator Group
Moderator Group


Joined: 15 August 2002
Location: Russian Federation
Online Status: Offline
Posts: 8119
Posted: 16 July 2006 at 4:54pm | IP Logged  

First, check IIS log to see how google requests pages. May be it supplies user agent that is not listed in your block list.
Back to Top View Lexey's Profile Search for other posts by Lexey
 

Sorry, you can NOT post a reply.
This topic is closed.

<< Prev Page of 2
  Post ReplyPost New Topic
Printable version Printable version

Forum Jump
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot delete your posts in this forum
You cannot edit your posts in this forum
You cannot create polls in this forum
You cannot vote in polls in this forum