We need some bug spray

Q

Q

New Member
There're a bajillion spiders crawling around here. I saw them snooping around when I was checking who's on line.
 
Leslie posted something in the shoutbox yesterday about it ,fury went "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA :banghead: ,so he's aware of the problem.Hopefully he can add a "No_bot" or something,I would hate to have everything we post be available via Google :eek13: :mope:
 
That is just a little creepy, I don't want EVERYTHING available via search engines either. Are we doing something about this?
 
PuterTutor said:
That is just a little creepy, I don't want EVERYTHING available via search engines either. Are we doing something about this?
Click to expand...
A lot of our traffic comes from people finding us through search engines, so I don't see a problem with it. Besides, search engines don't make EVERYTHING available to their users, they make a preview of the page with the search terms they're looking for and then link them to the actual page, so they will click on over to OTC.

A.B.: I was going AAAAAAAAAAAAAAAAAAAAH for totally unrelated reasons. :beerbang:
 
fury said:
A.B.: I was going AAAAAAAAAAAAAAAAAAAAH for totally unrelated reasons. :beerbang:
Click to expand...


My bad ,thought maybe the prospect of being probed was making you anxious :lloyd: .


Doesn't the constant Spider visits use up bandwidth that could be better spent on useless drivel?
 
We've only used 875 megs so far this month (that's over 4 days). I think we're doing fine in the bandwidth department. :lloyd:

(GZIP compression helps out a lot, plus I don't think spiders even download images except maybe attachments. ;))
 
I don't know too much about these spider-bot thingys...but the fact that they broke our record for users online and then broke their own record the last two days in a row...does indeed, creep me right the fuck out. That's just waaaay too many spiders. :blank:
 
Just give it alitte time Pad.;)
And-to you could join a few yahoo groups, and mention your
site once or twice, and you'll have all the crap you can handle. :D
 
I did submit a form for an exchange of links to everythingpcb.com.....that should pick things up a little soon.
 
fury said:
A lot of our traffic comes from people finding us through search engines, so I don't see a problem with it.
Click to expand...

Yeah, I guess so. After thinking about it I guess it isn't that strange. I do find it odd that 22 of their little spiders would be here on the same day though.
 
They do that for efficiency, so they don't have one spider crawling through the whole site (can you imagine how long that would take, even for a computer?)
 
still it is freaky though if they start going through our profile pages...if thats possible
 
Alright, all these spiders are starting to creep me out. I can understand they would need to catalog everything, but damn, they're still showing up 20 and 30 at a time. Is this going to be from now on since we got listed at Google?
 
No, just once every week or 2, depending on how often Google updates their indexes. They use so many because it would be inefficient for just one spider to crawl through the site, it would never get done.
 
is it a roaming IP or does each one have a temporary IP or is there a fixed IP for each spider? :confused:
 
You must log in or register to reply here.
Back
Top