[Az-Geocaching] Suggestion for statistics

Brian Casteel listserv@azgeocaching.com
Fri, 5 Sep 2003 11:35:29 -0700 

They don't like the mining for data basically.  In the past, they've had
people who really didn't know what they were doing running jobs to pull data
that were misconfigured.  This resulted in severe performance degredation
because the processes were running at inopportune hours (during most likely
caching hours), and some of them would become runaway processes and suck
more and more CPU power/bandwidth.  In the case of azgeocaching, the jobs
were running at something like 3am, when most caching activity is in
people's dreams.  I don't recall ever hearing about azgeocaching being a
problem with regards to a performance hit on the servers of geocaching.com.

In all honesty, it's probably a case of someone wanting resources to stay
there, and not share the wealth.  If the processes were configured
correctly, then I don't see there being a problem with it, but then again
I'm not a geocaching.com guru and have little say on the matter.  There's a
few threads in the gc.com forums now somewhat about this very issue, so I'll
see what I can find out.

Brian
Team A.I.


----- Original Message ----- 
From: "Regan L Smith" <buggers@mindspring.com>
To: <listserv@azgeocaching.com>
Sent: Friday, September 05, 2003 10:42 AM
Subject: Re: [Az-Geocaching] Suggestion for statistics


> do they consider AZGeocaching a threat? I don't understand you guys make
> their stuff much easier to use, and understand.
> ----- Original Message ----- 
> From: "Brian Casteel" <bcasteel@cox.net>
> To: <listserv@azgeocaching.com>
> Sent: Friday, September 05, 2003 10:27 AM
> Subject: Re: [Az-Geocaching] Suggestion for statistics
>
>
> > Brian,
> >
> > How much total data do you believe is being grabbed from geocaching.com
> > daily when you were pulling the 'usual' stuff?
> >
> > Brian
> > Team A.I.
> > ----- Original Message ----- 
> > From: "Brian Cluff" <brian@Snaptek.com>
> > To: <listserv@azgeocaching.com>
> > Sent: Friday, September 05, 2003 9:03 AM
> > Subject: Re: [Az-Geocaching] Suggestion for statistics
> >
> >
> > > I would prefer to just get them let us contiunue to do business as
> usual.
> > > They HAD to have made more money on the existance of azgeocaching.com
> > > than the money they can claim they were loosing if we weren't around,
> > > not to mention that azgeocaching.com off loads a heck of a lot more
> > > bandwidth than it's taking up by crawling the official site.
> > > There have also been a ton of people that wanted to donate to us in
the
> > > past that we just asked to get a membership instead.  Without
> > > geocaching.com's data our site is very boreing.  Jason and I feel that
> > > it's in their best interest in let us continue on as usual as I feel
> > > those same people will probably just not renew their membership in
> > > protest of us getting shut down potentialy costing geocaching.com
> > thousands.
> > >
> > > I would be open to starting our own deal, but thats just way more work
> > > and responsibility that I would like to deal with, and it probably
> > > doesn't have the numbers of people behind it to potentially make it
> > > profitable, so it would be a TON of time that just HAD to be put into
> > > it.  Don't get me wrong, I love doing azgeocaching.com, but there are
> > > some days that I just don't even want to look at it. I wouldn't want
to
> > > run a site that absolutly required maintainace every day without a
> > > little something extra to make it worth my while, and help pay the
> > > bills.  Spreading out the site is a good idea, but it doesn't get rid
o
> > > the fact that the site, however small it is would still need to be
> > > maintained, and having to do a couple of things a day is almost more
> > > annoying than haveing to do a bunch of things.  Thats always been the
> > > nice thing about azgeocaching.com, that for the most part, it's been
> > > self maintaining.  We could spend as much or as little time as our
> > > schedules dictated, and this time of the year it's been a real
blessing
> > > to not HAVE to babysit a website.
> > >
> > > Anyway, I guess it would take some further discussion.
> > >
> > > Brian Cluff
> > > Team Snaptek
> > >
> > > Brian LaFrance wrote:
> > > > I'm all for starting our own open source version specific to
> > Arizona...if we
> > > > could get enough resources together, it might be kind of fun to try.
> > > >
> > > > Brian
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: az-geocaching-admin@listserv.azgeocaching.com
> > > > [mailto:az-geocaching-admin@listserv.azgeocaching.com]On Behalf Of
> Jason
> > > > Poulter
> > > > Sent: Thursday, September 04, 2003 10:33 PM
> > > > To: listserv@azgeocaching.com
> > > > Subject: Re: [Az-Geocaching] Suggestion for statistics
> > > >
> > > >
> > > > maybe we should just dump geocaching.com all together and write our
> own
> > > > site!!! ;)
> > > >
> > > > or at least a distributed database that everyone and anyone could
> setup
> > > > and access all over the world with open source code etc...
> > > >
> > > > jason
> > > >
> > > >
> > > >
> > > > Brian LaFrance wrote:
> > > >
> > > >
> > > >>Maybe we could figure out a way to distribute the workload across
> > several
> > > >>servers.  I also have a server that I could run the code on.  It may
> be
> > > >>something where we could each query a range of cache numbers, user
> ID's,
> > or
> > > >>some other way of breaking the load up.  Not sure if that would help
> > > >
> > > > though.
> > > >
> > > >>We all may end up banned from scraping data off Geocaching.com...it
> > could
> > > >
> > > > be
> > > >
> > > >>a fun project to try though.
> > > >>
> > > >>Brian
> > > >>Team AZEvil
> > > >>
> > > >>-----Original Message-----
> > > >>From: az-geocaching-admin@listserv.azgeocaching.com
> > > >>[mailto:az-geocaching-admin@listserv.azgeocaching.com]On Behalf Of
> > > >>bcasteel@cox.net
> > > >>Sent: Thursday, September 04, 2003 4:35 PM
> > > >>To: listserv@azgeocaching.com
> > > >>Subject: Re: [Az-Geocaching] Suggestion for statistics
> > > >>
> > > >>
> > > >>That's a good idea, in theory.  However, not every cacher can write
> code
> > > >>like the Snaptek guys, nor can everyone understand how to run a
script
> > of
> > > >>this nature.  If they gave me the code, I'd gladly throw it up on my
> > server
> > > >>to run, and all would be happy.  :)
> > > >>
> > > >>Brian
> > > >>Team A.I.
> > > >>
> > > >>
> > > >>
> > > >>>From: Bob Renner <renner001@yahoo.com>
> > > >>>Date: 2003/09/04 Thu PM 05:34:29 EDT
> > > >>>To: Arizona Geocaching List <listserv@azgeocaching.com>
> > > >>>Subject: [Az-Geocaching] Suggestion for statistics
> > > >>>
> > > >>>Jason,
> > > >>>
> > > >>>I realize this would be a total rewrite of your
> > > >>>statistics gathering software, but it might be a way
> > > >>>to lessen the load it puts on the geocaching.com
> > > >>>website.  Instead of crawling the entire data base, go
> > > >>>to the individual profile pages of Arizona geocachers,
> > > >>>and get the information from the "User Stats" pages.
> > > >>>Anyone who wants to have their information included
> > > >>>into the AZgeocaching web page, would have to tell you
> > > >>>to include them in your list.  You could still include
> > > >>>the logs of others on the caches within Arizona.
> > > >>>
> > > >>>Another, approach would be for you to provide the code
> > > >>>for Arizona geocachers to run on their own computers
> > > >>>to get their basic information from geocaching.com and
> > > >>>then automatically forward this to you to be compiled
> > > >>>into one database.  The advantage of this would be
> > > >>>that the load on geocaching.com would be from multiple
> > > >>>IP addresses and only when a user knows his/her stats
> > > >>>need to be updated (i.e. when they have found or
> > > >>>placed a new cache).
> > > >>>
> > > >>>Just a thought that might help get around any blocking
> > > >>>you are experiencing.
> > > >>>
> > > >>>Bob Renner
> > > >>>
> > > >>>__________________________________
> > > >>>Do you Yahoo!?
> > > >>>Yahoo! SiteBuilder - Free, easy-to-use web site design software
> > > >>>http://sitebuilder.yahoo.com
> > > >>>____________________________________________________________
> > > >>>Az-Geocaching mailing list listserv@azgeocaching.com
> > > >>>To edit your setting, subscribe or unsubscribe visit:
> > > >>>http://listserv.azgeocaching.com/mailman/listinfo/az-geocaching
> > > >>>
> > > >>>Arizona's Geocaching Resource
> > > >>>http://www.azgeocaching.com
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >>____________________________________________________________
> > > >>Az-Geocaching mailing list listserv@azgeocaching.com
> > > >>To edit your setting, subscribe or unsubscribe visit:
> > > >>http://listserv.azgeocaching.com/mailman/listinfo/az-geocaching
> > > >>
> > > >>Arizona's Geocaching Resource
> > > >>http://www.azgeocaching.com
> > > >>
> > > >>
> > > >>
> > > >>____________________________________________________________
> > > >>Az-Geocaching mailing list listserv@azgeocaching.com
> > > >>To edit your setting, subscribe or unsubscribe visit:
> > > >>http://listserv.azgeocaching.com/mailman/listinfo/az-geocaching
> > > >>
> > > >>Arizona's Geocaching Resource
> > > >>http://www.azgeocaching.com
> > > >>
> > > >>
> > > >
> > > >
> > > >
> > > > ____________________________________________________________
> > > > Az-Geocaching mailing list listserv@azgeocaching.com
> > > > To edit your setting, subscribe or unsubscribe visit:
> > > > http://listserv.azgeocaching.com/mailman/listinfo/az-geocaching
> > > >
> > > > Arizona's Geocaching Resource
> > > > http://www.azgeocaching.com
> > > >
> > > >
> > > >
> > > > ____________________________________________________________
> > > > Az-Geocaching mailing list listserv@azgeocaching.com
> > > > To edit your setting, subscribe or unsubscribe visit:
> > > > http://listserv.azgeocaching.com/mailman/listinfo/az-geocaching
> > > >
> > > > Arizona's Geocaching Resource
> > > > http://www.azgeocaching.com
> > >
> > >
> > > ____________________________________________________________
> > > Az-Geocaching mailing list listserv@azgeocaching.com
> > > To edit your setting, subscribe or unsubscribe visit:
> > > http://listserv.azgeocaching.com/mailman/listinfo/az-geocaching
> > >
> > > Arizona's Geocaching Resource
> > > http://www.azgeocaching.com
> > >
> >
> >
> > ____________________________________________________________
> > Az-Geocaching mailing list listserv@azgeocaching.com
> > To edit your setting, subscribe or unsubscribe visit:
> > http://listserv.azgeocaching.com/mailman/listinfo/az-geocaching
> >
> > Arizona's Geocaching Resource
> > http://www.azgeocaching.com
>
> ____________________________________________________________
> Az-Geocaching mailing list listserv@azgeocaching.com
> To edit your setting, subscribe or unsubscribe visit:
> http://listserv.azgeocaching.com/mailman/listinfo/az-geocaching
>
> Arizona's Geocaching Resource
> http://www.azgeocaching.com
>