[westonwanderers]

The way Geocaching.com behaves relating to it's database is entirely the reason I will NOT pay for premium membership. I do not want to contribute to an organisation that serves to hurt the Geocaching community financially by not allowing third party access to their database, and by increasingly bloat their HTML pages.

Yes, Financially!. GC's policies are indirectly costing me more money than I would like. Case in point, While on holiday, the only internet access I have is via a Mobile Phone / Laptop. This system is a) Slow [even GPRS is only 56K] and B ) Costly...my Mobile phone provider charges £3 UK per megabyte GPRS. Just a simple download of a cache list from the main page, without pretty Icons or graphics, is a whopping 47K.....ie, The list of caches takes approx 15 seconds to download, and costs me 13p.... This is without taking into account the main title page, and browsing the individual caches. All in all, to find, and determine particular caches to hunt for on a particular day costs me between 50p and £1 in phone charges. Over a period of a 2 week holiday, the charges mount up quickly. (Orange Mobile Phone charges are here: http://www.orange.co.uk/service_plans/pay_...plan/index.html - Look for 'Pay as you Use')

It doesn't have to be this way at all..... If GC.com aloud access to their database for third parties (even if it required Users of said third party sites to be premium members of GC.com), then said third parties sites could provide this information in a form more data compact form, and everyone would be happy.

.... Yes I know there is the WAP service - If you've ever used it you will understand it's is pretty lame compared to using the main web site.... (You need to enter the co-ords in DD.DDDD , rather than DD MMM.MM format. Also, mechanisms prevent Bookmaking of Caches, so you have to go through the whole rigmarole of re-entering the cache waypoint to get the description/clue each time you want to check it!..... which works out about 10kilobytes of data per cache details/clue anyhow!).

My GC.com membership money is been given to the phone companies.....

This post has been edited by westonwanderers: 21 September 2004 - 01:12 AM

[New England n00b]

Honestly, why not simply make a PQ and carry the GPX files with you? Seems like a less expensive option to me... but then I have neither a WAP phone or internet browser enabled phone, so I'm probably missing the value of such a service.

On the other hand, if people are able to log their finds now (without scrapers logging down the site) and saving thus saving me time... That seems to me more valuable...

I suppose if I had a phone like you have maybe it'd be different... Dunno.

[westonwanderers]

In which case, I would have had to pre-plan my whole 2 weeks of caching at home.... Also, how could I be sure, for instance, that the caches I'm looking for 1 week into my holiday, haven't been archived in the preceding days?

[Keystone]

westonwanderers, on Sep 21 2004, 06:29 AM, said:

In which case, I would have had to pre-plan my whole 2 weeks of caching at home.... Also, how could I be sure, for instance, that the caches I'm looking for 1 week into my holiday, haven't been archived in the preceding days?

Would Buxley's site have told you about the caches that were archived in the past week?

Please stick to the topic. If you'd like to offer feedback on the new and improved version of the free Geocaching.com WAP site, there is a pinned thread in the Geocaching.com forum for that purpose. Thank you.

[westonwanderers]

If geocaching.com had provided Buxleys with this information, then yes they would..... As has been mentioned in this thread, finding archived caches is difficult using web-scrapeing methods (you can't scrape what's not there).

The example I gave was just another valid reason why I argue that GC.com should provide an official way for full database access to be provided to third party websites... Buxleys would not have been banned if such a way existed (and they had agreed to the terms of use of that service)..... They would have had no excuse for not removing archived caches.....If they had then continued to not delete the archived caches and were banned due to this, then it'd be their own fault, and I'd imagine they'd have much less support then they have now by the geocaching community, and GC.com would have much more kudos...

[BassoonPilot]

Keystone Approver, on Sep 21 2004, 05:36 AM, said:

Please stick to the topic.

Let's see ...

... Activity logged to caches over the past week is greatly reduced from the seasonal peak a few weeks back.

... We were told that "scrapers" were (in large part) the root cause of the gc.com access/speed issues.

... The scrapers are now being "throttled."

... Access/speed issues are at least as bad as they were.

What has the "throttling" accomplished, from the average user's perspective?

(For the record, I have never used Buxley's, but I did make use of other third party websites that "scraped" data, like Dan Miller's Leaderboard.)

This post has been edited by BassoonPilot: 21 September 2004 - 04:58 AM

[GrizzlyJohn]

I was also trying to stay out of this. But the usual nonsense from the usual people just keeps going on and on.

I think this part of the business is and should be open to discussion on the forums. If gc.com is going to offer the ability for another site to be able to easily download the data store of caches for a fee or otherwise I think this is something we all should know about.

I would like to know because I would like to have the same opportunity. But my use would be for just that, my use, and not to post on a third party web site. I have read the TOS and have asked if there are any times that TPTB would see that written permission would be granted in some of those instances where they say it would be needed. I have asked in email and on the forums and have never received a reply. I guess that is my answer so my point has been to just say that. But I do think that is of interest to the community and does not in any way bother gc.com’s “company secrets” or whatever their hang-up is.

I think if gc.com is going to start selling the data store that perhaps there are some people here that would want to know that. It is their work that creates the logs and caches and while it is clear that one has granted the right to gc.com to do so once they actually start doing it some people may not be happy about it.

I am always amused at the posts I see with people going on about how great a job TPTB are doing and how grateful they are to them to for all their hard work. Do these same people get on the boards of all the other companies they buy products from and praise the CEO and the board of directors? I doubt it.

Get real Dr. Phil -- this is a company, a business, a profit-driven venture. I completely support that concept and wish them well. I am not about to praise how wonderful this site is because I am really not overly impressed. But I can praise the business model they have come up with -- what a thing a beauty. They get a bunch of people all over the world to volunteer their time to approve caches. These caches are from people that volunteer their time and the cost of putting the cache together to then post them here. And let’s not forget the people that go and find them to add content for the company to display. They can then take all of that volunteer effort that has been put in and pretty much do whatever they want with it without concerning themselves with the people that created the content. But then when those same people that provide the content ask for something more they are met with silence, or are told it is none of their business on how the company operates. Maybe not, but maybe some people will start to think they are owed a little more than what they are getting.

And yes I have let my premium membership lapse. I will not be renewing until there is a way to pay with a credit card. The idea of using paypal is just silly. This has been promised for I am guessing nearly a year if not longer. I know I would never buy another book on Amazon if I had to use paypal and I don’t think many others would either.

[westonwanderers]

+1

[haggaeus]

I guess the overload issue is being exaggerated. Let's count how many pages Buxley needs to download:
210+50 country+state pages to get the list of new caches (with Canadian and Australian provinces, plus some large states which have more than one page of new caches, let's say the total is 400). The update was happening twice a day = 800 hits.
There are currently about 250 new caches approved daily (I counted the average interval between 1000 caches for a few recent cache IDs). It is probably more after a sunny weekend... 500? Buxley needs to get every cache page once.
This looks like 1300 hits in a day. Was this traffic really killing the website? One hit in a minute? If this is the case, then Buxley can use the $10 donation I sent him and buy Groundspeak a new dedicated server, I got my Sparcstation IPX for $5 off eBay.

[CO Admin]

GrizzlyJohn, on Sep 21 2004, 05:05 AM, said:

I was also trying to stay out of this. But the usual nonsense from the usual people just keeps going on and on.

I think this part of the business is and should be open to discussion on the forums. If gc.com is going to offer the ability for another site to be able to easily download the data store of caches for a fee or otherwise I think this is something we all should know about.

I would like to know because I would like to have the same opportunity. But my use would be for just that, my use, and not to post on a third party web site. I have read the TOS and have asked if there are any times that TPTB would see that written permission would be granted in some of those instances where they say it would be needed. I have asked in email and on the forums and have never received a reply. I guess that is my answer so my point has been to just say that. But I do think that is of interest to the community and does not in any way bother gc.com’s “company secrets” or whatever their hang-up is.

I think if gc.com is going to start selling the data store that perhaps there are some people here that would want to know that. It is their work that creates the logs and caches and while it is clear that one has granted the right to gc.com to do so once they actually start doing it some people may not be happy about it.

I am always amused at the posts I see with people going on about how great a job TPTB are doing and how grateful they are to them to for all their hard work. Do these same people get on the boards of all the other companies they buy products from and praise the CEO and the board of directors? I doubt it.

Get real Dr. Phil -- this is a company, a business, a profit-driven venture. I completely support that concept and wish them well. I am not about to praise how wonderful this site is because I am really not overly impressed. But I can praise the business model they have come up with -- what a thing a beauty. They get a bunch of people all over the world to volunteer their time to approve caches. These caches are from people that volunteer their time and the cost of putting the cache together to then post them here. And let’s not forget the people that go and find them to add content for the company to display. They can then take all of that volunteer effort that has been put in and pretty much do whatever they want with it without concerning themselves with the people that created the content. But then when those same people that provide the content ask for something more they are met with silence, or are told it is none of their business on how the company operates. Maybe not, but maybe some people will start to think they are owed a little more than what they are getting.

And yes I have let my premium membership lapse. I will not be renewing until there is a way to pay with a credit card. The idea of using paypal is just silly. This has been promised for I am guessing nearly a year if not longer. I know I would never buy another book on Amazon if I had to use paypal and I don’t think many others would either.

Only one comment.

I could be wrong, but your comment about getting volunteers to approve caches disturbed me. I am willing to admit that I might not be reading what you intended. I have always volunteered my time to something. Its the way I am. 25 years to the Boy scouts. 5 years to my church. Now its Geocaching. Its my choice. I'm not being forced or coerced into doing it. Its selfish, I like the rewards. Its my way of giving back.

If I read more into what you said then I am wrong. If not the I don't appreciate the comments. Until you respond I wont know. No matter how many other people here respond I wont know until you tell me.

Not a flame, not an argument, just a comment.

This post has been edited by CO Admin: 21 September 2004 - 06:06 AM

[GrizzlyJohn]

CO Admin, on Sep 21 2004, 10:04 AM, said:

I could be wrong, but your comment about getting volunteers to approve caches disturbed me. I am willing to admit that I might not be reading what you intended. I have always volunteered my time to something. Its the way I am. 25 years to the Boy scouts. 5 years to my church. Now its Geocaching. Its my choice. I'm not being forced or coerced into doing it. Its selfish, I like the rewards. Its my way of giving back.

If I read more into what you said then I am wrong. If not the I don't appreciate the comments. Until you respond I wont know. No matter how many other people here respond I wont know until you tell me.

Not a flame, not an argument, just a comment.

Whoa! I would like to respond but I am not sure how.

I never in any way meant anything negative by my comments about volunteering on this site. I was just pointing out that in fact that is a volunteer effort. And I would count myself in the group of volunteers by my adding my logs to this site. I will without a doubt yield the fact that approvers likely put in way more time approving then even the most prolific cacher and end up taking a lot more crap for it.

I have volunteered my time to activities and causes as well. I really don’t have a problem with the volunteering or people that do. And I certainly never meant to imply that you or anyone was being forced or coerced into do it. The same is true of anyone using this site, as an approver, hiding caches or finding them, the door swings both ways.

I also understand that this or any site would not be able to exist if all of the approvers had to be paid and all royalties for all of the people’s words would have to be compensated. I think we all understand that we all give a little to get back something.

I guess my main point was that often little to no respect is shown to the people that drive and create the content that a few profit from. Yes on that point we will likely disagree, but we will have to leave it at that because neither your or my mind will be changed.

If you took my remarks as a negative comment about volunteering in general or volunteering on this site, I apologize, that was never in any way shape or form my intent.

[CO Admin]

GrizzlyJohn, on Sep 21 2004, 06:37 AM, said:

CO Admin, on Sep 21 2004, 10:04 AM, said:

I could be wrong, but your comment about getting volunteers to approve caches disturbed me. I am willing to admit that I might not be reading what you intended. I have always volunteered my time to something. Its the way I am. 25 years to the Boy scouts. 5 years to my church. Now its Geocaching. Its my choice. I'm not being forced or coerced into doing it. Its selfish, I like the rewards. Its my way of giving back.

If I read more into what you said then I am wrong. If not the I don't appreciate the comments. Until you respond I wont know. No matter how many other people here respond I wont know until you tell me.

Not a flame, not an argument, just a comment.

Whoa! I would like to respond but I am not sure how.

I never in any way meant anything negative by my comments about volunteering on this site. I was just pointing out that in fact that is a volunteer effort. And I would count myself in the group of volunteers by my adding my logs to this site. I will without a doubt yield the fact that approvers likely put in way more time approving then even the most prolific cacher and end up taking a lot more crap for it.

I have volunteered my time to activities and causes as well. I really don’t have a problem with the volunteering or people that do. And I certainly never meant to imply that you or anyone was being forced or coerced into do it. The same is true of anyone using this site, as an approver, hiding caches or finding them, the door swings both ways.

I also understand that this or any site would not be able to exist if all of the approvers had to be paid and all royalties for all of the people’s words would have to be compensated. I think we all understand that we all give a little to get back something.

I guess my main point was that often little to no respect is shown to the people that drive and create the content that a few profit from. Yes on that point we will likely disagree, but we will have to leave it at that because neither your or my mind will be changed.

If you took my remarks as a negative comment about volunteering in general or volunteering on this site, I apologize, that was never in any way shape or form my intent.

I wasn't sure how to take it, Thats why I asked,
Thanks for the clarification.

No offense taken.

This post has been edited by CO Admin: 21 September 2004 - 07:31 AM

[Lazyboy & Mitey Mite]

From Buxleys site....

Quote

19 Sept 2004 (Sunday)
It's been several days and I still haven't received any reply from Groundspeak as to why my web site remains blocked (not throttled) from visiting their web pages.

I'm very curious to see if Groundspeak has any reasonable explanation for blocking my site. I myself really can't come up with any except that they must see Buxley's Geocaching Waypoint as a competitor. Strange... I never saw myself as being "competition", just providing a useful service for the community.

I am keeping up with the Buxley New Caches Not Being Updated? thread on the forums. Thanks to all for the supportive posting and emails. I agree with those who say that we (Groundspeak and I) should come up with some agreement so that my maps can be updated with current information without impacting their servers. I'm more than willing to have a discussion with the people at Groundspeak regarding how we can come up with such a solution.

Groundspeak? You have my email address. How about dropping me a line?

-Buxley

Sooo how about working something out????

[adampierson]

I've heard from a couple of people who claim that "web scrapers" are being throttled back. According to Buxley, his site is totally blocked. Who are these other web scapers?

[M-Class]

Considering that cachestats.made.nu got linked in this thread regarding the "one page per 10 seconds" quote, and considering that I wrote that site, I probably should chime in here and add some detail to that quote.

1. "10 seconds", as Robertlipe stated, is the maximum burst rate. It's a mere hard-coded delay in the software to wait before checking if there is another page to process in the queue. In the real world, there isn't always something in the queue, and the processing time to string parse / scrape the page has to be considered.
2. During a recent sampling of the busiest days on record (just after Geo Woodstock 2) the site never exceeded 2600 page views per day (which works out to about 33 seconds per page)
3. On the weekends, the software went into a restricted mode which only generated a maximum of 300 page views per day (essetially the 'recent 15 minutes' page and a handful of requests for new hides - a max of one per 5 minutes and usually far less)... all other needed data was queued up for processing during the early morning hours during the next week. The stats site never has been responsible for any weekend delays... period. Last April, Monday was added to the list of weekend days since the GC was begining to have frequent Monday problems as well- meaning the "heavy" data collection was only done 4 days per week, mostly during off-hours.
4. Whenver the software detected errors from geocaching.com, it would automatically "back off" and reduce it's request frequency to once per 3 minutes until GC returned to consistently healthy status.
5. In late August, GC implemented an automated throttling system that serves up blank white pages if a user exceeds an undisclosed threshold. At the time, testing seemed to indicate that on a 30 second interval, that threshold was hit between 300 and 400 requests. I've been informed of a test performed by another group which rated the threshold at 400 on a once per 5 second request rate. The automated block would automatically reset at around 12 hours in my tests.
6. All data collection was ceased for a week while I analyzed the problem and did some architecture changes on my end.
7. The site resumed partial operation on a 3 minute interval for page requests.
8. A "queue this up" page was added to allow end users to queue things up that were missed in the data collection (instead of the automated method of crawling down each users "finds by" page).
9. Since then, I've been very busy on other projects and haven't had the time to watch over the stats system. Unfortunately, some end users have been entering invalid GC numbers into the "queue this" page resulting in unnecessary GC page requests and a huge performance hit on the stats system (when you only have 1 page per 3 minutes available, the performance hit from these [possibly sabatoge] entries is obvious).
10. Data collection was completely turned off last week while I was repairing the data collection machine. As I said earlier, I've been busy with other projects, so the repairs weren't quick.

I've had nothing but the best intentions with providing stats for clubs. All data is collected by opt-in permission only, and I've purposefully built in mechanisms by which the site would not harm geocaching.com. Any problems with GC have not been the result of any activity from the stats site. In fact, during the week I was down entirely, GC was having worse than usual problems. I've always said that if geocaching.com wanted me to shut it down, I would. Back in August I kept the site down for almost a week while I waited to hear the official word on whether or not the stats site was locked out. The official word was that it was not, but an automated system had been added which then broke the site. Geocaching.com has not asked me to shut the site down, but they have implemented the throttling restrictions on the site... I was free to continue so long as I stayed within the restrictions. I've since retooled in a new manner (or should I say old, as it more resembles Dan's old manual queueing than the automated system).

For the record, additional tests on the throttling system revealed that it is still possible to manage more than 3000 pages per day (more than I ever requested in any given day) with special more sophisticated delay settings. Ironically, that method actually hits GC harder than my system ever did (it relies on hitting multiple concurrent page... like 10 at once... followed by a variable time delay on a sliding scale based upon byte received count). Even more ironic, the test was run during the time which everyone has stated that the site has been running better than it ever has! I have no plans to implement any such fancy code into the stats site, as it violates the intent of GC by adding the throttler in the first place. I hope the stats site doesn't die a slow death of data integrity issues (it doesn't look promising), but until GC allows a better mechanism to access the data (none of us really want to page scrape in the first place), these are the rules I have to live by. I have no problems with complying with them.

[Team Sagefox]

GrizzlyJohn, on Sep 21 2004, 05:05 AM, said:

I am always amused at the posts I see with people going on about how great a job TPTB are doing and how grateful they are to them to for all their hard work...

...Get real Dr. Phil -- this is a company, a business, a profit-driven venture... I am not about to praise how wonderful this site is because I am really not overly impressed.

I'm sure you believe that you are making a worthy argument. (That is questionable.) But for some reason you had to bury a couple of zingers in your discussion. Do you think that insulting a large majority of geocachers helps make your point? It turns people off is what it does.

Close variations on this theme come up often in the forums: Worldly wise commenter feels the call of duty to inform the masses who are blindly having fun that they are merely sheep following the greedy profit seeking PTB.

It is not like you suggest out in the real world of geocaching. You can tell that by simply reading the log entries people make on your geocache pages. You might not be happy with this company's policies and you might even have some worthy points to make but it sounds like sour grapes to many of us.

And you will let your membership lapse just because you don't approve of the payment system? That can't possibly be true. If you were happy with geocaching you would find a way to make the payments. Perhaps it is more likely that you don't make use of the services that the premium membership provides. No harm in that. You don't need to be a premium member to enjoy geocaching.

I feel fortunate in that it doesn't appear that your dissatisfaction is anywhere close to the norm out in the physical world of geocaching. I would hate it if I had to actually listen to this rant.

[lapaglia]

M-Class, on Sep 21 2004, 12:54 PM, said:

Considering that cachestats.made.nu got linked in this thread regarding the "one page per 10 seconds" quote, and considering that I wrote that site, I probably should chime in here and add some detail to that quote.

1. "10 seconds", as Robertlipe stated, is the maximum burst rate.  It's a mere hard-coded delay in the software to wait before checking if there is another page to process in the queue.  In the real world, there isn't always something in the queue, and the processing time to string parse / scrape the page has to be considered.
2. During a recent sampling of the busiest days on record (just after Geo Woodstock 2) the site never exceeded 2600 page views per day (which works out to about 33 seconds per page)
3. On the weekends, the software went into a restricted mode which only generated a maximum of 300 page views per day (essetially the 'recent 15 minutes' page and a handful of requests for new hides - a max of one per 5 minutes and usually far less)... all other needed data was queued up for processing during the early morning hours during the next week.  The stats site never has been responsible for any weekend delays... period.  Last April, Monday was added to the list of weekend days since the GC was begining to have frequent Monday problems as well- meaning the "heavy" data collection was only done 4 days per week, mostly during off-hours.
4. Whenver the software detected errors from geocaching.com, it would automatically "back off" and reduce it's request frequency to once per 3 minutes until GC returned to consistently healthy status.
5. In late August, GC implemented an automated throttling system that serves up blank white pages if a user exceeds an undisclosed threshold.  At the time, testing seemed to indicate that on a 30 second interval, that threshold was hit between 300 and 400 requests.  I've been informed of a test performed by another group which rated the threshold at 400 on a once per 5 second request rate.  The automated block would automatically reset at around 12 hours in my tests.
6. All data collection was ceased for a week while I analyzed the problem and did some architecture changes on my end.
7. The site resumed partial operation on a 3 minute interval for page requests.
8. A "queue this up" page was added to allow end users to queue things up that were missed in the data collection (instead of the automated method of crawling down each users "finds by" page).
9. Since then, I've been very busy on other projects and haven't had the time to watch over the stats system.  Unfortunately, some end users have been entering invalid GC numbers into the "queue this" page resulting in unnecessary GC page requests and a huge performance hit on the stats system (when you only have 1 page per 3 minutes available, the performance hit from these [possibly sabatoge] entries is obvious). 
10. Data collection was completely turned off last week while I was repairing the data collection machine.  As I said earlier, I've been busy with other projects, so the repairs weren't quick.

I've had nothing but the best intentions with providing stats for clubs.  All data is collected by opt-in permission only, and I've purposefully built in mechanisms by which the site would not harm geocaching.com.  Any problems with GC have not been the result of any activity from the stats site.  In fact, during the week I was down entirely, GC was having worse than usual problems.  I've always said that if geocaching.com wanted me to shut it down, I would.  Back in August I kept the site down for almost a week while I waited to hear the official word on whether or not the stats site was locked out.  The official word was that it was not, but an automated system had been added which then broke the site.  Geocaching.com has not asked me to shut the site down, but they have implemented the throttling restrictions on the site... I was free to continue so long as I stayed within the restrictions.  I've since retooled in a new manner (or should I say old, as it more resembles Dan's old manual queueing than the automated system).

For the record, additional tests on the throttling system revealed that it is still possible to manage more than 3000 pages per day (more than I ever requested in any given day) with special more sophisticated delay settings.  Ironically, that method actually hits GC harder than my system ever did (it relies on hitting multiple concurrent page... like 10 at once... followed by a variable time delay on a sliding scale based upon byte received count).  Even more ironic, the test was run during the time which everyone has stated that the site has been running better than it ever has!  I have no plans to implement any such fancy code into the stats site, as it violates the intent of GC by adding the throttler in the first place.  I hope the stats site doesn't die a slow death of data integrity issues (it doesn't look promising), but until GC allows a better mechanism to access the data (none of us really want to page scrape in the first place), these are the rules I have to live by.  I have no problems with complying with them.

Thats very thoughtfull of you. But, can you explain how what you are doing at any speed fits with the Terms of Use agreement? Are you not violating the Terms of use agreement by scrapping at any speed? " You agree that you will not use any robot, spider, scraper or other automated means to access the Site for any purpose without our express written permission." So no matter how nice you are in trying to not load the site are you not still in violation?

Quote

5. Access and Interference 
Much of the information on the Site is updated on a real time basis and is proprietary or is licensed to Groundspeak by our users or third parties. You agree that you will not use any robot, spider, scraper or other automated means to access the Site for any purpose without our express written permission. Additionally, you agree that you will not: (a) take any action that imposes, or may impose in our sole discretion an unreasonable or disproportionately large load on our infrastructure; or ( interfere or attempt to interfere with the proper working of the Site or any activities conducted on the Site or other measures we may use to prevent or restrict access to the Site.

http://www.geocachin...termsofuse.aspx

This post has been edited by Lapaglia: 21 September 2004 - 01:09 PM

[ju66l3r]

Thanks for the post and information, M-Class.

Have you (or Buxley) tried to e-mail GC.com directly and begin the discussion of a licensing system that would give you more regular access?

I think the problems were stemming from the combination of your hits, Buxley's, and more importantly less skilled programmers skimming for their own interests with little to no regard for consecutive user access or delay. While no one person was responsible for the lag/access issues, the combination along with increasing normal usage brought things to its knees at times.

This is why the development of a designated time to get semi-weekly updates or a determination of what each scraper's allowed rate is going to be needs to come from GC.com/Jeremy...and it's disappointing to see that he's uninterested in making the throttling parameters more transparent or finalizing a more structured update system for these valuable sites.

I'll give out 3 possible solutions:

a) special servers requiring a licensing agreement that would contain the update information that could be accessed realtime to keep other sites up-to-date.

B) special times (early AM) when a full scraping of the site would allow for update of other sites.

c) announced specific parameters that could allow access anytime (even weekends) but at a rate that guaranteed that together with normal access, the servers would never reach 90% access capacity.

This post has been edited by ju66l3r: 21 September 2004 - 03:19 PM

[M-Class]

It's been a while since I read the terms... back when I set up the site the terms didn't mention robots specifically. The basis then was that the terms stated that users owned their data and could with written permission give third parties permission to use that data (thus the opt-in permission requirement). When I checked into why I was getting the white pages back in August, I wasn't directed to this part of the terms but was told I could resume if I stayed under the threshold. Of course, that wasn't in writing, so I will stop my bots immediately until I can get the "written permission". Thanks for pointing this out to me.

[Robespierre]

Grizzly John, as far as I can tell YOU ARE NOT A VOLUNTEER, NOR AM I. We are just people who play the game (and forgive me if you are an approver, or something more than a player).
-Putting out caches is one of the most fun parts of the game. Play if you want, but no dividends are paid for playing. You have no investment here. (nor I). You use the site to play the game. That is all. You don't become a stockholder by playing.

Oh, man. So far I had been staying out of this one pretty well.

[lapaglia]

M-Class, on Sep 21 2004, 01:16 PM, said:

It's been a while since I read the terms... back when I set up the site the terms didn't mention robots specifically. The basis then was that the terms stated that users owned their data and could with written permission give third parties permission to use that data (thus the opt-in permission requirement). When I checked into why I was getting the white pages back in August, I wasn't directed to this part of the terms but was told I could resume if I stayed under the threshold. Of course, that wasn't in writing, so I will stop my bots immediately until I can get the "written permission". Thanks for pointing this out to me.

Thank you.
Your response is the most responsible one I have heard here so far. I am serious about this. Well done sir!

[robertlipe]

The licence can change AFTER after we agreed to it? Wow. Section 14 is a newsflash to me.

Those clauses were definitely NOT there when I clicked OK. Makes me wonder what I've agreed to every time I've edited a cache page - of course, we all kept hard copies of those and examined them for changes every time we click those boxes at the bottom of the page, right?

Now that I'm looking at the Waypoint License Agreement and the Terms of Use (everybody noticed they were different, right?) with a careful eye, I'm wondering if every program that reads and writes pocket queries has a license to

Quote

reverse engineer, decompile, or disassemble the GROUNDSPEAK-compatible data format(s) in an attempt to duplicate the proprietary and copyright-protected GROUNDSPEAK data model(s) and/or export format(s).

I could remove the groundspeak extensions to GPX (which IS an open format) from GPSBabel in about 10 minutes...

[GrizzlyJohn]

Team Sagefox, on Sep 21 2004, 05:01 PM, said:

I'm sure you believe that you are making a worthy argument.  (That is questionable.)  But for some reason you had to bury a couple of zingers in your discussion.  Do you think that insulting a large majority of geocachers helps make your point?  It turns people off is what it does.

Close variations on this theme come up often in the forums:  Worldly wise commenter feels the call of duty to inform the masses who are blindly having fun that they are merely sheep following the greedy profit seeking PTB.

It is not like you suggest out in the real world of geocaching.  You can tell that by simply reading the log entries people make on your geocache pages.  You might not be happy with this company's policies and you might even have some worthy points to make but it sounds like sour grapes to many of us. 

And you will let your membership lapse just because you don't approve of the payment system?  That can't possibly be true.  If you were happy with geocaching you would find a way to make the payments.  Perhaps it is more likely that you don't make use of the services that the premium membership provides.  No harm in that.  You don't need to be a premium member to enjoy geocaching.

I feel fortunate in that it doesn't appear that your dissatisfaction is anywhere close to the norm out in the physical world of geocaching.  I would hate it if I had to actually listen to this rant.

If people are turned off I am sorry, I just call it like I see it. This is a for profit venture. Will you deny that? And yes I am amused that people go out of their way to go on and on about how great TPTB are doing in running this site. Again do they go to other sites and sing the same praises?

Yes I do believe there are many sheep here. I also think there are many that are not and offer plenty of good ideas on how to improve things. And they do this not for themselves but to make a better time for everybody. If you wish to see it as sour grapes then so be it.

I look at this situation a lot like how I see professional sports. There was a feeling for a long time that the local team was there for the folks in the city. But we have since seen strikes, salaries and profits through the roofs and all at the cost to the fan in soaring ticket prices and obscene prices at the concession stands. Some have realized that it is not there as some kind of civic pride but is in fact a big business that involves lots of money which comes from the folks who put their backsides in the seats. I say this as person that did have a full season plan for four seats to the local major league baseball team for many years. Not anymore, I just got plain tired of being told to bend over again.

But the point that really got to me was questioning my honesty. Don’t ever do that! I used the services provided by a premium membership and I am sure there are plenty here that can tell you that I have actually been accused of using too much of the services provided by a premium membership. (Mopar feel free to jump in on that one.) So do not ever question what I say my reasons are. You don’t know me and you have no idea or anything to base your statements on. I don’t think I have ever had a problem with stating my reasons for doing anything that I do. You may not like what I have to say and you may not like how I say it, but that is your right and I will defend your ability to do that. But I will not sit by and allow you to call me a liar. Nothing I have said even comes close to thinking that is an acceptable thing to do.

And to Robespierre:
No I am not an approver (that will never happen) so no appology is necessary. We happen to disagree. I think we all have a stake here if we contribute. I am not asking to be a stockholder I just think it would be nice to get answers to questions asked. And I do think this problem is of interest to many people here, whatever side you happen to be on.

Edit to move my commment to Robespierre out of Team Sagefox's quote. Sorry Team Sagefox did not mean to include my comments as part of your quote. That was a misfiring of the finger and hitting the wrong edit box.

This post has been edited by GrizzlyJohn: 22 September 2004 - 04:25 AM

[CoyoteRed]

robertlipe, on Sep 21 2004, 05:29 PM, said:

I could remove the groundspeak extensions to GPX (which IS an open format) from GPSBabel in about 10 minutes...

A lot of things would come to a screeching halt and remove the reason that many people get PQs--and thusly pay their $30 a year.

Whatcha think?

[ju66l3r]

CoyoteRed, on Sep 21 2004, 07:06 PM, said:

robertlipe, on Sep 21 2004, 05:29 PM, said:

I could remove the groundspeak extensions to GPX (which IS an open format) from GPSBabel in about 10 minutes...

A lot of things would come to a screeching halt and remove the reason that many people get PQs--and thusly pay their $30 a year.

Whatcha think?

I think that unless you bundle it with an absolutely must-have feature, they'll all use and pass around the previous version...especially the previous version that would let them save it as another file format and then reimport into the new version that still supported that format...

[rickrich]

robertlipe, on Sep 21 2004, 04:29 PM, said:

I could remove the groundspeak extensions to GPX (which IS an open format) from GPSBabel in about 10 minutes...

I just sent a quick patch to the gpsbabel mailing list to do just that. Its gonna take me longer to rid the GPX extensions out of my own stuff. Fizzymagic's gotta pull gpx2html as well.

Lawyers.

-Rick

[Elias]

Uh, I'm not sure what just happened here. Honestly, I haven't been following this thread but looking back over today's posts it seems we have some kind of major misunderstanding. I could try to go back and piece together the details, but my gut tells me that there's more going on here and I'm not going to get the real picture and feelings from all of you if I do that. I'd like to insert myself into this thread and open a discussion to see if I can help resolve some of these issues.

If you have a question, concern, or issue you'd like to discuss, and can pose to me a direct question, I'll answer it as best I can.

Elias

[geojeeper74]

Elias, on Sep 21 2004, 04:36 PM, said:

Uh, I'm not sure what just happened here. Honestly, I haven't been following this thread but looking back over today's posts it seems we have some kind of major misunderstanding. I could try to go back and piece together the details, but my gut tells me that there's more going on here and I'm not going to get the real picture and feelings from all of you if I do that. I'd like to insert myself into this thread and open a discussion to see if I can help resolve some of these issues.

If you have a question, concern, or issue you'd like to discuss, and can pose to me a direct question, I'll answer it as best I can.

Elias

Would you please open a discussion with Ed Hall at Buxley's Waypoint so he can get his site up and running again without decreasing the performance of gc.com?

[Pto]

Quote

Would you please open a discussion with Ed Hall at Buxley's Waypoint so he can get his site up and running again without decreasing the performance of gc.com?

I second that~

[M-Class]

Elias,

I too wasn't aware of this thread until today. I've been running the cachestats.made.nu statistics site to provide geocaching clubs with statistics data for a year now. When I started, what I was doing was ok under the terms of use that was active at that point in time. Today (in this thread) the new TOU was pointed out to me which states the need for written permission. As a result, I've pulled the stats data collection and have sent an email to contact@geocaching.com (#136150) to request the necessary written permission. Although I've been told in the past that what I was doing is ok, I don't want to risk a lawsuit either. As a result I'm waiting for a response from Groundspeak on the whole written permission thing. That's my story... I can't speak for anyone else.

As for a direct question... as my contact@ email states, I'd like to know how to obtain written permission to collect data from geocaching.com for use with the statistics site so that I can comply with the current (and future) Terms of Use?

[Elias]

geojeeper74, on Sep 21 2004, 05:49 PM, said:

Would you please open a discussion with Ed Hall at Buxley's Waypoint so he can get his site up and running again without decreasing the performance of gc.com?

We did receive an email from Ed, but we have not responded yet. We will respond to him soon.

Elias

[geojeeper74]

Elias, on Sep 21 2004, 05:13 PM, said:

geojeeper74, on Sep 21 2004, 05:49 PM, said:

Would you please open a discussion with Ed Hall at Buxley's Waypoint so he can get his site up and running again without decreasing the performance of gc.com?

We did receive an email from Ed, but we have not responded yet. We will respond to him soon.

Elias

Thank you for your prompt reply.

[Elias]

M-Class, on Sep 21 2004, 05:56 PM, said:

As for a direct question... as my contact@ email states, I'd like to know how to obtain written permission to collect data from geocaching.com for use with the statistics site so that I can comply with the current (and future) Terms of Use?

The official answer:

Even though the TOU says to contact us for permission, we don't give permission to scrape the site. Its not supported, and we don't want anyone doing it.

The unofficial answer:

We know people scrape the site. We know that there are a number of sites, like yours, that use the scraped data to provide a useful and legitimate service to the community. But we can't enforce what we don't see. So if we don't see you scraping the site, we look the other way. But as soon as we see it, we'll contact you.

Elias

[M-Class]

Elias,

I understand. I'm sure you also understand that I'm not comfortable with that particular arrangement, meaning the useful service I provide has reached it's end.

Has Groundspeak considered the possibility of negotiating licensing agreements with third party sites (like mine) for access to the data? I'm thinking of something along the lines of a checkbox in user profiles for "allow third parties access to my data" (an opt-in) and a private interface for "recognized third parties" to query the data (hides/finds list, cache page info, logs, etc.) more responsibly? None of us really want to page scrape in the first place. I understand this would mean some cost on my behalf, meaning I would have to pass the cost on to the members. Such an arrangement could be beneficial for both Groundspeak and the geocaching community at large. For example:

1. It solves the whole "stats are great" vs. "stats are evil" debate by leaving the choice up to the end user.
2. It provides Groundspeak with an additional revenue stream through the licensing agreement.
3. It opens the door to additional value added services that GC doesn't have the time and/or desire to implement on the site (relieve the "I need this now" pressure from the developers)
4. It will actually reduce the GC load by distributing the data demand.... as an example: a stats site could implement a "give this to me in GPX" function that could reduce the PQ load on GC. I had considered such a function in my stats site but found that it would probably violate the waypoint license agreement and opted to never implement it.

I would like to discuss some sort of license agreement to access the data and provide value-added services.

[Team Sagefox]

GrizzlyJohn, on Sep 21 2004, 02:40 PM, said:

But the point that really got to me was questioning my honesty. Don’t ever do that!... ...So do not ever question what I say my reasons are. You don’t know me and you have no idea or anything to base your statements on.
... But I will not sit by and allow you to call me a liar. Nothing I have said even comes close to thinking that is an acceptable thing to do.

You are correct. I should not have written "That can't possibly be true." My post stands well enough on its own without having said that.

I do apologize. I try to prevent myself from going over the edge but it doesn't always work.

But we are too far off-topic and this is not the arena to debate these issues so I am checking out for now.

[Renegade Knight]

Good thread.

It will be interesting to see the outcome. I hope the outcome is made public.

[jeff35080]

Quote

We know people scrape the site. We know that there are a number of sites, like yours, that use the scraped data to provide a useful and legitimate service to the community. But we can't enforce what we don't see. So if we don't see you scraping the site, we look the other way. But as soon as we see it, we'll contact you.

Don't ask, don't tell?

I've been following this thread with quite a bit of interest. I'm the webmaster of a statewide association and have some code that would allow me to do exactly what others have done. Some of it is open source with some modifications by me. I have NEVER utilized this code to access gc.com, although I was planning to do some 'off hours' scraping for the benefit of our state association.

As the webmaster and as a member of a state association, I really would like to know how we (the association) can find a way to 'license' data from gc.com for use on our association's web site in the form of statistics.

Will gc.com ever license those of us that want to use the data for good, not evil? It may be time for gc.com to consider how to license data such as this to legitimate groups.

I know our state association would be willing to work with gc.com to come-up with a way for us to have 'live' stats at a fair price and not at the expense of hurting the performance of gc.com

Now... back to the lively discussion

[Elias]

M-Class, on Sep 21 2004, 06:59 PM, said:

Has Groundspeak considered the possibility of negotiating licensing agreements with third party sites (like mine) for access to the data?

Way back when, we did try to do this. We gave permission and we got burned. That's why we have taken the stance we now have.

That said, and I know this will sound a little like a broken record, we are working on creating web services which will allow better and direct access to some of the data. What "better" and "some" mean have yet to be completely defined, but it is in the works. While the first version of these services will be targeted to Premium Members to complement the existing Premium Member services we now have, its reasonable to expect that at some point they'll be extended to support other uses.

I'm sorry you feel you have to take your site down. I wish I could give you the official answer you want, but I can't. Its not our intent to prevent you from providing the service you give to the community, but until we have a supported method for accessing the data, its the only answer I can give you.

Elias

[jeff35080]

Quote

We gave permission and we got burned.

Did you have a legal and binding signed contract or simply a verbal gentlemen's agreement?

If a legitimate entity such a state organization would be willing to sign a contract for service that defined exactly how, what, where and when the data would be extracted would Groundspeak be willing to work with such a group?

[Renegade Knight]

Robespierre, on Sep 21 2004, 02:22 PM, said:

Grizzly John, as far as I can tell YOU ARE NOT A VOLUNTEER, NOR AM I. We are just people who play the game (and forgive me if you are an approver, or something more than a player).
-Putting out caches is one of the most fun parts of the game. Play if you want, but no dividends are paid for playing. You have no investment here. (nor I). You use the site to play the game. That is all. You don't become a stockholder by playing.

Oh, man. So far I had been staying out of this one pretty well.

What does it take to be "Something more than a player"?

A lot of the posters in this thread read like a Who's Who in geocaching. Ed Hall of course started up Buxleys when GC.com maps didn't much exist, M-Class was a pioneer in bringing back stats at some level when Dan Miller's site dropped off the net. Robert Lipe brings us GPSBabel, and amazing program that among other things gives a lot of the ability to GSAK. Several others run or volunteer for state, local, and national geocaching organizations.

There are a lot of players here and they are on both sides of the fence. All of them have a say in what geocaching should be. What exactly this site listens to will determine whether or not this site is much of anything in the long run. I'm not a player at GC.com but I do have to live with the results to play the game. That may not give me say here, but you know what? It should.

[Elias]

jeff35080, on Sep 21 2004, 07:17 PM, said:

Will gc.com ever license those of us that want to use the data for good, not evil?

I can tell you this is something we want to do. Its the how that's hard. And I'm not talking about the technical stuff. Its how to enforce and audit that the data that's out there is being used in the appropriate and intended manner. Its how to make sure that the rights of those who own the data (not Groundspeak, but all of you) are preserved and respected. There are environmental concerns, and data integrity concerns that have to be addressed; particularly the damage that "stale" data can cause. We have to be very careful to ensure that once the data is "out in the open" it doesn't damage or tarnish the integrity of the sport.

These are big issues that we at Groundspeak take very seriously. And after having been burned, we're going to proceed very slowly and carefully.

Quote

I know our state association would be willing to work with gc.com to come-up with a way for us to have 'live' stats at a fair price and not at the expense of hurting the performance of gc.com

I know there wasn't a direct question here, but I'd like to add something anyway. Groundspeak wants to promote and support local organizations like yours. I can't make any promises, but I'm pretty comfortable saying that any agreement we'd make with an org like yours would be purely for use of the data. Its highly unlikely we'd ever ask an org to pay a licensing fee for the data.

Elias

[jeff35080]

Quote

I know there wasn't a direct question here, but I'd like to add something anyway. Groundspeak wants to promote and support local organizations like yours. I can't make any promises, but I'm pretty comfortable saying that any agreement we'd make with an org like yours would be purely for use of the data. Its highly unlikely we'd ever ask an org to pay a licensing fee for the data.

I'm glad you were able to read between the lines Hopefully, this will all work-out where it ends-up being good for everyone Please, keep us posted!

[M-Class]

Don't worry... I'm not taking it personally Elias.

I understand the stance and the whole 'Don't ask, don't tell' policy. Legally, if you know something is going on and you don't enforce it, then when you do have to enforce it for some other reason your case may be weakened. Thus, you know sites like mine exist, but you have delicately chosen not to investigate and prove how the data is acquired... that's fair enough, and I respect that. If you had investigated, I might be in the unfortunate position of being in a courtroom today. I think it's fair to say that I'm quite exposed now and that it is in my best interest to simply walk away having never been put under the investigative scrutiny, less you be pressed on the issue from someone else and are forced into investigating and taking legal action. It's a very uncomfortable position to be in, and I don't want to risk any action on myself or any of the orginizations (clubs) participating in the system. Shutting down the site seems to be the only proper and ethical thing to do until such a time the geocaching.com is willing to make room at the table to discuss such things as data licensing. For the record, I would like to be one of the first at that table.

For those who want to know, I will be keeping the site available for historical/archival purposes, but no new data will be added to the system. The site just celebrated it's 1st birthday last month. I wasn't sure how long it would last when it began (considering the controversial stir that stats were at the time). I am happy to have been able to provide such a valuable service to the geocaching community for the year that it lasted. Thank you to all that provided support to keep things going. If/when geocaching.com decides to allow the site to return under a more proper arrangement, I will consider bringing the site back to service for the community to enjoy. Thank you.

[jslagle]

Mopar, on Sep 18 2004, 04:16 AM, said:

StarBrand, on Sep 18 2004, 12:30 AM, said:

Gee Mopar - that looks cool!! - too bad it doesn't exist anywhere.....

Quote

Wow, that is super useful to me...if only it existed.... 

That map, and the GPX file I loaded into my GPS for that trip, does exist. That was a real example from a few weeks ago. All the software exists right now to do that yourself for your own trip. It just has to be done on your own PC, using the PQs we have now. The only problem with that is it's wasteful. That leg of the trip required 2 PQs of 500 caches each just to get the 40 I was interested in. This is going way off-topic though, so if anyone is interested in how to plan a trip without staring at maps and clicking hundreds of little red dots, start with this thread. I basically use several of the ideas discussed there.

I recently took a trip from Vegas back to Toledo. I decided we would cache along the route back, so I went about doing just this. I created my route, and made an arc file from it. I created a set of 50 mile radius with a little overlap to cover congested areas on my route... This took 22 queries.

Over the next 5 or 6 days, I created each of the queries (I admit to the fact that I scriprted the creation of the queries. Doing it by hand is really slow.)

What I got back was 5.1 megs of zipped data, which I used several tools to prune down to my route.. The route started out with 3902 caches along it. Once pruned, that was reduced to 545 or so. With the duplicates because of the overlap, which is necessary, the PQ's pulled close to 4500 caches.

So basically, at LEAST 3400 caches weren't needed.

Now, suppose I went out and wrote a software package to take a route, create the necessary PQ entries for you to enter by hand, took the resulting GPX's, and generated a resulting GPX file of the caches along the route. This would put it within the realm of the normal non computer nerdy cacher to do; those that would be using buxley's site.

What do you think the resulting load on the site would be?

Moral of the story. Until they can offer a similiar service themselves, it probably makes sense for them to leave it alone. People have been using this mostly benign way of doing routes by hand. Now they're all looking for a new way, and it's only a matter of time before someone provides it.

[Elias]

jeff35080, on Sep 21 2004, 07:42 PM, said:

Did you have a legal and binding signed contract or simply a verbal gentlemen's agreement?

Honestly, it wasn't even much of an agreement (and it certainly had no legal component at all). It was a "hey, I can do this for you", and we thought that was cool, so we gave the go ahead. Then an issue developed and we tried to make changes after the fact, and we failed. At the time it seemed obvious to us that people would be respectful of others' wishes, but we were naive.

But it got us to thinking along the same lines as you are. What if we had a legal agreement? Does that really change our position in the situation? What can we gain from a successful outcome of a lawsuit?

We're of the opinion that it doesn't help all that much. If someone we have an agreement with chooses to give us the finger and out of spite posts the data to a newsgroup, what would be appropriate to seek in damages? At that point its not about money, the damage to the sport has already been done. People who come across that data who don't know to check our site to see if its valid, pose a potential risk to themselves, the environment, and possibly even the cache owner. And of course this is an international sport, and just the thought of trying to enforce a legal agreement outside this country makes my head spin.

Its been mentioned before, but Groundspeak feels that its role is to be the custodians of the data. We take that responsibility seriously and its why we're so protective of your data. There's a lot of legal stuff on the site and yeah, it was written by lawyers. But even though the agreements are real, we're relying a lot more on trust than anything else.

Quote

If a legitimate entity such a state organization would be willing to sign a contract for service that defined exactly how, what, where and when the data would be extracted would Groundspeak be willing to work with such a group?

Even with everything I've said above, the answer is yes. We do want to work with orgs and provide some level of access to the data. What, how, and when, I don't know. But we do want to do this, and we will work through our issues and find a way to make it happen.

Elias

[Pto]

I would just like to add that I, personally greatly appreciate the efforts of all the "3rd" parties around the country that are fighting this fight for ALL of us, the geocaching community. (OK, so "fight" may be a bit extreme, but IMHO-given my readings of these forums I think it fits)

YOU people are the people that make this whole concept of a hobby/sport/recreational activity even More enjoyable from the end users perspective. Sure, we can get maps. Sure we can create our own routes. Sure, it is technically possible to do all these things long hand, on your PC - but you folks have pushed the envelope, and created some truly Amazing! applications/tools/etc to create some very useful methods of using this data WE, as a whole have created. THANK YOU!

To All state and local groups, I want to THANK YOU for your hard work in this area. I am sure your locals appreciate you, maybe more than you might think.
Mr Minnesota,(there is only 1, right?) you are owed a special Thanks from me, as I have used your tools/info/cool things to "do my thing".

This thread has caused me to gain a new perspective on the whole "who's data is it anyways" argument, and the whole "pay up or shut up" principle many seem to live by.

If this site, or maybe even this sport is to survive, growing pains are to be expected- but must be oovercome. Its getting thru these issues, and in a timely matter - that could determine the outcome. Snowballs go downhill fast....we all understand that.

Thanks to Elias for contributing some straight forward dialogue to get this issue moving again, as it seemed stalled by lack of communication (as often seems the case)

Open dialouge with the users, makers of said data will go a LONG way in resolving these issues - hopefully sooner than later-
More of this open dialogue would also help alot of the "issues" around here be cleared up sooner, without as as much of the BS in between the users, and the "holders of the grail"

Now they're all looking for a new way, and it's only a matter of time before someone provides it.

Yes, I agree completely- There might only be one leap between gc.com and a "3rd party" provider. A few servers, some volunteers and it could be back to the garage again- only this time it could be a garage FULL of AWESOME tools, freely available to ALL who contribute thier cache data.....

enough for this entry. I apoliogize if I forgot anyone, or went off topic.

This post has been edited by Pto: 21 September 2004 - 08:05 PM

[jeff35080]

Cool! We have some awesome dialogue here... and, as far as I'm concerned, very positive dialogue.

To expand further Elias, I would like to suggest that Groundspeak work closely with someone such as M-Class or others who have written some very clean code that should be unobtrusive to gc.com By working with a small handful of developers, gc.com could have access to code and check to ensure that it would meet any requirements set-forth for regional groups to use i.e. number of accesses, method of access (when the issue is decided and finalized).

It would be nice for regional/state/local groups to have 3-4 'choices' of software that would bear the Groundspeak 'Seal of Approval'. The software developers could 'sell/license' their code to state groups. In return, these developers could possibly have to pay Groundspeak a 'fee' for approval of their code. I realize that M-Class did charge a nominal fee for the use of his services, but it probably wasn't enough to cover the costs of development and/or bandwidth.

Like I said, I have software that can access gc.com and scrape data in a myriad of ways, but I have always been afraid to actually use it since I value the good relationship the Alabama Geocachers Assn has with Groundspeak.

I'll admit that I'm tired and somewhat rambling with thoughts that probably don't make much sense, but I am glad to see this discussion take a positive turn with input from Groundspeak. Thanks for the candid and sincere responses!

[boulter]

Legal questions about sharing the data aside, one way to solve the technical problem of how to provide data to sites without making these webservices completely public would be to require a token or userID on each request.

Getting your token would involve an application process, giving information about the site and its purpose.

If abuse occured, you could revoke access for that token. Likewise, you could limit the request rate for that token.

Google and Amazon use similar schemes for their programmatic APIs.

[StarBrand]

Bravo!! We are talking with straightforward talk now!!! I am glad to see solutions and honest talk about the data and 3rd parties - good job with the systems Elias and thanks for responding!!! (now I am really out of here!!!)

[tobsas]

Elias, "Thank you!" for stepping in and answering questions! :-) It was IMO way to long without an official posting at least about the status. I'm glad to hear that you will talk and I hope you're able to do that soon as this pending status is creating more negative vibrations than needed, esp. as a good part arised out of the situation that Groundspeak seemed to be a black hole.

I have to admit that I was unaware about this :

robertlipe, on Sep 22 2004, 12:29 AM, said:

Now that I'm looking at the Waypoint License Agreement and the Terms of Use (everybody noticed they were different, right?) with a careful eye, I'm wondering if every program that reads and writes pocket queries has a license to

Quote

reverse engineer, decompile, or disassemble the GROUNDSPEAK-compatible data format(s) in an attempt to duplicate the proprietary and copyright-protected GROUNDSPEAK data model(s) and/or export format(s).

and I can't find where the quote is written. Google turns up empty searching for "decompile disassemble Groundspeak" and I failed while searching the homepage.

As I work with the Watchelist-eMails from Groundspeak to update the status of the TB race "Rund um Deutschland 2004", I'm very uncomfortable about this. IANAL, but I guess in legal terms you could say that I somehow decompile to (partly) duplicate data (coordinates, name of the cacher who placed the TB, link to cacher and cache). And I seriously have to overthink if I can live with that "as long as we don't contact you, we look away" as it really leaves me in the open and the risk of a lawsuit.

Greetings,
Tobias