App Engine Geospatial Datastore Search: A New Way!

One of my pet peeves with Google App Engine is its horrible support for geospatial indexes. Although you can store a GeoPt in the Datastore, you can’t really query it. You can use various hacks such as Geomodel, but they end up being slow and potentially expensive.

Last year Google released the beta API for Google App Engine search. This lets you search documents for text, HTML, numbers, dates, and location. However, it searches documents instead of datastore entities.

If documents are separate from your datastore, how do you use the new search API to do geospatial queries on your database? Simply store the location for each entity inside a document instead. To do this, make a document with a GeoField with the location and a string that contains the id of the associated datastore entity’s key (this code is based on Google’s own location example):

key = str(entity.key())
geopoint = search.GeoPoint(lat, lng)
index = search.Index(_INDEX_NAME)
document = search.Document( fields=[search.TextField(name='key', value=str(key)),
                                                    search.GeoField(name='loc', value=geopoint)])

Note that you have to store the key’s id as a string since you can’t store a long in a document.

Now, when you perform a geolocation search like this:

index = search.Index(_INDEX_NAME)
query = "distance(loc, geopoint(" + str(lat) + "," + str(lng) + ")) < 1000"
     results =
     for doc in results:'Document! ' + str(doc.field("key")))
except search.Error:

You can grab the id field from the document and query the datastore for it to get the rest of your data.

There are several problems with this method. First, it doesn’t work on the local dev_app_server. Currently, GeoField searches only work on the appspot production servers. Also, because the API is in beta you are restricted to the free quotas which don’t allow for very many operations. Finally, when Google reveals pricing changes, it can have disastrous results. It’s very risky to build an app using this method when you have no idea how much it costs.

At least it works! It’s still a mystery to me why they can’t add this feature to the datastore itself.

13 thoughts on “App Engine Geospatial Datastore Search: A New Way!

  1. Pingback: Social Recruiting: Google App Engine Experiments 3 | Ralph Barbagallo's Self Indulgent Blog

  2. We tested the api yesterday but we ran out off quota in 5 minutes with 3 users 😦 the reason:

    “Search API Complex Searches 100% 100 of 100 Limited”

  3. I really would like to know:
    why would I use this google search API knowing that I have to create document and indexes which for me is a kind of second database on which I have to perform queries.
    Why not just use an SQL database in this case?

  4. How would one go about querying by multiple properties on top of GeoPoint without having to put all the datastore properties in a search document?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s