Tuesday, November 8, 2011

Storing long strings in Google App Engine

Recently I am experimenting with Google App Engine development for a small cool project I involved in.
(p.s. GAE has been recently got out of preview mode into release mode)

As you probably know,  the data is persist in NoSQL type storage.
This has some implications on the state of mind one need to model and access its data.

I was surprised to learn the concept and constraint this model brings ( such as that queries will use only AND no OR..).

Once interesting issue that I faced was the fact that String field is limited to 500 characters.

Since I needed to store longer strings, I immediately turned into the next data type which contains higher amount of characters - Text.
However, following that change I faced another problem - I could not do any queries on that field, since it could not be indexed.

Therefore I sought another solution.
I decided to split the long string into smaller chunks (500 long each) and store it in StringList type.
That provided me a way to keep the string length and have queries by this field (Google Query Language allows to do queries on list).

So my question is - do you have any other solution for storing long Strings given the fact that the field required to be searchable?

p.s.
The project was written in python.... grrrr... I have nothing further to say.

No comments :

Post a Comment