FAQ Database Discussion Community


Google Appengine Search API to Datastore Query

google-app-engine,app-engine-ndb,google-search-api,google-app-engine-python
I construct the Search API data like search.SearchResults( results= [ search.ScoredDocument(doc_id=u'4873035534303232', fields=[search.TextField(name=u'name', value=u'a'), search.TextField(name=u'description', value=u'b'), search.DateField(name=u'date', value=datetime.datetime(2015, 2, 16, 7, 21, 58, 188000)), search.TextField(name=u'author', value=u'[email protected]')], language=u'en', rank=130231318L), search.ScoredDocument(doc_id=u'5717460464435200', fields=[search.TextField(name=u'name', value=u'a b c'),...

Does App Engine NDB keys in KeyProperty count against 1MB storage limit?

python,google-app-engine,app-engine-ndb
I'm creating NDB Model instances in Google App Engine that are hitting the 1MB size limit. It has to do with a repeated StringProperty which is 10-20k in length. Here's an example: class MyModel(ndb.Model): items = ndb.StringProperty(repeated=True) instance = MyModel() instance.items = [] For item in largeDataset: instance.items.append(item) instance.put() #...

python ndb classes that reference each other

python,google-app-engine,app-engine-ndb
I have two classes: class A(ndb.Model): first_prop = ndb.StructuredProperty(B) class B(ndb.Model): second_prop = ndb.StructuredProperty(A) putting the class name in quotes gives an error. What is a reasonable way to make it happen , which leaves code encapsulation intact ?...

Can I compare two properties of single kind in ndb of GAE?

google-app-engine,app-engine-ndb
Can I compare two properties of single kind in ndb of GAE? class GameLog(ndb.Model): duration = ndb.IntegerProperty(default=0) time = ndb.IntegerProperty(default=0) I need compare these two properties. How can I do this? GameLog.duration > GameLog.time ...

Fetching an image from google app engine ndb

python,image,google-app-engine,app-engine-ndb
Can someone verify/correct my method of accessing an image that is part of a ndb.Model class? This is my MainPage handler: class MainPage(webapp2.RequestHandler): def get(self): guestbook_name = self.request.get('guestbook_name', DEFAULT_GUESTBOOK_NAME) greetings_query = Greeting.query( ancestor=guestbook_key(guestbook_name)).order(-Greeting.date) greetings = greetings_query.fetch(10) template_values = { 'greetings': greetings, 'guestbook_name': urllib.quote_plus(guestbook_name), } template = JINJA_ENVIRONMENT.get_template('index.html')...

Structuring NDB models to make an “antijoin” query possible

python,google-app-engine,app-engine-ndb
I want to have a recommendation API for my app where a user can get a suggestion for an object they haven't yet seen, and I'm having trouble figuring out how I'm supposed to structure my data to make such a query efficient. Here's an example using books. Suppose this...

GAE/P: Migrating to NDB efficiently

python,google-app-engine,app-engine-ndb
I'm finally upgrading from db to ndb (it is a much bigger headache than I anticipated...). I used a lot of ReferenceProperty and I've converted these to KeyProperty. Now, every place where I used a ReferenceProperty I need to add an explicit get because it was previously done for me...

Google App Engine ndb order by length of StringProperty

sql-order-by,app-engine-ndb
Is it possible to order a query by the length of a StringProperty? I tried this: User.query(User.facebook_id = fb_id).order(-len(str(User.name))).fetch(10) but it displays me the error: TypeError: order() expects a Property or query Order;...

Updating an NDB's Model's Schema [closed]

python,google-app-engine,gae-datastore,app-engine-ndb
I have seen the solution to this question using App Engine's original Datastore, but cannot find a solution while using NDB. What is the best way to add migration support, so that I am able to migrate from an old version of a schema to a new version. Would it...

How to upload an image through appengine endpoints api?

google-app-engine,google-cloud-endpoints,app-engine-ndb,endpoints-proto-datastore
I am using Endpoints Proto Datastore API. Suppose I have code like this : Model : class MyModel(EndpointsModel): _message_fields_schema = ('name', 'image') name = ndb.StringProperty() image = ndb.BlobProperty() API : @endpoints.api(name='myapi', version='v1', description='my api') class MyApi(remote.Service): @MyModel.method(name='mymodel.insert', path='mymodel') def insert_mymodel(self, data): data.put() return data How can I upload image to...

Delete ndb EntitiesByProperty index table

google-app-engine,indexing,gae-datastore,app-engine-ndb
I have an NDB model with some obsolete EntitiesByProperty indexes. Some belong to properties that are no longer part of the schema (but they do exist on some old entities), and some belong to properties that are never used for queries (but they were with Indexed=True because this is ndb...

Fetching entities from datastore where Entity.key.IN([keys…])

database,google-app-engine,app-engine-ndb,datastore
I'm trying to fetch a long list of entities, and those entities all refer to one of a few different related entities. It's explained in the comments, but basically many "items" reference to a few "Company"s. I don't want to have to make multiple queries for each key in unique_key...

EndpointsAliasProperty filtering error

google-app-engine,app-engine-ndb,endpoints-proto-datastore
I am trying to implement custom query responses using EndpointsAliasProperty. Here is the code self._endpoints_query_info._filters.add(MyModel.hash_tag.IN(['a', 'b', 'c'])) It is giving me an error BadArgumentError: _MultiQuery with cursors requires __key__ order how can I solve this? Is there a way to implement above idea?...

How to use choices in Google App Engine NDB Model

python,django,google-app-engine,app-engine-ndb
I want to create a model of ndb.Model of Google App Engine in Python. One of the fields is a choice, there will be 4 options (0 to 3) and each option will have a string representing it. How do I define this field in the model? I want the...

Update for big count of NDB Entities fails

python,google-app-engine,bigdata,app-engine-ndb
I have very simple task. After migration and adding new field (repeated and composite property) to existing NDB Entity (~100K entities) I need to setup default value for it. I tried that code first: q = dm.E.query(ancestor=dm.E.root_key) for user in q.iter(batch_size=500): user.field1 = [dm.E2()] user.put() But it fails with such...

GAE NDB PickleProperty for dict: new entity taking old property?

python,google-app-engine,pickle,app-engine-ndb
Looks like a bug to me, but maybe someone will have a reasonable explanation for this. Consider the following code: class Test(ndb.Model): a= ndb.IntegerProperty() p = ndb.PickleProperty(default={}) Now do the following: >> t1 = Test() >> t1.p['a'] = 1 >> t1.p['b'] = 2 {'a': 1, 'b': 2} Up to here,...

Get properties of ndb model instance where Python name != datastore name

python,google-app-engine,app-engine-ndb,google-app-engine-python
Goal: get "Python names" of attributes from an instance of Example, where the model was defined with a different datastore name To provide some context, I have a custom to_dict() method for serializing an NDB Model. The core of the method is as follows: for key, prop in self._properties.iteritems(): if...

How to keep your code clean with denormalized data?

python,google-app-engine,app-engine-ndb,denormalization
I'm going to ask this question in two parts: first the general question, and than the question for my specific use case. The general question: I'm building a podcast app, where, hopefully, we'll have users. Users have subscripitons, settings, ... , which I'd like to store on the User object,...

App Engine - NDB query with projection requires subproperty?

google-app-engine,gae-datastore,app-engine-ndb,google-app-engine-python
I have the following objects: class Address(ndb.Model): type = ndb.StringProperty() # E.g., 'home', 'work' street = ndb.StringProperty() city = ndb.StringProperty() class Friend(ndb.Model): first_name = ndb.StringProperty() # E.g., 'home', 'work' last_name = ndb.StringProperty() class Contact(ndb.Model): name = ndb.StringProperty() addresses = ndb.StructuredProperty(Address, repeated=True) friends = ndb.StructuredProperty(Friend, repeated=True) And now to optimize the...

Use Google App Engine's NDB as a message queue?

google-app-engine,message-queue,app-engine-ndb,google-datastore
Has anyone tried to use NDB as a message queue? We have several consumers and producers, which may want to do broadcast, multicast, and publish-subscribe. I've read several documents on why using a RDBMS as a message queue is bad. But in my case, my app can tolerate latency of...

google app engine python: how to extend ndb User class

python,google-app-engine,google-cloud-endpoints,app-engine-ndb,endpoints-proto-datastore
I'm using google cloud endpoints and want to be able to extend the User class so that a call to get_current_user would return an AppUser object with my own extra properties. class AppUser(--?--): # what should i put here gcm = ndb.StringProperty() def send_notification(self): # do something with gcm ......

how to make ndb model declarations order independent

python,google-app-engine,app-engine-ndb
I am new to working with python and app engine. I have tried to implement some ndb models like this: class A(ndb.Model): some_property = ndb.KeyProperty(B , required=True) class B(ndb.Model): some_other_property = ndb.KeyProperty(A , required=True) of course you wouldn't do something like this in production as it's a very bad design...

Google App Engine - Get from repeated StructuredProperty

google-app-engine,app-engine-ndb
I have the following structures: class UserOther(ndb.Model): other_type = ndb.StringProperty(indexed = True) other_data = ndb.StringProperty(indexed = False) class User(ndb.Model): name = ndb.StringProperty(default = "NULL", indexed = False) email = ndb.StringProperty(default = "NULL", indexed = False) active = ndb.BooleanProperty(default = True) others = ndb.StructuredProperty(UserOther, repeated = True) updated_at = ndb.DateTimeProperty(auto_now =...

The error creating app Engine's DataStore Entity within GWT app

gwt,app-engine-ndb
I try to create the entity like this: DatastoreService ds = DatastoreServiceFactory.getDatastoreService(); Entity stock = new Entity("Stock", 1); stock.setProperty("Stock", "FCB"); ds.put(stock); but keep getting the error: No source code is available for type com.google.appengine.api.datastore.DatastoreService; did you forget to inherit a required module? ...

How can I use ndb outside of appengine?

python,database,flask,flask-sqlalchemy,app-engine-ndb
I really like GAE's object-relational database tools, db and ndb. I am currently preparing to develop a server which needs to be hosted from a local machine outside of Google's cloud environment. Namely, I am planning to use a python flask server. I've used other tools like SQLAlchemy to interact...

value error raised from using ndb.blobproperty

python,django,google-app-engine,app-engine-ndb
So I'm following the Python Image tutorial for Google App Engine and I'm getting the error: "raise ValueError('Name %r cannot contain period characters' % (name,))" I'm trying to save an image as part of my Model. I think it has to do with how I'm saving the uploaded image. I...

Is there any way to pass NDB kind as string?

google-app-engine,app-engine-ndb
Currently I use the code like below - Price.query().fetch(keys_only=True)) The Price here is hardcoded kind name. Is there any way to have something like: def get_kind_keys(kindname): return Kind(kindname).query().fetch(keys_only=True) ...

In Google App Engine, how to check input validity of Key created by urlsafe?

google-app-engine,app-engine-ndb
Suppose I create a key from user input websafe url key = ndb.Key(urlsafe=some_user_input) How can I check if the some_user_input is valid? My current experiment shows that statement above will throw ProtocolBufferDecodeError (Unable to merge from string.) exception if the some_user_input is invalid, but could not find anything about this...

ComputedProperty only updates on second put()

python,google-app-engine,app-engine-ndb,google-app-engine-python
I have a ComputedProperty inside a StructuredProperty that does not get updated when the object is first created. When I create the object address_components_ascii does not get saved. The field is not visible in the Datastore Viewer at all. But if I get() and then immediately put() again (even without...

NDB datastore write confirmation

google-app-engine,gae-datastore,app-engine-ndb
Is there any flag activated whenever a field is updated or added to the datastore? I have an edit handler that modifies fields(obviously) but when I return to the page where all the items are displayed, the fields appeared unchanged until I have refresh the page again. What I'm doing...

BadValueError when updating ndb entity

python,google-app-engine,app-engine-ndb
I am implementing a web application based on Google App Engine, Relying on ndb. I am facing a strange problem when trying to update one of the many entities in my db. When I try to update one of its properties (i.e. a string property) I get "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1715,...

Is it dangerous to allow user to give arguments to ndb.Query()?

security,google-app-engine,app-engine-ndb
I'm writing a search tool, and one of the things I have to do is to let the user search for ndb objects. Is it dangerous to let the user choose arguments to the ndb.Query() function? Can it be used to modify or delete data? Since I can supply the...

GAE: how do I query for an item that is referenced in a specific document?

python,google-app-engine,app-engine-ndb
I have City -> District and I need to find the District with a specified name in the specified City, if any. The code: class City(ndb.Model): districts = ndb.KeyProperty(kind=District, repeated=True) class District(ndb.Model): name = ndb.StringProperty() How do I do this?...