FAQ Database Discussion Community


Objectify transaction rollback not happening

gae-datastore,objectify,google-datastore
When I intentionally throw an exception within an Objectify transaction, my transaction is not being rolled back. What am I doing wrong? @Inject Dao dao ... public void testTransaction(){ dao.transact(new VoidWork() { @Override public void vrun() { Key aclKey= dao.save().entity(acl).now(); //expecting this to be rolled back //throw exception if(true) throw...

How are default values applied in app engine?

google-app-engine,gae-datastore
I'd like to know whether default values are applied at the time of instance creation, instance update, or only when attempting to read the value. I'd like to understand how this applies to queries. For example, suppose I initially create a model w/o default values class Foo (db.Model): bar =...

How to delete entity by Id from data-store in AppEngine?

google-app-engine,gae-datastore,objectify,objectify-delete
I created an API to delete an Entity by its key however I'm gettin Http 204 and Entity does not delete from data-store. This is my API, @ApiMethod(name = "deleteContact", path = "contact", httpMethod = ApiMethod.HttpMethod.DELETE) public void deleteContact(final @Named("id") long contactId) { ofy().delete().type(Contact.class).id(contactId).now(); } and my Contact class is...

Objectify list consistency in one entity

google-app-engine,gae-datastore,objectify
I try to make a chat solution on App Engine for my android app. A decided that instead of save all messages send to a topic in a separated entity like ChatMessage or something like this, I can save them in a List of Strings inside the Topic entity, like...

Querying immediately for saved entity returns null sometimes - datastore

java,ajax,google-app-engine,gae-datastore
I am saving an entity kind 'review' with following properties review { _id : 4e0590ae-7603-44ff-8333-b3d06546914f text : 'Nice review' } I am saving it with a AJAX call and on success response I am passing _id and querying for saved review in another AJAX call. Sometimes I am getting null...

App Engine: Mechanics of creating a unique entity if it doesn't exist

google-app-engine,transactions,gae-datastore
We have an issue where we want to lazily create an entity if it does not exist. There is some discussion going on about how to do this and I would like to clarify some things around app engine transactions. I will limit my query to single entity group transactions....

Retrieve an entity by entity id

python,google-app-engine,gae-datastore
In Datastore View, I can see Edit Entity: SaleRecord Decoded entity key: SaleRecord: id=4506503696351232 Entity key: ahJzfmF1dG90d2VldG1hY2hpbmVyFwsSClNhbGVSZWNvcmQYgICAwMLUgAgM How can I retrieve that entity if I only know the key id (4506503696351232). (I am using Python) I tried: s= SaleRecord.get('4506503696351232') but it returned errors...

Should I use NDB or Search API for storing User model?

python,google-app-engine,full-text-search,gae-datastore,gae-search
Suppose I have basic User model in ndb with following properties: ``` name = ndb.StringProperty(default='') username = ndb.StringProperty(required=True) email = ndb.StringProperty(default='') active = ndb.BooleanProperty(default=True) admin = ndb.BooleanProperty(default=False) permissions = ndb.StringProperty(repeated=True) verified = ndb.BooleanProperty(default=False) token = ndb.StringProperty(default='') password_hash = ndb.StringProperty(default='') bio = ndb.StringProperty(default='') location = ndb.StringProperty(default='') facebook...

Can Objectify library be used outside GAE?

google-app-engine,gae-datastore,google-cloud-platform
I want to use the Google Cloud Datastore. However, I am not using GAE. Can I still use the Objectify library? If so how? The documentation doesnt say anything about how to configure it to authenticate against Datastore....

Google datastore multiple values for the same property

android,google-app-engine,gae-datastore,google-datastore
I am using Google Datastore for an Android application, the backend is written in Java. In one table, I want to set multiple values to the same property: Entity newGroup = new Entity("group"); newGroup.setProperty("member", "A"); newGroup.setProperty("member", "B"); newGroup.setProperty("member", "C"); datastore.put(newGroup); I then want to query to find all groups a...

datastore - using queries and transactions

google-app-engine,gae-datastore
I'm working with objectify. In my app, i have Employee entities and Task entities. Every task is created for a given employee. Datastore is generating id for Task entities. But i have a restriction, i can't save a task for an employee if that task overlaps with another already existing...

What is the benefit of parent and child Entity in GAE?

java,google-app-engine,gae-datastore
How will I know if the entity I am creating should be a child entity or not? DatastoreService ds = DatastoreServiceFactory.getDatastoreService(); Entity E = new Entity("Person",52); E.setProperty("FirstName","Jack"); E.setProperty("LastName","Patrick"); E.setProperty("Gender","Male"); ds.put(E) Now I have an entity called User, what is the benefit I get if I create User as a child...

What is the correct way to define entities in the datastore when dealing with mass data?

java,android,database,google-app-engine,gae-datastore
I'm making an instant messaging app for android and using Java and app engine for the backend. To store conversations and messages in the backend, I have 2 options (as I see it) to store the data. Create 2 root entities: conversation (ID, message IDs) and message (ID, "text"). OR...

Java datastore write performance: Objectify vs. JPA

google-app-engine,jpa,gae-datastore,objectify
I ran two five minute long simple benchmarks against the datastore. One used the JPA implementation on top of Datanucleus provided by Google and the other used Objectify for persistence. Each request created 10 new indexed entities with the same 9 fields, each using another datatype. To avoid any effects...

How can I have dynamic properties in go on the google app engine datastore

google-app-engine,go,gae-datastore
I want to do something like the Expando Model that python supports on app engine. Sometimes you don't want to declare your properties ahead of time. A special model subclass, Expando, changes the behavior of its entities so that any attribute assigned (as long as it doesn't start with an...

JPA PersistenceException: Class name could not be resolved

java,google-app-engine,jpa,gae-datastore
I'm trying to create a method to get a project by the project name from GAE, but I'm getting the following error. javax.persistence.PersistenceException: Class name TestProject could not be resolved I get this error in the following piece of code on the line where the ifstatement starts public Project getProjectByProjectName(String...

App Engine - NDB query with projection requires subproperty?

google-app-engine,gae-datastore,app-engine-ndb,google-app-engine-python
I have the following objects: class Address(ndb.Model): type = ndb.StringProperty() # E.g., 'home', 'work' street = ndb.StringProperty() city = ndb.StringProperty() class Friend(ndb.Model): first_name = ndb.StringProperty() # E.g., 'home', 'work' last_name = ndb.StringProperty() class Contact(ndb.Model): name = ndb.StringProperty() addresses = ndb.StructuredProperty(Address, repeated=True) friends = ndb.StructuredProperty(Friend, repeated=True) And now to optimize the...

NDB datastore write confirmation

google-app-engine,gae-datastore,app-engine-ndb
Is there any flag activated whenever a field is updated or added to the datastore? I have an edit handler that modifies fields(obviously) but when I return to the page where all the items are displayed, the fields appeared unchanged until I have refresh the page again. What I'm doing...

How to pass the datastore value to javascript? i need to get data from the gae datastore and need to pass javascript

javascript,twitter-bootstrap,google-app-engine,gae-datastore
am working with GAE using webapp2, python. Now am working in google map using gae. Successfully done with multiple locations. This is my JavaScript: ====================== <script type="text/javascript"> var USER_POSITION; var DESTINATION_MARKER; var ALL_MARKERS = []; var coordinateArray = [ {coordinates: new google.maps.LatLng(12.9345494,77.6115174), id: "forum", title: "Forum", address_details: Bengaluru<br/>Karnataka-560095"}, {coordinates: new...

GAE Python - get_by_key_name no result - maybe Parent issue?

python,google-app-engine,gae-datastore,parent
I use a simple Model.get_by_key_name('item_key') to retrieve an item from the datastore. This all worked fine until today, whereas now the query is returning None, although I can confirm that there is an existing entity with that key name. As far as I know, the only thing I have changed...

Get list of all NDB model names ( class names ) in GAE Python

python,google-app-engine,gae-datastore
How to get the list of all NDB model names in GAE Python ? All NDB models are Python classes which inherit from ndb.Model. I thought we could use this info to fetch the names of all models. class BK (ndb.Model): property_1 = .. I tried below ( borrowed )...

Google App Engine Datastore delete entity by properties

python,google-app-engine,gae-datastore,gql
I made a simple login page with python and datastore. the User model is the following. class User(db.Model): name = db.StringProperty() password = db.StringProperty() Question 1: Lets say that I know the name and the password of a specific user. How can I delete that entity from the database that...

Datastore & Memcache consistency - Google App Engine & Objectify

java,google-app-engine,gae-datastore,objectify,consistency
I'm having a pretty tough problem ensuring consistency in the datastore. We're trying to do a sync job to BigQuery every 1 minute (cron) and are relying on the Datastore to store a timestamp for when the previous sync was completed. We are still seeing eventual consistency when the object...

I am working on small project with Google AppEngine (Python), tutored by Udacity. I am unable to render user comments to main page

python-2.7,google-app-engine,gae-datastore,jinja2
I am suppose to add forms to my HTML to allow users to add data (comments to my page) and also store the data via GAE-Datastore. I have been able to store the data but not able to get the user comments posted to the main page. Seperate code files:...

Delete ndb EntitiesByProperty index table

google-app-engine,indexing,gae-datastore,app-engine-ndb
I have an NDB model with some obsolete EntitiesByProperty indexes. Some belong to properties that are no longer part of the schema (but they do exist on some old entities), and some belong to properties that are never used for queries (but they were with Indexed=True because this is ndb...

Checking username availability - Handling of AJAX requests (Google App Engine)

python,ajax,google-app-engine,memcached,gae-datastore
I want to add the 'check username available' functionality on my signup page using AJAX. I have few doubts about the way I should implement it. With which event should I register my AJAX requests? We can send the requests when user focus out of the 'username' input field (blur...

Get Google Cloud Storage File from ObjectName

java,google-app-engine,gae-datastore,google-cloud-storage
I'm migrating my GAE app from the deprecated File API to Google Cloud Storage Client Library. I used to persist the blobKey, but since there is partial support for it (as specified here) from now on I'll have to persist the object name. Unfortunately the object name that comes from...

Local Datastore Editor [closed]

google-app-engine,gae-datastore
I'm stuck trying to figure out a way to edit my local datastore within the Java SDK. Is there some kind of interactive interface (command line or GUI) that will allow me to modify data to help with testing and scenario creation? The online datastore editor is awesome in the...

Update entity from appengine .GetAll and save to datastore

google-app-engine,go,gae-datastore
I have a model: type UserProfile struct { UserID string `datastore:"user_id" json:"user_id,omitempty"` Username string `datastore:"username" json:"username,omitempty"` StripUsername string `datastore:"strip_username" json:"strip_username,omitempty"` Email string `datastore:"email" json:"email,omitempty"` LoginType string `datastore:"login_type" json:"login_type,omitempty"` CurrentSession string `datastore:"current_session" json:"current_session,omitempty"` FBAcessToken string `datastore:"fb_access_token"...

Verifying user accounts using email (Google App Engine)

python,google-app-engine,authentication,gae-datastore
I am building custom accounts system, where when a user creates an account I save it to Datastore but with the 'verified' field default set to false (one cannot login with an Account until its verified field is set to true.) class Account(ndb.Model): username = ndb.StringProperty(required = True) password =...

Java - GAE Datastore persisting Date with specific format

java,google-app-engine,date,gae-datastore
I have a Google app engine JDO datastore entity with a date field and it looks like the following. @PersistenceCapable public class Test { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Long id; @Persistent private Date dateTime; } The problem I'm having is it defaults to datetime (2014-08-30 00:00:00.000Z) format which is...

Export Google Cloud Datastore and import to BigQuery programmatically

gae-datastore,google-bigquery,google-datastore
I'm looking for a method to export my Cloud DataStore and import it into Big Query daily. The manual way is described at google page. I do not find a clean way to automate it.

Programmatically get Google Datastore record count and used storage

python,google-app-engine,gae-datastore,gcloud-python
I need to get Google Datastore usage statistics - storage usage, and record count for my application. At the moment I only can count fetch all items and count them. But I have no idea how I can get storage amount used. Is there any reports API I can use?...

Running Datastore queries on Google App Engine

java,google-app-engine,gae-datastore
How we can run queries from local machine to remote app deployed on Google app engine? (not talking for admin web console) Traditionally we can connect to remote db as easy as local, just by using remote IP....

GAE DatastoreService Query not getting any results?

java,google-app-engine,gae-datastore
UPDATE: It appears to work if I remove transaction related code, any reason why that would be? I'm having trouble with a simple hello world using GAE DatastoreService. I'm just trying to put stuff in and then list it, the following code should create two entities and puts them and...

Storing the value with the Ref, as long as it's not in the datastore

google-app-engine,gae-datastore,objectify
I'm have a List<Ref<Entity>>. I add new entries to the list like this: entities.add(Ref.create(new_entry)); modified.add(new_entry); When I store the entity that contains the list, I store the list itself and all the entities that are in the modified list. This works fine. The problem is, that I have to work...

How to paginate in GCS when using GAE Python GCS Client Library for access ?

python,google-app-engine,gae-datastore,google-cloud-storage
GCS = Google Cloud Storage GAE = Google App Engine If there is a huge number of files in a given directory (emulated directory since no real ones exist), how do I manage : Listing all files for some processing in my GAE Python code ? Sort in descending order...

Managing Autogenerated Indexes

python,google-app-engine,gae-datastore
I'm doing an advanced search like the one detailed in https://cloud.google.com/appengine/articles/indexselection I'm increasing the number of filters from 2 to 4, with 4 sort orders. If I create a perfect index on this search, I'll need 64 indexes in my index.yaml. I followed the advice in the Advanced Search article...

Unable to check for already existing usernames google app engine

python,google-app-engine,gae-datastore,registration,user-registration
Here is my code to check for already existing usernames : def post(self): have_error = False username = self.request.get('username') password = self.request.get('password') verify = self.request.get('verify') email = self.request.get('email') params = dict(username= username, email = email) usernames = db.GqlQuery(' select * from Users where username = :1 ', username) if not...

503 Error ObjectifyService NoClassDefFoundError

java,google-app-engine,maven,gae-datastore,objectify
I am having issues with GAE in the api explorer.I am trying to use objectify to save to a datastore but I am getting this error: { "error": { "message": "java.lang.NoClassDefFoundError: Could not initialize class Project.service.OfyService", "code": 503, "errors": [ { "domain": "global", "reason": "backendError", "message": "java.lang.NoClassDefFoundError: Could not initialize...

What/Where can I find GAE (python) Reserved Keys (key path element)

python,google-app-engine,gae-datastore
I am creating an app that will involve a lot of users which they can Assign any username they want to, which will be used as the entity ID/Name. But when I tried the username '__something__' , GAE gave me an error: raised BadRequestError(The key path element name "__something__" is...

Objectify doesn't always return results

google-app-engine,gae-datastore,objectify
I am using Objectify to store data on Google App Engine's datastore. I have been trying to implement a one-to-many relationship between two classes, but by storing a list of parameterised keys. The method below works perfectly some of the time, but returns an empty array other times - does...

How to fetch the latest 25 entries from an App-Engine datastore entity kind

java,google-app-engine,sorting,gae-datastore
I have an entity kind (i.e. ~table) with thousands of entries. I want to fetch the latest 25 entries. How do I do that? The following logic is not working: it returns the oldest 25. Filter timestampFilter = new FilterPredicate("lastestEditDate", FilterOperator.GREATER_THAN_OR_EQUAL, 0); Query query = new Query(CatMeow.class.getSimpleName()).setFilter(timestampFilter); FetchOptions options =...

How to build datastore indexes (PHP GAE)

php,google-app-engine,gae-datastore
I am using Tom Walder's Google Datastore Library for PHP to insert data into my Google App Engine Datastore. $obj_schema = (new GDS\Schema('Add Log')) ->addString('name', TRUE) ->addDatetime('time', TRUE); $obj_store = new GDS\Store($obj_gateway, $obj_schema); $obj_store->upsert($obj_store->createEntity(['name' => "test",'time' => date('Y-m-d H:i:s', time())])); When I insert data like the above code, everything seems...

How can you retrieve key of a datastore Entity

java,google-app-engine,gae-datastore,datastore
I have created datastore entity like below. I am trying to get the key of the entity by looping the query result. Also Is this the right way to get key of an entity? DatastoreService ds = DatastoreServiceFactory.getDatastoreService(); Entity e = new Entity("User"); e.setProperty("userName", user.getUserName()); e.setProperty("email", user.getEmail()); ds.put(e); Query q...

Find Maximum Index in Google Datastore (Pagination in Blog System)

python,google-app-engine,gae-datastore,datastore
I have a series of blog posts as entities. I will receive a URL that looks like this: /blog/page/1. I would like to access the most recent 5 posts in this case. In the case of /blog/page/2, I want the 6-10th most recent. So allow me to do an X-Y,...

Where is data stored when installed while logged on to a Google Cloud Compute instance?

gae-datastore,google-datastore,gcloud
I'm following this tutorial to get started with the Google Cloud Datastore: https://cloud.google.com/datastore/docs/getstarted/start_java/. At this point I have my Cloud account all set up with the APIs needed to use Datastore and Endpoints for a project I am working on. I was following the setup instructions and so far I...

Handle RequestTooLargeError exception in Appengine

python,google-app-engine,gae-datastore
When i try to insert 1000+ list into appengine datastore, i gout this error RequestTooLargeError: The request to API call datastore_v3.Put() was too large. Tried to import this from google.appengine.ext.db import RequestTooLargeError try: ..... except RequestTooLargeError: logging.error("RequestTooLargeError") But getting ImportError: cannot import name RequestTooLargeError How to catch this RequestTooLargeError exception...

db.Model.all() returns an empty list

html,python-2.7,google-app-engine,gae-datastore,jinja2
This is a fragment of my tips.html template, included in index.html: {% for t in tips %} <div class="col-lg-4 col-sm-6"> <form action="comment.html" method="get" class="portfolio-box"> <img src="img/portfolio/4.jpg" class="img-responsive" alt=""> <div class="portfolio-box-caption"> <div class="portfolio-box-caption-content"> <div class="project-category text-faded"> {{ t.title }} </div> <div class="project-name"> {{ t.content }} </div> <div><br /></div> {% if user...

persisting relationships in datastore - app engine and objectify

java,google-app-engine,gae-datastore,objectify
my app is using objectify. i'm really new to NoSql . i have a data model like this. pay no attention to lack of getters and setters, lack of builder pattern, etc. is just an example. as you can see, ReallyWeirdCar is the root of a quite deep object graph....

Trouble deleting entity with specific property set in Java GAE

java,google-app-engine,gae-datastore
I have the entities created like this: public String addNewStockName(String newStock) throws DelistedException { Entity stock = new Entity("Stocks"); stock.setProperty("Name", newStock); ds.put(stock); return "OK"; } Trying delete the specific entity like this: public String deleteStockName(String stockName){ Key key = KeyFactory.createKey("Stocks", stockName); ds.delete(key); return "OK"; } And it does not delete...

Google App Engine returning odd values from datastore

python,google-app-engine,python-2.7,gae-datastore
I'm trying to do a simple save string to datastore, redirect on that entity's key, then fetch it in the next handler. When I make the call in PDFHandler to fetch the string (a.name()) "w" is returned everytime, no matter what I enter. class Pdf(db.Model): name = db.StringProperty(required=True) class InputHandler(Handler):...

Google App Engine Datastore get value of property

python,python-2.7,google-app-engine,gae-datastore
I have some questions. Suppose we have this Model for Datastore: class SessionID(db.Model): SESSIONID = db.StringProperty() userid = db.StringProperty() userip = db.StringProperty() exp_time = db.IntegerProperty() Lets say we registered the SESSIONID property to 1234, assigned userid to bob, the userip to 123.123.123.123, and the exp_time to 1 Question 1: Suppose...

Is there a limit on the number of entities you can query from the GAE datastore?

google-app-engine,gae-datastore,objectify
My GCM Endpoint is derived from the code at /github.com/GoogleCloudPlatform/gradle-appengine-templates/tree/master/GcmEndpoints/root/src/main. Each Android client device registers with the endpoint. A message can be sent to the first 10 registered devices using this code: @Api(name = "messaging", version = "v1", namespace = @ApiNamespace(ownerDomain = "${endpointOwnerDomain}", ownerName = "${endpointOwnerDomain}", packagePath="${endpointPackagePath}")) public class MessagingEndpoint...

datastore index exception though I have defined index for the same

java,google-app-engine,gae-datastore
I have an entity king review as follows properties and value Review { merchantId : 'X', appId : 'Y', productId : 'Z', reviewId : '12345', timeStamp : '1493482784728', viewtimestamp : '148834892374' } and I have defined index as follows <datastore-index kind="review" ancestor="false" source="manual"> <property name="merchantId" direction="asc"/> <property name="appId" direction="asc"/> <property...

Updating an NDB's Model's Schema [closed]

python,google-app-engine,gae-datastore,app-engine-ndb
I have seen the solution to this question using App Engine's original Datastore, but cannot find a solution while using NDB. What is the best way to add migration support, so that I am able to migrate from an old version of a schema to a new version. Would it...

PHP-GDS Sort queries from AppEngine DataStore

php,google-app-engine,gae-datastore
I've created an index using a schema using the awesome PHP-GDS: $obj_product_schema = (new Schema('Product')) ->addString('url') ->addString('date') ->addString('name', TRUE) ->addInteger('votes', TRUE); However if I try to return a subset of objects with a sorted query as follows I get a 'no matching index found' error. $queryStr = "SELECT * FROM...

Google App Engine - ByteString query fails

google-app-engine,go,gae-datastore,google-datastore
I am working on a Go application in which I have an Entity with a Property that holds an identifying token which is a random string of bytes. I am storing this property as a ByteString, and in my development environment I have been able to query for this property...

migrating images to google app engine datastore or blobstore

python,google-app-engine,gae-datastore,blobstore
I have property model, containing a field image_url. class Property(ndb.Model): date_created=data.UTCDateTimeProperty(auto_now_add=True) # some other fields here image_url = ndb.StringProperty(indexed=False) and image model, class Image(ndb.Model): property = ndb.KeyProperty() file = ndb.KeyProperty(indexed=False) # some other fields image_url = ndb.StringProperty(indexed=False) Now I have 'n' number of images for each property in my local...

How to check if entity exist in gae datastore?

python,google-app-engine,gae-datastore
I need to check datastore if entity kind "Kayit" exist to create new . my db.model class: class Kayit(db.Model): url=db.StringProperty() date = db.DateTimeProperty(auto_now_add=True) currently using this to check but it doesnt seem elegant class MainPage(webapp2.RequestHandler): def get(self): r_name="none" q=db.Query(Kayit) try: self.response.write(q[0].url) except: kay=Kayit(parent=_DefterKey(r_name),key_name='alibaba') kay.url="some url" kay.put() I guess we get...

How to view app data on google app engine datastore?

google-app-engine,gae-datastore,datastore
I have made an android app which saves user input data in a googele app engine datastore. I want to view the stored entity on google developers console. How can I do it? On left hand side menu I can see Query menu which takes me to my entity page...

Migration from Google cloud sql to datastore

gae-datastore,google-cloud-sql,google-datastore
Is there an easy way to migrate a large (100G) Google cloud sql database to Google datastore? The way that comes to mind is to write a python appengine script for each database and table and then put it into the datastore. That sounds tedious but maybe it has to...

Google App Engine Java HTTP Post Image from API Method to Servlet

java,android,google-app-engine,servlets,gae-datastore
I hope someone can help me out. I want to send a url as a string to the client endpoint function and then I want the endpoint function to download the image and send it then via a HTTP Post request to my servlet (also running on GAE). The problem...

In Google App Engine for Go, how can a property have values of more than one type?

google-app-engine,go,gae-datastore
The Google App Engine for Go datastore docs say, " A property can have values of more than one type". There is no example or further explanation. (Version: appengine 1.9.19.) How can a property have more than one type if you must declare that property with a specific type in...

Objectify: Handle race condition to prevent duplicate account creation

google-app-engine,gae-datastore,objectify
I'm using Google App Engine and Datastore with objectify. I'm trying to create an account for a user (Google user), so I need to check if that users exist, and if not, create an account for that user, but I'm facing the fact that sometimes the account is created twice...