Skip to content

[Bug] incorrect conversion of batch insertion in Datastore #875

Closed
Closed
[Bug] incorrect conversion of batch insertion in Datastore#875
Assignees
yoshi-automation
Labels
api: datastoreIssues related to the Datastore API.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Description

@dg3feiko
dg3feiko
opened on Sep 20, 2015

Env

Node: 2.3.3
gcloud: 0.21.0

Problem

for single insert

var res;

        data = [
            {
                name: "foo",
                value: {
                    bar: [
                        {hello: 1},
                        {hello: 2}
                    ]
                }
            }
        ];

        //save to datastore
        res = yield dataset.saveAsync([
                {
                    key: dataset.key({
                        path: ['MyKind', "123"]
                    }),
                    data: data
                },

            ]
        );

        res = yield dataset.getAsync(
            dataset.key(['MyKind', "123"])
        );

        console.log(JSON.stringify(res, null, 2));

output

{
  "key": {
    "path": [
      "MyKind",
      123
    ]
  },
  "data": {
    "foo": {
      "bar": [
        {
          "hello": 1
        },
        {
          "hello": 2
        }
      ]
    }
  }
}

but for multiple inserts

var res;

        data = [
            {
                name: "foo",
                value: {
                    bar: [
                        {hello: 1},
                        {hello: 2}
                    ]
                }
            }
        ];

        //save to datastore
        res = yield dataset.saveAsync([
                {
                    key: dataset.key({
                        path: ['MyKind', "123"]
                    }),
                    data: data
                },
                {
                    key: dataset.key({
                        path: ['MyKind', "234"]
                    }),
                    data: data
                }
            ]
        );

        res = yield dataset.getAsync(
            dataset.key(['MyKind', "123"])
        );

        console.log(JSON.stringify(res, null, 2));

the output is

{
  "key": {
    "path": [
      "MyKind",
      123
    ]
  },
  "data": {
    "foo": {
      "entity_value": {
        "property": [
          {
            "name": "bar",
            "value": {
              "list_value": [
                {
                  "entity_value": {
                    "property": [
                      {
                        "name": "hello",
                        "value": {
                          "integer_value": 1
                        }
                      }
                    ]
                  },
                  "indexed": false
                },
                {
                  "entity_value": {
                    "property": [
                      {
                        "name": "hello",
                        "value": {
                          "integer_value": 2
                        }
                      }
                    ]
                  },
                  "indexed": false
                }
              ]
            }
          }
        ]
      },
      "indexed": false
    }
  }
}

Activity

  1. dg3feiko commented on Sep 20, 2015

    @dg3feiko
    dg3feiko
    on Sep 20, 2015
    Author

    for single insert
    the raw request body is

    {
    "mutation": {
        "insert": [],
        "update": [],
        "upsert": [
            {
                "property": [
                    {
                        "name": "foo",
                        "value": {
                            "entity_value": {
                                "property": [
                                    {
                                        "name": "bar",
                                        "value": {
                                            "list_value": [
                                                {
                                                    "entity_value": {
                                                        "property": [
                                                            {
                                                                "name": "hello",
                                                                "value": {
                                                                    "integer_value": 1
                                                                }
                                                            }
                                                        ]
                                                    },
                                                    "indexed": false
                                                },
                                                {
                                                    "entity_value": {
                                                        "property": [
                                                            {
                                                                "name": "hello",
                                                                "value": {
                                                                    "integer_value": 2
                                                                }
                                                            }
                                                        ]
                                                    },
                                                    "indexed": false
                                                }
                                            ]
                                        }
                                    }
                                ]
                            },
                            "indexed": false
                        }
                    }
                ],
                "key": {
                    "path_element": [
                        {
                            "kind": "MyKind",
                            "id": "123"
                        }
                    ]
                }
            }
        ],
        "insert_auto_id": []
    }
}

    for multiple one
    the body is

    {
    "mutation": {
        "insert": [],
        "update": [],
        "upsert": [
            {
                "property": [
                    {
                        "name": "foo",
                        "value": {
                            "entity_value": {
                                "property": [
                                    {
                                        "name": "entity_value",
                                        "value": {
                                            "entity_value": {
                                                "property": [
                                                    {
                                                        "name": "property",
                                                        "value": {
                                                            "list_value": [
                                                                {
                                                                    "entity_value": {
                                                                        "property": [
                                                                            {
                                                                                "name": "name",
                                                                                "value": {
                                                                                    "string_value": "bar"
                                                                                }
                                                                            },
                                                                            {
                                                                                "name": "value",
                                                                                "value": {
                                                                                    "entity_value": {
                                                                                        "property": [
                                                                                            {
                                                                                                "name": "list_value",
                                                                                                "value": {
                                                                                                    "list_value": [
                                                                                                        {
                                                                                                            "entity_value": {
                                                                                                                "property": [
                                                                                                                    {
                                                                                                                        "name": "entity_value",
                                                                                                                        "value": {
                                                                                                                            "entity_value": {
                                                                                                                                "property": [
                                                                                                                                    {
                                                                                                                                        "name": "property",
                                                                                                                                        "value": {
                                                                                                                                            "list_value": [
                                                                                                                                                {
                                                                                                                                                    "entity_value": {
                                                                                                                                                        "property": [
                                                                                                                                                            {
                                                                                                                                                                "name": "name",
                                                                                                                                                                "value": {
                                                                                                                                                                    "string_value": "hello"
                                                                                                                                                                }
                                                                                                                                                            },
                                                                                                                                                            {
                                                                                                                                                                "name": "value",
                                                                                                                                                                "value": {
                                                                                                                                                                    "entity_value": {
                                                                                                                                                                        "property": [
                                                                                                                                                                            {
                                                                                                                                                                                "name": "integer_value",
                                                                                                                                                                                "value": {
                                                                                                                                                                                    "integer_value": 1
                                                                                                                                                                                }
                                                                                                                                                                            }
                                                                                                                                                                        ]
                                                                                                                                                                    },
                                                                                                                                                                    "indexed": false
                                                                                                                                                                }
                                                                                                                                                            }
                                                                                                                                                        ]
                                                                                                                                                    },
                                                                                                                                                    "indexed": false
                                                                                                                                                }
                                                                                                                                            ]
                                                                                                                                        }
                                                                                                                                    }
                                                                                                                                ]
                                                                                                                            },
                                                                                                                            "indexed": false
                                                                                                                        }
                                                                                                                    },
                                                                                                                    {
                                                                                                                        "name": "indexed",
                                                                                                                        "value": {
                                                                                                                            "boolean_value": false
                                                                                                                        }
                                                                                                                    }
                                                                                                                ]
                                                                                                            },
                                                                                                            "indexed": false
                                                                                                        },
                                                                                                        {
                                                                                                            "entity_value": {
                                                                                                                "property": [
                                                                                                                    {
                                                                                                                        "name": "entity_value",
                                                                                                                        "value": {
                                                                                                                            "entity_value": {
                                                                                                                                "property": [
                                                                                                                                    {
                                                                                                                                        "name": "property",
                                                                                                                                        "value": {
                                                                                                                                            "list_value": [
                                                                                                                                                {
                                                                                                                                                    "entity_value": {
                                                                                                                                                        "property": [
                                                                                                                                                            {
                                                                                                                                                                "name": "name",
                                                                                                                                                                "value": {
                                                                                                                                                                    "string_value": "hello"
                                                                                                                                                                }
                                                                                                                                                            },
                                                                                                                                                            {
                                                                                                                                                                "name": "value",
                                                                                                                                                                "value": {
                                                                                                                                                                    "entity_value": {
                                                                                                                                                                        "property": [
                                                                                                                                                                            {
                                                                                                                                                                                "name": "integer_value",
                                                                                                                                                                                "value": {
                                                                                                                                                                                    "integer_value": 2
                                                                                                                                                                                }
                                                                                                                                                                            }
                                                                                                                                                                        ]
                                                                                                                                                                    },
                                                                                                                                                                    "indexed": false
                                                                                                                                                                }
                                                                                                                                                            }
                                                                                                                                                        ]
                                                                                                                                                    },
                                                                                                                                                    "indexed": false
                                                                                                                                                }
                                                                                                                                            ]
                                                                                                                                        }
                                                                                                                                    }
                                                                                                                                ]
                                                                                                                            },
                                                                                                                            "indexed": false
                                                                                                                        }
                                                                                                                    },
                                                                                                                    {
                                                                                                                        "name": "indexed",
                                                                                                                        "value": {
                                                                                                                            "boolean_value": false
                                                                                                                        }
                                                                                                                    }
                                                                                                                ]
                                                                                                            },
                                                                                                            "indexed": false
                                                                                                        }
                                                                                                    ]
                                                                                                }
                                                                                            }
                                                                                        ]
                                                                                    },
                                                                                    "indexed": false
                                                                                }
                                                                            }
                                                                        ]
                                                                    },
                                                                    "indexed": false
                                                                }
                                                            ]
                                                        }
                                                    }
                                                ]
                                            },
                                            "indexed": false
                                        }
                                    },
                                    {
                                        "name": "indexed",
                                        "value": {
                                            "boolean_value": false
                                        }
                                    }
                                ]
                            },
                            "indexed": false
                        }
                    }
                ],
                "key": {
                    "path_element": [
                        {
                            "kind": "MyKind",
                            "id": "123"
                        }
                    ]
                }
            },
            {
                "property": [
                    {
                        "name": "foo",
                        "value": {
                            "entity_value": {
                                "property": [
                                    {
                                        "name": "entity_value",
                                        "value": {
                                            "entity_value": {
                                                "property": [
                                                    {
                                                        "name": "property",
                                                        "value": {
                                                            "list_value": [
                                                                {
                                                                    "entity_value": {
                                                                        "property": [
                                                                            {
                                                                                "name": "name",
                                                                                "value": {
                                                                                    "string_value": "bar"
                                                                                }
                                                                            },
                                                                            {
                                                                                "name": "value",
                                                                                "value": {
                                                                                    "entity_value": {
                                                                                        "property": [
                                                                                            {
                                                                                                "name": "list_value",
                                                                                                "value": {
                                                                                                    "list_value": [
                                                                                                        {
                                                                                                            "entity_value": {
                                                                                                                "property": [
                                                                                                                    {
                                                                                                                        "name": "entity_value",
                                                                                                                        "value": {
                                                                                                                            "entity_value": {
                                                                                                                                "property": [
                                                                                                                                    {
                                                                                                                                        "name": "property",
                                                                                                                                        "value": {
                                                                                                                                            "list_value": [
                                                                                                                                                {
                                                                                                                                                    "entity_value": {
                                                                                                                                                        "property": [
                                                                                                                                                            {
                                                                                                                                                                "name": "name",
                                                                                                                                                                "value": {
                                                                                                                                                                    "string_value": "hello"
                                                                                                                                                                }
                                                                                                                                                            },
                                                                                                                                                            {
                                                                                                                                                                "name": "value",
                                                                                                                                                                "value": {
                                                                                                                                                                    "entity_value": {
                                                                                                                                                                        "property": [
                                                                                                                                                                            {
                                                                                                                                                                                "name": "integer_value",
                                                                                                                                                                                "value": {
                                                                                                                                                                                    "integer_value": 1
                                                                                                                                                                                }
                                                                                                                                                                            }
                                                                                                                                                                        ]
                                                                                                                                                                    },
                                                                                                                                                                    "indexed": false
                                                                                                                                                                }
                                                                                                                                                            }
                                                                                                                                                        ]
                                                                                                                                                    },
                                                                                                                                                    "indexed": false
                                                                                                                                                }
                                                                                                                                            ]
                                                                                                                                        }
                                                                                                                                    }
                                                                                                                                ]
                                                                                                                            },
                                                                                                                            "indexed": false
                                                                                                                        }
                                                                                                                    },
                                                                                                                    {
                                                                                                                        "name": "indexed",
                                                                                                                        "value": {
                                                                                                                            "boolean_value": false
                                                                                                                        }
                                                                                                                    }
                                                                                                                ]
                                                                                                            },
                                                                                                            "indexed": false
                                                                                                        },
                                                                                                        {
                                                                                                            "entity_value": {
                                                                                                                "property": [
                                                                                                                    {
                                                                                                                        "name": "entity_value",
                                                                                                                        "value": {
                                                                                                                            "entity_value": {
                                                                                                                                "property": [
                                                                                                                                    {
                                                                                                                                        "name": "property",
                                                                                                                                        "value": {
                                                                                                                                            "list_value": [
                                                                                                                                                {
                                                                                                                                                    "entity_value": {
                                                                                                                                                        "property": [
                                                                                                                                                            {
                                                                                                                                                                "name": "name",
                                                                                                                                                                "value": {
                                                                                                                                                                    "string_value": "hello"
                                                                                                                                                                }
                                                                                                                                                            },
                                                                                                                                                            {
                                                                                                                                                                "name": "value",
                                                                                                                                                                "value": {
                                                                                                                                                                    "entity_value": {
                                                                                                                                                                        "property": [
                                                                                                                                                                            {
                                                                                                                                                                                "name": "integer_value",
                                                                                                                                                                                "value": {
                                                                                                                                                                                    "integer_value": 2
                                                                                                                                                                                }
                                                                                                                                                                            }
                                                                                                                                                                        ]
                                                                                                                                                                    },
                                                                                                                                                                    "indexed": false
                                                                                                                                                                }
                                                                                                                                                            }
                                                                                                                                                        ]
                                                                                                                                                    },
                                                                                                                                                    "indexed": false
                                                                                                                                                }
                                                                                                                                            ]
                                                                                                                                        }
                                                                                                                                    }
                                                                                                                                ]
                                                                                                                            },
                                                                                                                            "indexed": false
                                                                                                                        }
                                                                                                                    },
                                                                                                                    {
                                                                                                                        "name": "indexed",
                                                                                                                        "value": {
                                                                                                                            "boolean_value": false
                                                                                                                        }
                                                                                                                    }
                                                                                                                ]
                                                                                                            },
                                                                                                            "indexed": false
                                                                                                        }
                                                                                                    ]
                                                                                                }
                                                                                            }
                                                                                        ]
                                                                                    },
                                                                                    "indexed": false
                                                                                }
                                                                            }
                                                                        ]
                                                                    },
                                                                    "indexed": false
                                                                }
                                                            ]
                                                        }
                                                    }
                                                ]
                                            },
                                            "indexed": false
                                        }
                                    },
                                    {
                                        "name": "indexed",
                                        "value": {
                                            "boolean_value": false
                                        }
                                    }
                                ]
                            },
                            "indexed": false
                        }
                    }
                ],
                "key": {
                    "path_element": [
                        {
                            "kind": "MyKind",
                            "id": "234"
                        }
                    ]
                }
            }
        ],
        "insert_auto_id": []
    }
}
  2. stephenplusplus
    added
    api: datastoreIssues related to the Datastore API.
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    on Sep 20, 2015

  3. stephenplusplus commented on Sep 21, 2015

    @stephenplusplus
    stephenplusplus
    on Sep 21, 2015
    Contributor

    Fix incoming!

  4. callmehiphop
    closed this as completedin #881on Sep 21, 2015
  5. JustinBeckwith
    assigned
    yoshi-automation
    on Feb 1, 2021
  6. sofisl
    added a commit that references this issue on Nov 10, 2022

    chore(deps): update dependency @types/node to v18 (#875)

    827f771
  7. sofisl
    added 8 commits that reference this issue on Jan 27, 2026

    chore(deps): update dependency c8 to v8 (#875)

    e927c3c

    chore(deps): update dependency c8 to v8 (#875)

    c059ec0

    chore(deps): update dependency c8 to v8 (#875)

    2d4d423

    chore(deps): update dependency c8 to v8 (#875)

    3c86582

    chore(deps): update dependency c8 to v8 (#875)

    240ba2e

    chore(deps): update dependency c8 to v8 (#875)

    3a49e55

    chore(deps): update dependency c8 to v8 (#875)

    e61a30a

    chore(deps): update dependency c8 to v8 (#875)

    8b529e1
  8. sofisl
    added 3 commits that reference this issue on Feb 24, 2026

    refactor: use loop for transactions (#875)

    1379be1

    fix(deps)!: update dependency @google-cloud/common to v3 (#875)

    727acc4

    build: add missing docs and docs-test scripts (#875)

    9673b95
  9. GautamSharda
    added a commit that references this issue on Mar 11, 2026

    test: modernize mocha config (#875)

    378c59a
  10. feywind
    added a commit that references this issue on Mar 27, 2026

    test: modernize mocha config (#875)

    442fa65
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: datastoreIssues related to the Datastore API.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions