This repository was archived by the owner on Mar 19, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Mar 19, 2026. It is now read-only.
3.5.8 causes an OOM when used with @google-cloud/datastore #248
Description
- Is this a client library issue or a product issue?
This is an issue that occurs with a specific version (3.5.8) of google-gax.
- Did someone already solve this?
no
- Do you have a support contract?
no
Environment details
> uname -a
Linux f86fa67c0fcb 5.15.0-1030-aws googleapis/gax-nodejs#34~20.04.1-Ubuntu SMP Tue Jan 24 15:16:46 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
> node --version
v18.6.0
> npm --version
8.13.2
> yarn --version
3.3.1
google-gax: 3.5.8
Steps to reproduce
> cat test.js
var ds = require('@google-cloud/datastore');
var ds2 = new ds.Datastore({ namespace: 'record-properties-test' });
var root = ds2.key(['RecordProperty','Root']);
var q = ds2.createQuery('RecordProperty').hasAncestor(root);
ds2.runQuery(q).then(console.log).catch(console.error);
> node --max-old-space-size=512 test.js
<--- Last few GCs --->
[6481:0x6e52790] 7136 ms: Scavenge (reduce) 508.9 (520.8) -> 507.5 (518.0) MB, 8.0 / 0.0 ms (average mu = 0.282, current mu = 0.256) allocation failure;
[6481:0x6e52790] 7195 ms: Scavenge (reduce) 508.2 (518.0) -> 507.6 (518.5) MB, 4.8 / 0.0 ms (average mu = 0.282, current mu = 0.256) allocation failure;
[6481:0x6e52790] 7260 ms: Scavenge (reduce) 508.6 (518.5) -> 507.9 (518.5) MB, 1.6 / 0.0 ms (average mu = 0.282, current mu = 0.256) allocation failure;
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0xb65a10 node::Abort() [node]
2: 0xa761b5 node::FatalError(char const*, char const*) [node]
3: 0xd5713e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
4: 0xd574b7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
5: 0xf34545 [node]
6: 0xf35448 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
7: 0xf45953 [node]
8: 0xf467c8 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
9: 0xf49995 v8::internal::Heap::HandleGCRequest() [node]
10: 0xec7aef v8::internal::StackGuard::HandleInterrupts() [node]
11: 0x12c64c5 v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [node]
12: 0x16f87f9 [node]
Aborted (core dumped)
when i use yarn's resolutions to force google-gax 3.5.2 it works fine:
> node --max-old-space-size=512 test.js
[ [], { moreResults: 'MORE_RESULTS_AFTER_LIMIT', endCursor: 'CgA=' } ]
Making sure to follow these steps will guarantee the quickest resolution possible.
Thanks!
Activity
alexander-fenster commentedon Mar 30, 2023
Hi @scottbessler,
I cannot easily reproduce it (neither with an empty list of properties nor with non-empty list), so I will need to ask you to help us figure out what's going on.
Would you be able to try versions of
google-gaxbetween 3.5.2 and 3.5.8 and see which exact version starts failing? I believe there were no changes ingoogle-gaxthat could change anything related to memory allocation, but there were two updates of dependencies (@grpc/grpc-jsandprotobufjs) and it would be very useful to see if we need to dig into the changes in either of them.weyert commentedon Apr 11, 2023
We have been having a memory leak to in one of the other google packages, probably, the
@google-cloud/pubsubpackage and after downgrading to 3.5.7 the memory disappeared. Before we were using 3.6.0alexander-fenster commentedon Apr 11, 2023
@weyert Just to confirm - no memory leak on 3.5.7 but leaking on 3.5.8, with no other changes? The only difference between them is related to the HTTP transport https://github.com/googleapis/gax-nodejs/pull/1431/files which is very unlikely to be used by
@google-cloud/pubsub.Could you share more details, preferably a repro, in a new issue since it seems not directly related to what @scottbessler describes here?
weyert commentedon Apr 11, 2023
@alexander-fenster I haven't tested 3.5.8. I will see what I can do to test the versions between 3.5.7 and 3.6.0. I can only say that there was an issue in 3.6.0 and force a downgrade to 3.5.7 resolved the issue by adding
[email protected]as main dependency of the project.I will report back as soon as possible. Now that a solution has been found for the memory leak I need to work through my backlog of tasks before I can do the above, though. I hope to be able to get back end of the week or early next week.
leahecole commentedon May 21, 2024
going to close as obsolete given that nodejs-datastore now depends on 4.0.5 or greater, but please reopen if this is an issue still.
chore: remove duplicate mocha config (#248)