configStore.get(that.name) returns undefined causes uncaught exception

[31merror 2015-03-10T18:05:24.806Z: 5c00c7dd3ce2 pid:560 uncaughtExceptionHandler (requirements/uncaughtExceptionHandler.js:15) Aborting on uncaught top-level exception:  TypeError: Cannot read property 'firstChunk' of undefined
    at DestroyableTransform._transform (/opt/bb/experiments/build/node_modules/gcloud/lib/storage/file.js:866:58)
    at DestroyableTransform.Transform._read (/opt/bb/experiments/build/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:184:10)
    at DestroyableTransform.Transform._write (/opt/bb/experiments/build/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:172:12)
    at doWrite (/opt/bb/experiments/build/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:237:10)
    at writeOrBuffer (/opt/bb/experiments/build/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:227:5)
    at DestroyableTransform.Writable.write (/opt/bb/experiments/build/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:194:11)
    at write (/opt/bb/experiments/build/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:623:24)
    at flow (/opt/bb/experiments/build/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:632:7)
    at /opt/bb/experiments/build/node_modules/gcloud/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:600:7
    at process._tickCallback (node.js:419:13)�[39m
�```

One possible cause is that the startupUpload()->makeAuthorizedRequest()'s error handler is handleError(), which can call setTimeout(resumeUpload) if the err.code is between > 499 and < 600. And skip the configStore.set(that.name).

One fix is: if handleError() is triggered by the startupUpload(), it should never trigger resumeUpload().

I think I know the cause. The file.js use configure store ('gcloud-node') to keep track of the object upload process. If there are multiple node processes running at the same time, they may overwrite others' config file and cause problems. This effectively prohibit multiple node process running (under one user) to access the GCS at the same time. This is a serious limitation and is not documented anywhere.

I don't understand this design. Why do you need to use persist configure store to keep track every object's upload process? Why cannot you use a simple in-memory-only object to do that? Resumable upload is nice to have, but do you really need to think resumable-upload cross node process restart (at a such high expense)?

I make some changes like below and my stress test that uses multiple node processes is now happy.

diff --git a/node_modules/gcloud/lib/storage/file.js b/node_modules/gcloud/lib/storage/file.js
index 7c04437..adfdf35 100644
--- a/node_modules/gcloud/lib/storage/file.js
+++ b/node_modules/gcloud/lib/storage/file.js
@@ -21,7 +21,14 @@
 'use strict';

 var bufferEqual = require('buffer-equal');
-var ConfigStore = require('configstore');
+//var ConfigStore = require('configstore');
+var all = {};
+var configStore = {
+   all: {},
+   get: function(key) { return all[key]; },
+   del: function(key) { delete all[key]; },
+   set: function(key, value) { all[key] = value; },
+};
 var crc = require('fast-crc32c');
 var crypto = require('crypto');
 var duplexify = require('duplexify');
@@ -766,7 +773,7 @@ File.prototype.startResumableUpload_ = function(stream, metadata) {
   metadata = metadata || {};

   var that = this;
-  var configStore = new ConfigStore('gcloud-node');
+  //var configStore = new ConfigStore('gcloud-node');
   var config = configStore.get(that.name);
   var makeAuthorizedRequest = that.bucket.storage.makeAuthorizedRequest_;

I suppose the real important question here is: Do we really need to support resumable across node process restarts? I'm going to assume yes, so we just likely need better error checking to ensure that the value exists at the time we need to use it, otherwise resumable will not be used.