Issue with sse4_crc32

A user reported that the gcloud library was problematic to install over AWS as it depends upon a natively compiled library https://www.npmjs.com/package/sse4_crc32

The dependency tree is:
https://www.npmjs.com/package/gcloud -> https://www.npmjs.com/package/hash-stream-validation -> https://www.npmjs.com/package/fast-crc32c -> https://www.npmjs.com/package/sse4_crc32

Please see:
parse-community/parse-server#1050

Any known workaround, besides the obvious optional dependency.

@anandsuresh

That dependency has caused some trouble. In general, native dependencies for some users will be problematic. We use fast-crc32c because it marks sse4_crc32 as an optional dependency, falling back to a software implementation if the install fails.

We've redirected most issues with the installation process to @anandsuresh, author of sse4 _crc32, who has been greatly helpful. If anyone has such issues that can provide logs and details about the environment, they can open an issue on his repo: https://github.com/Voxer/sse4_crc32/issues/new

@flovilmart

@flovilmart Would love to help out any way I can. Can you please post some logs to debug the issue? Also, it seems there are multiple github issues being reported on multiple repositories. Can you please post the link to the open/active thread for this issue?

@anandsuresh

@anandsuresh Thanks! (for them)
It seems to be mostly related to deployments on CentOS 6 over AWS elastic beanstalk,

parse-community/parse-server#1082

parse-community/parse-server#1050

As you can see, I had to shut down the convo on the last one as it was getting out of control.

I've also opened parse-community/parse-server#1331 so we can gather as much evidence as possible

@flovilmart

@flovilmart No shit! Some people bear the weight of the world on their shoulders, it seems.

Checking those 2 threads, neither one seems to have any logs for the npm install process. It would help to get some logs to debug the issue.

@anandsuresh

@anandsuresh surreal right? had to report the guy, that's just unacceptable. Anyhow, I opened another thread, hope that will stay clean, and if we have more reports, I'll ping you.
Thanks for your time! really appreciated.

@flovilmart

No worries @flovilmart. Can you also add me as a follower on the new thread? Also, feel free to open an issue directly at https://github.com/Voxer/sse4_crc32/issues. Being a native dep, this has had some issues on different platforms from time-to-time. I'd love to ensure that this library works on all platforms/systems.

Just quoted you on the thread. If you need more logs just ping me :)

@flovilmart

@flovilmart I checked the gist and didn't see any logs that directly mentioned the SSE4_CRC32 build process. Just a line that says that it failed.

Is there any way we can get more detailed logs that would expose the errors from the build/install process for SSE4_CRC32? Alternately, can you please try running npm install sse4_crc32 directly and post the logs?

https://gist.github.com/flovilmart/39c9720cc747b63ed2cfd20f3ff25056

wait a sec, that's maybe not you, maybe it's node-gyp that's not found on EB

~~Dummy me.... http://stackoverflow.com/questions/34084814/npm-error-when-deploying-nodejs-to-elastic-beanstalk~~

doesn't fix either...

@flovilmart

@flovilmart Looking at the logs, it seems that node-gyp could be the missing dependency, but I can't be sure. The ENOENT errno returned by the spawn call seem to be the key log line to focus on.

Yup... So that don't seem to be on you, I noticed node-gyp and crc32 are as devDependencies, on your repo, that may imply that with NPM_CONFIG_PRODUCTION=true and running npm rebuild we'd miss the dependencies?

crc32 is used only for benchmarks. So I wouldn't worry about that. However, node-gyp needs to be present for the compilation to work, along with a C++ compiler.

I tested the process locally on my mac with NPM_CONFIG_PRODUCTION=true with no node-gyp installed, and the install process worked just fine. But then this could be indicative of some difference between our environments as well.

What version of node/npm are you using for tests?

We're targeting node 4.3, and the corresponding npm version. The more it goes the more it seems to the Elastic Beanstalk toolchain, specifically with AMi / Centos 6. We don have other issues on heroku, nor Google App Engine and friends