Skip to content

Add more file format support for FileStreamSource - #30

Closed
zsxwing wants to merge 1 commit into
marmbrus:streaming-dffrom
zsxwing:file-format
Closed

zsxwing wants to merge 1 commit into
marmbrus:streaming-dffrom
zsxwing:file-format

Conversation

@zsxwing

@zsxwing zsxwing commented Jan 26, 2016

Copy link
Copy Markdown

This PR adds more file format support for FileStreamSource. Because it reuses HadoopFsRelationProvider, all file formats in SQL will be supported. Also add a unit test to read from json files.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this before start to avoid missing any exception.

@zsxwing

zsxwing commented Jan 26, 2016

Copy link
Copy Markdown
Author

cc @marmbrus @tdas

@AmplabJenkins

Copy link
Copy Markdown

Can one of the admins verify this patch?

@tdas

tdas commented Jan 26, 2016

Copy link
Copy Markdown
Collaborator

jenkins ok to test.

@AmplabJenkins

Copy link
Copy Markdown

Merged build finished. Test FAILed.

@AmplabJenkins

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/spark-streaming-df-test/52/
Test FAILed.

@zsxwing

zsxwing commented Jan 26, 2016

Copy link
Copy Markdown
Author

Looks merging master (#29) was not sucessful

@zsxwing zsxwing closed this Feb 2, 2016
@zsxwing
zsxwing deleted the file-format branch November 23, 2016 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants