Skip to content

Full Text Search Support - #3904

Merged
flovilmart merged 14 commits into
parse-community:masterfrom
dplewis:full-text
Jun 14, 2017
Merged

flovilmart merged 14 commits into
parse-community:masterfrom
dplewis:full-text

Conversation

@dplewis

@dplewis dplewis commented Jun 3, 2017 •

Copy link
Copy Markdown
Member

PR for #3747

Adds Full Text Support for mongo and postgres

@dplewis dplewis changed the title Full Text Support for #3747 Full Text Support Jun 3, 2017
@dplewis dplewis changed the title Full Text Support Full Text Search Support Jun 3, 2017
@seriph

seriph commented Jun 3, 2017

Copy link
Copy Markdown

Does this support the textScore from MongoDB? I don't see a test for it or any of the code, though everything else is pretty similar to my PR

@dplewis

dplewis commented Jun 3, 2017 •

Copy link
Copy Markdown
Member Author

Working on a test case now. Unless you want to do it.

@flovilmart

Copy link
Copy Markdown
Contributor

Yeah try to coordinate into a single PR if possible @dplewis, you seem well versed into Postgres, why don't you add support to it on @seriph's PR?

@codecov

codecov Bot commented Jun 3, 2017 •

Copy link
Copy Markdown

Codecov Report

Merging #3904 into master will increase coverage by 0.02%.
The diff coverage is 96.29%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3904      +/-   ##
==========================================
+ Coverage   90.26%   90.28%   +0.02%     
==========================================
  Files         114      114              
  Lines        7588     7641      +53     
==========================================
+ Hits         6849     6899      +50     
- Misses        739      742       +3
Impacted Files Coverage Δ
src/Adapters/Storage/Mongo/MongoStorageAdapter.js 92.98% <100%> (+0.08%) ⬆️
src/RestQuery.js 94.94% <100%> (+0.02%) ⬆️
src/Adapters/Storage/Mongo/MongoCollection.js 95.12% <100%> (+0.38%) ⬆️
src/Controllers/DatabaseController.js 94.35% <100%> (+0.01%) ⬆️
src/Adapters/Storage/Mongo/MongoTransform.js 83.5% <95.23%> (+0.51%) ⬆️
...dapters/Storage/Postgres/PostgresStorageAdapter.js 95.7% <95.83%> (ø) ⬆️
src/RestWrite.js 93.12% <0%> (-0.2%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f991e9...c54865f. Read the comment docs.

@dplewis

dplewis commented Jun 3, 2017 •

Copy link
Copy Markdown
Member Author

@flovilmart I would do that but I don't like how @seriph copy and pasted my PR an hour after I posted it.

He had over a month to do a PR.

Also I announced that I was going to do a PR so something like this wouldn't happen.

@flovilmart

Copy link
Copy Markdown
Contributor

@seriph's code is way old, and that's open source, let's work together to be more efficient, it's not a time for disputes guys.

@flovilmart

Copy link
Copy Markdown
Contributor

You gotta treat people with respect man, the way you're acting and telling @seriph down is not acceptable, not matter how many commits you have in the project. @seriph had a fork with the feature in months ago, and I expect he's now pissed and not likely to contribute to the project again. Read our CODE_OF_CONDUCT. I can not tolerate your behaviour with other contributors.

@dplewis

dplewis commented Jun 3, 2017 •

Copy link
Copy Markdown
Member Author

@flovilmart I understand it won't happen again. @seriph I'll help you get this PR in

@flovilmart

Copy link
Copy Markdown
Contributor

Also, on the code side, $text should not be a special key, but the REST format should be:

{
  "myKey": {
     "$text": {
        "$search": ...
      }
  }
}

@seriph

seriph commented Jun 3, 2017

Copy link
Copy Markdown

Haha, to tell you the truth @dplewis I suspected you did the same when I saw you'd posted in the thread where someone had asked me about my fork, then your PR shows up with almost identical code. The textScore code has been live in production on an ecommerce site for months, so surprised you're not thinking it works. Thanks for the professionalism @flovilmart, happy to contribute in the future. I have matching code for the JS API if it helps.

@flovilmart

flovilmart commented Jun 3, 2017 •

Copy link
Copy Markdown
Contributor

So we're all good then, if you guys can coordinate, and work together to get the best of that PR, I'll happily create a new full-text-search branch, where you can both merge your PR's before we merge the full-text-search in master. What do you think?

@flovilmart
flovilmart changed the base branch from master to full-text-search June 3, 2017 16:51
@dplewis

dplewis commented Jun 3, 2017 •

Copy link
Copy Markdown
Member Author

@flovilmart since you can only have 1 Text Index "myKey" doesn't matter.

@seriph I just added a commit for textScore and a test for it. Look at it and you'll see why I thought your textScore didn't work. I didn't see a keyword 'textScore' in any of the docs.

@flovilmart thanks for creating that branch.

@dplewis

dplewis commented Jun 3, 2017

Copy link
Copy Markdown
Member Author

@flovilmart do we commit to that branch and move the conversation over to @seriph PR.

@seriph

seriph commented Jun 3, 2017 via email

Copy link
Copy Markdown

@flovilmart

Copy link
Copy Markdown
Contributor

Is that true as well for Postgres? (The single index?)

@dplewis

dplewis commented Jun 3, 2017 •

Copy link
Copy Markdown
Member Author

Postgres doesn't need an index but can only have 1 index

@dplewis

dplewis commented Jun 3, 2017

Copy link
Copy Markdown
Member Author

Mongo throws an error if index doesn't exist. I'm writing a test case for it now.

@flovilmart

Copy link
Copy Markdown
Contributor

And on Postgres, there's no need either to specify which key of the table will be looked up for search?

@dplewis

dplewis commented Jun 3, 2017 •

Copy link
Copy Markdown
Member Author

I'm still doing research on it. But we'll know soon enough.

From the looks of it were going to use SELECT [mykey] FROM [className] or SELECT to_tsvector()

@dplewis

dplewis commented Jun 3, 2017

Copy link
Copy Markdown
Member Author

I still have 4 test cases to write then we can merge.

Index Exists
Multiple Index Error
Space Delimiter
Phrases

@SebC99

SebC99 commented Jun 3, 2017

Copy link
Copy Markdown
Contributor

@dplewis are you sure you need to to change the find function of the mongoStorageAdapter? @seriph's version with an update of mongoCollection seems simpler, but I'm not good enough in JS to judge :)
I just feel adding those 16 lines in mongoStorageAdapter is less clean than the rest of the code :)
My 2 cents

@dplewis

dplewis commented Jun 3, 2017 •

Copy link
Copy Markdown
Member Author

@SebC99 For some reason I couldn't get Mongocollection.find(query...) to handle 2 projections mongoWhere, {{score...}} Which I found to be neccessary after research.

@seriph can you run the test cases against your code? You may have to change the REST format.

@dplewis

dplewis commented Jun 9, 2017 •

Copy link
Copy Markdown
Member Author

@flovilmart I've added support for Postgres. Users have to keep in mind that this will have unexpected consequences if their indexes aren't set up properly

Also is travis not running because of the parse-community:full-text-search branch?

@flovilmart

Copy link
Copy Markdown
Contributor

closing to force travis to pick up the build

@flovilmart flovilmart closed this Jun 13, 2017
@flovilmart flovilmart reopened this Jun 13, 2017
@flovilmart
flovilmart changed the base branch from full-text-search to master June 13, 2017 14:44
Comment thread spec/ParseQuery.spec.js Outdated
}, done.fail);
});

it_exclude_dbs(['postgres'])('fullTextSearch: $caseSensitive', (done) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why still excluding PG?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or can you group together the tests related to mongo (excluding PG) in a sub describe?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are 3 excludes here. I've added comments in the postgres adapter for $caseSensitive and $diraticSensitive.

No Index... You don't need index to do PG
Case Sensitive... Full text doesn't support case sensitive on PG. You could use regex or make a second column for lower case string.
Diratic Sensitive... Can be automated on a per table / class basis based on setup and index on PG.

Should I replace the comments with error instructions in postgresAdapter or leave it be?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably you should Error the query and point to instructions / have a clear way to resolution

@flovilmart

Copy link
Copy Markdown
Contributor

It's looking super great IMHO, @seriph what do you think?

@dplewis

dplewis commented Jun 13, 2017

Copy link
Copy Markdown
Member Author

I separated the tests and added the error messages

@flovilmart

Copy link
Copy Markdown
Contributor

Sorry to ask for something more, could we put all the tests in a new file? The ParseQuery.spec.js is already huge :) perhaps ParseQuery.FullTextSearch.spec.js?

@flovilmart flovilmart left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really amazing!!!! Thanks!
Would you be keen to take care of the docs as well?

@dplewis

dplewis commented Jun 13, 2017 •

Copy link
Copy Markdown
Member Author

@flovilmart Sure no problem. Glad I could help. How should we format this?
query.fullTextSearch(key, options)?

@seriph can you work on updating the javascript sdk?

@flovilmart

Copy link
Copy Markdown
Contributor

For now let's start with the REST docs, we'll see the SDK's after right?

@flovilmart
flovilmart merged commit 8b21d5a into parse-community:master Jun 14, 2017
@dplewis

dplewis commented Jun 14, 2017

Copy link
Copy Markdown
Member Author

Right

@srameshr

srameshr commented Jan 8, 2018

Copy link
Copy Markdown
Contributor

Any chance that an API for this will come into the JS SDK?

@NicksonYap

Copy link
Copy Markdown

https://docs.parseplatform.org/js/guide/#full-text-search

It seems there is a query.fullText()

but returns empty array

@flovilmart

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants