Skip to content

Parse.Query.and Problem with nested "equalTo relational fields" condition #7579

Description

@Kingtous

New Issue Checklist

Issue Description

I'm doing a data collection and publication application. It has two roles, data uploader, and reader. When readers retrieve data, the server need to remove data that is not reviewed by the administrator and only show data which had been reviewed. But for data uploader, he/she can see data uploaded by him/herself although the data has not been reviewed by the administrator. So we defined beforeFind trigger in a class named Topic(data class), it has a nested "equalTo relational fields" in Parse.Query.and (Parse.Query.or is ok). But the server sdk will treat nested "equalTo relational fields" as "equalTo pointer fields", other than relation.

Steps to reproduce

  1. define a class 'A', class 'B'
  2. add some test data
  3. set a relational field 'rel' in A, which is the relation between 'A' and 'B'
  4. set an arbitrary field(String, Boolean, etc) named 'C'
  5. define a beforeFind trigger in 'A'
...
const q1 = request.query; // class A
q1.equalTo('rel',<Parse.Object for 'B'>);
const q2 = new Parse.Query('A');
q2.equalTo('C',true);
const finalReturnQuery = Parse.Query.and(q1,q2);
return finalReturnQuery;

Actual Outcome

zero results found.
explain:

{
    "results":{
    "queryPlanner": {
    "parsedQuery":{
    "$and":[
   {
    "rel" : {
    { "$eq" : "B$<id>"}
...
}}]}}}}

the 'rel' not be parsed to containedIn but be treated as a pointer.

Expected Outcome

multiple results found.
expected explain:

{
    "results":{
    "queryPlanner": {
    "parsedQuery":{
    "$and":[
   {
    "rel" : {
    "_id" : { "$in" : [<A objectId 1>,<A objectId 2>]}
...
}}]}}}}

Environment

^4.10.3

Server

  • Parse Server version: 4.10.3
  • Operating system: macOS Catalina 10.15.7
  • Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): local

Database

  • System (MongoDB or Postgres): MongoDB
  • Database version: 4.4.6
  • Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): local

Client

  • SDK (iOS, Android, JavaScript, PHP, Unity, etc): Flutter
  • SDK version: ^3.0.0

test using Postman and restful-api for parse server too.

Logs

Topic is the target class, and 'cls' is a relational field to the class named 'Class'.

verbose: REQUEST for [GET] /parse/classes/Topic: {} 
verbose: RESPONSE from [GET] /parse/classes/Topic: {
  "response": {
    "results": {
      "queryPlanner": {
        "plannerVersion": 1,
        "namespace": "parse.Topic",
        "indexFilterSet": false,
        "parsedQuery": {
          "$and": [
            {
              "cls": {
                "$eq": "Class$5E1lG8FaQo"
              }
            },
            {
              "_rperm": {
                "$in": [
                  null,
                  "*",
                  "role:schoolAdmin",
                  "role:teacher",
                  "role:topAdmin",
                  "sLBwuv5J6B"
                ]
              }
            }
          ]
        },
        "winningPlan": {
          "stage": "LIMIT",
          "limitAmount": 100,
          "inputStage": {
            "stage": "COLLSCAN",
            "filter": {
              "$and": [
                {
                  "cls": {
                    "$eq": "Class$5E1lG8FaQo"
                  }
                },
                {
                  "_rperm": {
                    "$in": [
                      null,
                      "*",
                      "role:schoolAdmin",
                      "role:teacher",
                      "role:topAdmin",
                      "sLBwuv5J6B"
                    ]
                  }
                }
              ]
            },
            "direction": "forward"
          }
        },
        "rejectedPlans": []
      },
      "executionStats": {
        "executionSuccess": true,
        "nReturned": 0,
        "executionTimeMillis": 0,
        "totalKeysExamined": 0,
        "totalDocsExamined": 22,
        "executionStages": {
          "stage": "LIMIT",
          "nReturned": 0,
          "executionTimeMillisEstimate": 0,
          "works": 24,
          "advanced": 0,
          "needTime": 23,
          "needYield": 0,
          "saveState": 0,
          "restoreState": 0,
          "isEOF": 1,
          "limitAmount": 100,
          "inputStage": {
            "stage": "COLLSCAN",
            "filter": {
              "$and": [
                {
                  "cls": {
                    "$eq": "Class$5E1lG8FaQo"
                  }
                },
                {
                  "_rperm": {
                    "$in": [
                      null,
                      "*",
                      "role:schoolAdmin",
                      "role:teacher",
                      "role:topAdmin",
                      "sLBwuv5J6B"
                    ]
                  }
                }
              ]
            },
            "nReturned": 0,
            "executionTimeMillisEstimate": 0,
            "works": 24,
            "advanced": 0,
            "needTime": 23,
            "needYield": 0,
            "saveState": 0,
            "restoreState": 0,
            "isEOF": 1,
            "direction": "forward",
            "docsExamined": 22
          }
        },
        "allPlansExecution": []
      },
      "serverInfo": {
        "host": "kingtousdeMBP.lan",
        "port": 27017,
        "version": "4.4.6",
        "gitVersion": "72e66213c2c3eab37d9358d5e78ad7f5c1d0d0d7"
      },
      "ok": 1
    }
  }
} {"result":{"response":{"results":{"queryPlanner":{"plannerVersion":1,"namespace":"parse.Topic","indexFilterSet":false,"parsedQuery":{"$and":[{"cls":{"$eq":"Class$5E1lG8FaQo"}},{"_rperm":{"$in":[null,"*","role:schoolAdmin","role:teacher","role:topAdmin","sLBwuv5J6B"]}}]},"winningPlan":{"stage":"LIMIT","limitAmount":100,"inputStage":{"stage":"COLLSCAN","filter":{"$and":[{"cls":{"$eq":"Class$5E1lG8FaQo"}},{"_rperm":{"$in":[null,"*","role:schoolAdmin","role:teacher","role:topAdmin","sLBwuv5J6B"]}}]},"direction":"forward"}},"rejectedPlans":[]},"executionStats":{"executionSuccess":true,"nReturned":0,"executionTimeMillis":0,"totalKeysExamined":0,"totalDocsExamined":22,"executionStages":{"stage":"LIMIT","nReturned":0,"executionTimeMillisEstimate":0,"works":24,"advanced":0,"needTime":23,"needYield":0,"saveState":0,"restoreState":0,"isEOF":1,"limitAmount":100,"inputStage":{"stage":"COLLSCAN","filter":{"$and":[{"cls":{"$eq":"Class$5E1lG8FaQo"}},{"_rperm":{"$in":[null,"*","role:schoolAdmin","role:teacher","role:topAdmin","sLBwuv5J6B"]}}]},"nReturned":0,"executionTimeMillisEstimate":0,"works":24,"advanced":0,"needTime":23,"needYield":0,"saveState":0,"restoreState":0,"isEOF":1,"direction":"forward","docsExamined":22}},"allPlansExecution":[]},"serverInfo":{"host":"kingtousdeMBP.lan","port":27017,"version":"4.4.6","gitVersion":"72e66213c2c3eab37d9358d5e78ad7f5c1d0d0d7"},"ok":1}}}}

Activity

  1. parse-github-assistant commented on Sep 18, 2021

    @parse-github-assistant

    Thanks for opening this issue!

    • 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.
  2. added
    type:bugImpaired feature or lacking behavior that is likely assumed
    on Sep 19, 2021
  3. mtrezza commented on Sep 19, 2021

    @mtrezza
    Member

    Can you please change the code in "Steps to reproduce" to a specific example that one can copy/paste to reproduce the issue? Instead of using placeholders in const q1 = request.query; and q1.equalTo('rel',<Parse.Object for 'B'>); please use specific code. The code should be as simple and short as possible, just enough to reproduce the issue. It would be best if you could open a PR with a failing test that uses such a simple example and shows exactly where it fails.

  4. added and removed
    type:bugImpaired feature or lacking behavior that is likely assumed
    on Sep 19, 2021
  5. Kingtous commented on Sep 24, 2021

    @Kingtous
    ContributorAuthor

    here is the pr with a new test case

  6. Kingtous commented on Sep 24, 2021

    @Kingtous
    ContributorAuthor

    I don't know which branch should be chosen to be the target branch, so I opened 2 PRs. one is 'release-4.x.x', the other is 'master' branch. I just want to prove that this issue can be reproduced on both branches. If there is no need to open PR for both of them, I will close one of them. :)

  7. mtrezza commented on Sep 24, 2021

    @mtrezza
    Member

    We usually work and complete a fix for the master branch, and after merging backport it to 4.x branch. But you did well thinking about both branches! Maybe we can close the 4.x PR for now, until we have completed the master PR.

  8. Kingtous commented on Sep 25, 2021

    @Kingtous
    ContributorAuthor

    done.

  9. added
    type:bugImpaired feature or lacking behavior that is likely assumed
    on Oct 5, 2021
  10. Kingtous commented on Oct 23, 2021

    @Kingtous
    ContributorAuthor

    I've pushed some commits to #7593, and please check it out. :)

  11. Kingtous commented on Oct 23, 2021

    @Kingtous
    ContributorAuthor

    I think It's necessary to backport this bugfix to release-4.x.x because it totally affects the process of $and identifier. Lacking this commit, Parse.Query.and will not be processed by parse server when querying with relational keys.

  12. mtrezza commented on Oct 23, 2021

    @mtrezza
    Member

    Thanks for pointing this out! Let's first merge this into the master branch, then try to backport it into 4.x.

  13. Kingtous commented on Oct 28, 2021

    @Kingtous
    ContributorAuthor

    any plans to merge this into alpha branch? #7593

  14. mtrezza commented on Oct 28, 2021

    @mtrezza
    Member

    Sure, I'll take a look at the PR, apologies for the wait.

  15. Kingtous commented on Oct 30, 2021

    @Kingtous
    ContributorAuthor

    Thanks for merging the pr into alpha, I'll try using the newest pre-release for my production. Should I close this issue now? These days I've noticed that the issue was labeled by bounty:$20, is it a bonus for resolving this issue? :)

  16. mtrezza commented on Oct 30, 2021

    @mtrezza
    Member

    Amazing, we can't have enough alpha testers, so please let us know in case you find any issues with the alpha release. There is indeed a bounty on this issue, the Parse Bounty Program explains how you can request the payout.

    This issue can be closed as we assume the merged PR has solved it. Thanks for your contribution!

  17. cbaker6 commented on Nov 22, 2021

    @cbaker6
    Contributor

    @mtrezza there's a PR that broke the parse-server and I've tracked it to this PR. 5.0.0-alpha.1 works for me, 5.0.0-alpha.2 I haven't tested yet, but it only adds a GraphQL change which I'm not using. Leaving this one in which my servers don't work anymore for 5.0.0-alpha.3. Below is is my error logs:

    parse-hipaa-parse-1      | info: afterSave triggered for _User for user undefined:
    parse-hipaa-parse-1      |   Input: {"authData":{"anonymous":{"id":"9601869f-2eef-4fff-8519-690a422310fa"}},"ACL":{"*":{"read":true,"write":false},"aAnt8JX15Q9vjix6kGdZgjRQWm5QGrPY":{"read":true,"write":true}},"createdAt":"2021-11-22T22:21:46.478Z","username":"YBBMbganzC1KGlFhhKXWFRjNn","sessionToken":"r:97d78ad4fac78246134dc96ed8c08cac","updatedAt":"2021-11-22T22:21:46.478Z","objectId":"aAnt8JX15Q9vjix6kGdZgjRQWm5QGrPY"}
    parse-hipaa-parse-1      |   Result: undefined {"className":"_User","triggerType":"afterSave"}
    parse-hipaa-parse-1      | /parse-server/lib/ParseServer.js:258
    parse-hipaa-parse-1      |           throw err;
    parse-hipaa-parse-1      |           ^
    parse-hipaa-parse-1      | 
    parse-hipaa-parse-1      | TypeError: Cannot read properties of undefined (reading 'type')
    parse-hipaa-parse-1      |     at /parse-server/lib/Adapters/Storage/Postgres/PostgresStorageAdapter.js:1344:40
    parse-hipaa-parse-1      |     at Array.forEach (<anonymous>)
    parse-hipaa-parse-1      |     at PostgresStorageAdapter.createObject (/parse-server/lib/Adapters/Storage/Postgres/PostgresStorageAdapter.js:1303:25)
    parse-hipaa-parse-1      |     at /parse-server/lib/Controllers/DatabaseController.js:782:29
    parse-hipaa-parse-1      |     at processTicksAndRejections (node:internal/process/task_queues:96:5)
    parse-hipaa-parse-1 exited with code 7

    Note, this completely crashes the server. I'm using the Swift SDK, but Query.and is implemented like the rest of the Client SDKs, so this a major breaking change. In addition, skimming the additions in this PR, I highly doubt the server wasn't handling $and before this PR. If I remove the code that uses triggers, none of my client side queries work correctly because of this PR.

    I'll open an issue soon.

  18. mtrezza commented on Nov 26, 2021

    @mtrezza
    Member

    I highly doubt the server wasn't handling $and before this PR

    The PR #7593 demoed the failing test, so the issue was apparently there before the PR. See this test: c262c77

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bounty:$20Bounty applies for fixing this issue (Parse Bounty Program)type:bugImpaired feature or lacking behavior that is likely assumed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions