Skip to content

Search API Not Returning Fields #442

Description

@blowmage

We have a couple blocking issues due to the Cloud Search API not behaving according to the documentation.

v1.projects.indexes.documents.list view Parameter

The view parameter is used to control whether the document fields are returned. This is not working. Only the docId and rank values are in the returned data. The fields are returned on v1.projects.indexes.documents.get calls, however. We need the fields to be returned on this call.

v1.projects.indexes.list view Parameter

The view parameter here is used to return all the fields that are indexed on the index, but they are not returned. There is no other way to get this information. Users are unable to get this data as-is.

v1.projects.indexes.search returnFields Parameter

The returnFields parameter on document searches also is not behaving. No matter what values are sent the only data returned is docId and nextPageToken. This is a major blocker.

Activity

  1. jgeewax commented on Nov 30, 2015

    @jgeewax

    /cc @chrisnb to help us sort these out.

  2. yyr2015 commented on Nov 30, 2015

    @yyr2015

    v1.projects.indexes.documents.list view works for me. For example, when view is set to FULL, it returns documents with fields:

    {
      "nextPageToken": "CgRkb2Mz", 
      "documents": [
        {
          "fields": {
            "field2": {
              "values": [
                {
                  "stringValue": "<html>helloworld</html>", 
                  "stringFormat": "HTML"
                }
              ]
            }, 
            "field1": {
              "values": [
                {
                  "stringValue": "helloworld", 
                  "stringFormat": "TEXT"
                }, 
                {
                  "timestampValue": "2014-08-18T21:19:55.000Z"
                }, 
                {
                  "numberValue": 10
                }, 
                {
                  "geoValue": "40.6894, -74.0447"
                }
              ]
            }
          }, 
          "docId": "doc1", 
          "rank": 140052271
        }, 

    Similarly, for v1.projects.indexes.list to return indexed fields, please set view=FULL.

    returnFields in v1.projcts.indexes.search works for me too. Setting "returnFields=*" returns all fields in each document. Setting "returnFields=field1" returns only field1 in each document, and setting "returnFields=field1&returnFields=field2" returns field1 and field2 in each document.

  3. blowmage commented on Nov 30, 2015

    @blowmage
    ContributorAuthor

    @yyr2015 Thanks! I think I have identified the issue. I haven't been able to fully resolve this yet. Can you verify some additional behavior? You said:

    returnFields in v1.projcts.indexes.search works for me too. Setting "returnFields=*" returns all fields in each document.

    Yep. I'm seeing that now.

    Setting "returnFields=field1" returns only field1 in each document,

    I see this as well.

    and setting "returnFields=field1&returnFields=field2" returns field1 and field2 in each document.

    I'm seeing only field2 returned. If we list more than two, we only see the last one in the query string returned. So "returnFields=username&returnFields=email&returnFields=bio" returns only "bio". Can you confirm?

  4. yyr2015 commented on Nov 30, 2015

    @yyr2015

    Can you please give more detail about how to reproduce your issue, including the full search request?

  5. blowmage commented on Nov 30, 2015

    @blowmage
    ContributorAuthor

    Sure. The following are the requests made to the Search API and their responses. The requests are all authenticated with signet.

    https://cloudsearch.googleapis.com/v1/projects/nodal-almanac-725/indexes/test/search?query=ruby

    {
      "results": [
        {
          "docId": "gcloud-ruby",
          "nextPageToken": "..."
        }
      ],
      "matchedCount": "1"
    }

    https://cloudsearch.googleapis.com/v1/projects/nodal-almanac-725/indexes/test/search?query=ruby&returnFields=%2A

    {
      "results": [
        {
          "docId": "gcloud-ruby",
          "fields": {
            "summary": {
              "values": [
                {
                  "stringValue": "Idiomatic Ruby client for Google Cloud Platform services.",
                  "stringFormat": "TEXT"
                }
              ]
            },
            "title": {
              "values": [
                {
                  "stringValue": "gcloud-ruby",
                  "stringFormat": "TEXT"
                }
              ]
            },
            "description": {
              "values": [
                {
                  "stringValue": "Gcloud is the official library for interacting with Google Cloud.",
                  "stringFormat": "TEXT"
                }
              ]
            }
          },
          "nextPageToken": "..."
        }
      ],
      "matchedCount": "1"
    }

    https://cloudsearch.googleapis.com/v1/projects/nodal-almanac-725/indexes/test/search?query=ruby&returnFields=title

    {
      "results": [
        {
          "docId": "gcloud-ruby",
          "fields": {
            "title": {
              "values": [
                {
                  "stringValue": "gcloud-ruby",
                  "stringFormat": "TEXT"
                }
              ]
            }
          },
          "nextPageToken": "..."
        }
      ],
      "matchedCount": "1"
    }

    https://cloudsearch.googleapis.com/v1/projects/nodal-almanac-725/indexes/test/search?query=ruby&returnFields=title&returnFields=description

    {
      "results": [
        {
          "docId": "gcloud-ruby",
          "fields": {
            "description": {
              "values": [
                {
                  "stringValue": "Gcloud is the official library for interacting with Google Cloud.",
                  "stringFormat": "TEXT"
                }
              ]
            }
          },
          "nextPageToken": "..."
        }
      ],
      "matchedCount": "1"
    }

    https://cloudsearch.googleapis.com/v1/projects/nodal-almanac-725/indexes/test/search?query=ruby&returnFields=title&returnFields=description&returnFields=summary

    {
      "results": [
        {
          "docId": "gcloud-ruby",
          "fields": {
            "summary": {
              "values": [
                {
                  "stringValue": "Idiomatic Ruby client for Google Cloud Platform services.",
                  "stringFormat": "TEXT"
                }
              ]
            }
          },
          "nextPageToken": "..."
        }
      ],
      "matchedCount": "1"
    }
  6. jgeewax commented on Dec 1, 2015

    @jgeewax

    Just out of curiosity, what happens when we do

    ?returnFields=title,description (comma separated)
    or
    ?returnFields[]=title&returnFields[]=description (array notation)

    ?

  7. blowmage commented on Dec 1, 2015

    @blowmage
    ContributorAuthor

    Both of those options raise an invalid argument error. The comma separated values doesn't have much info, but the jQuery array notation messaging is more explicit about it not being supported.

    https://cloudsearch.googleapis.com/v1/projects/nodal-almanac-725/indexes/test/search?query=ruby&returnFields=title%2Cdescription%2Csummary

    {
      "error": {
        "code": 400,
        "message": "Request contains an invalid argument.",
        "status": "INVALID_ARGUMENT"
      }
    }

    https://cloudsearch.googleapis.com/v1/projects/nodal-almanac-725/indexes/test/search?query=ruby&returnFields%5B%5D=title&returnFields%5B%5D=description&returnFields%5B%5D=summary

    {
      "error": {
        "code": 400,
        "message": "Invalid JSON payload received. Unknown name \"returnFields[]\": Cannot bind query parameter. Field 'returnFields[]' could not be found in request message.",
        "status": "INVALID_ARGUMENT",
        "details": [
          {
            "@type": "type.googleapis.com/google.rpc.BadRequest",
            "fieldViolations": [
              {
                "description": "Invalid JSON payload received. Unknown name \"returnFields[]\": Cannot bind query parameter. Field 'returnFields[]' could not be found in request message."
              }
            ]
          }
        ]
      }
    }
  8. blowmage commented on Dec 2, 2015

    @blowmage
    ContributorAuthor

    I am closing this issue since it has been solved. The multiple returnFields values will be moved to a separate issue. Thanks all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

🚨This issue needs some love.status: blockedResolving the issue is dependent on other work.triage meI really want to be triaged.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions