Skip to content

Cypress is running flaky tests in CircleCI Test Environment  #1276

Description

  • Operating System: OSX
  • Cypress Version: 1.4.1

Is this a Feature or Bug?

not sure

Current behavior:

We are currently using Cypress to test both our customer facing website and our Admin-Panel. These tests are run each time someone pushes their branch up to staging by CIRCLECI. Currently, the tests are producing inconsistent results(passing 1 time then failing the next) in this environment even though they are all passing locally.

Desired behavior:

We would like to be able to run Cypress tests in the CIrcleCI env consistently.

How to reproduce:

We are using the following configuration for our CIRCLECI cypress test setup

  test_cardash-web:
    <<: *cardash_circleci
    steps:
      - attach_workspace:
          at: ~/

      - *setup_docker
      - *create_ssh_key
      - *set_permissions_ssh_key
      - *npm_link_cardash-cli
      - *load_db

      - run:
          name: Load cardash-api
          command: |
            # Sets the env variables in our current login shell
            `cardash setup api`

            # Open the tunnel for this step
            cardash tunnel up

            export DB_PORT="3306"
            export DB_PASS="root"
            export VCDB_PORT="3306"
            export VCDB_PASS="root"
            export NODE_ENV=development
            env | sort
            # Instantiate cardash-api
            PORT=3000 node scripts/dev.js &> /tmp/cardash-api-output.log
          working_directory: packages/cardash-api
          background: true

      - run:
          name: Boot cardash-web
          # Redirect both stdout and stderr to webserver-output.log with "&>"
          command: |
            `node ~/repo/packages/cardash-cli/cardash-setup.js api`
            node scripts/generateSitemap.js
            NODE_ENV=local ./node_modules/.bin/webpack-dev-server --env.dev --hot &> /tmp/cardash-web-output.log
          background: true
          working_directory: packages/cardash-web

      - run:
          name: Run Cypress tests
          command: |
            Wait 30s for cardash-api to finish loading
            sleep 30
            ./node_modules/.bin/cypress run --headed
             exit $?
          working_directory: packages/cardash-web

      - store_artifacts:
          path: packages/cardash-web/cypress/videos

      - store_artifacts:
          path: packages/cardash-web/cypress/screenshots

      - store_artifacts:
          path: /tmp/cardash-api-output.log

      - store_artifacts:
          path: /tmp/cardash-web-output.log

We are also using the following script command to run the tests locally:

"test": "node_modules/.bin/cypress run --headed"

Please let me know if you need any additional info.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions