Skip to content

SPARK-1699: Python relative independent, becomes a subproject - #631

Closed
witgo wants to merge 1 commit into
apache:masterfrom
witgo:SPARK-1699
Closed

witgo wants to merge 1 commit into
apache:masterfrom
witgo:SPARK-1699

Conversation

@witgo

@witgo witgo commented May 4, 2014

Copy link
Copy Markdown
Contributor

No description provided.

@AmplabJenkins

Copy link
Copy Markdown

Can one of the admins verify this patch?

@witgo witgo closed this May 7, 2014
@witgo
witgo deleted the SPARK-1699 branch October 15, 2014 10:24
andrewor14 pushed a commit to andrewor14/spark that referenced this pull request Jan 8, 2015
The current doc hints spark doesn't support accumulators of type `Long`, which is wrong.

JIRA: https://spark-project.atlassian.net/browse/SPARK-1117

Author: Xiangrui Meng <[email protected]>

Closes apache#631 from mengxr/acc and squashes the following commits:

45ecd25 [Xiangrui Meng] update accumulator docs
(cherry picked from commit aaec7d4)

Signed-off-by: Patrick Wendell <[email protected]>
johnhany97 pushed a commit to johnhany97/spark that referenced this pull request Jan 10, 2020
Cherry pick SPARK-27291 and SPARK-27990
MaxGekk added a commit to MaxGekk/spark that referenced this pull request Oct 5, 2026
### What changes were proposed in this pull request?

`dev/varka_vector_bits.java`, the probe the gate runs to print the narrow step's vector width (VARKA-286, apache#629), moves to `dev/varka_canary/VectorBits.java`, beside the other JVM probes. `dev/varka_gate.sh` runs it from there.

**Why it broke MiMa.** Spark's sbt build defines the project `oldDeps` with `dev/` as its base directory (`project/SparkBuild.scala`, `Project("oldDeps", file("dev"))`). `dev/mima` builds that project, and sbt compiles any Java file sitting directly in a project's base directory. The probe imports `jdk.incubator.vector`, which that project doesn't have, so the step fails with "Failed to export oldDeps classpath". sbt doesn't compile subdirectories of the base, so `dev/varka_canary/` is safe.

**Why apache#629 was green.** Pull requests that change only Varka's files skip Spark's linter job, MiMa included. The failure shows up only on pull requests that run Spark's full checks, such as apache#631, which changes the workflow; it has failed MiMa twice.

### Why are the changes needed?

So that MiMa passes again on every pull request that runs it, starting with apache#631.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

`dev/mima` passes locally on this branch; on apache#631's branch it fails at the probe ("package jdk.incubator.vector.IntVector does not exist"). The moved probe prints 128 with `-XX:MaxVectorSize=16` and 512 without, as before, and `bash -n dev/varka_gate.sh` is clean.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5.5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants