[GLUTEN-11550][UT] Enable 15 disabled test suites for Spark 4.x#11816
Open
baibaichen wants to merge 7 commits intoapache:mainfrom
Open
[GLUTEN-11550][UT] Enable 15 disabled test suites for Spark 4.x#11816baibaichen wants to merge 7 commits intoapache:mainfrom
baibaichen wants to merge 7 commits intoapache:mainfrom
Conversation
baibaichen
changed the title
|
Run Gluten Clickhouse CI on x86 |
baibaichen
force-pushed
the
fix/11550-enable-all
branch
from
5782b90 to
451760d
Compare
|
Run Gluten Clickhouse CI on x86 |
1 similar comment
|
Run Gluten Clickhouse CI on x86 |
baibaichen
force-pushed
the
fix/11550-enable-all
branch
from
431721f to
bb137ad
Compare
baibaichen
changed the title
|
Run Gluten Clickhouse CI on x86 |
This was referenced Apr 10, 2026
baibaichen
force-pushed
the
fix/11550-enable-all
branch
from
bb137ad to
c0c68ff
Compare
|
Run Gluten Clickhouse CI on x86 |
baibaichen
force-pushed
the
fix/11550-enable-all
branch
2 times, most recently
from
8ec0857 to
0c2ca3d
Compare
|
Run Gluten Clickhouse CI on x86 |
1 similar comment
|
Run Gluten Clickhouse CI on x86 |
baibaichen
force-pushed
the
fix/11550-enable-all
branch
from
0c2ca3d to
d9b6122
Compare
|
Run Gluten Clickhouse CI on x86 |
Direct enable (no code changes needed, ANSI fallback or trait fix): - GlutenTPCDSQueryWithStatsSuite: fix trait GlutenTestsCommonTrait -> GlutenSQLTestsTrait - GlutenTPCDSQueryANSISuite: fix trait - GlutenToPrettyStringSuite: ANSI fallback - GlutenCollationRegexpExpressionsSuite: ANSI fallback (spark41-only) - GlutenSimpleSQLViewSuite: ANSI fallback - GlutenDataSourceScanExecRedactionSuite: ANSI fallback - GlutenDataSourceV2ScanExecRedactionSuite: ANSI fallback - Disable GlutenStreamingQueryHashPartitionVerifySuite (wrong trait, runs as vanilla Spark) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wrap exception in glutenCheckExpression with fail() to match Spark's checkEvaluationWithoutCodegen behavior. No testGluten override needed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…extFileV2Suite Override testFile() to use getWorkspaceFilePath() instead of default jar-based path. The default testFile() returns jar:file: URI which Hadoop Path does not support. Same pattern used by GlutenCSVSuite, GlutenJsonSuite, GlutenParquetIOSuite. V1 3/3 passed, V2 3/3 passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Override test to find ColumnarToRowExecBase instead of ColumnarToRowExec. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
6 tests rewritten with testGluten (match TakeOrderedAndProjectExecTransformer, LimitExecTransformer, ColumnarCollectLimitBaseExec). Original tests excluded. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
testGluten rewrite for 'join strategy hint - shuffle-replicate-nl': Gluten replaces CartesianProductExec with CartesianProductExecTransformer, so assertShuffleReplicateNLJoin needs to match both types. Original test excluded, testGluten 18/18 passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
testGluten rewrite for 'explain formatted - check presence of subquery in case of DPP': Gluten uses different warehouse path (no suite class name) and DPP subquery is named 'dynamicpruning' instead of 'subquery'. Original test excluded, testGluten 24/24 passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
baibaichen
force-pushed
the
fix/11550-enable-all
branch
from
d9b6122 to
666ce08
Compare
|
Run Gluten Clickhouse CI on x86 |
baibaichen
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Enable 15 previously disabled test suites for Spark 4.0/4.1, organized by root cause:
Commit 1: VTS-only changes (8 suites)
Direct enable with no code changes needed:
GlutenTPCDSQueryWithStatsSuite/GlutenTPCDSQueryANSISuite— fix wrong trait (GlutenTestsCommonTrait->GlutenSQLTestsTrait)GlutenToPrettyStringSuite,GlutenCollationRegexpExpressionsSuite(spark41),GlutenSimpleSQLViewSuite,GlutenDataSourceScanExecRedactionSuite,GlutenDataSourceV2ScanExecRedactionSuite— ANSI fallback, all tests passGlutenStreamingQueryHashPartitionVerifySuite— uses wrong trait, runs as vanilla Spark (no GlutenPlugin loaded)Commit 2: Exception wrapper fix (1 suite)
GlutenCsvExpressionsSuite— modifyGlutenTestsTrait.glutenCheckExpressionto unwrapSparkExceptionbeforefail(), matching Spark'scheckEvaluationWithoutCodegenbehaviorCommit 3: Test resource path fix (2 suites)
GlutenWholeTextFileV1Suite/GlutenWholeTextFileV2Suite— overridetestFile()to usegetWorkspaceFilePath()instead of default jar-based path (same pattern as GlutenCSVSuite/GlutenJsonSuite/GlutenParquetIOSuite)Commits 4-7: testGluten rewrites (4 suites)
GlutenSparkPlanSuite— matchColumnarToRowExecBaseinstead ofColumnarToRowExecGlutenInsertSortForLimitAndOffsetSuite— 6 testGluten matchingTakeOrderedAndProjectExecTransformer+LimitExecTransformerGlutenJoinHintSuite— match bothCartesianProductExecandCartesianProductExecTransformerGlutenExplainSuite— fix warehouse path pattern + DPP subquery naming (dynamicpruningvssubquery)How was this patch tested?
Each suite verified on spark41 and spark40 via
run-scala-test.sh. All excluded tests have correspondingtestGlutenreplacements.Related issue: #11550