Fix clickhouse-jdbc: lex heredoc strings as a single literal in the JavaCC parser - #3040
Open
polyglotAI-bot wants to merge 1 commit into
Open
Fix clickhouse-jdbc: lex heredoc strings as a single literal in the JavaCC parser#3040polyglotAI-bot wants to merge 1 commit into
polyglotAI-bot wants to merge 1 commit into
Conversation
…avaCC parser The legacy (v1) JavaCC grammar had no heredoc token, so the body of a $$body$$ / $tag$body$tag$ literal was lexed as ordinary SQL: a ';' in the body ended the statement (select $$a;b$$ was truncated to select $$a and prepareStatement sent that to the server) and another body character that is not a valid token in that position left the statement classified as UNKNOWN with no table name. Fixes: #3039
|
Client V2 CoverageCoverage Report
Class Coverage
|
JDBC V2 CoverageCoverage Report
Class Coverage
|
JDBC V1 CoverageCoverage Report
Class Coverage
|
Client V1 CoverageCoverage Report
Class Coverage
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #3039.
The legacy
clickhouse-jdbc(v1) JavaCC grammar has no heredoc token, so the body of a$$body$$/$tag$body$tag$literal is lexed as ordinary SQL. A;in the body therefore ends thestatement —
select $$a;b$$parses to a single statement whose SQL is onlyselect $$a, and sinceClickHouseConnectionImpl.prepareStatementbuilds the prepared query fromparsedStmt.getSQL(), thedriver sends that truncated SQL to the server (
Syntax error: failed at position 8 ($): $$a). A bodycharacter that is not a valid token in that position (e.g.
?) aborts the parse instead, leaving thestatement classified as
UNKNOWNwith no table name, which disables the table-name based paths. Aheredoc is now lexed as one string literal, so its body is opaque to the parser.
This is the v1 counterpart of #3030 (jdbc-v2 JavaCC) and #3032 (jdbc-v2 ANTLR4), and uses the same
token definition as #3030. It is a different layer from #3036 / #3035 (the v1
JdbcParameterizedQueryplaceholder scan) — both are needed for the v1 driver to accept such astatement end to end; this PR only removes the truncation/misclassification in the parser.
Changes
clickhouse-jdbc/src/main/javacc/ClickHouseSqlParser.jj: added theHEREDOC_LITERALtoken (with aprivate
HEREDOC_TAG) declared beforeIDENTIFIER, and accepted it inliteral(). Matchedloosely like the rest of this grammar: the tags are not required to be equal and a body cannot
contain
$; an unterminated tag ($foo$bar) still lexes as an identifier, which is how the serverreads it too.
CHANGELOG.md: entry under0.11.0-rc1→ Bug Fixes.Known limitation, unchanged by this PR: a heredoc used in a position that only accepts a quoted
string literal (
ON CLUSTER,INFILE/OUTFILE,COMPRESSION,INPUT(...)) is still notaccepted there — supporting it also needs heredoc-aware unescaping of the extracted value, which is
out of scope here. The truncation this PR fixes no longer happens in those positions either, since
the body is a single token.
Test
ClickHouseSqlParserFacadeTest(moduleclickhouse-jdbc, groupunit):testHeredocStringLiteral— a@DataProvider-driven test asserting the parsed SQL is nottruncated and the statement type / table name are correct for a heredoc body containing
;,?,parentheses and commas, whitespace, an empty body, and the tagged form, both in a
SELECTlist andin an
INSERT ... VALUESlist. Contrast rows pin the behaviour that must not change: anunterminated tag (
select $foo$bar from tbl) and a$inside a name (select a$b from tbl) stayidentifiers, and a quoted string literal (
select 'a;b', 1 from tbl) keeps its existing handling.testHeredocDoesNotSplitStatements—select $$a;b$$;select 2splits at the real separator only.Unpatched, 5 of the 11 provider rows and
testHeredocDoesNotSplitStatementsfail (truncated SQL /UNKNOWN); with the fix all pass. Full module suite: 101 tests, 0 failures. Also verified at thedriver level against a 26.5.1 server with
clickhouse.jdbc.v1=true:select $$a;b$$ as sused toreach the server as
select $$a(syntax error) and no longer does.Pre-PR validation gate
AGENTS.md(targeted module tests,@DataProviderfornear-identical cases, no issue references inside test code, CHANGELOG updated)
Notes
clickhouse-jdbc(v1) is the legacy stack — close this if v1 grammar changes are out of scope.