Recently, the project team asked to write an SQL script, and the relevant parsing tools can customize the verification rules. They began to study the learning of Irony framework based on C#; Due to my weak foundation, there are not many resources for this framework on the Internet. Finally, the Apache compute framework was abandoned. I borrowed some resources and blogs on the Internet and finally finished the demo of custom rules. Now the learning process and problems are sorted out as follows. I hope it can help new partners who also want to learn from calcium.
-
1> Download IDE and configure maven environment
My demo is developed based on eclipse, and other IDE environments can study by themselves
1. Download maven
http://maven.apache.org/download.cgi
1.2. Unzip the downloaded compressed package and configure the settings in maven xml
Partial configuration descriptionD:\JAVA\MavenJar
Set the local warehouse address and Alibaba cloud warehouse address
1.3 configuring maven environment in eclipse
Window–>Preferences–>Maven–>User Settings
-
2> Download the calcite source package
Download address: https://gitee.com/Bomy/calcite
2.1: copy all files under code\src\main\codegen to your own code path
-
3> Open eclipse to create Maven project
3.1 project creation
3.2. Copy all codes in code \ SRC \ main \ codegrand in the cacite source code to the main folder, and refresh the project after copying
3.3. Build maven project. Note here that you need to configure the compilation plug-ins, mainly including freemaker and javacc. Here is my POM xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>cn.wupeng</groupId> <artifactId>FirstCalciteProject</artifactId> <version>0.0.1-SNAPSHOT</version> <dependencies> <dependency> <groupId>org.apache.calcite</groupId> <artifactId>calcite-core</artifactId> <version>1.19.0</version> <!--<version>1.11.0</version>--> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>compile</scope> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>2.4</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>javacc-maven-plugin</artifactId> <executions> <execution> <id>javacc</id> <goals> <goal>javacc</goal> </goals> <configuration> <sourceDirectory>${project.build.directory}/generated-sources/fmpp</sourceDirectory> <includes> <include>**/Parser.jj</include> </includes> <lookAhead>2</lookAhead> <isStatic>false</isStatic> </configuration> </execution> <execution> <id>javacc-test</id> <phase>generate-test-sources</phase> <goals> <goal>javacc</goal> </goals> <configuration> <sourceDirectory>${project.build.directory}/generated-test-sources/fmpp</sourceDirectory> <outputDirectory>${project.build.directory}/generated-test-sources/javacc</outputDirectory> <includes> <include>**/Parser.jj</include> </includes> <lookAhead>2</lookAhead> <isStatic>false</isStatic> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.drill.tools</groupId> <artifactId>drill-fmpp-maven-plugin</artifactId> <executions> <execution> <configuration> <config>src/main/codegen/config.fmpp</config> <output>${project.build.directory}/generated-sources/fmpp</output> <templates>src/main/codegen/templates</templates> </configuration> <id>generate-fmpp-sources</id> <phase>validate</phase> <goals> <goal>generate</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </project>
pom. If there is an error in execution in XML, please follow the following method
Window–>Preferences–>Maven–>Errors/Warnings
3.4. Create SQL parsing class SqlWuPeng, and the package path is org apache. calcite. sql
package org.apache.calcite.sql; import org.apache.calcite.sql.parser.SqlParserPos; import org.apache.calcite.sql.util.SqlVisitor; import org.apache.calcite.sql.validate.SqlValidator; import org.apache.calcite.sql.validate.SqlValidatorScope; import org.apache.calcite.util.Litmus; public class SqlWuPeng extends SqlNode { private String jackyString; private SqlParserPos pos; public SqlWuPeng(SqlParserPos pos, String jackyString){ super(pos); this.pos = pos; this.jackyString = jackyString; } public String getJackyString(){ System.out.println("getJackyString"); return this.jackyString; } @Override public SqlNode clone(SqlParserPos sqlParserPos) { System.out.println("clone"); return null; } @Override public void unparse(SqlWriter sqlWriter, int i, int i1) { sqlWriter.keyword("hello"); sqlWriter.keyword("calcite"); sqlWriter.print("\n"); sqlWriter.keyword("" + jackyString + ""); } @Override public void validate(SqlValidator sqlValidator, SqlValidatorScope sqlValidatorScope) { System.out.println("validate"); } @Override public <R> R accept(SqlVisitor<R> sqlVisitor) { System.out.println("accept"); return null; } @Override public boolean equalsDeep(SqlNode sqlNode, Litmus litmus) { System.out.println("equalsDeep"); return false; } }
3.5. Modify config FMPP file
Find the package: "org. Apache. Cite. SQL. Parser. Impl" and modify the following class to WPSqlParserImpl
3.6. Modify parser JJ file
3.6.1 - > Add resolution class reference
3.6.2 > Add parsing logic
/** *Custom parsing logic **/ SqlNode SqlJacky() : { SqlNode stringNode; } { <hello> <calcite> stringNode = StringLiteral() { return new SqlJacky(getPos(), token.image); } }
3.6.3 > find SqlNode SqlStmt() and add custom parsing logic
3.6.4 > set keyword
3.6.5 > compiling maven project
Compiled project directory
3.6.6 > create a test class WupengParser
package wupeng.test; import org.apache.calcite.avatica.util.Casing; import org.apache.calcite.avatica.util.Quoting; import org.apache.calcite.schema.SchemaPlus; import org.apache.calcite.sql.SqlNode; import org.apache.calcite.sql.parser.SqlParser; import org.apache.calcite.tools.FrameworkConfig; import org.apache.calcite.tools.Frameworks; public class WupengParser { public static void main(String[] args) { SchemaPlus rootSchema = Frameworks.createRootSchema(true); final FrameworkConfig config = Frameworks.newConfigBuilder() .parserConfig(SqlParser.configBuilder() //.setLex(Lex.ORACLE) .setParserFactory(WPSqlParserImpl.FACTORY) .setCaseSensitive(false) .setQuoting(Quoting.BACK_TICK) .setQuotedCasing(Casing.TO_UPPER) .setUnquotedCasing(Casing.TO_UPPER) //.setConformance(SqlConformanceEnum.ORACLE_12) .build()) .build(); // "jacky 'select ids, name from test where id < 5'"; String sql = "hello calcite 'select ids, name from test where id < 5'"; SqlParser parser = SqlParser.create(sql, config.getParserConfig()); try { SqlNode sqlNode = parser.parseStmt(); System.out.println(sqlNode.toString()); } catch (Exception e) { e.printStackTrace(); } } }
Handling of error reporting by referencing user-defined parsing classes
Right click Project – Properties – Java Build Path --Source – Link Source, and then find javacc directory under target to add
3.6.7 > execute test class WupengParser
Acknowledgment: This demo mainly refers to the code of Yu Qi Great God, which has benefited a lot.
Reference connection:
https://blog.csdn.net/ccllcaochong1/article/details/93367343
https://github.com/yuqi1129/calcite-test
https://github.com/quxiucheng/apache-calcite-tutorial/tree/a7d63273d0c7585fc65ad250c99a67a201bcb8b5
https://dafei1288.blog.csdn.net/article/details/102735371?utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7EBlogCommendFromMachineLearnPai2%7Edefault-1.control&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7EBlogCommendFromMachineLearnPai2%7Edefault-1.control