1, Database localization background
Oracle is still the greatest database by far. The Oracle versions we have seen or heard of in our generation are 8i and 9i. The release dates are 1998 and 2001. The earliest release date of Oracle can be traced back to the 1960s and 1970s. At that time, oracle was called RSi (Relational Software, Inc). Oracle database has changed from simple function to complex function, From unstable to stable, it has experienced the precipitation of time.
Mr. Gai Guoqiang Gai called "the first year of domestic databases" in 2019. Although domestic databases started late, they have a history of about 40 years. With the guidance, support and support of the state in recent years, Internet enterprises have entered the market with high enthusiasm, the participation of emerging independent database start-ups, the participation of head technology enterprises, and the huge user base of local databases, I believe that the development of domestic databases will catch up one day, from trial, usable, to dare to use, easy to use and easy to use.
With the impact of the "trade war" in 2019, a number of technologies are "stuck", especially in the chip industry, resulting in the situation that domestic enterprises have no core available. As one of the three basic software, the database also has the problem of "neck sticking". Under the guidance of national policies, more and more enterprises enter the Bureau, pursue self-control and get rid of foreign monopoly.
In addition, in many segmentation scenarios, Oracle alone can no longer meet the needs of domestic unique business scenarios. Domestic databases have made efforts in distributed and cloud native. At present, they have made good achievements in finance, Internet, operators and other industries, and even opened markets at home and abroad.
The slogan of going to IOE has been mentioned in recent years. With the wave of domestic databases, the "oracle" in going to IOE has been put on the agenda by many large, medium and small enterprises. A common problem is that the more heavily Oracle is applied, the more users use features, and the greater the migration complexity. For example, with the accumulation of time, the PL/SQL function of Oracle, the PL/SQL code of some customers has reached the level of millions or even tens of millions. The human and material resources spent on the code transformation of heterogeneous migration are unacceptable to many customers. This function alone discourages many users who want to go to "O". The current situation of "O" removal is that eight immortals cross the sea to show their powers. There is no good tool to unify the migration process. Huawei cloud recently launched two products UGO+DRS, which I believe can help users complete and simplify the process of "O".
2, What are UGO, DRS, GaussDB
Database and Application Migration UGO (UGO) is a professional cloud service focusing on heterogeneous database structure migration and application SQL transformation. Through UGO's pre migration evaluation and automatic syntax conversion, users can identify migration risks in advance, improve migration efficiency, and minimize users' database migration costs.
Data Replication Service (DRS) is an easy-to-use, stable and efficient cloud service for online database migration and real-time database synchronization. DRS focuses on cloud database, reduces the complexity of data flow between databases, and effectively helps you reduce the cost of data transmission.
GaussDB(for openGauss) is a self-developed enterprise level distributed relational database built by Huawei. The product has the mixed load capacity of enterprise level complex transactions. At the same time, it supports excellent distributed transactions, cross AZ deployment in the same city, data 0 loss, 1000 + expansion capability, PB level mass storage and other enterprise level database features. It has the key capabilities of high availability, high reliability, high security, elastic scalability, one click deployment, rapid backup and recovery, monitoring and alarm, and can provide enterprises with enterprise level database services with comprehensive functions, stability, reliability, strong scalability and superior performance. At the same time, Huawei open source openGauss stand-alone active and standby community version to encourage more partners and developers to jointly prosper China's database ecology.
With UGO+DRS, Huawei has achieved the highest level of industrial standards set by the Institute in terms of business database migration.
3, Migration test
This test simulates the heterogeneous migration from Oracle database in the production environment to Huawei cloud GaussDB(for openGauss), uses UGO to migrate objects, uses DRS to migrate and synchronize data in real time, and evaluates and verifies the migration process and data.
1. Architecture diagram
The test architecture is shown above. The source end is Oracle 19C installed in Docker, and the target end is the GaussDB(for openGauss) instance purchased on Huawei cloud. Ugos and DRS are purchased through Huawei cloud and operated directly in the web console.
2. Build source side environment
Package image
[root@ecs-ugo ~]# git clone https://github.com/oracle/docker-images.git[root@ecs-ugo ~]# cd docker-images-main/OracleDatabase/SingleInstance/dockerfiles/19.3.0/[root@ecs-ugo 19.3.0]# lscheckDBStatus.sh Checksum.ee createDB.sh db_inst.rsp installDBBinaries.sh relinkOracleBinary.sh runUserScripts.sh setupLinuxEnv.shcheckSpace.sh Checksum.se2 dbca.rsp.tmpl Dockerfile LINUX.X64_193000_db_home.zip runOracle.sh setPassword.sh startDB.sh[root@ecs-ugo 19.3.0]# pwd/root/docker-images-main/OracleDatabase/SingleInstance/dockerfiles/19.3.0[root@ecs-ugo 19.3.0]# cd ..[root@ecs-ugo dockerfiles]# ./buildContainerImage.sh --help./buildContainerImage.sh: illegal option -- - Usage: buildContainerImage.sh -v [version] -t [image_name:tag] [-e | -s | -x] [-i] [-o] [container build option]Builds a container image for Oracle Database. Parameters: -v: version to build Choose one of: 11.2.0.2 12.1.0.2 12.2.0.1 18.3.0 18.4.0 19.3.0 21.3.0 -t: image_name:tag for the generated docker image -e: creates image based on 'Enterprise Edition' -s: creates image based on 'Standard Edition 2' -x: creates image based on 'Express Edition' -i: ignores the MD5 checksums -o: passes on container build option * select one edition only: -e, -s, or -x LICENSE UPL 1.0 Copyright (c) 2014,2021 Oracle and/or its affiliates. [root@ecs-ugo dockerfiles]# ./buildContainerImage.sh -v 19.3.0 -i -e[root@ecs-ugo dockerfiles]# docker image lsREPOSITORY TAG IMAGE ID CREATED SIZEoracle/database 21.3.0-ee 0855ee2749de 9 days ago 7.94GBswr.cn-north-4.myhuaweicloud.com/oracle/oracle 21.3.0 0855ee2749de 9 days ago 7.94GBoracle/database 19.3.0-ee cc930c1e5ee2 10 days ago 6.53GBswr.cn-north-4.myhuaweicloud.com/oracle/oracle 19.3.0 cc930c1e5ee2 10 days ago 6.53GBswr.cn-north-4.myhuaweicloud.com/oracle/oracle 19c cc930c1e5ee2 10 days ago 6.53GBoraclelinux 7-slim 078d6e3ae75e 5 weeks ago 132MB
Start Oracle 19c
docker run --name oracle21c -d \ -p 15221:1521 \ -e ORACLE_SID=LEE \ -e ORACLE_PDB=oracle19c \ -e ORACLE_PWD=Enmotech \ -e ORACLE_CHARACTERSET=ZHS16GBK \ swr.cn-north-4.myhuaweicloud.com/oracle/oracle:19.3.0
Test connection
[root@ecs-ugo ~]# docker exec -it oracle19c bash[oracle@666d4b0868ff ~]$ export ORACLE_SID=LEE[oracle@666d4b0868ff ~]$ sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Fri Dec 3 05:35:53 2021Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to:Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - ProductionVersion 19.3.0.0.0
SQL> show pdbs;
CON_ID CON_NAME OPEN MODE RESTRICTED---------- ------------------------------ ---------- ---------- 2 PDB$SEED READ ONLY NO 3 ORACLE19C READ WRITE NOSQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - ProductionVersion 19.3.0.0.0[oracle@666d4b0868ff ~]$ lsnrctl status
LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 03-DEC-2021 05:36:29
Copyright (c) 1991, 2019, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1)))STATUS of the LISTENER------------------------Alias LISTENERVersion TNSLSNR for Linux: Version 19.0.0.0.0 - ProductionStart Date 03-DEC-2021 03:31:42Uptime 0 days 2 hr. 4 min. 46 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /opt/oracle/product/19c/dbhome_1/network/admin/listener.oraListener Log File /opt/oracle/diag/tnslsnr/666d4b0868ff/listener/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=666d4b0868ff)(PORT=5500))(Security=(my_wallet_directory=/opt/oracle/admin/LEE/xdb_wallet))(Presentation=HTTP)(Session=RAW))Services Summary...Service "LEE" has 1 instance(s). Instance "LEE", status READY, has 1 handler(s) for this service...Service "LEEXDB" has 1 instance(s). Instance "LEE", status READY, has 1 handler(s) for this service...Service "d16cf25e0b510cc0e053020011ace43e" has 1 instance(s). Instance "LEE", status READY, has 1 handler(s) for this service...Service "oracle19c" has 1 instance(s). Instance "LEE", status READY, has 1 handler(s) for this service...The command completed successfully3. Purchase target end GaussDB for openGauss
Select the cloud database GaussDB on the console
On the instance management page, click purchase database instance
On the create instance page, select the billing mode, fill in and select the instance related information, and then click buy now.
Submit
4. Test and log in to GaussDB for openGauss
DAS platform
command line
[root@ecs-ugo Euler2.5_X86_64]# gsql -d enmotech -h 192.168.0.202 -Uroot -p8000 -r -WEnmotech2021@gsql ((GaussDB Kernel V500R001C20 build 327f8401) compiled at 2021-09-15 00:29:02 commit 1094 last mr 6379 )SSL connection (cipher: DHE-RSA-AES128-GCM-SHA256, bits: 128)Type "help" for help. enmotech=> select version(); version ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ PostgreSQL 9.2.4 (GaussDB Kernel V500R001C20 build 327f8401) compiled at 2021-09-15 00:28:41 commit 1094 last mr 6379 on aarch64-unknown-linux-gnu, compiled by g++ (GCC) 7.3.0, 64-bit(1 row)
5. UGO object migration
The whole object migration process of UGO is as follows:
Create evaluation project
- Support multiple versions of Oracle database (11g, 12c, 18c, 19c)
Fill in database connection information
Pre inspection
- Verify the necessary conditions for migration
Select evaluation scope
- Support to customize the evaluation of multiple database objects
Task confirmation, creating
Confirm target library evaluation
- For the selection of target database types for intelligent recommendation, the perspective of user selection is multi-dimensional. UGO provides compatibility recommendation, which can provide key quantitative dimensions.
Source database analysis
- The details of the portrait in the original database are recorded
Target database evaluation
- Analyze the compatibility and performance of different kinds of target databases
- Workload and man day evaluation, statistics of object information, and solving customer pain points
- List the existing risk points and give corresponding modification suggestions
Summary report
Create migration project
Conversion plan
- In the industry, UGO generally imitates O racle syntax at the database syntax layer. In fact, this practice is connected. Huawei's scheme is completely switched to an autonomous and controllable ecology, so the transformation of UGO belongs to "true transformation".
- Huawei also mentioned in the public live broadcast course that UGO training library has unique advantages. It is the top user of Oracle all over the world. It grinds in the core field of IOE and replaces the core business of Oracle Jushi system, that is, class A business. At present, the conversion rate of ten million row PLSQL in landmark projects has reached 95%.
Comparison and modification suggestions before and after SQL
grammatical transformation
Object correction
Prospect of new functions It is understood that the following functions are planned to be supported in the future:
- You can see the compatibility of 51 Oracle objects at one time, i.e. GaussDB (for openaauss), rather than just showing which objects are currently used in Oracle. Users who plan to replace Oracle in batches can understand the compatibility problem at one time.
- For the general situation of integrating Oracle system views into business PL/SQL. UGO compares the compatibility relationship between Oracle and GaussDB (for opencauss) system views in detail, and makes targeted design according to the key demands of users in the process of de-O.
- The attribute level compatibility analysis and comparison for object compatibility solves a problem that can not be answered in the industry: for example, how to prove that an index conversion is successful? This function of UGO can show that the index has several attributes in Oracle and corresponding attributes in GaussDB(for opengauss), so they are equivalent.
Report display
6. DRS data synchronization
The RDS data synchronization process is as follows:
Create synchronization task
Fill in the connection information and create an SMN
Fill in source library and target library information
Edit synchronization task
advanced setting
Data processing
Pre inspection
- There are many items for pre inspection, which can avoid stepping on many pits.
Task confirmation
Screenshot of console
Source end
Target end
7. DRS data verification
- Object level comparison: it supports the comparison of database, index, table, view, stored procedure and function, table sorting rules and other objects.
- Data level comparison: it supports the comparison of the number of rows and contents of a table or set. (content comparison is not open yet)
- View the comparison report and export the report.
4, Test summary
Traditional Oracle heterogeneous migration requires a large number of people to participate in research, evaluation, testing and verification. The project implementation cycle is also very long, and the average labor cost is about 500000. Under the current trend of "going to O" and going to the cloud, Huawei cloud's UGO and DRS tools can quickly migrate to heterogeneous databases such as Huawei cloud GaussDB(for openGauss) and RDS. The tools completely cover the whole project cycle of heterogeneous object migration, data synchronization and data verification, For the most complex object migration, you only need to rectify some objects according to the rewriting prompt and evaluation person days, which greatly reduces the cost of "de O" and reduces the threshold for domestic databases to replace traditional databases. It is expected that the tool will support more databases and support cloud migration in the future.
Ink Sky Wheel original link: https://www.modb.pro/db/172953?sjhy
About the author
Li Hongda, the committer of openGauss open source community, is mainly engaged in Oracle, openGauss and cloud services. Now it has Oracle OCP, OceanBase OBCA, openGauss OGCA, PostgreSQL PGCA and Huawei cloud HCIP (Solutions Architect) certification.