The DSC is a CLI tool running on the Linux or Windows OS. It is dedicated to providing customers with simple, fast, and reliable application SQL script migration services. It parses the SQL scripts of source database applications using the built-in syntax migration logic, and converts them to SQL scripts applicable to GaussDB(DWS) databases. You do not need to connect the DSC to a database. It can migrate data in offline mode without service interruption. In GaussDB(DWS), you can run the migrated SQL scripts to restore the database, thereby easily migrating offline databases to the cloud.
-The DSC can migrate SQL scripts of Teradata, Oracle, Netezza, MySQL, and DB2 databases.
-Downloading the DSC SQL Migration Tool
- Log in to the GaussDB(DWS) management console.
- In the navigation tree on the left, click Connection Management.
- In the Download Client and Driver area, click here to download the DSC migration tool.
If you have clusters of different versions, the system displays a dialog box, prompting you to select the cluster version and download the client corresponding to the cluster version. In the cluster list on the Cluster Management page, click the name of the specified cluster and click the Basic Information tab to view the cluster version.
-Figure 1 Downloading the tool- -
- After downloading the DSC tool to the local PC, use WinSCP to upload it to a Linux host.
The user who uploads the tool must have the full control permission on the target directory of the Linux host.
+The DSC is a CLI tool running on the Linux or Windows OS. It is dedicated to providing customers with simple, fast, and reliable application SQL script migration services. It parses the SQL scripts of source database applications using the built-in syntax migration logic, and converts them to SQL scripts applicable to GaussDB(DWS) databases. You do not need to connect the DSC to a database. It can migrate data in offline mode without service interruption. In GaussDB(DWS), you can run the migrated SQL scripts to restore the database, thereby easily migrating offline databases to the cloud.
+The DSC can migrate SQL scripts of Teradata, Oracle, Netezza, MySQL, and DB2 databases.
+-Downloading the DSC SQL Migration Tool
- Log in to the GaussDB(DWS) management console.
- In the navigation tree on the left, click Connection Management.
- In the Download Client and Driver area, click here to download the DSC migration tool.
If you have clusters of different versions, the system displays a dialog box, prompting you to select the cluster version and download the client corresponding to the cluster version. In the cluster list on the Cluster Management page, click the name of the specified cluster and click the Basic Information tab to view the cluster version.
+Figure 1 Downloading the tool+ +
- After downloading the DSC tool to the local PC, use WinSCP to upload it to a Linux host.
The user who uploads the tool must have the full control permission on the target directory of the Linux host.
Operation Guide for the DSC SQL Syntax Migration Tool
For details, see "DSC - SQL Syntax Migration Tool" in the Data Warehouse Service Tool Guide.
+Operation Guide for the DSC SQL Syntax Migration Tool
For details, see "DSC - SQL Syntax Migration Tool" in the Data Warehouse Service Tool Guide.
diff --git a/docs/dws/dev/dws_04_0001.html b/docs/dws/dev/dws_04_0001.html index 1a251b98..b42ec35f 100644 --- a/docs/dws/dev/dws_04_0001.html +++ b/docs/dws/dev/dws_04_0001.html @@ -1,4 +1,4 @@ - +diff --git a/docs/dws/dev/dws_04_0002.html b/docs/dws/dev/dws_04_0002.html index f4c59221..4cb03c98 100644 --- a/docs/dws/dev/dws_04_0002.html +++ b/docs/dws/dev/dws_04_0002.html @@ -1,15 +1,15 @@ - +Welcome
@@ -8,14 +8,8 @@- Reading Guide
-
- Conventions
-- Prerequisites
- -
-Parent topic: Developer Guide-Target Readers
-This document is intended for database designers, application developers, and database administrators, and provides information required for designing, building, querying and maintaining data warehouses.
-As a database administrator or application developer, you need to be familiar with:
-- Knowledge about OSs, which is the basis for everything.
- SQL syntax, which is the necessary skill for database operation.
Statement
When writing documents, the writers of GaussDB(DWS) try their best to provide guidance from the perspective of commercial use, application scenarios, and task completion. Even so, references to PostgreSQL content may still exist in the document. For this type of content, the following PostgreSQL Copyright is applicable:
-Postgres-XC is Copyright © 1996-2013 by the PostgreSQL Global Development Group.
-PostgreSQL is Copyright © 1996-2013 by the PostgreSQL Global Development Group.
-Postgres95 is Copyright © 1994-5 by the Regents of the University of California.
-IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS-IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
+This document is intended for database designers, application developers, and database administrators, and provides information required for designing, building, querying and maintaining data warehouses.
+As a database administrator or application developer, you need to be familiar with:
+- Knowledge about OSs, which is the basis for everything.
- SQL syntax, which is the necessary skill for database operation.
Statement
When writing documents, the writers of GaussDB(DWS) try their best to provide guidance from the perspective of commercial use, application scenarios, and task completion. Even so, references to PostgreSQL content may still exist in the document. For this type of content, the following PostgreSQL Copyright is applicable:
+Postgres-XC is Copyright © 1996-2013 by the PostgreSQL Global Development Group.
+PostgreSQL is Copyright © 1996-2013 by the PostgreSQL Global Development Group.
+Postgres95 is Copyright © 1994-5 by the Regents of the University of California.
+IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS-IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
diff --git a/docs/dws/dev/dws_04_0004.html b/docs/dws/dev/dws_04_0004.html index 07e319a3..b3dd93ea 100644 --- a/docs/dws/dev/dws_04_0004.html +++ b/docs/dws/dev/dws_04_0004.html @@ -1,48 +1,48 @@ - +Reading Guide
-If you are a new GaussDB(DWS) user, you are advised to read the following contents first:
-- Sections describing the features, functions, and application scenarios of GaussDB(DWS).
- "Getting Started": guides you through creating a data warehouse cluster, creating a database table, uploading data, and testing queries.
If you intend to or are migrating applications from other data warehouses to GaussDB(DWS), you might want to know how GaussDB(DWS) differs from them.
-You can find useful information from the following table for GaussDB(DWS) database application development.
+If you are a new GaussDB(DWS) user, you are advised to read the following contents first:
+- Sections describing the features, functions, and application scenarios of GaussDB(DWS).
- "Getting Started": guides you through creating a data warehouse cluster, creating a database table, uploading data, and testing queries.
If you intend to or are migrating applications from other data warehouses to GaussDB(DWS), you might want to know how GaussDB(DWS) differs from them.
+You can find useful information from the following table for GaussDB(DWS) database application development.
-If you want to...
+If you want to...
Query Suggestions
+Query Suggestions
Quickly get started with GaussDB(DWS).
+Quickly get started with GaussDB(DWS).
Deploy a cluster, connect to the database, and perform some queries by following the instructions provided in "Getting Started" in the Data Warehouse Service (DWS) User Guide.
-When you are ready to construct a database, load data to tables and compile the query content to operate the data in the data warehouse. Then, you can return to the Data Warehouse Service Database Developer Guide.
+Deploy a cluster, connect to the database, and perform some queries by following the instructions provided in "Getting Started" in the Data Warehouse Service (DWS) User Guide.
+When you are ready to construct a database, load data to tables and compile the query content to operate the data in the data warehouse. Then, you can return to the Data Warehouse Service Database Developer Guide.
Understand the internal architecture of a GaussDB(DWS) data warehouse.
+Understand the internal architecture of a GaussDB(DWS) data warehouse.
To know more about GaussDB(DWS), go to the GaussDB(DWS) home page.
+To know more about GaussDB(DWS), go to the GaussDB(DWS) home page.
Learn how to design tables to achieve the excellent performance.
+Learn how to design tables to achieve the excellent performance.
Development and Design Proposal introduces the design specifications that should be complied with during the development of database applications. Modeling compliant with these specifications fits the distributed processing architecture of GaussDB(DWS) and provides efficient SQL code.
-To facilitate service execution through optimization, you can refer to Query Performance Optimization. Successful performance optimization depends more on database administrators' experience and judgment than on instructions and explanation. However, Query Performance Optimization still tries to systematically illustrate the performance optimization methods for application development personnel and new GaussDB(DWS) database administrators.
+Development and Design Proposal introduces the design specifications that should be complied with during the development of database applications. Modeling compliant with these specifications fits the distributed processing architecture of GaussDB(DWS) and provides efficient SQL code.
+To facilitate service execution through optimization, you can refer to Performance Tuning. Successful performance optimization depends more on database administrators' experience and judgment than on instructions and explanation. However, Performance Tuning still tries to systematically illustrate the performance optimization methods for application development personnel and new GaussDB(DWS) database administrators.
Load data.
+Load data.
Data Import describes how to import data to GaussDB(DWS).
+Importing Data describes how to import data to GaussDB(DWS).
Manage users, groups, and database security.
+Manage users, groups, and database security.
Database Security Management covers database security topics.
+Database Security Management covers database security topics.
Monitor and optimize system performance.
+Monitor and optimize system performance.
System Catalogs and System Views describes the system catalogs where you can query the database status and monitor the query content and process.
-You can learn how to check the system running status and monitoring metrics on the GaussDB(DWS) console by referring to "Monitoring Clusters" in the Data Warehouse Service (DWS) User Guide.
+System Catalogs and System Views describes the system catalogs where you can query the database status and monitor the query content and process.
+You can learn how to check the system running status and monitoring metrics on the GaussDB(DWS) console by referring to "Monitoring Clusters" in the Data Warehouse Service (DWS) User Guide.
Conventions
---Example Conventions
--
-Example
-Description
-dbadmin
-Indicates the user operating and maintaining GaussDB(DWS) appointed when the cluster is created.
-8000
-Indicates the port number used by GaussDB(DWS) to monitor connection requests from the client.
-SQL examples in this manual are developed based on the TPC-DS model. Before you execute the examples, install the TPC-DS benchmark by following the instructions on the official website https://www.tpc.org/tpcds/.
--SQL Syntax Text Conventions
To better understand the syntax usage, you can refer to the SQL syntax text conventions described as follows:
- --
-Format
-Description
-Uppercase characters
-Indicates that keywords must be in uppercase.
-Lowercase characters
-Indicates that parameters must be in lowercase.
-[ ]
-Indicates that the items in brackets [] are optional.
-...
-Indicates that preceding elements can appear repeatedly.
-[ x | y | ... ]
-Indicates that one item is selected from two or more options or no item is selected.
-{ x | y | ... }
-Indicates that one item is selected from two or more options.
-[x | y | ... ] [ ... ]
-Indicates that multiple parameters or no parameter can be selected. If multiple parameters are selected, separate them with spaces.
-[ x | y | ... ] [ ,... ]
-Indicates that multiple parameters or no parameter can be selected. If multiple parameters are selected, separate them with commas (,).
-{ x | y | ... } [ ... ]
-Indicates that at least one parameter can be selected. If multiple parameters are selected, separate them with spaces.
-{ x | y | ... } [ ,... ]
-Indicates that at least one parameter can be selected. If multiple parameters are selected, separate them with commas (,).
--- diff --git a/docs/dws/dev/dws_04_0006.html b/docs/dws/dev/dws_04_0006.html index 547a9cc5..663ab841 100644 --- a/docs/dws/dev/dws_04_0006.html +++ b/docs/dws/dev/dws_04_0006.html @@ -1,9 +1,9 @@ - +--Parent topic: Welcome-Prerequisites
-Complete the following tasks before you perform operations described in this document:
-- Create a GaussDB(DWS) cluster.
- Install an SQL client.
- Connect the SQL client to the default database of the cluster.
For details about the preceding tasks, see "Getting Started" in the Data Warehouse Service (DWS) User Guide.
+Complete the following tasks before you perform operations described in this document:
+- Create a GaussDB(DWS) cluster.
- Install an SQL client.
- Connect the SQL client to the default database of the cluster.
For details about how to perform the preceding tasks, see "Getting Started" in the Data Warehouse Service (DWS) User Guide.
diff --git a/docs/dws/dev/dws_04_0007.html b/docs/dws/dev/dws_04_0007.html index 61c48b17..1bb92382 100644 --- a/docs/dws/dev/dws_04_0007.html +++ b/docs/dws/dev/dws_04_0007.html @@ -1,7 +1,7 @@ - +System Overview
-+diff --git a/docs/dws/dev/dws_04_0011.html b/docs/dws/dev/dws_04_0011.html index b91e6713..8255df1d 100644 --- a/docs/dws/dev/dws_04_0011.html +++ b/docs/dws/dev/dws_04_0011.html @@ -1,12 +1,12 @@ - +-
@@ -12,9 +12,5 @@
- Related Concepts
-Parent topic: Developer Guide-Highly Reliable Transaction Processing
-GaussDB(DWS) manages cluster transactions, the basis of HA and failovers. This ensures speedy fault recovery, guarantees the Atomicity, Consistency, Isolation, Durability (ACID) properties for transactions and after a recovery, and enables concurrent control.
-Fault Rectification
-GaussDB(DWS) provides an HA mechanism to reduce the service interruption time when a cluster is faulty. It protects key user programs to continuously provide external services, minimizing the impact of hardware, software, and human faults on services and ensuring service continuity.
-- Hardware HA: Disk RAID, switch stacking, NIC bond, and uninterruptible power supply (UPS)
- Software HA: HA mechanism used for instances in the GaussDB(DWS) cluster, such as CNs, GTMs, and DNs)
Transaction Management
-- Transaction blocks are supported. You can run start transaction to make the startup of a transaction block explicit.
- Single-statement transactions are supported. If you do not explicitly start a transaction, a single statement is processed as a transaction.
- Distributed transaction management and global transaction information management are supported. This includes gxid, snapshot, timestamp management, distributed transaction status management, and gxid overflow processing.
- Distributed transactions have ACID properties.
- Deadlocks are prevented in the distributed system. A transaction will be unlocked immediately after a deadlock (if any).
GaussDB(DWS) manages cluster transactions, the basis of HA and failovers. This ensures speedy fault recovery, guarantees the Atomicity, Consistency, Isolation, Durability (ACID) properties for transactions and after a recovery, and enables concurrent control.
+Fault Rectification
+GaussDB(DWS) provides an HA mechanism to reduce the service interruption time when a cluster is faulty. It protects key user programs to continuously provide external services, minimizing the impact of hardware, software, and human faults on services and ensuring service continuity.
+- Hardware HA: Disk RAID, switch stacking, NIC bond, and uninterruptible power supply (UPS)
- Software HA: HA mechanism used for instances in the GaussDB(DWS) cluster, such as CNs, GTMs, and DNs)
Transaction Management
+- Transaction blocks are supported. You can run start transaction to make the startup of a transaction block explicit.
- Single-statement transactions are supported. If you do not explicitly start a transaction, a single statement is processed as a transaction.
- Distributed transaction management and global transaction information management are supported. This includes gxid, snapshot, timestamp management, distributed transaction status management, and gxid overflow processing.
- Distributed transactions have ACID properties.
- Deadlocks are prevented in the distributed system. A transaction will be unlocked immediately after a deadlock (if any).
diff --git a/docs/dws/dev/dws_04_0012.html b/docs/dws/dev/dws_04_0012.html index 68717d7c..b397895b 100644 --- a/docs/dws/dev/dws_04_0012.html +++ b/docs/dws/dev/dws_04_0012.html @@ -1,15 +1,15 @@ - +High Query Performance
-The following GaussDB(DWS) features help achieve high query performance.
-Fully Parallel Query
GaussDB(DWS) is an MPP system with the shared-nothing architecture. It consists of multiple independent logical nodes that do not share system resources, such as the CPU, memory, and storage units. In such a system architecture, service data is separately stored on numerous nodes. Data analysis tasks are executed in parallel on the nodes where data is stored. The massively parallel data processing significantly improves response speed.
-In addition, GaussDB(DWS) improves data query performance by executing operators in parallel, executing commands in registers in parallel, and using LLVM to dynamically compile the logical conditions of redundancy prune.
+The following GaussDB(DWS) features help achieve high query performance.
+-Fully Parallel Query
GaussDB(DWS) is an MPP system with the shared-nothing architecture. It consists of multiple independent logical nodes that do not share system resources, such as the CPU, memory, and storage units. In such a system architecture, service data is separately stored on numerous nodes. Data analysis tasks are executed in parallel on the nodes where data is stored. The massively parallel data processing significantly improves response speed.
+In addition, GaussDB(DWS) improves data query performance by executing operators in parallel, executing commands in registers in parallel, and using LLVM to dynamically compile the logical conditions of redundancy prune.
Hybrid Row-Column Storage
GaussDB(DWS) supports both the row and column storage models. You can choose a row- or column-store table as needed.
-The hybrid row-column storage engine achieves higher data compression ratio (column storage), index performance (column storage), and point update and point query (row storage) performance.
+-Hybrid Row-Column Storage
GaussDB(DWS) supports both the row and column storage models. You can choose a row- or column-store table as needed.
+The hybrid row-column storage engine achieves higher data compression ratio (column storage), index performance (column storage), and point update and point query (row storage) performance.
Data Compression in Column Storage
You can compress old, inactive data to free up space, reducing procurement and O&M costs.
-In GaussDB(DWS), data can be compressed using the Delta Value Encoding, Dictionary, RLE, LZ4, and ZLIB algorithms. The system automatically selects a compression algorithm based on data characteristics. The average compression ratio is 7:1. Compressed data can be directly accessed and is transparent to services, greatly reducing the preparation time before accessing historical data.
+Data Compression in Column Storage
You can compress old, inactive data to free up space, reducing procurement and O&M costs.
+In GaussDB(DWS), data can be compressed using the Delta Value Encoding, Dictionary, RLE, LZ4, and ZLIB algorithms. The system automatically selects a compression algorithm based on data characteristics. The average compression ratio is 7:1. Compressed data can be directly accessed and is transparent to services, greatly reducing the preparation time before accessing historical data.
diff --git a/docs/dws/dev/dws_04_0015.html b/docs/dws/dev/dws_04_0015.html index 084c2b82..ac35d367 100644 --- a/docs/dws/dev/dws_04_0015.html +++ b/docs/dws/dev/dws_04_0015.html @@ -1,17 +1,17 @@ - +Related Concepts
-Database
A database manages data objects and is isolated from other databases. While creating a database, you can specify a tablespace. If you do not specify it, database objects will be saved to the PG_DEFAULT by default. Objects managed by a database can be distributed to multiple tablespaces.
+-Database
A database manages data objects and is isolated from other databases. While creating an object, you can specify a tablespace for it. If you do not specify it, the object will be saved to the PG_DEFAULT space by default. Objects managed by a database can be distributed to multiple tablespaces.
Instance
In GaussDB(DWS), instances are a group of database processes running in the memory. An instance can manage one or more databases that form a cluster. A cluster is an area in the storage disk. This area is initialized during installation and composed of a directory. The directory, called data directory, stores all data and is created by initdb. Theoretically, one server can start multiple instances on different ports, but GaussDB(DWS) manages only one instance at a time. The start and stop of an instance rely on the specific data directory. For compatibility purposes, the concept of instance name may be introduced.
+-Instance
In GaussDB(DWS), instances are a group of database processes running in the memory. An instance can manage one or more databases that form a cluster. A cluster is an area in the storage disk. This area is initialized during installation and composed of a directory. The directory, called data directory, stores all data and is created by initdb. Theoretically, one server can start multiple instances on different ports, but GaussDB(DWS) manages only one instance at a time. The start and stop of an instance rely on the specific data directory. For compatibility purposes, the concept of instance name may be introduced.
Tablespaces
In GaussDB(DWS), a tablespace is a directory storing physical files of the databases the tablespace contains. Multiple tablespaces can coexist. Files are physically isolated using tablespaces and managed by a file system.
+-Tablespaces
In GaussDB(DWS), a tablespace is a directory storing physical files of the databases the tablespace contains. Multiple tablespaces can coexist. Files are physically isolated using tablespaces and managed by a file system.
schema
GaussDB(DWS) schemas logically separate databases. All database objects are created under certain schemas. In GaussDB(DWS), schemas and users are loosely bound. When you create a user, a schema with the same name as the user will be created automatically. You can also create a schema or specify another schema.
+-schema
GaussDB(DWS) schemas logically separate databases. All database objects are created under certain schemas. In GaussDB(DWS), schemas and users are loosely bound. When you create a user, a schema with the same name as the user will be created automatically. You can also create a schema or specify another schema.
User and Role
GaussDB(DWS) uses users and roles to control the access to databases. A role can be a database user or a group of database users, depending on role settings. In GaussDB(DWS), the difference between roles and users is that a role does not have the LOGIN permission by default. In GaussDB(DWS), one user can have only one role, but you can put a user's role under a parent role to grant multiple permissions to the user.
+-User and Role
GaussDB(DWS) uses users and roles to control the access to databases. A role can be a database user or a group of database users, depending on role settings. In GaussDB(DWS), the difference between roles and users is that a role does not have the LOGIN permission by default. In GaussDB(DWS), one user can have only one role, but you can put a user's role under a parent role to grant multiple permissions to the user.
Transaction Management
In GaussDB(DWS), transactions are managed by multi-version concurrency control (MVCC) and two-phase locking (2PL). It enables smooth data reads and writes. In GaussDB(DWS), MVCC saves historical version data together with the current tuple version. GaussDB(DWS) uses the VACUUM process instead of rollback segments to routinely delete historical version data. Unless in performance optimization, you do not need to pay attention to the VACUUM process. Transactions are automatically submitted in GaussDB(DWS).
+Transaction Management
In GaussDB(DWS), transactions are managed by multi-version concurrency control (MVCC) and two-phase locking (2PL). It enables smooth data reads and writes. In GaussDB(DWS), MVCC saves historical version data together with the current tuple version. GaussDB(DWS) uses the VACUUM process instead of rollback segments to routinely delete historical version data. Unless in performance optimization, you do not need to pay attention to the VACUUM process. Transactions are automatically submitted in GaussDB(DWS).
diff --git a/docs/dws/dev/dws_04_0016.html b/docs/dws/dev/dws_04_0016.html new file mode 100644 index 00000000..442513d9 --- /dev/null +++ b/docs/dws/dev/dws_04_0016.html @@ -0,0 +1,30 @@ + + +Defining Database Objects
+++++ diff --git a/docs/dws/dev/dws_04_0025.html b/docs/dws/dev/dws_04_0025.html new file mode 100644 index 00000000..80bf4381 --- /dev/null +++ b/docs/dws/dev/dws_04_0025.html @@ -0,0 +1,52 @@ + + +-
+
- Creating and Managing Databases
+
+ - Creating and Managing Schemas
+
+ - Creating and Managing Tables
+
+ - Selecting a Table Storage Mode
+
+ - Defining Table Partitions
+
+ - Creating and Managing Indexes
+
+ - Creating and Using Sequences
+
+ - Creating and Managing Views
+
+ - Creating and Managing Scheduled Tasks
+
+ - Viewing a System Catalog
+
+
Creating and Managing Databases
++A database is a collection of objects such as tables, indexes, views, stored procedures, and operators. GaussDB (DWS) supports the creation of multiple databases. However, a client program can connect to and access only one database at a time, and cross-database query is not supported.
++Template and Default Databases
- GaussDB (DWS) provides two template databases template0 and template1 and a default database gaussdb.
- By default, each newly created database is based on a template database. The GaussDB(DWS) database uses template1 as the template by default. The encoding format is SQL_ASCII, and user-defined character encoding is not allowed. If you need to specify the character encoding when creating a database, use template0 to create the database.
- Do not use a client or any other tools to connect to or to perform operations on both the two template databases.+
You can run the show server_encoding command to view the current database encoding.
+
+Creating a Database.
Run the CREATE DATABASE statement to create a database.
+++ +1CREATE DATABASE mydatabase; +
+ - When you create a database, if the length of the database name exceeds 63 bytes, the server truncates the database name and retains the first 63 bytes. Therefore, you are advised to set the length of the database name to a value less than or equal to 63 bytes. Do not use multi-byte characters as object names. If an object whose name is truncated mistakenly cannot be deleted, delete the object using the name before the truncation, or manually delete it from the corresponding system catalog on each node.
- Database names must comply with the naming convention of SQL identifiers. The current user automatically becomes the owner of this new database.
- If a database system is used to support independent users and projects, store them in different databases.
- If the projects or users are associated with each other and share resources, store them in different schemas in the same database.
- A maximum of 128 databases can be created in GaussDB(DWS).
- You must have the permission to create a database or the permission that the system administrator owns.
+Viewing Databases
To view databases, perform the following steps:+- Run the \l meta-command to view the database list of the database system.++ +
1\l +
- Querying the database list using the pg_database system catalog++ +
1SELECT datname FROM pg_database; +
+Modifying a Database
You can use the ALTER DATABASE statement modify database configuration such as the database owner, name, and default settings.
+- Run the following command to set the default search path for the database:++ +
1ALTER DATABASE mydatabase SET search_path TO pa_catalog,public; +
- Rename the database.++ +
1ALTER DATABASE mydatabase RENAME TO newdatabase; +
+Deleting a Database
You can run DROP DATABASE statement to delete a database. This statement deletes the system catalog of the database and the database directory on the disk. Only the database owner or system administrator can delete a database. A database being accessed by users cannot be deleted, You need to connect to another database before deleting this database.
+Run the DROP DATABASE statement to delete a database:+++ +1DROP DATABASE newdatabase; +
++ diff --git a/docs/dws/dev/dws_04_0026.html b/docs/dws/dev/dws_04_0026.html new file mode 100644 index 00000000..3736f485 --- /dev/null +++ b/docs/dws/dev/dws_04_0026.html @@ -0,0 +1,129 @@ + + +++Parent topic: Defining Database Objects+Selecting a Table Storage Mode
++GaussDB(DWS) supports hybrid row and column storage. When creating a table, you can set the table storage mode to row storage or column storage.
+Row storage stores tables to disk partitions by row, and column storage stores tables to disk partitions by column. By default, a table is created in row storage mode. For details about differences between row storage and column storage, see Figure 1.
+ +In the preceding figure, the upper left part is a row-store table, and the upper right part shows how the row-store table is stored on a disk; the lower left part is a column-store table, and the lower right part shows how the column-store table is stored on a disk.
+The row/column storage of a table is specified by the orientation attribute in the table definition. The value row indicates a row-store table and column indicates a column-store table. The default value is row. Each storage mode applies to specific scenarios. Select an appropriate mode when creating a table.
+ ++
+Table 1 Table storage modes and scenarios Storage Mode
+Benefit
+Drawback
+Application Scenarios
+Row storage
+Data is stored by row. When you query a row of data, you can quickly locate the target row.
+All data in the queried row is read while only a few columns are needed.
+- The number of columns in the table is small, and most fields in the table are queried.
- Point queries (simple index–based query that returns only a few records) are performed.
- Add, Delete, Modify, and Query operations on entire rows are frequently performed.
Column storage
+- Only necessary columns in a query are read.
- The homogeneity of data within a column facilitates efficient compression.
It is not suitable for INSERT or UPDATE operations on a small amount of data.
+- Query a few columns in a table that contains a large number of columns.
- Statistical analysis queries (requiring a large number of association and grouping operations)
- Ad hoc queries (using uncertain query conditions and unable to utilize indexes to scan row-store tables)
+Creating a Row-store Table
For example, to create a row-store table named customer_t1, run the following command:
+++ +1 +2 +3 +4 +5 +6
CREATE TABLE customer_t1 +( + state_ID CHAR(2), + state_NAME VARCHAR2(40), + area_ID NUMBER +); +
+Creating a column-store table.
For example, to create a column-store table named customer_t2, run the following command:
+++ +1 +2 +3 +4 +5 +6 +7
CREATE TABLE customer_t2 +( + state_ID CHAR(2), + state_NAME VARCHAR2(40), + area_ID NUMBER +) +WITH (ORIENTATION = COLUMN); +
+Table Compression
Table compression can be enabled when a table is created. Table compression enables data in the table to be stored in compressed format to reduce memory usage.
+In scenarios where I/O is large (much data is read and written) and CPU is sufficient (little data is computed), select a high compression ratio. In scenarios where I/O is small and CPU is insufficient, select a low compression ratio. Based on this principle, you are advised to select different compression ratios and test and compare the results to select the optimal compression ratio as required. Specify a compressions ratio using the COMPRESSION parameter. The supported values are as follows:
+- The valid value of column-store tables is YES, NO, LOW, MIDDLE, or HIGH, and the default value is LOW.
- The valid values of row-store tables are YES and NO, and the default is NO. (The row-store table compression function is not put into commercial use. To use this function, contact technical support.)
The service scenarios applicable to each compression level are described in the following table.
+ ++
+Compression Level
+Application Scenario
+LOW
+The system CPU usage is high and the disk storage space is sufficient.
+MIDDLE
+The system CPU usage is moderate and the disk storage space is insufficient.
+HIGH
+The system CPU usage is low and the disk storage space is insufficient.
+For example, to create a compressed column-store table named customer_t3, run the following command:
+++ +1 +2 +3 +4 +5 +6 +7
CREATE TABLE customer_t3 +( + state_ID CHAR(2), + state_NAME VARCHAR2(40), + area_ID NUMBER +) +WITH (ORIENTATION = COLUMN,COMPRESSION=middle); +
++ diff --git a/docs/dws/dev/dws_04_0028.html b/docs/dws/dev/dws_04_0028.html new file mode 100644 index 00000000..282ae4f9 --- /dev/null +++ b/docs/dws/dev/dws_04_0028.html @@ -0,0 +1,287 @@ + + +++Parent topic: Defining Database Objects+Creating and Managing Tables
+++Creating a Table
You can run the CREATE TABLE command to create a table. When creating a table, you can define the following information:
+- Columns and data type of the table.
- Table or column constraints that restrict a column or the data contained in a table. For details, see Definition of Table Constraints.
- Distribution policy of a table, which determines how the GaussDB (DWS) database divides data between segments. For details, see Definition of Table Distribution.
- Table storage format. For details, see Selecting a Table Storage Mode.
- Partition table information. For details, see Defining Table Partitions.
Example: Use CREATE TABLE to create a table web_returns_p1, use wr_item_sk as the distribution key, and sets the range distribution function through wr_returned_date_sk.
+++ +1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14
CREATE TABLE web_returns_p1 +( + wr_returned_date_sk integer, + wr_returned_time_sk integer, + wr_item_sk integer NOT NULL, + wr_refunded_customer_sk integer +) +WITH (orientation = column) +DISTRIBUTE BY HASH (wr_item_sk) +PARTITION BY RANGE(wr_returned_date_sk) +( + PARTITION p2019 START(20191231) END(20221231) EVERY(10000), + PARTITION p0 END(maxvalue) +); +
+Definition of Table Constraints
You can define constraints on columns and tables to restrict data in a table. However, there are the following restrictions:
+- The primary key constraint and unique constraint in the table must contain a distribution column.
- Column-store tables support the PARTIAL CLUSTER KEY and table-level primary key and unique constraints, but do not support table-level foreign key constraints.
- Only the NULL, NOT NULL, and DEFAULT constant values can be used as column-store table column constraints.
+
+Table 1 Table constraints Constraint
+Description
+Example
+Check constraint
+A CHECK constraint allows you to specify that values in a specific column must satisfy a Boolean (true) expression.
+Create the products table. The price column must be positive.
+++ +1 +2 +3 +4 +5 +6
CREATE TABLE products +( + product_no integer, + name text, + price numeric CHECK (price > 0) +); +
NOT NULL constraint
+A NOT NULL constraint specifies that a column cannot have null values. A non-null constraint is always written as a column constraint.
+Create the products table. The values of product_no and name cannot be null.
+++ +1 +2 +3 +4 +5 +6
CREATE TABLE products +( + product_no integer NOT NULL, + name text NOT NULL, + price numeric +); +
UNIQUE constraint
+A UNIQUE constraint specifies that the values in a column or a group of columns are all unique. If DISTRIBUTE BY REPLICATION is not specified, the column table that contains only unique values must contain distribution columns.
+Create the products table. The values of product_no must be unique.
+++ +1 +2 +3 +4 +5 +6
CREATE TABLE products +( + product_no integer UNIQUE, + name text, + price numeric +)DISTRIBUTE BY HASH(product_no); +
Primary key constraint
+A primary key constraint is the combination of a UNIQUE constraint and a NOT NULL constraint. If DISTRIBUTE BY REPLICATION is not specified, the column set with a primary key constraint must contain distributed columns. If a table has a primary key, the column (or group of columns) of the primary key is selected as the distribution keys of the table by default.
+Create the products table. The primary key constraint is product_no.
+++ +1 +2 +3 +4 +5 +6
CREATE TABLE products +( + product_no integer PRIMARY KEY, + name text, + price numeric +)DISTRIBUTE BY HASH(product_no); +
Partial cluster key
+Partial cluster key can minimize or maximize sparse indexes to quickly filter base tables. Partial cluster key can specify multiple columns, but you are advised to specify no more than two columns.
+Create the products table with PCK set to product_no:
+++ +1 +2 +3 +4 +5 +6 +7
CREATE TABLE products +( + product_no integer, + name text, + price numeric, + PARTIAL CLUSTER KEY(product_no) +) WITH (ORIENTATION = COLUMN); +
+Definition of Table Distribution
GaussDB(DWS) supports the following distribution modes: replication, hash, and roundrobin.++ + The roundrobin distribution mode is supported only by cluster version 8.1.2 or later.
++
+Policy
+Description
+Scenario
+Advantages/Disadvantages
+Replication
+Full data in a table is stored on each DN in the cluster.
+Small tables and dimension tables
+- The advantage of replication is that each DN has full data of the table. During the join operation, data does not need to be redistributed, reducing network overheads and reducing plan segments (each plan segment starts a corresponding thread).
- The disadvantage of replication is that each DN retains the complete data of the table, resulting in data redundancy. Generally, replication is only used for small dimension tables.
Hash
+Table data is distributed on all DNs in the cluster.
+Fact tables containing a large amount of data
+- The I/O resources of each node can be used during data read/write, greatly improving the read/write speed of a table.
- Generally, a large table (containing over 1 million records) is defined as a hash table.
Polling (Round-robin)
+Each row in the table is sent to each DN in turn. Data can be evenly distributed on each DN.
+Fact tables that contain a large amount of data and cannot find a proper distribution column in hash mode
+- Round-robin can avoid data skew, improving the space utilization of the cluster.
- Round-robin does not support local DN optimization like a hash table does, and the query performance of Round-robin is usually lower than that of a hash table.
- If a proper distribution column can be found for a large table, use the hash distribution mode with better performance. Otherwise, define the table as a round-robin table.
Selecting a Distribution Key
+If the hash distribution mode is used, a distribution key must be specified for the user table. When a record is inserted, the system hashes it based on the distribution key and then stores it on the corresponding DN.
+Select a hash distribution key based on the following principles:
+- The values of the distribution key should be discrete so that data can be evenly distributed on each DN. You can select the primary key of the table as the distribution key. For example, for a person information table, choose the ID number column as the distribution key.
- Do not select the column that has a constant filter. For example, if a constant constraint (for example, zqdh= '000001') exists on the zqdh column in some queries on the dwcjk table, you are not advised to use zqdh as the distribution key.
- With the above principles met, you can select join conditions as distribution keys, so that join tasks can be pushed down to DNs for execution, reducing the amount of data transferred between the DNs.
For a hash table, an inappropriate distribution key may cause data skew or poor I/O performance on certain DNs. Therefore, you need to check the table to ensure that data is evenly distributed on each DN. You can run the following SQL statements to check for data skew:
+++ +1 +2 +3 +4 +5
select +xc_node_id, count(1) +from tablename +group by xc_node_id +order by xc_node_id desc; +
xc_node_id corresponds to a DN. Generally, over 5% difference between the amount of data on different DNs is regarded as data skew. If the difference is over 10%, choose another distribution key.
+ - You are not advised to add a column as a distribution key, especially add a new column and use the SEQUENCE value to fill the column. (Sequences may cause performance bottlenecks and unnecessary maintenance costs.)
+View the data in the table.
- Run the following command to query information about all tables in a database in the system catalog pg_tables:++ +
1SELECT * FROM pg_tables; +
- Run the \d+ command of the gsql tool to query table attributes:++ +
1\d+ customer_t1; +
- Run the following command to query the data volume of table customer_t1:++ +
1SELECT count(*) FROM customer_t1; +
- Run the following command to query all data in table customer_t1:++ +
1SELECT * FROM customer_t1; +
- Run the following command to query data in column c_customer_sk:++ +
1SELECT c_customer_sk FROM customer_t1; +
- Run the following command to filter repeated data in column c_customer_sk:++ +
1SELECT DISTINCT( c_customer_sk ) FROM customer_t1; +
- Run the following command to query all data whose column c_customer_sk is 3869:++ +
1SELECT * FROM customer_t1 WHERE c_customer_sk = 3869; +
- Run the following command to sort data based on column c_customer_sk.++ +
1SELECT * FROM customer_t1 ORDER BY c_customer_sk; +
+Deleting Data in a Table
You can delete outdated data from a table by row.
+SQL statements can only access and delete an independent row by declaring conditions that match the row. If a table has a primary key column, you can use it to specify a row. You can delete several rows that match the specified condition or delete all the rows from a table.
+- For example, to delete all the rows whose c_customer_sk column is 3869 from table customer_t1, run the following statement:++ +
1DELETE FROM customer_t1 WHERE c_customer_sk = 3869; +
- To delete all rows from the table, run either of the following statements:++ +
1DELETE FROM customer_t1; +
++ +1TRUNCATE TABLE customer_t1; +
+ If you need to delete an entire table, you are advised to use the TRUNCATE statement rather than DELETE. Using TRUNCATE to delete a large volume of data is much faster than using DELETE.
+- Delete the created table.++ +
1DROP TABLE customer_t1; +
++ diff --git a/docs/dws/dev/dws_04_0034.html b/docs/dws/dev/dws_04_0034.html new file mode 100644 index 00000000..6d026995 --- /dev/null +++ b/docs/dws/dev/dws_04_0034.html @@ -0,0 +1,133 @@ + + +++Parent topic: Defining Database Objects+Viewing a System Catalog
++In addition to the created tables, a database contains many system catalogs These system catalogs contain cluster installation information and information about various queries and processes in GaussDB(DWS). You can collect information about the database by querying the system catalog.
++Querying Database Tables
For example, query the PG_TABLES system catalog for all tables in the public schema.
+++ +1SELECT distinct(tablename) FROM pg_tables WHERE SCHEMANAME = 'public'; +
Information similar to the following is displayed:
+++ +1 +2 +3 +4 +5 +6 +7 +8 +9
tablename +------------------- + err_hr_staffs + test + err_hr_staffs_ft3 + web_returns_p1 + mig_seq_table + films4 +(6 rows) +
+Viewing Database Users
You can run the PG_USER command to view the list of all users in the database, and view the user ID (USESYSID) and permissions.
+++ +1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10
SELECT * FROM pg_user; +usename | usesysid | usecreatedb | usesuper | usecatupd | userepl | passwd | valbegin | valuntil | respool | parent | spacelimit | useconfig | nodegroup | tempspacelimit | spillspacelim +it +---------+----------+-------------+----------+-----------+---------+----------+----------+----------+--------------+--------+------------+-----------+-----------+----------------+-------------- +--- + Ruby | 10 | t | t | t | t | ******** | | | default_pool | 0 | | | | | + dbadmin | 16393 | f | f | f | f | ******** | | | default_pool | 0 | | | | | + lily | 16691 | f | f | f | f | ******** | | | default_pool | 0 | | | | | + jack | 70694 | f | f | f | f | ******** | | | default_pool | 0 | | | | | +(4 rows) +
GaussDB(DWS) uses Ruby to perform routine management and maintenance. You can add WHERE usesysid > 10 to the SELECT statement to filter queries so that only specified user names are displayed.
+++ +1 +2 +3 +4 +5 +6 +7 +8 +9
SELECT * FROM pg_user WHERE usesysid > 10; + usename | usesysid | usecreatedb | usesuper | usecatupd | userepl | passwd | valbegin | valuntil | respool | parent | spacelimit | useconfig | nodegroup | tempspacelimit | spillspacelim +it +---------+----------+-------------+----------+-----------+---------+----------+----------+----------+--------------+--------+------------+-----------+-----------+----------------+-------------- +--- + dbadmin | 16393 | f | f | f | f | ******** | | | default_pool | 0 | | | | | + lily | 16691 | f | f | f | f | ******** | | | default_pool | 0 | | | | | + jack | 70694 | f | f | f | f | ******** | | | default_pool | 0 | | | | | +(3 rows) +
+Viewing and Stopping the Running Query Statements
You can view the running query statements in the PG_STAT_ACTIVITY view. Do as follows:
+- Set the parameter track_activities to on.++ +
1SET track_activities = on; +
The database collects the running information about active queries only if the parameter is set to on.
+ - View the running query statements. Run the following command to view the database names, users, query statuses, and PIDs of the running query statements:++ +
1SELECT datname, usename, state,pid FROM pg_stat_activity; +
If the state column is idle, the connection is idle and requires a user to enter a command.
+To identify only active query statements, run the following command:
+++ +1SELECT datname, usename, state FROM pg_stat_activity WHERE state != 'idle'; +
- To cancel queries that have been running for a long time, use the PG_TERMINATE_BACKEND function to end sessions based on the thread ID.++ +
1SELECT PG_TERMINATE_BACKEND(139834759993104); +
If information similar to the following is displayed, the session is successfully terminated:
+++ +1 +2 +3 +4
PG_TERMINATE_BACKEND +---------------------- + t +(1 row) +
If information similar to the following is displayed, a user has terminated the current session.+++ +1 +2
FATAL: terminating connection due to administrator command +FATAL: terminating connection due to administrator command +
+ If the PG_TERMINATE_BACKEND function is used to terminate the backend threads of the current session, the gsql client will be reconnected automatically rather than be logged out. The message "The connection to the server was lost." is returned. Attempting reset: Succeeded."
+++ +1 +2 +3
FATAL: terminating connection due to administrator command +FATAL: terminating connection due to administrator command +The connection to the server was lost. Attempting reset: Succeeded. +
++ diff --git a/docs/dws/dev/dws_04_0036.html b/docs/dws/dev/dws_04_0036.html new file mode 100644 index 00000000..5e691b62 --- /dev/null +++ b/docs/dws/dev/dws_04_0036.html @@ -0,0 +1,133 @@ + + +++Parent topic: Defining Database Objects+Creating and Managing Schemas
++A schema is the logical organization of objects and data in a database. Schema management allows multiple users to use the same database without interfering with each other. Third-party applications can be added to corresponding schemas to avoid conflicts.
+The same database object name can be used in different schemas in a database without causing conflicts. For example, both a_schema and b_schema can contain a table named mytable. Users with required permissions can access objects across multiple schemas in a database.
+If a user is created, a schema named after the user will also be created in the current database.
++The Default Schema Public
Each database has a schema named Public. If you do not create any schema, the object will be created in the schema named public. All database roles (users) have the CREATE and USAGE permissions in the public schema. When creating a schema, you need to grant the access permission to users.
++Creating a Schema
- Run the CREATE SCHEMA command to create a schema.++ +
1CREATE SCHEMA myschema; +
To create or access an object in the schema, the object name in the command should be composed of the schema name and the object name, which are separated by a dot (.), for example, myschema.table.
+ - Users can create a schema owned by others. For example, run the following command to create a schema named myschema and set the owner of the schema to user jack:++ +
1CREATE SCHEMA myschema AUTHORIZATION jack; +
If authorization username is not specified, the schema owner is the user who runs the command.
+
+Modifying a Schema
- Run the ALTER SCHEMA command to change the schema name. Only the schema owner can change the schema name.++ +
1ALTER SCHEMA schema_name RENAME TO new_name; +
- Run the ALTER SCHEMA command to change the schema owner.++ +
1ALTER SCHEMA schema_name OWNER TO new_owner; +
+Setting the Schema Search Path
The GUC parameter search_path specifies the schema search sequence. The parameter value is a series of schema names separated by commas (,). If no schema is specified during object creation, the object will be added to the first schema displayed in the search path. If there are objects with the same name in different schemas and no schema is specified for an object query, the object will be returned from the first schema containing the object in the search path.
+- Run the SHOW command to view the current search path.++ +
1 +2 +3 +4 +5
SHOW SEARCH_PATH; + search_path +---------------- + "$user",public +(1 row) +
The default value of search_path is "$user",public. $user indicates the name of the schema with the same name as the current session user. If the schema does not exist, $user will be ignored. By default, after a user connects to a database that has schemas with the same name, objects will be added to all the schemas. If there are no such schemas, objects will be added to only to the public schema.
+ - Run the SET command to modify the default schema of the current session. For example, if the search path is set to "myschema, public", myschema is searched first.++ +
1SET SEARCH_PATH TO myschema, public; +
You can also run the ALTER ROLE command to set search_path for a role (user). For example:+++ +1ALTER ROLE jack SET search_path TO myschema, public; +
+Using a Schema
If you want to create or access an object in a specified schema, the object name must contain the schema name. To be specific, the name consists of a schema name and an object name, which are separated by a dot (.).
+- Create a table mytable in myschema. Create a table in schema_name.table_name format.++ +
1CREATE TABLE myschema.mytable(id int, name varchar(20)); +
- Query all data in the table mytable in myschema.++ +
1 +2 +3 +4
SELECT * FROM myschema.mytable; + id | name +----+------ +(0 rows) +
+Viewing a Schema
- Use the current_schema() function to view the current schema.++ +
1 +2 +3 +4 +5
SELECT current_schema(); + current_schema +---------------- + myschema +(1 row) +
- To view the owner of a schema, perform the following join query on the system catalogs PG_NAMESPACE and PG_USER. Replace schema_name in the statement with the name of the schema to be queried.++ +
1SELECT s.nspname,u.usename AS nspowner FROM PG_NAMESPACE s, PG_USER u WHERE nspname='schema_name' AND s.nspowner = u.usesysid; +
- To view a list of all schemas, query the system catalog PG_NAMESPACE.++ +
1SELECT * FROM PG_NAMESPACE; +
- Use the PGXC_TOTAL_SCHEMA_INFO view to query the space usage of schemas in the cluster.++ +
1SELECT * FROM PGXC_TOTAL_SCHEMA_INFO; +
- To view a list of tables in a schema, query the system catalog PG_TABLES. For example, the following query will return a table list from PG_CATALOG in the schema.++ +
1SELECT distinct(tablename),schemaname FROM PG_TABLES where schemaname = 'pg_catalog'; +
+Schema Permission Control
By default, a user can only access database objects in its own schema. To access objects in other schemas, the user must have the usage permission of the corresponding schema.
+By granting the CREATE permission for a schema to a user, the user can create objects in this schema.
+- Grant the usage permission of myschema to user jack.++ +
1GRANT USAGE ON schema myschema TO jack; +
- Run the following command to revoke the USAGE permission for myschema from jack:++ +
1REVOKE USAGE ON schema myschema FROM jack; +
+Drop Schema
- Run the DROP SCHEMA command to delete an empty schema (no database objects in the schema).++ +
1DROP SCHEMA IF EXISTS myschema; +
- By default, a schema must be empty before being deleted. To delete a schema and all its objects (such as tables, data, and functions), use the CASCADE keyword.++ +
1DROP SCHEMA myschema CASCADE; +
+System Schema
- Each database has a pg_catalog schema, which contains system catalogs and all built-in data types, functions, and operators. pg_catalog is a part of the search path and has the second highest search priority. It is searched after the schema of temporary tables and before other schemas specified in search_path. This search order ensures that database built-in objects can be found. To use a custom object that has the same name as a built-in object, you can specify the schema of the custom object.
- The information_schema consists of a collection of views that contain object information in a database. These views obtain system information from the system catalogs in a standardized way.
++ diff --git a/docs/dws/dev/dws_04_0037.html b/docs/dws/dev/dws_04_0037.html new file mode 100644 index 00000000..282d2fba --- /dev/null +++ b/docs/dws/dev/dws_04_0037.html @@ -0,0 +1,216 @@ + + +++Parent topic: Defining Database Objects+Defining Table Partitions
++Partitioning refers to splitting what is logically one large table into smaller physical pieces based on specific schemes. The table based on the logic is called a partition cable, and a physical piece is called a partition. Data is stored on these smaller physical pieces, namely, partitions, instead of the larger logical partitioned table. During conditional query, the system scans only the partitions that meet the conditions rather than scanning the entire table improving query performance.
+Advantages of partitioned tables:
+- Improved query performance. You can search in specific partitions, improving the search efficiency.
- Enhanced availability. If a partition is faulty, data in other partitions is still available.
- Improved maintainability. For expired historical data that needs to be periodically deleted, you can quickly delete it by dropping or truncate partitions.
+Supported Table Partition Types
- Range partitioning: partitions are created based on a numeric range, for example, by date or price range.
- List partitioning: partitions are created based on a list of values, such as sales scope or product attribute. Only clusters of 8.1.3 and later versions support this function.
+Choosing to Partition a Table
You can choose to partition a table when the table has the following characteristics:
+- There are obvious ranges among the fields of the table.
A table is partitioned based on obvious rangeable fields. Generally, columns such as date, area, and value are used for partitioning. The time column is most commonly used.
+ - Queries to the table have obvious range characteristics.
If the queried data fall into specific ranges, its better tables are partitioned so that through partition pruning, only the queried partition needs to be scanned, improving data scanning efficiency and reducing the I/O overhead of data scanning.
+ - The table contains a large amount of data.
Scanning small tables does not take much time, therefore the performance benefits of partitioning are not significant. Therefore, you are advised to partition only large tables. In column-store table, each column is an independent file storage unit, and the minimum storage unit CU can store 60,000 rows of data. Therefore, for column-store partitioned tables, it is recommended that the data volume in each partition be greater than or equal to the number of DNs multiplied by 60,000.
+
+Creating a Range Partitioned Table
Example: Create a table web_returns_p1 partitioned by the range wr_returned_date_sk.
+++ +1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17
CREATE TABLE web_returns_p1 +( + wr_returned_date_sk integer, + wr_returned_time_sk integer, + wr_item_sk integer NOT NULL, + wr_refunded_customer_sk integer +) +WITH (orientation = column) +DISTRIBUTE BY HASH (wr_item_sk) +PARTITION BY RANGE (wr_returned_date_sk) +( + PARTITION p2016 VALUES LESS THAN(20161231), + PARTITION p2017 VALUES LESS THAN(20171231), + PARTITION p2018 VALUES LESS THAN(20181231), + PARTITION p2019 VALUES LESS THAN(20191231), + PARTITION pxxxx VALUES LESS THAN(maxvalue) +); +
Create partitions in batches, with fixed partition ranges. The following example can be used:
+++ +1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14
CREATE TABLE web_returns_p2 +( + wr_returned_date_sk integer, + wr_returned_time_sk integer, + wr_item_sk integer NOT NULL, + wr_refunded_customer_sk integer +) +WITH (orientation = column) +DISTRIBUTE BY HASH (wr_item_sk) +PARTITION BY RANGE(wr_returned_date_sk) +( + PARTITION p2016 START(20161231) END(20191231) EVERY(10000), + PARTITION p0 END(maxvalue) +); +
+Creating a List Partitioned Table
A list partitioned table can use any column that allows value comparison as the partition key column. When creating a list partitioned table, you must declare the value partition for each partition.
+Example: Create a list partitioned table sales_info.
+++ +1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15
CREATE TABLE sales_info +( +sale_time timestamptz, +period int, +city text, +price numeric(10,2), +remark varchar2(100) +) +DISTRIBUTE BY HASH(sale_time) +PARTITION BY LIST (period, city) +( +PARTITION province1_202201 VALUES (('202201', 'city1'), ('202201', 'city2')), +PARTITION province2_202201 VALUES (('202201', 'city3'), ('202201', 'city4'), ('202201', 'city5')), +PARTITION rest VALUES (DEFAULT) +); +
+Partitioning an Existing Table
A table can be partitioned only when it is created. If you want to partition a table, you must create a partitioned table, load the data in the original table to the partitioned table, delete the original table, and rename the partitioned table as the name of the original table. You must also re-grant permissions on the table to users. For example:
+++ +1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14
CREATE TABLE web_returns_p2 +( + wr_returned_date_sk integer, + wr_returned_time_sk integer, + wr_item_sk integer NOT NULL, + wr_refunded_customer_sk integer +) +WITH (orientation = column) +DISTRIBUTE BY HASH (wr_item_sk) +PARTITION BY RANGE(wr_returned_date_sk) +( + PARTITION p2016 START(20161231) END(20191231) EVERY(10000), + PARTITION p0 END(maxvalue) +); +
++ +1 +2 +3 +4 +5
INSERT INTO web_returns_p2 SELECT * FROM web_returns_p1; +DROP TABLE web_returns_p1; +ALTER TABLE web_returns_p2 RENAME TO web_returns_p1; +GRANT ALL PRIVILEGES ON web_returns_p1 TO dbadmin; +GRANT SELECT ON web_returns_p1 TO jack; +
+Adding a Partition
Run the ALTER TABLE statement to add a partition to a partitioned table. For example, to add partition P2020 to the web_returns_p1 table, run the following command:
+++ +1ALTER TABLE web_returns_p1 ADD PARTITION P2020 VALUES LESS THAN (20201231); +
+Splitting a Partition
The syntax for splitting a partition varies between a range partitioned table and a list partitioned table.
+- Run the ALTER TABLE statement to split a partition in a range partitioned table. For example, the partition pxxxx of the table web_returns_p1 is split into two partitions p2020 and p20xx at the splitting point 20201231.++ +
1ALTER TABLE web_returns_p1 SPLIT PARTITION pxxxx AT(20201231) INTO (PARTITION p2020,PARTITION p20xx); +
- Run the ALTER TABLE statement to split a partition in a list partitioned table. For example, split the partition province2_202201 of table sales_inf into two partitions province3_202201 and province4_202201.++ +
1ALTER TABLE sales_info SPLIT PARTITION province2_202201 VALUES(('202201', 'city5')) INTO (PARTITION province3_202201,PARTITION province4_202201); +
+Merging Partitions
Run the ALTER TABLE statement to merge two partitions in a partitioned table. For example, merge partitions p2016 and p2017 of table web_returns_p1 into one partition p20162017.
+++ +1ALTER TABLE web_returns_p1 MERGE PARTITIONS p2016,p2017 INTO PARTITION p20162017; +
+Deleting a Partition
Run the ALTER TABLE statement to delete a partition from a partitioned table. For example, run the following command to delete partition P2020 from the web_returns_p1 table:+++ +1ALTER TABLE web_returns_p1 DROP PARTITION P2020; +
+Querying a Partition
- Query partition p2019.++ +
1 +2
SELECT * FROM web_returns_p1 PARTITION (p2019); +SELECT * FROM web_returns_p1 PARTITION FOR (20201231); +
- View partitioned tables using the system catalog dba_tab_partitions.++ +
1SELECT * FROM dba_tab_partitions where table_name='web_returns_p1'; +
+Deleting a Partitioned Table
Run the DROP TABLE statement to delete a partitioned table.
+++ +1DROP TABLE web_returns_p1; +
++ diff --git a/docs/dws/dev/dws_04_0038.html b/docs/dws/dev/dws_04_0038.html new file mode 100644 index 00000000..80c3abf3 --- /dev/null +++ b/docs/dws/dev/dws_04_0038.html @@ -0,0 +1,124 @@ + + +++Parent topic: Defining Database Objects+Creating and Managing Indexes
++Indexes accelerate the data access speed but also add the processing time of the insert, update, and delete operations. Therefore, before creating an index, consider whether it is necessary and determine the columns where indexes will be created. You can determine whether to add an index for a table by analyzing the service processing and data use of applications, as well as columns that are frequently used as search criteria or need to be sorted.
++Index type
- btree: The B-tree index uses a structure that is similar to the B+ tree structure to store data key values, facilitating index search. btree supports comparison queries with ranges specified.
- gin: GIN indexes are reverse indexes and can process values that contain multiple keys (for example, arrays).
- gist: GiST indexes are suitable for the set data type and multidimensional data types, such as geometric and geographic data types.
- Psort: psort index. It is used to perform partial sort on column-store tables.
Row-based tables support the following index types: btree (default), gin, and gist. Column-based tables support the following index types: Psort (default), btree, and gin.
++ Create a B-tree index for point queries.
++Index Selection Principles
Indexes are created based on columns in database tables. When creating indexes, you need to determine the columns, which can be:
+- Columns that are frequently searched: The search efficiency can be improved.
- The uniqueness of the columns and the data sequence structures is ensured.
- Columns that usually function as foreign keys and are used for connections. Then the connection efficiency is improved.
- Columns that are usually searched for by a specified scope. These indexes have already been arranged in a sequence, and the specified scope is contiguous.
- Columns that need to be arranged in a sequence. These indexes have already been arranged in a sequence, so the sequence query time is accelerated.
- Columns that usually use the WHERE clause. Then the condition decision efficiency is increased.
- Fields that are frequently used after keywords, such as ORDER BY, GROUP BY, and DISTINCT.+
- After an index is created, the system automatically determines when to reference it. If the system determines that indexing is faster than sequenced scanning, the index will be used.
- After an index is successfully created, it must be synchronized with the associated table to ensure new data can be accurately located. Therefore, data operations increase. Therefore, delete unnecessary indexes periodically.
+Creating an Index
GaussDB(DWS) supports four methods for creating indexes. For details, see Table 1.
++ + - After an index is created, the system automatically determines when to reference it. If the system determines that indexing is faster than sequenced scanning, the index will be used.
- After an index is successfully created, it must be synchronized with the associated table to ensure new data can be accurately located. Therefore, data operations increase. Therefore, delete unnecessary indexes periodically.
+
+Table 1 Indexing Method Indexing Method
+Description
+Unique index
+Refers to an index that constrains the uniqueness of an index attribute or an attribute group. If a table declares unique constraints or primary keys, GaussDB(DWS) automatically creates unique indexes (or composite indexes) for columns that form the primary keys or unique constraints. Currently, only B-tree can create a unique index in GaussDB(DWS).
+Composite index
+Refers to an index that can be defined for multiple attributes of a table. Currently, composite indexes can be created only for B-tree in GaussDB(DWS) and a maximum of 32 columns can share a composite index.
+Partial index
+Refers to an index that can be created for subsets of a table. This indexing method contains only tuples that meet condition expressions.
+Expression index
+Refers to an index that is built on a function or an expression calculated based on one or more attributes of a table. An expression index works only when the queried expression is the same as the created expression.
+- Run the following command to create an ordinary table:++ +
1CREATE TABLE tpcds.customer_address_bak AS TABLE tpcds.customer_address; +
- Create a common index.You need to query the following information in the tpcds.customer_address_bak table:+++ +
1SELECT ca_address_sk FROM tpcds.customer_address_bak WHERE ca_address_sk=14888; +
Generally, the database system needs to scan the tpcds.customer_address_bak table row by row to find all matched tuples. If the size of the tpcds.customer_address_bak table is large but only a few (possibly zero or one) of the WHERE conditions are met, the performance of this sequential scan is low. If the database system uses an index to maintain the ca_address_sk attribute, the database system only needs to search a few tree layers for the matched tuples. This greatly improves data query performance. Furthermore, indexes can improve the update and delete operation performance in the database.
+Run the following command to create an index:
+++ +1CREATE INDEX index_wr_returned_date_sk ON tpcds.customer_address_bak (ca_address_sk); +
- Create a multi-column index.Assume you need to frequently query records with ca_address_sk being 5050 and ca_street_number smaller than 1000 in the tpcds.customer_address_bak table. Run the following command:+++ +
1SELECT ca_address_sk,ca_address_id FROM tpcds.customer_address_bak WHERE ca_address_sk = 5050 AND ca_street_number < 1000; +
Run the following command to define a multiple-column index on ca_address_sk and ca_street_number columns:+++ +1CREATE INDEX more_column_index ON tpcds.customer_address_bak(ca_address_sk ,ca_street_number ); +
- Create a partition index.
If you only want to find records whose ca_address_sk is 5050, you can create a partial index to facilitate your query.
+++ +1CREATE INDEX part_index ON tpcds.customer_address_bak(ca_address_sk) WHERE ca_address_sk = 5050; +
- Create an expression index.Assume you need to frequently query records with ca_street_number smaller than 1000, run the following command:+++ +
1SELECT * FROM tpcds.customer_address_bak WHERE trunc(ca_street_number) < 1000; +
The following expression index can be created for this query task:+++ +1CREATE INDEX para_index ON tpcds.customer_address_bak (trunc(ca_street_number)); +
+Querying an Index
- Run the following command to query all indexes defined by the system and users:++ +
1SELECT RELNAME FROM PG_CLASS WHERE RELKIND='i'; +
- Run the following command to query information about a specified index:++ +
1\di+ index_wr_returned_date_sk +
+Recreating an Index
- Recreate the index index_wr_returned_date_sk.++ +
1REINDEX INDEX index_wr_returned_date_sk; +
- Recreate all indexes of a table.++ +
1REINDEX TABLE tpcds.customer_address_bak; +
+Deleting an Index
You can use the DROP INDEX statement to delete indexes.+++ +1DROP INDEX index_wr_returned_date_sk; +
++ diff --git a/docs/dws/dev/dws_04_0039.html b/docs/dws/dev/dws_04_0039.html new file mode 100644 index 00000000..344481ff --- /dev/null +++ b/docs/dws/dev/dws_04_0039.html @@ -0,0 +1,65 @@ + + +++Parent topic: Defining Database Objects+Creating and Managing Views
++Views allow users to save queries. Views are not physically stored on disks. Queries to a view run as subqueries. A database only stores the definition of a view and does not store its data. The data is still stored in the original base table. If data in the base table changes, the data in the view changes accordingly. In this sense, a view is like a window through which users can know their interested data and data changes in the database. A view is triggered every time it is referenced.
++Creating a view
Run the CREATE VIEW command to create a view.+++ +1CREATE OR REPLACE VIEW MyView AS SELECT * FROM tpcds.customer WHERE c_customer_sk < 150; +
+ The OR REPLACE parameter in this command is optional. It indicates that if the view exists, the new view will replace the existing view.
++View Details
- View the MyView view. Real-time data will be returned.++ +
1SELECT * FROM myview; +
- Run the following command to query the views in the current user:++ +
1SELECT * FROM user_views; +
- Run the following command to query all views:++ +
1SELECT * FROM dba_views; +
- View details about a specified view.Run the following command to view details about the dba_users view:+++ +
1 +2 +3 +4 +5 +6 +7 +8
\d+ dba_users + View "PG_CATALOG.DBA_USERS" + Column | Type | Modifiers | Storage | Description +----------+-----------------------+-----------+----------+------------- + USERNAME | CHARACTER VARYING(64) | | extended | +View definition: + SELECT PG_AUTHID.ROLNAME::CHARACTER VARYING(64) AS USERNAME + FROM PG_AUTHID; +
+Rebuilding a View
Run the ALTER VIEW command to rebuild a view without entering query statements.
+++ +1ALTER VIEW myview REBUILD; +
+Deleting a View
Run the DROP VIEW command to delete a view.+++ +1DROP VIEW myview; +
DROP VIEW ... The CASCADE command can be used to delete objects that depend on the view. For example, view A depends on view B. If view B is deleted, view A will also be deleted. Without the CASCADE option, the DROP VIEW command will fail.
+++ diff --git a/docs/dws/dev/dws_04_0040.html b/docs/dws/dev/dws_04_0040.html new file mode 100644 index 00000000..aa72f17e --- /dev/null +++ b/docs/dws/dev/dws_04_0040.html @@ -0,0 +1,95 @@ + + +++Parent topic: Defining Database Objects+Creating and Using Sequences
++A sequence is a database object that generates unique integers according to a certain rule and is usually used to generate primary key values.
+You can create a sequence for a column in either of the following methods:+- Set the data type of a column to sequence integer. A sequence will be automatically created by the database for this column.
- Use CREATE SEQUENCE to create a new sequenc. Use the nextval('sequence_name') function to increment the sequence and return a new value. Specify the default value of the column as the sequence value returned by the nextval('sequence_name') function. In this way, this column can be used as a unique identifier.
+Creating a Sequence.
Method 1: Set the data type of a column to a sequence integer. For example:+++ +1 +2 +3 +4 +5
CREATE TABLE T1 +( + id serial, + name text +); +
Method 2: Create a sequence and set the initial value of the nextval('sequence_name') function to the default value of a column. You can cache a specific number of sequence values to reduce the requests to the GTM, improving the performance.
+- Create a sequence.++ +
1CREATE SEQUENCE seq1 cache 100; +
- Set the initial value of the nextval('sequence_name') function to the default value of a column.++ +
1 +2 +3 +4 +5
CREATE TABLE T2 +( + id int not null default nextval('seq1'), + name text +); +
+ Methods 1 and 2 are similar except that method 2 specifies cache for the sequence. A sequence using cache has holes (non-consecutive values, for example, 1, 4, 5) and cannot keep the order of the values. After a sequence is deleted, its sub-sequences will be deleted automatically. A sequence shared by multiple columns is not forbidden in a database, but you are not advised to do that.
+Currently, the preceding two methods cannot be used for existing tables.
++Modifying a Sequence
The ALTER SEQUENCE statement changes the attributes of an existing sequence, including the owner, owning column, and maximum value.
+ ++Deleting a Sequence
Run the DROP SEQUENCE command to delete a sequence. For example, to delete the sequence named seq1, run the following command:
+++ +1DROP SEQUENCE seq1; +
+Precautions
Sequence values are generated by the GTM. By default, each request for a sequence value is sent to the GTM. The GTM calculates the result of the current value plus the step and then returns the result. As GTM is a globally unique node, generating default sequence numbers can cause performance issues. For operations that need frequent sequence number generation, such as bulkload data import, this is not recommended. For example, the INSERT FROM SELECT statement has poor performance in the following scenario:
+++ +1 +2 +3 +4 +5 +6 +7
CREATE SEQUENCE newSeq1; +CREATE TABLE newT1 + ( + id int not null default nextval('newSeq1'), + name text + ); +INSERT INTO newT1(name) SELECT name from T1; +
To improve the performance, run the following statements (assume that data of 10,000 rows will be imported from T1 to newT1):
+++ +1 +2
INSERT INTO newT1(id, name) SELECT id,name from T1; +SELECT SETVAL('newSeq1',10000); +
+ Rollback is not supported by sequence functions, including nextval() and setval(). The value of the setval function immediately takes effects on nextval in the current session in any cases and takes effects in other sessions only when no cache is specified for them. If cache is specified for a session, it takes effect only after all the cached values have been used. To avoid duplicate values, use setval only when necessary. Do not set it to an existing sequence value or a cached sequence value.
+If BulkLoad is used, set sufficient cache for newSeq1 and do not set Maxvalue or Minvalue. To improve the performance, database may push down the invocation of nextval('sequence_name') to DNs. Currently, the concurrent connection requests that can be processed by the GTM are limited. If there are too many DNs, a large number of concurrent connection requests will be sent to the GTM. In this case, you need to limit the concurrent connection of BulkLoad to save the GTM connection resources. If the target table is a replication table (DISTRIBUTE BY REPLICATION), pushdown cannot be performed. If the data volume is large, this will be a disaster for the database. In addition, the database space may be exhausted. After the import is complete, do VACUUM FULL. Therefore, you are not advised to use sequences when BulkLoad is used.
+After a sequence is created, a single-row table is maintained on each node to store the sequence definition and value, which is obtained from the last interaction with the GTM rather than updated in real time. The single-row table on a node does not update when other nodes request a new value from the GTM or when the sequence is modified using setval.
+++ diff --git a/docs/dws/dev/dws_04_0041.html b/docs/dws/dev/dws_04_0041.html new file mode 100644 index 00000000..445bf616 --- /dev/null +++ b/docs/dws/dev/dws_04_0041.html @@ -0,0 +1,184 @@ + + +++Parent topic: Defining Database Objects+Creating and Managing Scheduled Tasks
++GaussDB(DWS) allows users to create scheduled tasks, which are automatically executed at specified time points, reducing O&M workload.
+Database complies with the Oracle scheduled task function using the DBMS.JOB interface, which can be used to create scheduled tasks, execute tasks automatically, delete a task, and modify task attributes(including task ID, enable/disable a task, the task triggering time/interval and task contents).
++ - The hybrid data warehouse (standalone) does not support scheduled tasks.
- The execution statements of scheduled tasks are not recorded in the Real-time Top SQL logs. The statements can be recorded only in versions later than 8.2.1.
- By default, GaussDB(DWS) uses the UTC time. The execution time of the scheduled task needs to be converted to the time zone of the user.
+Periodic Task Management
- Creates a test table.++ +
1CREATE TABLE test(id int, time date); +
If the following information is displayed, the table has been created.
+++ +1CREATE TABLE +
- Create the customized storage procedure.++ +
1 +2 +3 +4 +5 +6 +7 +8 +9
CREATE OR REPLACE PROCEDURE PRC_JOB_1() +AS +N_NUM integer :=1; +BEGIN +FOR I IN 1..1000 LOOP +INSERT INTO test VALUES(I,SYSDATE); +END LOOP; +END; +/ +
If the following information is displayed, the procedure has been created.
+++ +1CREATE PROCEDURE +
- Create a task.
- Create a task with unspecified job_id and execute the PRC_JOB_1 storage procedure every two minutes.++ +
1 +2 +3 +4 +5
call dbms_job.submit('call public.prc_job_1(); ', sysdate, 'interval ''1 minute''', :a); +job +----- +1 +(1 row) +
- Create task with specified job_id.++ +
1 +2 +3 +4 +5
call dbms_job.isubmit(2,'call public.prc_job_1(); ', sysdate, 'interval ''1 minute'''); +isubmit +--------- + +(1 row) +
- Create a task with unspecified job_id and execute the PRC_JOB_1 storage procedure every two minutes.
- View the created task information about the current user in the USER_JOBS view.
Only the system administrator can access this system view. For details about the fields, see Table 1.
+++ +1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10
select job,dbname,start_date,last_date,this_date,next_date,broken,status,interval,failures,what from user_jobs; + job | dbname | start_date | last_date | this_date | next_date | broken | status | interval | failures | what + +-----+----------+----------------------------+----------------------------+----------------------------+---------------------+--------+--------+---------------------+----------+---------------- +----------- + 1 | db_demo | 2022-03-25 07:58:01.829436 | 2022-03-25 07:58:03.174817 | 2022-03-25 07:58:01.829436 | 2022-03-25 07:59:01 | n | s | interval '1 minute' | 0 | call public.prc +_job_1(); + 2 | db_demo | 2022-03-25 07:58:15.893383 | 2022-03-25 07:58:16.608959 | 2022-03-25 07:58:15.893383 | 2022-03-25 07:59:15 | n | s | interval '1 minute' | 0 | call public.prc +_job_1(); +(2 rows) +
- Stop a task.++ +
1 +2 +3 +4 +5
call dbms_job.broken(1,true); +broken +-------- + +(1 row) +
- Start a task.++ +
1 +2 +3 +4 +5
call dbms_job.broken(1,false); +broken +-------- + +(1 row) +
- Modify attributes of a task.
- Modify the Next_date parameter information about a task. For example, change the value of Next_date of Job1 to 1 hour.++ +
1 +2 +3 +4 +5
call dbms_job.next_date(1, sysdate+1.0/24); +next_date +----------- + +(1 row) +
- Modify the Interval parameter information of a task. For example, change the value of Interval of Job1 to 1 hour.++ +
1 +2 +3 +4 +5
call dbms_job.interval(1,'sysdate + 1.0/24'); +interval +---------- + +(1 row) +
- Modify the What parameter information of a JOB. For example, change What of Job1 to insert into public.test values(333, sysdate+5).++ +
1 +2 +3 +4 +5
call dbms_job.what(1,'insert into public.test values(333, sysdate+5);'); +what +------ + +(1 row) +
- Modify Next_date, Interval, and What parameter information of JOB.++ +
1 +2 +3 +4 +5
call dbms_job.change(1, 'call public.prc_job_1();', sysdate, 'interval ''1 minute'''); +change +-------- + +(1 row) +
- Modify the Next_date parameter information about a task. For example, change the value of Next_date of Job1 to 1 hour.
- Delete a job.++ +
1 +2 +3 +4 +5
call dbms_job.remove(1); +remove +-------- + +(1 row) +
- Set job permissions.
- During the creation of a job, the job is bound to the user and database that created the job. Accordingly, the user and database are added to dbname and log_user columns in the pg_job system view, respectively.
- If the current user is a DBA user, system administrator, or the user who created the job (log_user in pg_job), the user has the permissions to delete or modify parameter settings of the job using the remove, change, next_data, what, or interval interface. Otherwise, the system displays a message indicating that the current user has no permission to perform operations on the JOB.
- If the current database is the one that created a job, (that is, dbname in pg_job), you can delete or modify parameter settings of the job using the remove, change, next_data, what, or interval interface.
- When deleting the database that created a job, (that is, dbname in pg_job), the system associatively deletes the job records of the database.
- When deleting the user who created a job, (that is, log_user in pg_job), the system associatively deletes the job records of the user.
++ diff --git a/docs/dws/dev/dws_04_0042.html b/docs/dws/dev/dws_04_0042.html index faa3efb3..c7d3b69a 100644 --- a/docs/dws/dev/dws_04_0042.html +++ b/docs/dws/dev/dws_04_0042.html @@ -1,246 +1,284 @@ - +++Parent topic: Defining Database Objects+Syntax Compatibility Differences Among Oracle, Teradata, and MySQL
-GaussDB(DWS) is compatible with Oracle, Teradata and MySQL syntax, of which the syntax behavior is different.
+GaussDB(DWS) is compatible with Oracle, Teradata and MySQL syntax, of which the syntax behavior is different.
-Table 1 Compatibility differences Compatibility Item
+-Table 1 Compatibility differences Compatibility Item
Oracle
+Oracle
Teradata
+Teradata
MySQL
+MySQL
Empty string
+Empty string
An empty string is treated as NULL.
+Only null is available.
An empty string is distinguished from NULL.
+An empty string is distinguished from null.
An empty string is distinguished from NULL.
+An empty string is distinguished from null.
Conversion of an empty string to a number
+Conversion of an empty string to a number
NULL
+null
0
+0
0
+0
Automatic truncation of overlong characters
+Automatic truncation of overlong characters
Not supported
+Not supported
Supported (set GUC parameter td_compatible_truncation to ON)
+Supported (set GUC parameter td_compatible_truncation to ON)
Not supported
+Not supported
NULL concatenation
+null concatenation
Returns a non-NULL object after combining a non-NULL object with NULL.
-For example, 'abc'||NULL returns 'abc'.
+Returns a non-null object after combining a non-null object with null.
+For example, 'abc'||null returns 'abc'.
The strict_text_concat_td option is added to the GUC parameter behavior_compat_options to be compatible with the Teradata behavior. After the NULL type is concatenated, NULL is returned.
-For example, 'abc'||NULL returns NULL.
+The strict_text_concat_td option is added to the GUC parameter behavior_compat_options to be compatible with the Teradata behavior. After the null type is concatenated, null is returned.
+For example, 'abc'||null returns null.
Is compatible with MySQL behavior. After the NULL type is concatenated, NULL is returned.
-For example, 'abc'||NULL returns NULL.
+Compatible with MySQL behavior. After the null type is concatenated, null is returned.
+For example, 'abc'||null returns null.
Concatenation of the char(n) type
+Concatenation of the char(n) type
Removes spaces and placeholders on the right when the char(n) type is concatenated.
-For example, cast('a' as char(3))||'b' returns 'ab'.
+Removes spaces and placeholders on the right when the char(n) type is concatenated.
+For example, cast('a' as char(3))||'b' returns 'ab'.
After the bpchar_text_without_rtrim option is added to the GUC parameter behavior_compat_options, when the char(n) type is concatenated, spaces are reserved and supplemented to the specified length n.
-Currently, ignoring spaces at the end of a string for comparison is not supported. If the concatenated string contains spaces at the end, the comparison is space-sensitive.
-For example, cast('a' as char(3))||'b' returns 'a b'.
+After the bpchar_text_without_rtrim option is added to the GUC parameter behavior_compat_options, when the char(n) type is concatenated, spaces are reserved and supplemented to the specified length n.
+Currently, ignoring spaces at the end of a string for comparison is not supported. If the concatenated string contains spaces at the end, the comparison is space-sensitive.
+For example, cast('a' as char(3))||'b' returns 'a b'.
Removes spaces and placeholders on the right.
+Removes spaces and placeholders on the right.
concat(str1,str2)
+concat(str1,str2)
Returns the concatenation of all non-NULL strings.
+Returns the concatenation of all non-null strings.
Returns the concatenation of all non-NULL strings.
+Returns the concatenation of all non-null strings.
If an input parameter is NULL, NULL is returned.
+If an input parameter is null, null is returned.
left and right processing of negative values
+left and right processing of negative values
Returns all characters except the first and last |n| characters.
+Returns all characters except the first and last |n| characters.
Returns all characters except the first and last |n| characters.
+Returns all characters except the first and last |n| characters.
Returns an empty string.
+Returns an empty string.
lpad(string text, length int [, fill text])
-rpad(string text, length int [, fill text])
+lpad(string text, length int [, fill text])
+rpad(string text, length int [, fill text])
Fills up the string to the specified length by appending the fill characters (a space by default). If the string is already longer than length then it is truncated (on the right). If fill is an empty string or length is a negative number, null is returned.
+Fills up the string to the specified length by appending the fill characters (a space by default). If the string is already longer than length then it is truncated (on the right). If fill is an empty string or length is a negative number, null is returned.
If fill is an empty string and the string length is less than the specified length, the original string is returned. If length is a negative number, an empty string is returned.
+If fill is an empty string and the string length is less than the specified length, the original string is returned. If length is a negative number, an empty string is returned.
If fill is an empty string and the string length is less than the specified length, an empty string is returned. If length is a negative number, null is returned.
+If fill is an empty string and the string length is less than the specified length, an empty string is returned. If length is a negative number, null is returned.
log(x)
+substr(str, s[, n])
Returns the logarithm with 10 as the base.
+If s is set to 0, the first n characters are returned.
Returns the logarithm with 10 as the base.
+If s is set to 0, the first n characters are returned.
Returns the natural logarithm.
+If s is set to 0, an empty string is returned.
mod(x, 0)
+substring(str, s[, n])
+substring(str [from s] [for n])
Returns x if the divisor is 0.
+If s is set to 0, the first n - 1 characters are returned.
+If s is < 0, the first s + n - 1 characters are returned.
+If n is < 0, an error is reported.
Returns x if the divisor is 0.
+If s is set to 0, the first n - 1 characters are returned.
+If s is < 0, the first s + n - 1 characters are returned.
+If n is < 0, an error is reported.
Reports an error if the divisor is 0.
+If s is set to 0, an empty string is returned.
+If s is < 0, n characters starting from the last |s| character are truncated.
+If n is < 0, an empty string is returned.
Data type DATE
+trim, ltrim, rtrim, btrim(string[,characters])
Converts the DATE data type to the TIMESTAMP data type which stores year, month, day, hour, minute, and second values.
+Removes the longest string that contains only the characters (a space by default) in the characters from a specified position of the string.
Stores year and month values.
+Removes the longest string that contains only the characters (a space by default) in the characters from a specified position of the string.
Stores year and month values.
+Removes the string that is equivalent to characters (a space by default) from a specified position of the string.
to_char(date)
+log(x)
The maximum value of the input parameter can only be the maximum value of the timestamp type. The maximum value of the date type is not supported. The return value is of the timestamp type.
+Returns the logarithm with 10 as the base.
The maximum value of the input parameter can only be the maximum value of the timestamp type. The maximum value of the date type is not supported. The return value is of the date type in YYYY/MM/DD format. (The GUC parameter convert_empty_str_to_null_td is enabled.)
+Returns the logarithm with 10 as the base.
The maximum value of the input parameter can only be the maximum value of the timestamp type. The maximum value of the date type is not supported. The return value is of the date type.
+Returns the natural logarithm.
to_date, to_timestamp, and to_number processing of empty strings
+mod(x, 0)
Returns NULL.
+Returns x if the divisor is 0.
Returns NULL. (The convert_empty_str_to_null_td parameter is enabled.)
+Returns x if the divisor is 0.
to_date and to_timestamp returns NULL. If the parameter passed to to_number is an empty string, 0 is returned.
+Reports an error if the divisor is 0.
Return value types of last_day and next_day
+date data type
Returns values of the timestamp type.
+Converts the date data type to the timestamp data type which stores year, month, day, hour, minute, and second values.
Returns values of the timestamp type.
+Stores year and month values.
Returns values of the date type.
+Stores year and month values.
Return value type of add_months
+to_char(date)
Returns values of the timestamp type.
+The maximum value of the input parameter can only be the maximum value of the timestamp type. The maximum value of the date type is not supported. The return value is of the timestamp type.
Returns values of the timestamp type.
+The maximum value of the input parameter can only be the maximum value of the timestamp type. The maximum value of the date type is not supported. The return value is of the date type in YYYY/MM/DD format. (The GUC parameter convert_empty_str_to_null_td is enabled.)
If the input parameter is of the date type, the return value is of the date type.
-If the input parameter is of the timestamp type, the return value is of the timestamp type.
-If the input parameter is of the timestamptz type, the return value is of the timestamptz type.
+Only the timestamp type and the date type support the maximum input value. The return value is of the date type.
CURRENT_TIME
-CURRENT_TIME(p)
+to_date, to_timestamp, and to_number processing of empty strings
Obtains the time of the current transaction. The return value type is timetz.
+Returns null.
Obtains the time of the current transaction. The return value type is timetz.
+Returns null. (The convert_empty_str_to_null_td parameter is enabled.)
Obtains the execution time of the current statement. The return value type is time.
+to_date and to_timestamp returns null. If the parameter passed to to_number is an empty string, 0 is returned.
CURRENT_TIMESTAMP
-CURRENT_TIMESTAMP(p)
+Return value types of last_day and next_day
Obtains the execution time of the current statement. The return value type is timestamptz.
+Returns values of the timestamp type.
Obtains the execution time of the current statement. The return value type is timestamptz.
+Returns values of the timestamp type.
Obtains the execution time of the current statement. The return value type is timestamp.
+Returns values of the date type.
LOCALTIME
-LOCALTIME(p)
+Return value type of add_months
Obtains the time of the current transaction. The return value type is time.
+Returns values of the timestamp type.
Obtains the time of the current transaction. The return value type is time.
+Returns values of the timestamp type.
Obtains the execution time of the current statement. The return value type is time.
+If the input parameter is of the date type, the return value is of the date type.
+If the input parameter is of the timestamp type, the return value is of the timestamp type.
+If the input parameter is of the timestamptz type, the return value is of the timestamptz type.
LOCALTIMESTAMP
-LOCALTIMESTAMP(p)
+CURRENT_TIME
+CURRENT_TIME(p)
Obtains the time of the current transaction. The return value type is timestamp.
+Obtains the time of the current transaction. The return value is of the timetz type.
Obtains the time of the current transaction. The return value type is timestamp.
+Obtains the time of the current transaction. The return value is of the timetz type.
Obtains the execution time of the current statement. The return value type is timestamp.
+Obtains the execution time of the current statement. The return value is of the time type.
SYSDATE
-SYSDATE(p)
+CURRENT_TIMESTAMP
+CURRENT_TIMESTAMP(p)
Obtains the execution time of the current statement. The return value type is timestamp(0).
+Obtains the execution time of the current statement. The return value is of the timestamptz type.
Obtains the execution time of the current statement. The return value type is timestamp(0).
+Obtains the execution time of the current statement. The return value is of the timestamptz type.
Obtains the current system time. The return value type is timestamp(0).
+Obtains the execution time of the current statement. The return value is of the timestamp type.
NOW()
+LOCALTIME
+LOCALTIME(p)
Obtains the time of the current transaction. The return value type is timestamptz.
+Obtains the time of the current transaction. The return value is of the time type.
Obtains the time of the current transaction. The return value type is timestamptz.
+Obtains the time of the current transaction. The return value is of the time type.
Obtains the statement execution time. The return value type is timestamptz.
+Obtains the execution time of the current statement. The return value is of the timestamp type.
Operator ^
+LOCALTIMESTAMP
+LOCALTIMESTAMP(p)
Performs exponentiation.
+Obtains the time of the current transaction. The return value is of the timestamp type.
Performs exponentiation.
+Obtains the time of the current transaction. The return value is of the timestamp type.
Performs the exclusive OR operation.
+Obtains the execution time of the current statement. The return value is of the timestamp type.
Different input parameter types of CASE, COALESCE, IF, and IFNULL expressions
+SYSDATE
+SYSDATE(p)
Reports error.
+Obtains the execution time of the current statement. The return value is of the timestamp(0) type.
Is compatible with behavior of Teradata and supports type conversion between digits and strings. For example, if input parameters for COALESCE are of INT and VARCHAR types, the parameters are resolved as VARCHAR type.
+Obtains the execution time of the current statement. The return value is of the timestamp(0) type.
Is compatible with behavior of MySQL and supports type conversion between strings and other types. For example, if input parameters for COALESCE are of DATE, INT, and VARCHAR types, the parameters are resolved as VARCHAR type.
+Obtains the current system time. The return value is of the timestamp(0) type. This function cannot be pushed down. You are advised to use current_date instead.
+now()
+Obtains the time of the current transaction. The return value is of the timestamptz type.
+Obtains the time of the current transaction. The return value is of the timestamptz type.
+Obtains the statement execution time. The return value is of the timestamptz type.
+Operator ^
+Performs exponentiation.
+Performs exponentiation.
+Performs the exclusive OR operation.
+Expressions GREATEST and LEAST
+Returns the comparison results of all non-null input parameters.
+Returns the comparison results of all non-null input parameters.
+If an input parameter is null, null is returned.
+Different input parameter types of CASE, COALESCE, IF, and IFNULL expressions
+Reports error.
+Is compatible with behavior of Teradata and supports type conversion between digits and strings. For example, if input parameters for COALESCE are of INT and VARCHAR types, the parameters are resolved as VARCHAR type.
+Is compatible with behavior of MySQL and supports type conversion between strings and other types. For example, if input parameters for COALESCE are of DATE, INT, and VARCHAR types, the parameters are resolved as VARCHAR type.
-diff --git a/docs/dws/dev/dws_04_0043.html b/docs/dws/dev/dws_04_0043.html index 7b520a0c..e6125ee0 100644 --- a/docs/dws/dev/dws_04_0043.html +++ b/docs/dws/dev/dws_04_0043.html @@ -1,7 +1,7 @@ - +--Parent topic: Developer Guide-Database Security Management
-+diff --git a/docs/dws/dev/dws_04_0053.html b/docs/dws/dev/dws_04_0053.html index f682daaa..7346ea68 100644 --- a/docs/dws/dev/dws_04_0053.html +++ b/docs/dws/dev/dws_04_0053.html @@ -1,25 +1,19 @@ - +-
@@ -10,9 +10,5 @@
- Sensitive Data Management
-Parent topic: Developer Guide-Managing Users and Their Permissions
-
-
- Default Permission Mechanism
+ - Database Users
- - System Administrator
+ - User Management
+
+ - User-defined Password Policy
+
+ - Permissions Management
- Separation of Permissions
- - Users
-
- - Roles
-
- - Schema
-
- - User Permission Setting
-
- - Setting Security Policies
-
diff --git a/docs/dws/dev/dws_04_0054.html b/docs/dws/dev/dws_04_0054.html index d0a13347..5d2fb04b 100644 --- a/docs/dws/dev/dws_04_0054.html +++ b/docs/dws/dev/dws_04_0054.html @@ -1,12 +1,90 @@ - + -Default Permission Mechanism
-A user who creates an object is the owner of this object. By default, Separation of Permissions is disabled after cluster installation. A database system administrator has the same permissions as object owners. After an object is created, only the object owner or system administrator can query, modify, and delete the object, and grant permissions for the object to other users through GRANT by default.
-To enable another user to use the object, grant required permissions to the user or the role that contains the user.
-GaussDB(DWS) supports the following permissions: SELECT, INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, CREATE, CONNECT, EXECUTE, USAGE and ANALYZE|ANALYSE. Permission types are associated with object types. For permission details, see GRANT.
-To remove permissions, use REVOKE. Object owner permissions such as ALTER, DROP, GRANT, and REVOKE are implicit and cannot be granted or revoked. That is, you have the implicit permissions for an object if you are the owner of the object. Object owners can remove their own common permissions, for example, making tables read-only to themselves or others.
-System catalogs and views are visible to either system administrators or all users. System catalogs and views that require system administrator permissions can be queried only by system administrators. For details, see System Catalogs and System Views.
-The database provides the object isolation feature. If this feature is enabled, users can view only the objects (tables, views, columns, and functions) that they have the permission to access. System administrators are not affected by this feature. For details, see ALTER DATABASE.
+Permissions Management
++Permission Overview
Permissions are used to control whether a user is allowed to access a database object (including schemas, tables, functions, and sequences) to perform operations such as adding, deleting, modifying, querying, and creating a database object.
+Permission management in GaussDB(DWS) falls into three categories:
+- System permissions
System permissions are also called user attributes, including SYSADMIN, CREATEDB, CREATEROLE, AUDITADMIN, and LOGIN.
+They can be specified only by the CREATE ROLE or ALTER ROLE syntax. The SYSADMIN permission can be granted and revoked using GRANT ALL PRIVILEGE and REVOKE ALL PRIVILEGE, respectively. System permissions cannot be inherited by a user from a role, and cannot be granted using PUBLIC.
+
- Object permissions
Permissions on a database object (table, view, column, database, function, schema, or tablespace) can be granted to a role or user. The GRANT command can be used to grant permissions to a user or role. These permissions granted are added to the existing ones.
+ - Permissions
Grant a role's or user's permissions to one or more roles or users. In this case, every role or user can be regarded as a set of one or more database permissions.
+If WITH ADMIN OPTION is specified, the member can in turn grant permissions in the role to others, and revoke permissions in the role as well. If a role or user granted with certain permissions is changed or revoked, the permissions inherited from the role or user also change.
+A database administrator can grant permissions to and revoke them from any role or user. Roles having CREATEROLE permission can grant or revoke membership in any role that is not an administrator.
+
+Hierarchical Permission Management
GaussDB(DWS) implements a hierarchical permission management on databases, schemas, and data objects.
+- Databases cannot communicate with each other and share very few resources. Their connections and permissions can be isolated. The database cluster has one or more named databases. Users and roles are shared within the entire cluster, but their data is not shared. That is, a user can connect to any database, but after the connection is successful, any user can access only the database declared in the connection request.
- Schemas share more resources than databases do. User permissions on schemas and subordinate objects can be flexibly configured using the GRANT and REVOKE syntax. Each database has one or more schemas. Each schema contains various types of objects, such as tables, views, and functions. To access an object contained in a specified schema, a user must have the USAGE permission on the schema.
- After an object is created, by default, only the object owner or system administrator can query, modify, and delete the object. To access a specific database object, for example, table1, other users must be granted the CONNECT permission of database, the USAGE permission of schema, and the SELECT permission of table1. To access an object at the bottom layer, a user must be granted the permission on the object at the upper layer. To create or delete a schema, you must have the CREATE permission on its database.
Figure 1 Hierarchical Permission Management+
+Roles
The permission management model of GaussDB(DWS) is a typical implementation of the role-based permission control (RBAC). It manages users, roles, and permissions through this model.
+A role is a set of permissions.
+- The concept of "user" is equivalent to that of "role". The only difference is that "user" has the login permission while "role" has the nologin permission.
- Roles are assigned with different permissions based on their responsibilities in the database system. A role is a set of database permissions and represents the behavior constraints of a database user or a group of data users.
- Roles and users can be converted. You can use ALTER to assign the login permission to a role.
- After a role is granted to a user through GRANT, the user will have all the permissions of the role. It is recommended that roles be used to efficiently grant permissions. For example, you can create different roles of design, development, and maintenance personnel, grant the roles to users, and then grant specific data permissions required by different users. When permissions are granted or revoked at the role level, these permission changes take effect for all the members of the role.
- In non-separation-of-duty scenarios, a role can be created, modified, and deleted only by a system administrator or a user with the CREATEROLE attribute. In separation-of-duty scenarios, a role can be created, modified, and deleted only by a user with the CREATEROLE attribute.
To view all roles, query the system catalog PG_ROLES.
+++ +1SELECT * FROM PG_ROLES; +
For details about how to create, modify, and delete a role, see CREARE ROLE/ALTER ROLE/DROP ROLE in SQL Syntax Reference.
++Preset Roles
GaussDB(DWS) provides a group of preset roles. Their names start with gs_role_. These roles allow access to operations that require high permissions. You can grant these roles to other users or roles in the database for them to access or use specific information and functions. Exercise caution and ensure security when using preset roles.
+The following table describes the permissions of preset roles.
+ ++
+Table 1 Permissions of preset roles Role
+Permission
+gs_role_signal_backend
+Invokes functions such as pg_cancel_backend, pg_terminate_backend, pg_terminate_query, pg_cancel_query, pgxc_terminate_query, and pgxc_cancel_query to cancel or terminate sessions, excluding those of the initial users.
+gs_role_read_all_stats
+Reads the system status view and uses various extension-related statistics, including information that is usually visible only to system administrators. For example:
+Resource management views:
+- pgxc_wlm_operator_history
- pgxc_wlm_operator_info
- pgxc_wlm_operator_statistics
- pgxc_wlm_session_info
- pgxc_wlm_session_statistics
- pgxc_wlm_workload_records
- pgxc_workload_sql_count
- pgxc_workload_sql_elapse_time
- pgxc_workload_transaction
Status information views:
+- pgxc_stat_activity
- pgxc_get_table_skewness
- table_distribution
- pgxc_total_memory_detail
- pgxc_os_run_info
- pg_nodes_memory
- pgxc_instance_time
- pgxc_redo_stat
gs_role_analyze_any
+A user with the system-level ANALYZE permission can skip the schema permission check and perform ANALYZE on all tables.
+gs_role_vacuum_any
+A user with the system-level VACUUM permission can skip the schema permission check and perform ANALYZE on all tables.
+Restrictions on using preset roles:
+- gs_role_ is the name field dedicated to preset roles in the database. Do not create users or roles starting with gs_role_ or rename existing users or roles starting with gs_role_.
- Do not perform ALTER or DROP operations on preset roles.
- By default, a preset role does not have the LOGIN permission, so there is no preset login password for the role.
- The gsql meta-commands \du and \dg do not display information about preset roles. However, if PATTERN is specified, information about preset roles will be displayed.
- If the separation of permissions is disabled, the system administrator and users with the ADMIN OPTION permission of preset roles are allowed to perform GRANT and REVOKE operations on preset roles. If the separation of permissions is enabled, the security administrator (with the CREATEROLE attribute) and users with the ADMIN OPTION permission of preset roles are allowed to perform GRANT and REVOKE operations on preset roles. Example:++ +
1 +2
GRANT gs_role_signal_backend TO user1; +REVOKE gs_role_signal_backend FROM user1; +
Granting or Revoking Permissions
A user who creates an object is the owner of this object. By default, Separation of Permissions is disabled after cluster installation. A database system administrator has the same permissions as object owners.
+After an object is created, only the object owner or system administrator can query, modify, and delete the object, and grant permissions for the object to other users through GRANT by default. To enable a user to use an object, the object owner or administrator can run the GRANT or REVOKE command to grant permissions to or revoke permissions from the user or role.
+- Run the GRANT statement to grant permissions.For example, grant the permission of schema myschema to role u1, and grant the SELECT permission of table myschema.t1 to role u1.+++ +
1 +2
GRANT USAGE ON SCHEMA myschema TO u1; +GRANT SELECT ON TABLE myschema.t1 to u1; +
- Run the REVOKE command to revoke a permission that has been granted.For example, revoke all permissions of user u1 on the myschema.t1 table.+
REVOKE ALL PRIVILEGES ON myschema.t1 FROM u1;
+
diff --git a/docs/dws/dev/dws_04_0055.html b/docs/dws/dev/dws_04_0055.html index c5b21c9f..2647babf 100644 --- a/docs/dws/dev/dws_04_0055.html +++ b/docs/dws/dev/dws_04_0055.html @@ -1,19 +1,55 @@ - - -System Administrator
-A system administrator is an account with the SYSADMIN permission. After a cluster is installed, a system administrator has the permissions of all object owners by default.
-The user dbadmin created upon GaussDB(DWS) startup is a system administrator.
-To create a database administrator, connect to the database as an administrator and run the CREATE USER or ALTER statement with SYSADMIN specified.
-+ +1CREATE USER sysadmin WITH SYSADMIN password 'password'; -
Database Users
+Without separation of permissions, GaussDB(DWS) supports two types of database accounts: administrator and common user. For details about user types and permissions under separation of permissions, see Separation of Permissions.
+- The administrator can manage all common users and databases.
- Common users can connect to and access the database, and perform specific database operations and execute SQL statements after being authorized.
Users are authenticated when they log in to the GaussDB(DWS) database. A user can own databases and database objects (such as tables), and grant permissions of these objects to other users and roles. In addition to system administrators, users with the CREATEDB attribute can create databases and grant permissions to these databases.
+Database User Types
+-Table 1 Database user types User Type
+Description
+Allowed Operations
+How to Create
+Administrator dbadmin
+An administrator, also called a system administrator, is an account with the SYSADMIN attribute.
+If separation of permissions is not enabled, this account has the highest permission in the system and can perform all operations. The system administrator has the same permissions as the object owner.
+- User dbadmin created during cluster creation on the GaussDB(DWS) management console is a system administrator.
- Use the CREATE USER or ALTER USER syntax to create an administrator.
CREATE USER sysadmin WITH SYSADMIN password '{Password}';+ALTER USER u1 SYSADMIN;
+
Common user
+ +Common user
+- Use a tool to connect to the database.
- Have the attributes of specific database system operations, such as CREATEDB, CREATEROLE, and SYSADMIN.
- Access database objects.
- Run SQL statements.
Run the CREATE USER syntax to create a common user.
+CREATE USER u1 PASSWORD '{Password}';+Private user
+A user created with the INDEPENDENT attribute in non-separation-of-permissions mode.
+Database administrators can manage (DROP, ALTER, and TRUNCATE) objects of private users but cannot access (INSERT, DELETE, SELECT, UPDATE, COPY, GRANT, REVOKE, and ALTER OWNER) the objects before being authorized.
+Use the CREATE USER syntax to create a private user.
+CREATE USER user_independent WITH INDEPENDENT IDENTIFIED BY '{Password}';+Alternatively, you can run the following statement:
---1ALTER USER joe SYSADMIN; -
To run the ALTER USER statement, the user must exist.
diff --git a/docs/dws/dev/dws_04_0056.html b/docs/dws/dev/dws_04_0056.html index ef665f3d..03b89e88 100644 --- a/docs/dws/dev/dws_04_0056.html +++ b/docs/dws/dev/dws_04_0056.html @@ -1,146 +1,145 @@ - +Separation of Permissions
-Descriptions in Default Permission Mechanism and System Administrator are about the initial situation after a cluster is created. By default, a system administrator with the SYSADMIN attribute has the highest-level permissions.
-To avoid risks caused by centralized permissions, you can enable the separation of permissions to delegate system administrator permissions to security administrators and audit administrators.
-After the separation of permissions is enabled, a system administrator does not have the CREATEROLE attribute (security administrator) and AUDITADMIN attribute (audit administrator). That is, you do not have the permissions for creating roles and users and the permissions for viewing and maintaining database audit logs. For details about the CREATEROLE and AUDITADMIN attributes, see CREATE ROLE.
-After the separation of permissions is enabled, system administrators have the permissions only for the objects owned by them.
-For details, see Separating Rights of Roles.
-For details about permission changes before and after enabling the separation of permissions, see Table 1 and Table 2.
+By default, the system administrator with the SYSADMIN attribute has the highest permission in the system. To avoid risks caused by centralized permissions, you can enable the separation of permissions to delegate system administrator permissions to security administrators and audit administrators.
+- After the separation of permissions is enabled, a system administrator does not have the CREATEROLE attribute (security administrator) and AUDITADMIN attribute (audit administrator). That is, you do not have the permissions for creating roles and users and the permissions for viewing and maintaining database audit logs. For details about the CREATEROLE and AUDITADMIN attributes, see CREATE ROLE.
- After the separation of permissions is enabled, system administrators have the permissions only for the objects owned by them.
For details about how to configure separation of permissions, see "Managing a Cluster > Cluster Security Configuration > Configuring Separation of Permissions" in the Data Warehouse Service (DWS) User Guide.
+For details about permission changes before and after enabling the separation of permissions, see Table 1 and Table 2.
-Table 1 Default user permissions Object
+-Table 1 Default user permissions Object
System Administrator
+System Administrator
Security Administrator
+Security Administrator
Audit Administrator
+Audit Administrator
Common User
+Common User
Tablespace
+Tablespace
Can create, modify, delete, access, and allocate tablespaces.
+Can create, modify, delete, access, and allocate tablespaces.
Cannot create, modify, delete, or allocate tablespaces, with authorization required for accessing tablespaces.
+Cannot create, modify, delete, or allocate tablespaces, with authorization required for accessing tablespaces.
Table
+Table
Has permissions for all tables.
+Has permissions for all tables.
Has permissions for its own tables, but does not have permissions for other users' tables.
+Has permissions for its own tables, but does not have permissions for other users' tables.
Index
+Index
Can create indexes on all tables.
+Can create indexes on all tables.
Can create indexes on their own tables.
+Can create indexes on their own tables.
Schema
+Schema
Has permissions for all schemas.
+Has permissions for all schemas.
Has all permissions for its own schemas, but does not have permissions for other users' schemas.
+Has all permissions for its own schemas, but does not have permissions for other users' schemas.
Function
+Function
Has permissions for all functions.
+Has permissions for all functions.
Has permissions for its own functions, has the call permission for other users' functions in the public schema, but does not have permissions for other users' functions in other schemas.
+Has permissions for its own functions, has the call permission for other users' functions in the public schema, but does not have permissions for other users' functions in other schemas.
Customized view
+Customized view
Has permissions for all views.
+Has permissions for all views.
Has permissions for its own views, but does not have permissions for other users' views.
+Has permissions for its own views, but does not have permissions for other users' views.
System catalog and system view
+System catalog and system view
Has permissions for querying all system catalogs and views.
+Has permissions for querying all system catalogs and views.
Has permissions for querying only some system catalogs and views. For details, see System Catalogs and System Views.
+Has permissions for querying only some system catalogs and views. For details, see System Catalogs and System Views.
Table 2 Changes in permissions after the separation of permissions Object
+Table 2 Changes in permissions after the separation of permissions Object
System Administrator
+System Administrator
Security Administrator
+Security Administrator
Audit Administrator
+Audit Administrator
Common User
+Common User
Tablespace
+Tablespace
No change
+No change
No change
+No change
Table
+Table
Permissions reduced
-Has all permissions for its own tables, but does not have permissions for other users' tables in their schemas.
+Permissions reduced
+Has all permissions for its own tables, but does not have permissions for other users' tables in their schemas.
No change
+No change
Index
+Index
Permissions reduced
-Can create indexes on its own tables.
+Permissions reduced
+Can create indexes on its own tables.
No change
+No change
Schema
+Schema
Permissions reduced
-Has all permissions for its own schemas, but does not have permissions for other users' schemas.
+Permissions reduced
+Has all permissions for its own schemas, but does not have permissions for other users' schemas.
No change
+No change
Function
+Function
Permissions reduced
-Has all permissions for its own functions, but does not have permissions for other users' functions in their schemas.
+Permissions reduced
+Has all permissions for its own functions, but does not have permissions for other users' functions in their schemas.
No change
+No change
Customized view
+Customized view
Permissions reduced
-Has all permissions for its own views and other users' views in the public schema, but does not have permissions for other users' views in their schemas.
+Permissions reduced
+Has all permissions for its own views and other users' views in the public schema, but does not have permissions for other users' views in their schemas.
No change
+No change
System catalog and system view
+System catalog and system view
No change
+No change
No change
+No change
No change
+No change
Has no permission for viewing any system catalogs or views.
+Has no permission for viewing any system catalogs or views.
Users
-You can use CREATE USER and ALTER USER to create and manage database users, respectively. The database cluster has one or more named databases. Users and roles are shared within the entire cluster, but their data is not shared. That is, a user can connect to any database, but after the connection is successful, any user can access only the database declared in the connection request.
-In non-separation-of-duty scenarios, a GaussDB(DWS) user account can be created and deleted only by a system administrator or a security administrator with the CREATEROLE attribute. In separation-of-duty scenarios, a user account can be created only by a security administrator.
-When a user logs in, GaussDB(DWS) authenticates the user. A user can own databases and database objects (such as tables), and grant permissions of these objects to other users and roles. In addition to system administrators, users with the CREATEDB attribute can create databases and grant permissions to these databases.
-Adding, Modifying, and Deleting Users
- To create a user, use the SQL CREATE USER statement.For example, create a user joe and set the CREATEDB attribute for the user.
1CREATE USER joe WITH CREATEDB PASSWORD 'password'; +
User Management
+You can use CREATE USER and ALTER USER to create and manage database users.
+- In the non-separation-of-permission mode, a GaussDB(DWS) user account can be created and deleted only by a system administrator or a security administrator with the CREATEROLE attribute.
- In separation-of-permission mode, a user account can be created only by a security administrator.
-Creating a User
The CREATE USER statement is used to create a GaussDB (DWS) user. After creating a user, you can use the user to connect to the database.
+- Create common user u1 and assign the CREATEDB attribute to the user.-
1CREATE USER u1 WITH CREATEDB PASSWORD '{Password}';
- To create a system administrator, use the CREATE USER statement with the SYSADMIN parameter.
- To delete an existing user, use the DROP USER statement.
- To change a user account (for example, rename the user or change the password), use the ALTER USER statement.
- To view a user list, query the PG_USER view.
1SELECT * FROM pg_user; +
- To create the system administrator mydbadmin, you need to specify the SYSADMIN parameter.
-1CREATE USER mydbadmin sysadmin PASSWORD '{Password}';
- To view user attributes, query the system catalog PG_AUTHID.
1-SELECT * FROM pg_authid; +
- View the created user in the PG_USER view.
++ +1SELECT * FROM pg_user; +
- To view user attributes, query the system catalog PG_AUTHID.
1SELECT * FROM pg_authid;
Private Users
If multiple service departments use different database user accounts to perform service operations and a database maintenance department at the same level uses database administrator accounts to perform maintenance operations, service departments may require that database administrators, without specific authorization, can manage (DROP, ALTER, and TRUNCATE) their data but cannot access (INSERT, DELETE, UPDATE, SELECT, and COPY) the data. That is, the management permissions of database administrators for tables need to be isolated from their access permissions to improve the data security of common users.
-In Separation of Permissions mode, a database administrator does not have permissions for the tables in schemas of other users. In this case, database administrators have neither management permissions nor access permissions, which does not meet the requirements of the service departments mentioned above. Therefore, GaussDB(DWS) provides private users to solve the problem. That is, create private users with the INDEPENDENT attribute in non-separation-of-duties mode.
-1CREATE USER user_independent WITH INDEPENDENT IDENTIFIED BY "password"; +
+Altering User Attributes
The ALTER USER statement is used to alter user attributes, such as changing user passwords or permissions.
+Example:
+- Rename user u1 to u2.-
1ALTER USER u1 RENAME TO u2;
Database administrators can manage (DROP, ALTER, and TRUNCATE) objects of private users but cannot access (INSERT, DELETE, SELECT, UPDATE, COPY, GRANT, REVOKE, and ALTER OWNER the objects before being authorized.
+ - Grant the CREATEROLE permission to user u1:++ +
1ALTER USER u1 CREATEROLE; +
- For details about how to change the user password, see Setting and Changing a Password.
+Locking a User
The ACCOUNT LOCK | ACCOUNT UNLOCK parameter in the statement is used to lock or unlock a user. A locked user cannot log in to the system. If an account is stolen or illegally accessed, the administrator can manually lock the account. After the account is secured, the administrator can manually unlock the account.
+Example:
+- To lock user u1, run the following command:++ +
1ALTER USER u1 ACCOUNT LOCK; +
- To unlock user u1, run the following command:++ +
1ALTER USER u1 ACCOUNT UNLOCK; +
Deleting a User
The DROP USER statement is used to delete one or more GaussDB(DWS) users. An administrator can delete an account that is no longer used. Deleted users cannot be restored.
+- If multiple users are deleted at the same time, separate them with commas (,).
- After a user is deleted successfully, all the permissions of the user are also deleted.
- When an account to be deleted is in the active state, it is deleted after the session is disconnected.
- When CASCADE is specified in the DROP USER statement, objects such as tables that depend on the user will be deleted. That is, the objects whose owner is the user are deleted, and the authorizations of other objects to the user are also deleted.
Example:
+- -- Delete user u1.++ +
1DROP USER u1; +
- Delete account u2 in a cascading manner.++ +
1DROP USER u2 CASCADE; +
diff --git a/docs/dws/dev/dws_04_0058.html b/docs/dws/dev/dws_04_0058.html deleted file mode 100644 index 01883662..00000000 --- a/docs/dws/dev/dws_04_0058.html +++ /dev/null @@ -1,20 +0,0 @@ - - -Roles
--A role is a set of permissions. After a role is granted to a user through GRANT, the user will have all the permissions of the role. It is recommended that roles be used to efficiently grant permissions. For example, you can create different roles of design, development, and maintenance personnel, grant the roles to users, and then grant specific data permissions required by different users. When permissions are granted or revoked at the role level, these changes take effect on all members of the role.
-GaussDB(DWS) provides an implicitly defined group PUBLIC that contains all roles. By default, all new users and roles have the permissions of PUBLIC. For details about the default permissions of PUBLIC, see GRANT. To revoke permissions of PUBLIC from a user or role, or re-grant these permissions to them, add the PUBLIC keyword in the REVOKE or GRANT statement.
-To view all roles, query the system catalog PG_ROLES.
--- -1SELECT * FROM PG_ROLES; -
-Adding, Modifying, and Deleting Roles
In non-separation-of-duty scenarios, a role can be created, modified, and deleted only by a system administrator or a user with the CREATEROLE attribute. In separation-of-duty scenarios, a role can be created, modified, and deleted only by a user with the CREATEROLE attribute.
-- To create a role, use CREATE ROLE.
- To add or delete users in an existing role, use ALTER ROLE.
- To delete a role, use DROP ROLE. DROP ROLE deletes only a role, rather than member users in the role.
-- diff --git a/docs/dws/dev/dws_04_0059.html b/docs/dws/dev/dws_04_0059.html deleted file mode 100644 index afbf34f7..00000000 --- a/docs/dws/dev/dws_04_0059.html +++ /dev/null @@ -1,52 +0,0 @@ - - ---Parent topic: Managing Users and Their Permissions-Schema
--Schemas function as models. Schema management allows multiple users to use the same database without mutual impacts, to organize database objects as manageable logical groups, and to add third-party applications to the same schema without causing conflicts.
-Each database has one or more schemas. Each schema contains tables and other types of objects. When a database is created, a schema named public is created by default, and all users have permissions for this schema. You can group database objects by schema. A schema is similar to an OS directory but cannot be nested.
-The same database object name can be used in different schemas of the same database without causing conflicts. For example, both a_schema and b_schema can contain a table named mytable. Users with required permissions can access objects across multiple schemas of the same database.
-If a user is created, a schema named after the user will also be created in the current database.
-Database objects are generally created in the first schema in a database search path. For details about the first schema and how to change the schema order, see Search Path.
--Creating, Modifying, and Deleting Schemas
- To create a schema, use CREATE SCHEMA. Any user can create a schema.
- To change the name or owner of a schema, use ALTER SCHEMA. Only the schema owner can do so.
- To delete a schema and its objects, use DROP SCHEMA. Only the schema owner can do so.
- To create a table in a schema, use the schema_name.table_name format to specify the table. If schema_name is not specified, the table will be created in the first schema in Search Path.
- To view the owner of a schema, perform the following join query on the system catalogs PG_NAMESPACE and PG_USER. Replace schema_name in the statement with the name of the schema to be queried.-- -
1SELECT s.nspname,u.usename AS nspowner FROM pg_namespace s, pg_user u WHERE nspname='schema_name' AND s.nspowner = u.usesysid; -
- To view a list of all schemas, query the system catalog PG_NAMESPACE.-- -
1SELECT * FROM pg_namespace; -
- To view a list of tables in a schema, query the system catalog PG_TABLES. For example, the following query will return a table list from PG_CATALOG in the schema.-- -
1SELECT distinct(tablename),schemaname from pg_tables where schemaname = 'pg_catalog'; -
-Search Path
A search path is defined in the search_path parameter. The parameter value is a list of schema names separated by commas (,). If no target schema is specified during object creation, the object will be added to the first schema listed in the search path. If there are objects with the same name across different schemas and no schema is specified for an object query, the object will be returned from the first schema containing the object in the search path.
-- To view the current search path, use SHOW.-- -
1 -2 -3 -4 -5
SHOW SEARCH_PATH; - search_path ----------------- - "$user",public -(1 row) -
The default value of search_path is "$user",public. $user indicates the name of the schema with the same name as the current session user. If the schema does not exist, $user will be ignored. By default, after a user connects to a database that has schemas with the same name, objects will be added to all the schemas. If there are no such schemas, objects will be added to only to the public schema.
- - To change the default schema of the current session, run the SET statement.Run the following command to set search_path to myschema and public (myschema will be searched first):--- -
1 -2
SET SEARCH_PATH TO myschema, public; -SET -
-- diff --git a/docs/dws/dev/dws_04_0060.html b/docs/dws/dev/dws_04_0060.html deleted file mode 100644 index e6c6ffb2..00000000 --- a/docs/dws/dev/dws_04_0060.html +++ /dev/null @@ -1,38 +0,0 @@ - - ---Parent topic: Managing Users and Their Permissions-User Permission Setting
--- To grant the permission for an object directly to a user, use GRANT.
When permissions for a table or view in a schema are granted to a user or role, the USAGE permission of the schema must be granted together. Otherwise, the user or role can only see the names of the objects but cannot actually access them.
-In the following example, permissions for the schema tpcds are first granted to the user joe, and then the SELECT permission for the tpcds.web_returns table is also granted.
--- -1 -2
GRANT USAGE ON SCHEMA tpcds TO joe; -GRANT SELECT ON TABLE tpcds.web_returns to joe; -
- Granting a role to a user allows the user to inherit the object permissions of the role.
- Create a role.
Create a role lily and grant the system permission CREATEDB to the role.
--- -1CREATE ROLE lily WITH CREATEDB PASSWORD 'password'; -
- To grant object permissions to a role, use GRANT.
For example, first grant permissions for the schema tpcds to the role lily, and then grant the SELECT permission of the tpcds.web_returns table to lily.
--- -1 -2
GRANT USAGE ON SCHEMA tpcds TO lily; -GRANT SELECT ON TABLE tpcds.web_returns to lily; -
- Grant the role permissions to a user.-- -
1GRANT lily to joe; -
- When the permissions of a role are granted to a user, the attributes of the role are not transferred together.
-
- Create a role.
- To revoke user permissions, use REVOKE.
-- diff --git a/docs/dws/dev/dws_04_0061.html b/docs/dws/dev/dws_04_0061.html index 28f1ee91..063d9bce 100644 --- a/docs/dws/dev/dws_04_0061.html +++ b/docs/dws/dev/dws_04_0061.html @@ -1,11 +1,91 @@ - +--Parent topic: Managing Users and Their Permissions-Row-Level Access Control
-The row-level access control feature enables database access control to be accurate to each row of data tables. In this way, the same SQL query may return different results for different users.
-You can create a row-level access control policy for a data table. The policy defines an expression that takes effect only for specific database users and SQL operations. When a database user accesses the data table, if a SQL statement meets the specified row-level access control policies of the data table, the expressions that meet the specified condition will be combined by using AND or OR based on the attribute type (PERMISSIVE | RESTRICTIVE) and applied to the execution plan in the query optimization phase.
-Row-level access control is used to control the visibility of row-level data in tables. By predefining filters for data tables, the expressions that meet the specified condition can be applied to execution plans in the query optimization phase, which will affect the final execution result. Currently, the SQL statements that can be affected include SELECT, UPDATE, and DELETE.
-Scenario 1: A table summarizes the data of different users. Users can view only their own data.
-1 +
The row-level access control feature restricts users to access only specific data rows in the data table, ensuring data read and write security.
++Configuring Row-Level Access Control
Row-level access control is used to control the visibility of row-level data in tables. By predefining filters for data tables, the expressions that meet the specified condition can be applied to execution plans in the query optimization phase, which will affect the final execution result. Currently, the SQL statements that can be affected include SELECT, UPDATE, and DELETE.
+- You can use the CREATE ROW LEVEL SECURITY POLICY statement to create a row-level security policy on a table.
This policy works only for expressions that take effect for specific database users and SQL operations. When a database user accesses the data table, if a SQL statement meets the specified row-level access control policies of the data table, the expressions that meet the specified condition will be combined by using AND or OR based on the attribute type (PERMISSIVE | RESTRICTIVE) and applied to the execution plan in the query optimization phase.
+ - After a row-level access control policy is created for a table, it takes effect only when the row-level access control switch (ALTER TABLE...ENABLE ROW LEVEL SECURITY) of the table is turned on.
Example of Row-Level Access Control
The data of all users is aggregated in table all_data. Implement row-level access control on this table so that different users can view only their own data.
+- Create users alice, bob, and peter.++ +
1 +2 +3
CREATE ROLE alice PASSWORD '********'; +CREATE ROLE bob PASSWORD '********'; +CREATE ROLE peter PASSWORD '********'; +
Create table all_data and insert data of different users into it.
+++ +1 +2 +3 +4 +5
CREATE TABLE public.all_data(id int, role varchar(100), data varchar(100)); + +INSERT INTO all_data VALUES(1, 'alice', 'alice data'); +INSERT INTO all_data VALUES(2, 'bob', 'bob data'); +INSERT INTO all_data VALUES(3, 'peter', 'peter data'); +
- Grant the read permission on table all_data to users alice, bob, and peter.++ +
1GRANT SELECT ON all_data TO alice, bob, peter; +
- Run the ALTER TABLE tablename ENABLE ROW LEVEL SECURITY statement to enable the row-level access control.++ +
1ALTER TABLE all_data ENABLE ROW LEVEL SECURITY; +
- Run the CREATE ROW LEVEL SECURITY POLICY statement to create a row-level access control policy so that the current user can view only its own data.++ +
1CREATE ROW LEVEL SECURITY POLICY all_data_rls ON all_data USING(role = CURRENT_USER); +
- View information about the all_data table.++ +
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14
\d+ all_data + Table "public.all_data" + Column | Type | Modifiers | Storage | Stats target | Description +--------+------------------------+-----------+----------+--------------+------------- + id | integer | | plain | | + role | character varying(100) | | extended | | + data | character varying(100) | | extended | | +Row Level Security Policies: + POLICY "all_data_rls" + USING (((role)::name = "current_user"())) +Has OIDs: no +Distribute By: ROUND ROBIN +Location Nodes: ALL DATANODES +Options: orientation=row, compression=no, enable_rowsecurity=true +
- Switch to user alice and query the data in table all_data. The query result shows that the row-level access control policy takes effect. User alice can only view its own data.++ +
1 +2 +3 +4 +5 +6
SET ROLE alice PASSWORD '********'; + +SELECT * FROM all_data; + id | role | data +----+-------+------------ + 1 | alice | alice data +
The execution plan of the query is displayed, indicating that access to table all_data is under the row-level access control.
+++1 2 3 4 @@ -24,154 +104,89 @@ 17 18 19 -20 -21 -22 -23 -24 -25 -26 -27 -28 -29 -30 -31 -32 -33 -34 -35 -36 -37 -38 -39 -40 -41 -42 -43 -44 -45 -46 -47 -48 -49 -50 -51 -52 -53 -54 -55 -56 -57 -58 -59 -60 -61 -62 -63 -64 -65 -66 -67 -68 -69 -70 -71 -72 -73 -74 -75 -76 -77 -78 -79 -80 -81 -82 -83
-- Create users alice, bob, and peter. -CREATE ROLE alice PASSWORD 'password'; -CREATE ROLE bob PASSWORD 'password'; -CREATE ROLE peter PASSWORD 'password'; - --- Create the public.all_data table that contains user information. -CREATE TABLE public.all_data(id int, role varchar(100), data varchar(100)); - --- Insert data into the data table. -INSERT INTO all_data VALUES(1, 'alice', 'alice data'); -INSERT INTO all_data VALUES(2, 'bob', 'bob data'); -INSERT INTO all_data VALUES(3, 'peter', 'peter data'); - --- Grant the read permission for the all_data table to users alice, bob, and peter. -GRANT SELECT ON all_data TO alice, bob, peter; - --- Enable row-level access control. -ALTER TABLE all_data ENABLE ROW LEVEL SECURITY; - --- Create a row-level access control policy to specify that the current user can view only their own data. -CREATE ROW LEVEL SECURITY POLICY all_data_rls ON all_data USING(role = CURRENT_USER); - --- View table details. -\d+ all_data - Table "public.all_data" - Column | Type | Modifiers | Storage | Stats target | Description ---------+------------------------+-----------+----------+--------------+------------- - id | integer | | plain | | - role | character varying(100) | | extended | | - data | character varying(100) | | extended | | -Row Level Security Policies: - POLICY "all_data_rls" - USING (((role)::name = "current_user"())) -Has OIDs: no -Distribute By: HASH(id) -Location Nodes: ALL DATANODES -Options: orientation=row, compression=no, enable_rowsecurity=true - --- Switch to user alice and run SELECT * FROM all_data. -SET ROLE alice PASSWORD 'password'; -SELECT * FROM all_data; - id | role | data -----+-------+------------ - 1 | alice | alice data -(1 row) - -EXPLAIN(COSTS OFF) SELECT * FROM all_data; - QUERY PLAN +20
EXPLAIN(COSTS OFF) SELECT * FROM all_data; + QUERY PLAN ---------------------------------------------------------------- - id | operation + id | operation ----+------------------------------ - 1 | -> Streaming (type: GATHER) - 2 | -> Seq Scan on all_data + 1 | -> Streaming (type: GATHER) + 2 | -> Seq Scan on all_data - Predicate Information (identified by plan id) + Predicate Information (identified by plan id) -------------------------------------------------------------- 2 --Seq Scan on all_data - Filter: ((role)::name = 'alice'::name) - Notice: This query is influenced by row level security feature -(10 rows) + Filter: ((role)::name = 'alice'::name) + Notice: This query is influenced by row level security feature --- Switch to user peter and run SELECT * FROM .all_data. -SET ROLE peter PASSWORD 'password'; -SELECT * FROM all_data; - id | role | data -----+-------+------------ - 3 | peter | peter data -(1 row) - -EXPLAIN(COSTS OFF) SELECT * FROM all_data; - QUERY PLAN ----------------------------------------------------------------- - id | operation - ----+------------------------------ - 1 | -> Streaming (type: GATHER) - 2 | -> Seq Scan on all_data - - Predicate Information (identified by plan id) - -------------------------------------------------------------- - 2 --Seq Scan on all_data - Filter: ((role)::name = 'peter'::name) - Notice: This query is influenced by row level security feature -(10 rows) + ====== Query Summary ===== + ------------------------------- + System available mem: 4833280KB + Query Max mem: 4833280KB + Query estimated mem: 1024KB +(16 rows)
- Switch to user peter and query the data in table all_data. The query result shows that the row-level access control policy takes effect. User peter can only view its own data.++ +
1 +2 +3 +4 +5 +6 +7
SET ROLE peter PASSWORD '********'; + +SELECT * FROM all_data; + id | role | data +----+-------+------------ + 3 | peter | peter data +(1 row) +
The execution plan of the table query is displayed, indicating that the query of table all_data is under the row-level access control.
+++ +1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20
EXPLAIN(COSTS OFF) SELECT * FROM all_data; + QUERY PLAN +---------------------------------------------------------------- + id | operation + ----+------------------------------ + 1 | -> Streaming (type: GATHER) + 2 | -> Seq Scan on all_data + + Predicate Information (identified by plan id) + -------------------------------------------------------------- + 2 --Seq Scan on all_data + Filter: ((role)::name = 'peter'::name) + Notice: This query is influenced by row level security feature + + ====== Query Summary ===== + ------------------------------- + System available mem: 4833280KB + Query Max mem: 4833280KB + Query estimated mem: 1024KB +(16 rows) +
diff --git a/docs/dws/dev/dws_04_0062.html b/docs/dws/dev/dws_04_0062.html index 7eb887e5..ab73ff0a 100644 --- a/docs/dws/dev/dws_04_0062.html +++ b/docs/dws/dev/dws_04_0062.html @@ -1,109 +1,251 @@ - +Data Redaction
-GaussDB(DWS) provides the column-level dynamic data masking (DDM) function. For sensitive data, such as the ID card number, mobile number, and bank card number, the DDM function is used to redact the original data to protect data security and user privacy.
-- You can create a redaction policy for a specified table object and set the effective scope of the policy. For details, see section "CREATE REDACTION POLICY" in SQL Syntax Reference.
- You can modify a redaction policy using the provided syntax, including modifying expressions for the policy to take effect, renaming the policy, and adding, modifying, and deleting redacted columns. For details, see section "ALTER REDACTION POLICY" in SQL Syntax Reference.
- You can delete a redaction policy by deleting data redaction function information of the policy from all columns of a table. For details, see "DROP REDACTION POLICY" in SQL Syntax Reference.
- You can use the built-in masking functions MASK_NONE, MASK_FULL, and MASK_PARTIAL, or create your own masking functions by using PL/pgSQL. For details, see "Data Masking Functions" in SQL Syntax Reference.
- Redaction policy information is stored in the system catalog PG_REDACTION_POLICY, and redacted column information is stored in the system catalog PG_REDACTION_COLUMN.
- You can view information about the redaction policy and redacted columns in the system views REDACTION_POLICIES and REDACTION_COLUMNS.
- Generally, you can execute a SELECT statement to view the data redaction result. If the statement has the following features, sensitive data may be deliberately obtained. In this case, an error will be reported during statement execution.
- The GROUP BY clause references a Target Entry that contains redacted columns as the target column.
- The DISTINCT clause is executed on the output redacted columns.
- The statement contains CTE.
- Set operations are involved.
- The target columns of a subquery are not redacted columns of the base table, but are the expressions or function calls for the redacted columns of the base table.
- You can use COPY TO or GDS to export the redacted data. As the redacted data is irreversible, any secondary operation on the redacted data is meaningless.
- The target columns of UPDATE, MERGE INTO, and DELETE statements cannot contain redacted columns.
- The UPSERT statement allows you to update data using EXCLUDED. If data in the base table is updated by referencing redacted columns, the data may be modified by mistake. As a result, an error will be reported during the execution.
GaussDB(DWS) provides the column-level dynamic data masking (DDM) function. For sensitive data (such as the ID card number, mobile number, and bank card number), the DDM function is used to redact the original data to protect data security and user privacy.
+- Creating a data masking policy for a table
GaussDB(DWS) uses the CREATE REDACTION POLICY syntax to create a data masking policy on a table. (MASK_NONE: Do not perform masking. MASK_FULL: Mask data into a fixed value. MASK_PARTIAL: Perform partial masking based on the character type, numeric type, or time type.)
+ - Modifying the data masking policy of a table
The ALTER REDACTION POLICY syntax is used to modify the expression for enabling a masking policy, rename a masking policy, and add, modify, or delete masked columns.
+ - Deleting the masking policy of a table
The DROP REDACTION POLICY syntax is used to delete the masking function information of a masking policy on all columns of a table.
+ - Viewing the masking policy and masked columns
Redaction policy information is stored in the system catalog PG_REDACTION_POLICY, and redacted column information is stored in the system catalog PG_REDACTION_COLUMN. You can view information about the redaction policy and redacted columns in the system views REDACTION_POLICIES and REDACTION_COLUMNS.
+
- - Generally, you can run the SELECT statement to view the data redaction result. If a statement has the following features, sensitive data may be deliberately obtained. In this case, an error will be reported during statement execution.
- The GROUP BY clause references the Target Entry containing redaction columns as the target column.
- DISTINCT works on the output redaction columns.
- The statement contains CTE.
- Operations on sets are involved.
- The target columns of a subquery are not redaction columns of the base table, but the expressions or function calls for redaction columns of the base table.
- You can use COPY TO or GDS to export the redacted data. Due to the irreversibility of the data redaction, secondary redaction of the data is meaningless.
- Do not set target columns of UPDATE, MERGE INTO, and DELETE statements to redaction columns.
- The UPSERT statement allows you to insert update data through EXCLUDED. If data in the base table is updated by referencing redaction columns, the data may be modified by mistake. As a result, an error will be reported during the execution.
Examples
The following uses the employee table emp, administrator alice, and common users matu and july as examples to describe the data redaction process. The user alice is the owner of the emp table. The emp table contains private data such as the employee name, mobile number, email address, bank card number, and salary.
-- Create users alice, matu, and july:
CREATE ROLE alice PASSWORD 'password'; -CREATE ROLE matu PASSWORD 'password'; -CREATE ROLE july PASSWORD 'password';
- - Create the emp table as user alice, and insert three employee records into the table.
CREATE TABLE emp(id int, name varchar(20), phone_no varchar(11), card_no number, card_string varchar(19), email text, salary numeric(100, 4), birthday date); +
Examples
The following uses the employee table emp, table owner alice, and roles matu and july as an example to illustrate the data masking process. The emp table contains private data such as the employee name, mobile number, email address, bank card number, and salary.
+- After connecting to the database as the administrator, create roles alice, matu, and july.+-INSERT INTO emp VALUES(1, 'anny', '13420002340', 1234123412341234, '1234-1234-1234-1234', 'smithWu@163.com', 10000.00, '1999-10-02'); -INSERT INTO emp VALUES(2, 'bob', '18299023211', 3456345634563456, '3456-3456-3456-3456', '66allen_mm@qq.com', 9999.99, '1989-12-12'); -INSERT INTO emp VALUES(3, 'cici', '15512231233', NULL, NULL, 'jonesishere@sina.com', NULL, '1992-11-06'); -
1 +2 +3
CREATE ROLE alice PASSWORD '{Password}'; +CREATE ROLE matu PASSWORD '{Password}'; +CREATE ROLE july PASSWORD '{Password}'; +
- User alice grants the emp table read permission to users matu and july.
GRANT SELECT ON emp TO matu, july;
-- Only user alice can view all employee information. Users matu and july cannot view bank card numbers and salary data of the employees. Create a redaction policy for the emp table and bind the redaction function to card_no, card_string, and salary columns, respectively.
CREATE REDACTION POLICY mask_emp ON emp WHEN (current_user IN ('matu', 'july')) -ADD COLUMN card_no WITH mask_full(card_no), -ADD COLUMN card_string WITH mask_partial(card_string, 'VVVVFVVVVFVVVVFVVVV','VVVV-VVVV-VVVV-VVVV','#',1,12), -ADD COLUMN salary WITH mask_partial(salary, '9', 1, length(salary) - 2);-- Switch to users matu and july and view the emp table, respectively.
SET ROLE matu PASSWORD 'password'; -SELECT * FROM emp; - id | name | phone_no | card_no | card_string | email | salary | birthday -----+------+-------------+---------+---------------------+----------------------+------------+--------------------- - 1 | anny | 13420002340 | 0 | ####-####-####-1234 | smithWu@163.com | 99999.9990 | 1999-10-02 00:00:00 - 2 | bob | 18299023211 | 0 | ####-####-####-3456 | 66allen_mm@qq.com | 9999.9990 | 1989-12-12 00:00:00 - 3 | cici | 15512231233 | | | jonesishere@sina.com | | 1992-11-06 00:00:00 -(3 rows) +
- User alice grants the emp table read permission to users matu and july.
- Grant schema permissions on the current database to alice, matu, and july.+-SET ROLE july PASSWORD 'password'; -SELECT * FROM emp; - id | name | phone_no | card_no | card_string | email | salary | birthday -----+------+-------------+---------+---------------------+----------------------+------------+--------------------- - 1 | anny | 13420002340 | 0 | ####-####-####-1234 | smithWu@163.com | 99999.9990 | 1999-10-02 00:00:00 - 2 | bob | 18299023211 | 0 | ####-####-####-3456 | 66allen_mm@qq.com | 9999.9990 | 1989-12-12 00:00:00 - 3 | cici | 15512231233 | | | jonesishere@sina.com | | 1992-11-06 00:00:00 -(3 rows) -
1GRANT ALL PRIVILEGES on schema public to alice,matu,july; +
- User matu also has the permission for viewing all employee information, but user july does not. Modify the effective scope of the redaction policy.
ALTER REDACTION POLICY mask_emp ON emp WHEN(current_user = 'july');
-- Switch to users matu and july and view the emp table again, respectively.
SET ROLE matu PASSWORD 'password'; -SELECT * FROM emp; - id | name | phone_no | card_no | card_string | email | salary | birthday -----+------+-------------+------------------+---------------------+----------------------+------------+--------------------- - 1 | anny | 13420002340 | 1234123412341234 | 1234-1234-1234-1234 | smithWu@163.com | 10000.0000 | 1999-10-02 00:00:00 - 2 | bob | 18299023211 | 3456345634563456 | 3456-3456-3456-3456 | 66allen_mm@qq.com | 9999.9900 | 1989-12-12 00:00:00 - 3 | cici | 15512231233 | | | jonesishere@sina.com | | 1992-11-06 00:00:00 -(3 rows) +
- User matu also has the permission for viewing all employee information, but user july does not. Modify the effective scope of the redaction policy.
- Switch to role alice, create the emp table, and insert three pieces of employee information.+-CREATE FUNCTION mask_regexp_salary(salary_info text) RETURNS text AS -$$ - SELECT regexp_replace($1, '[0-9]+','*','g'); -$$ -LANGUAGE SQL -STRICT SHIPPABLE; +
1 +2 +3 +4 +5 +6 +7
SET ROLE alice PASSWORD '{Password}'; -SET ROLE july PASSWORD 'password'; -SELECT * FROM emp; - id | name | phone_no | card_no | card_string | email | salary | birthday -----+------+-------------+---------+---------------------+----------------------+------------+--------------------- - 1 | anny | 13420002340 | 0 | ####-####-####-1234 | smithWu@163.com | 99999.9990 | 1999-10-02 00:00:00 - 2 | bob | 18299023211 | 0 | ####-####-####-3456 | 66allen_mm@qq.com | 9999.9990 | 1989-12-12 00:00:00 - 3 | cici | 15512231233 | | | jonesishere@sina.com | | 1992-11-06 00:00:00 -(3 rows)
-- The information in the phone_no, email, and birthday columns is private data. Update the redaction policy mask_emp and add three redacted columns.
ALTER REDACTION POLICY mask_emp ON emp ADD COLUMN phone_no WITH mask_partial(phone_no, '*', 4); -ALTER REDACTION POLICY mask_emp ON emp ADD COLUMN email WITH mask_partial(email, '*', 1, position('@' in email)); -ALTER REDACTION POLICY mask_emp ON emp ADD COLUMN birthday WITH mask_full(birthday);-- Switch to user july and view the emp table data.
SET ROLE july PASSWORD 'password'; -SELECT * FROM emp; - id | name | phone_no | card_no | card_string | email | salary | birthday -----+------+-------------+---------+---------------------+----------------------+------------+--------------------- - 1 | anny | 134******** | 0 | ####-####-####-1234 | ********163.com | 99999.9990 | 1970-01-01 00:00:00 - 2 | bob | 182******** | 0 | ####-####-####-3456 | ***********qq.com | 9999.9990 | 1970-01-01 00:00:00 - 3 | cici | 155******** | | | ************sina.com | | 1970-01-01 00:00:00 -(3 rows)
-- Query redaction_policies and redaction_columns to view details about the current redaction policy mask_emp.
SELECT * FROM redaction_policies; - object_schema | object_owner | object_name | policy_name | expression | enable | policy_description ----------------+--------------+-------------+-------------+-----------------------------------+--------+-------------------- - public | alice | emp | mask_emp | ("current_user"() = 'july'::name) | t | -(1 row) +CREATE TABLE emp(id int, name varchar(20), phone_no varchar(11), card_no number, card_string varchar(19), email text, salary numeric(100, 4), birthday date); -SELECT object_name, column_name, function_info FROM redaction_columns; - object_name | column_name | function_info --------------+-------------+------------------------------------------------------------------------------------------------------- - emp | card_no | mask_full(card_no) - emp | card_string | mask_partial(card_string, 'VVVVFVVVVFVVVVFVVVV'::text, 'VVVV-VVVV-VVVV-VVVV'::text, '#'::text, 1, 12) - emp | email | mask_partial(email, '*'::text, 1, "position"(email, '@'::text)) - emp | salary | mask_partial(salary, '9'::text, 1, (length((salary)::text) - 2)) - emp | birthday | mask_full(birthday) - emp | phone_no | mask_partial(phone_no, '*'::text, 4) -(6 rows)-- Add the salary_info column. To replace the salary information in text format with *.*, you can create a user-defined redaction function. In this step, you can use the PL/pgSQL to define the redaction function mask_regexp_salary. To create a redaction column, you simply need to customize the function name and parameter list. For details, see User-Defined Functions.
ALTER TABLE emp ADD COLUMN salary_info TEXT; -UPDATE emp SET salary_info = salary::text; +INSERT INTO emp VALUES(1, 'anny', '13420002340', 1234123412341234, '1234-1234-1234-1234', 'smithWu@163.com', 10000.00, '1999-10-02'); +INSERT INTO emp VALUES(2, 'bob', '18299023211', 3456345634563456, '3456-3456-3456-3456', '66allen_mm@qq.com', 9999.99, '1989-12-12'); +INSERT INTO emp VALUES(3, 'cici', '15512231233', NULL, NULL, 'jonesishere@sina.com', NULL, '1992-11-06'); +
- The information in the phone_no, email, and birthday columns is private data. Update the redaction policy mask_emp and add three redacted columns.
- alice grants the read permission on the emp table to matu and july.+-ALTER REDACTION POLICY mask_emp ON emp ADD COLUMN salary_info WITH mask_regexp_salary(salary_info); +
1GRANT SELECT ON emp TO matu, july; +
- Create the masking policy mask_emp: Only user alice can view all employee information. User matu and july cannot view employee bank card numbers and salary data. The card_no column is of the numeric type and all of its data is masked into 0 by the MASK_FULL function. The card_string column is of the character type and part of its data is masked by the MASK_PARTIAL function based on the specified input and output formats. The salary column is of the numeric type and the MASK_PARTIAL function is used to mask all digits before the penultimate digit using the number 9.+-SET ROLE july PASSWORD 'password'; -SELECT id, name, salary_info FROM emp; - id | name | salary_info -----+------+------------- - 1 | anny | *.* - 2 | bob | *.* - 3 | cici | -(3 rows) -
1 +2 +3 +4
CREATE REDACTION POLICY mask_emp ON emp WHEN (current_user IN ('matu', 'july')) + ADD COLUMN card_no WITH mask_full(card_no), + ADD COLUMN card_string WITH mask_partial(card_string, 'VVVVFVVVVFVVVVFVVVV','VVVV-VVVV-VVVV-VVVV','#',1,12), + ADD COLUMN salary WITH mask_partial(salary, '9', 1, length(salary) - 2); +
- If there is no need to set a redaction policy for the emp table, delete the redaction policy mask_emp.
+DROP REDACTION POLICY mask_emp ON emp;
- - If there is no need to set a redaction policy for the emp table, delete the redaction policy mask_emp.
- Switch to matu and july and view the employee table emp.++ +
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17
SET ROLE matu PASSWORD '{Password}'; +SELECT * FROM emp; + id | name | phone_no | card_no | card_string | email | salary | birthday +----+------+-------------+---------+---------------------+----------------------+------------+--------------------- + 1 | anny | 13420002340 | 0 | ####-####-####-1234 | smithWu@163.com | 99999.9990 | 1999-10-02 00:00:00 + 2 | bob | 18299023211 | 0 | ####-####-####-3456 | 66allen_mm@qq.com | 9999.9990 | 1989-12-12 00:00:00 + 3 | cici | 15512231233 | | | jonesishere@sina.com | | 1992-11-06 00:00:00 +(3 rows) + +SET ROLE july PASSWORD '{Password}'; +SELECT * FROM emp; + id | name | phone_no | card_no | card_string | email | salary | birthday +----+------+-------------+---------+---------------------+----------------------+------------+--------------------- + 1 | anny | 13420002340 | 0 | ####-####-####-1234 | smithWu@163.com | 99999.9990 | 1999-10-02 00:00:00 + 2 | bob | 18299023211 | 0 | ####-####-####-3456 | 66allen_mm@qq.com | 9999.9990 | 1989-12-12 00:00:00 + 3 | cici | 15512231233 | | | jonesishere@sina.com | | 1992-11-06 00:00:00 +(3 rows) +
- If you want matu to have the permission to view all employee information, but do not want july to have. In this case, you only need to modify the effective scope of the policy.++ +
1 +2
SET ROLE alice PASSWORD '{Password}'; +ALTER REDACTION POLICY mask_emp ON emp WHEN(current_user = 'july'); +
- Switch to users matu and july and view the emp table again, respectively.++ +
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17
SET ROLE matu PASSWORD '{Password}'; +SELECT * FROM emp; + id | name | phone_no | card_no | card_string | email | salary | birthday +----+------+-------------+------------------+---------------------+----------------------+------------+--------------------- + 1 | anny | 13420002340 | 1234123412341234 | 1234-1234-1234-1234 | smithWu@163.com | 10000.0000 | 1999-10-02 00:00:00 + 2 | bob | 18299023211 | 3456345634563456 | 3456-3456-3456-3456 | 66allen_mm@qq.com | 9999.9900 | 1989-12-12 00:00:00 + 3 | cici | 15512231233 | | | jonesishere@sina.com | | 1992-11-06 00:00:00 +(3 rows) + +SET ROLE july PASSWORD '{Password}'; +SELECT * FROM emp; + id | name | phone_no | card_no | card_string | email | salary | birthday +----+------+-------------+---------+---------------------+----------------------+------------+--------------------- + 1 | anny | 13420002340 | 0 | ####-####-####-1234 | smithWu@163.com | 99999.9990 | 1999-10-02 00:00:00 + 2 | bob | 18299023211 | 0 | ####-####-####-3456 | 66allen_mm@qq.com | 9999.9990 | 1989-12-12 00:00:00 + 3 | cici | 15512231233 | | | jonesishere@sina.com | | 1992-11-06 00:00:00 +(3 rows) +
- The information in the phone_no, email, and birthday columns is private data. Update redaction policy mask_emp and add three redaction columns.++ +
1 +2 +3 +4
SET ROLE alice PASSWORD '{Password}'; +ALTER REDACTION POLICY mask_emp ON emp ADD COLUMN phone_no WITH mask_partial(phone_no, '*', 4); +ALTER REDACTION POLICY mask_emp ON emp ADD COLUMN email WITH mask_partial(email, '*', 1, position('@' in email)); +ALTER REDACTION POLICY mask_emp ON emp ADD COLUMN birthday WITH mask_full(birthday); +
- Switch to july and view data in the emp table.++ +
1 +2 +3 +4 +5 +6 +7 +8
SET ROLE july PASSWORD '{Password}'; +SELECT * FROM emp; + id | name | phone_no | card_no | card_string | email | salary | birthday +----+------+-------------+---------+---------------------+----------------------+------------+--------------------- + 1 | anny | 134******** | 0 | ####-####-####-1234 | ********163.com | 99999.9990 | 1970-01-01 00:00:00 + 2 | bob | 182******** | 0 | ####-####-####-3456 | ***********qq.com | 9999.9990 | 1970-01-01 00:00:00 + 3 | cici | 155******** | | | ************sina.com | | 1970-01-01 00:00:00 +(3 rows) +
- Query redaction_policies and redaction_columns to view details about the current redaction policy mask_emp.++ +
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16
SELECT * FROM redaction_policies; + object_schema | object_owner | object_name | policy_name | expression | enable | policy_description | inherited +---------------+--------------+-------------+-------------+-----------------------------------+--------+--------------------+----------- + public | alice | emp | mask_emp | ("current_user"() = 'july'::name) | t | | f +(1 row) + +SELECT object_name, column_name, function_info FROM redaction_columns; + object_name | column_name | function_info +-------------+-------------+------------------------------------------------------------------------------------------------------- + emp | card_no | mask_full(card_no) + emp | card_string | mask_partial(card_string, 'VVVVFVVVVFVVVVFVVVV'::text, 'VVVV-VVVV-VVVV-VVVV'::text, '#'::text, 1, 12) + emp | email | mask_partial(email, '*'::text, 1, "position"(email, '@'::text)) + emp | salary | mask_partial(salary, '9'::text, 1, (length((salary)::text) - 2)) + emp | birthday | mask_full(birthday) + emp | phone_no | mask_partial(phone_no, '*'::text, 4) +(6 rows) +
- Add the salary_info column. To replace the salary information in text format with *.*, you can create a user-defined redaction function. In this step, you can use the PL/pgSQL to define the redaction function mask_regexp_salary. To create a redaction column, you simply need to customize the function name and parameter list. For details, see User-Defined Functions.++ +
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20 +21 +22
SET ROLE alice PASSWORD '{Password}'; + +ALTER TABLE emp ADD COLUMN salary_info TEXT; +UPDATE emp SET salary_info = salary::text; + +CREATE FUNCTION mask_regexp_salary(salary_info text) RETURNS text AS +$$ + SELECT regexp_replace($1, '[0-9]+','*','g'); +$$ + LANGUAGE SQL +STRICT SHIPPABLE; + +ALTER REDACTION POLICY mask_emp ON emp ADD COLUMN salary_info WITH mask_regexp_salary(salary_info); + +SET ROLE july PASSWORD '{Password}'; +SELECT id, name, salary_info FROM emp; + id | name | salary_info +----+------+------------- + 1 | anny | *.* + 2 | bob | *.* + 3 | cici | +(3 rows) +
- If there is no need to set a redaction policy for the emp table, delete redaction policy mask_emp.++ +
1 +2
SET ROLE alice PASSWORD '{Password}'; +DROP REDACTION POLICY mask_emp ON emp; +
- After connecting to the database as the administrator, create roles alice, matu, and july.
diff --git a/docs/dws/dev/dws_04_0063.html b/docs/dws/dev/dws_04_0063.html index 7bc8e0e7..ef9d2bd1 100644 --- a/docs/dws/dev/dws_04_0063.html +++ b/docs/dws/dev/dws_04_0063.html @@ -1,17 +1,380 @@ - + -Setting Security Policies
- +User-defined Password Policy
+When creating or modifying a user, you need to specify a password. GaussDB(DWS) has default password complexity requirements. You can also define database account password policies.
++Default GaussDB(DWS) Password Policy
By default, GaussDB(DWS) verifies the password complexity (that is, the GUC parameter password_policy is set to 1 by default). The default password policy requires that the password:
+- Contain 8 to 32 characters.
- Contain at least three types of the following characters: uppercase letters, lowercase letters, digits, and special characters.
- Cannot be the same as the user name or the user name in reverse order, case insensitive.
- Cannot be the current password or the current password in reverse order.
+User-defined Password Policy
The password policy includes the password complexity requirements, password validity period, password reuse settings, password encryption mode, and password retry and lock policies. Different policy items are controlled by the corresponding GUC parameters. For details, see Security and Authentication (postgresql.conf).
+ ++ +
+Table 1 User-defined password policies and corresponding GUC parameters Password Policy
+Parameter
+Description
+Value Range
+Default Value in GaussDB(DWS)
+Password complexity check
+password_policy
+Specifies whether to check the password complexity when a GaussDB(DWS) account is created or modified.
+- 0 indicates that no password complexity policy is used. Setting this parameter to 0 leads to security risks. You are advised not to set this parameter to 0.
- 1 indicates that the default password complexity policy is used.
1
+Password complexity requirement
+ + + + + +password_min_length
+Specifies the minimum password length.
+An integer ranging from 6 to 999
+8
+password_max_length
+Specifies the maximum password length.
+An integer ranging from 6 to 999
+32
+password_min_uppercase
+Minimum number of uppercase letters (A-Z)
+An integer ranging from 0 to 999
+- 0 means no requirements.
- 1-999 indicates the minimum number of uppercase letters in the password.
0
+password_min_lowercase
+Minimum number of lowercase letters (a-z)
+An integer ranging from 0 to 999
+- 0 means no requirements.
- 1-999 indicates the minimum number of lower letters in the password.
0
+password_min_digital
+Minimum number of digits (0-9)
+An integer ranging from 0 to 999
+- 0 means no requirements.
- 1-999 indicates the minimum number of digits in the password.
0
+password_min_special
+Minimum number of special characters (password_min_special)
+An integer ranging from 0 to 999
+- 0 means no requirements.
- 1-999 indicates the minimum number of special characters in the password.
0
+Password validity
+ +password_effect_time
+Password validity period When the number of days in advance a user is notified that the password is about to expire reaches the value of password_notify_time, the system prompts the user to change the password when the user logs in to the database.
+The value is a floating point number ranging from 0 to 999. The unit is day.
+- 0 indicates the validity period is disabled.
- A floating point number from 1 to 999 indicates the validity period of the password. When the password is about to expire or has expired, the system prompts the user to change the password.
90
+password_notify_time
+Specifies for how many days you are reminded of the password expiry.
+The value is an integer ranging from 0 to 999. The unit is day.
+- 0 indicates the reminder is disabled.
- A value ranging from 1 to 999 indicates the number of days prior to password expiration that a user will receive a notification.
7
+Password reuse settings
+password_reuse_time
+Specifies the number of days after which the password cannot be reused.
+A Floating point number ranging from 0 to 3650. The unit is day.
+- 0 indicates that the password reuse days are not checked.
- A positive number indicates that the new password cannot be chosen from passwords in history that are newer than the specified number of days.
60
+password_reuse_max
+Specifies the number of the most recent passwords that the new password cannot be chosen from.
+An integer ranging from 0 to 1000
+- 0 indicates that the password reuse times are not checked.
- A positive number indicates that the new password cannot be chosen from the specified number of the most recent passwords.
0
+Encryption mode
+password_encryption_type
+Specifies the password storage encryption mode.
+0, 1, 2
+- 0 indicates that passwords are encrypted in MD5 mode. The password is encrypted using MD5. This mode is not recommended for users.
- 1 indicates that passwords are encrypted with SHA-256, which is compatible with the MD5 user authentication method of the PostgreSQL client. The password is stored in ciphertext encrypted by MD5 and SHA256.
- 2 indicates that passwords are encrypted using SHA-256. The password is encrypted using SHA256.
1
+Retry and lock
+ +password_lock_time
+Specifies the duration for a locked account to be automatically unlocked.
+A Floating point number ranging from 0 to 365. The unit is day.
+- 0 indicates that the account is not automatically locked if the password verification fails.
- A positive number indicates the duration after which a locked account is automatically unlocked.NOTE:+
The integral part of the value of the password_lock_time parameter indicates the number of days and its decimal part can be converted into hours, minutes, and seconds.
+
1
+failed_login_attempts
+If the number of incorrect password attempts reaches the value of failed_login_attempts, the account is locked and will be automatically unlocked in X (which indicates the value of password_lock_time) seconds.
+An integer ranging from 0 to 1000
+- 0 indicates that the automatic locking function does not take effect.
- A positive number indicates that an account is locked when the number of incorrect password attempts reaches the value of failed_login_attempts.
10
++
+Table 2 Special characters No.
+Character
+No.
+Character
+No.
+Character
+No.
+Character
+1
+~
+9
+*
+17
+|
+25
+<
+2
+!
+10
+(
+18
+[
+26
+.
+3
+@
+11
+)
+19
+{
+27
+>
+4
+#
+12
+-
+20
+}
+28
+/
+5
+$
+13
+_
+21
+]
+29
+?
+6
+%
+14
+=
+22
+;
+-
+-
+7
+^
+15
++
+23
+:
+-
+-
+8
+&
+16
+\
+24
+,
+-
+-
++Example of User-defined Password Policies
Example 1: Configure the password complexity parameter password_policy.
+- Log in to the GaussDB(DWS) management console.
- In the navigation pane on the left, choose Clusters.
- In the cluster list, find the target cluster and click the cluster name. The Cluster Information page is displayed.
- Click the Parameters tab, change the value of password_policy, and click Save. The password_policy parameter takes effect immediately after being modified. You do not need to restart the cluster.
Figure 1 password_policy+
Example 2: Configure password_effect_time for password validity period.
+- Log in to the GaussDB(DWS) management console.
- In the navigation pane on the left, choose Clusters.
- In the cluster list, find the target cluster and click the cluster name. The Cluster Information page is displayed.
- Click the Parameters tab, change the value of password_effect_time, and click Save. The modification of password_effect_time takes effect immediately. You do not need to restart the cluster.Figure 2 password_effect_time+
+Setting and Changing a Password
- Both system administrators and common users need to periodically change their passwords to prevent the accounts from being stolen.
For example, to change the password of the user user1, connect to the database as the administrator and run the following command:
+++ +1ALTER USER user1 IDENTIFIED BY 'newpassword' REPLACE 'oldpassword'; +
+ The password must meet input requirements, or the execution will fail.
+ - An administrator can change its own password and other accounts' passwords. With the permission for changing other accounts' passwords, the administrator can resolve a login failure when a user forgets its password.
To change the password of the user joe, run the following command:
+++ +1ALTER USER joe IDENTIFIED BY 'password'; +
+ - System administrators are not allowed to change passwords for each other.
- When a system administrator changes the password of a common user, the original password is not required.
- However, when a system administrator changes its own password, the original password is required.
- Password verification
Password verification is required when you set the user or role in the current session. If the entered password is inconsistent with the stored password of the user, an error is reported.
+To set the password of the user joe, run the following command:
+++ +1SET ROLE joe PASSWORD 'password'; +
If the following information is displayed, the role setting has been modified:
+SET ROLE
+
--
-
- Setting Account Security Policies
-
- - Setting the Validity Period of an Account
-
- - Setting a User Password
-
-
diff --git a/docs/dws/dev/dws_04_0064.html b/docs/dws/dev/dws_04_0064.html deleted file mode 100644 index 39a31ca3..00000000 --- a/docs/dws/dev/dws_04_0064.html +++ /dev/null @@ -1,35 +0,0 @@ - - -Parent topic: Managing Users and Their PermissionsSetting Account Security Policies
---Background
For data security purposes, GaussDB(DWS) provides a series of security measures, such as automatically locking and unlocking accounts, manually locking and unlocking abnormal accounts, and deleting accounts that are no longer used.
--Automatically Locking and Unlocking Accounts
- If a user fails to enter the correct password for over 10 times during database connection, the system automatically locks the account.
- An account is automatically unlocked one day after it was locked.
-Manually Locking and Unlocking Accounts
If administrators detect an abnormal account that may be stolen or illegally accesses the database, they can manually lock the account.
-The administrator can also manually unlock the account if the account becomes normal again.
-For details about how to create a user, see Users. To manually lock and unlock user joe, run commands in the following format:
-- To manually lock the user:-- -
1ALTER USER joe ACCOUNT LOCK; -
- To manually unlock the user:-- -
1ALTER USER joe ACCOUNT UNLOCK; -
-Deleting Accounts that Are No Longer Used
An administrator can delete an account that is no longer used. This operation cannot be rolled back.
-When an account to be deleted is in the active state, it is deleted after the session is disconnected.
-For example, if you want to delete account joe, run the command in the following format:
--- -1DROP USER joe CASCADE; -
-- diff --git a/docs/dws/dev/dws_04_0065.html b/docs/dws/dev/dws_04_0065.html deleted file mode 100644 index 9e4f6ac8..00000000 --- a/docs/dws/dev/dws_04_0065.html +++ /dev/null @@ -1,25 +0,0 @@ - - ---Parent topic: Setting Security Policies-Setting the Validity Period of an Account
---Precautions
When creating a user, you need to specify the validity period of the user, including the start time and end time.
-To enable a user not within the validity period to use its account, set a new validity period.
--Procedure
- Run the following command to create a user and specify the start time and end time.-- -
1CREATE USER joe WITH PASSWORD 'password' VALID BEGIN '2015-10-10 08:00:00' VALID UNTIL '2016-10-10 08:00:00'; -
- If the user is not within the specified validity period, run the following command to set the start time and end time of a new validity period.-- -
1ALTER USER joe WITH VALID BEGIN '2016-11-10 08:00:00' VALID UNTIL '2017-11-10 08:00:00'; -
- If VALID BEGIN is not specified in the CREATE ROLE or ALTER ROLE statement, the start time of the validity period is not limited. If VALID UNTIL is not specified, the end time of the validity period is not limited. If both of the parameters are not specified, the user is always valid.
--- diff --git a/docs/dws/dev/dws_04_0066.html b/docs/dws/dev/dws_04_0066.html index afd91c73..eb12af39 100644 --- a/docs/dws/dev/dws_04_0066.html +++ b/docs/dws/dev/dws_04_0066.html @@ -1,15 +1,15 @@ - +--Parent topic: Setting Security Policies-Overview
-MRS is a big data cluster running based on the open-source Hadoop ecosystem. It provides the industry's latest cutting-edge storage and analytical capabilities of massive volumes of data, satisfying your data storage and processing requirements. For details, see the MapReduce Service User Guide.
-You can use Hive/Spark (analysis cluster of MRS) to store massive volumes of service data. Hive/Spark data files are stored on HDFS. On GaussDB(DWS), you can connect a GaussDB(DWS) cluster to an MRS cluster, read data from HDFS files, and write the data to GaussDB(DWS) when the clusters are on the same network.
-
Ensure that MRS can communicate with DWS:
-Scenario 1: If MRS and DWS are in the same region and VPC, they can communicate with each other by default.
-Scenario 2: If MRS and DWS are in the same region but in different VPCs, you need to create a VPC peering connection. For details, see "VPC Peering Connection Overview" in Virtual Private Cloud User Guide.
-Scenario 3: If MRS and DWS are not in the same region. You need to use Cloud Connect (CC) to create network connections. For details, see the user guide of the corresponding service.
-Scenario 4: If MRS is deployed on-premises, you need to use Direct Connect (DC) or Virtual Private Network (VPN) to create network connections. For details, see the user guide of the corresponding service.
+MRS is a big data cluster running based on the open-source Hadoop ecosystem. It provides the industry's latest cutting-edge storage and analytical capabilities of massive volumes of data, satisfying your data storage and processing requirements. For details, see the MapReduce Service User Guide.
+You can use Hive/Spark (analysis cluster of MRS) to store massive volumes of service data. Hive/Spark data files are stored on HDFS. On GaussDB(DWS), you can connect a GaussDB(DWS) cluster to an MRS cluster, read data from HDFS files, and write the data to GaussDB(DWS) when the clusters are on the same network.
+- Ensure that MRS can communicate with DWS:
+Scenario 1: If MRS and DWS are in the same region and VPC, they can communicate with each other by default.
+Scenario 2: If MRS and GaussDB(DWS) are in the same region but in different VPCs, you need to create a VPC peering connection. For details, see "VPC Peering Connection Overview" in Virtual Private Cloud User Guide.
+Scenario 3: If MRS and GaussDB(DWS) are not in the same region, you need to use Cloud Connect (CC) to create network connections. For details, see the user guide of the corresponding service.
+Scenario 4: If MRS is deployed on-premises, you need to use Direct Connect (DC) or Virtual Private Network (VPN) to connect the network. For details, see the User Guide of the corresponding service.
Importing Data from MRS to a GaussDB(DWS) Cluster
- Preparing Data in an MRS Cluster
- (Optional) Manually Creating a Foreign Server
- Creating a Foreign Table
- Importing Data
- Deleting Resources
Importing Data from MRS to a GaussDB(DWS) Cluster
diff --git a/docs/dws/dev/dws_04_0067.html b/docs/dws/dev/dws_04_0067.html deleted file mode 100644 index 70892859..00000000 --- a/docs/dws/dev/dws_04_0067.html +++ /dev/null @@ -1,202 +0,0 @@ - - -diff --git a/docs/dws/dev/dws_04_0075.html b/docs/dws/dev/dws_04_0075.html index ea886ac8..39a06571 100644 --- a/docs/dws/dev/dws_04_0075.html +++ b/docs/dws/dev/dws_04_0075.html @@ -1,9 +1,9 @@ - +Setting a User Password
--User passwords are stored in the system catalog pg_authid. To prevent password leakage, GaussDB(DWS) encrypts and stores the user passwords.
-- Password complexity
The password complexity requirements are as follows:
-- The number of uppercase letters (A–Z) ranges from 0 to 999, the number of lowercase letters (a–z) ranges from 0 to 999, the number of digits ranges from 0 to 999, and the number of special characters ranges from 0 to 999. For details about the special characters, see Table 1.-
A password must contain at least three types of the preceding characters (uppercase letters, lowercase letters, digits, and special characters).
- - A password must contain at least 6 characters, and the default length is 8 characters.
- A password can contain a maximum of 999 characters, and the default length is 32 characters.
- A password must differ from the username or the username in reverse order.
- A new password must be different from the current password or the current password in reverse order.
- The number of uppercase letters (A–Z) ranges from 0 to 999, the number of lowercase letters (a–z) ranges from 0 to 999, the number of digits ranges from 0 to 999, and the number of special characters ranges from 0 to 999. For details about the special characters, see Table 1.
- Password reuse
When a user changes the password, the user can reuse a password only if it has not been used for over 60 days.
- - Password validity period
A validity period (90 days by default) is set for each database user password. If the password is about to expire (in seven days), the system displays a message reminding the user to change it upon login.
-- Considering the usage and service continuity of a database, the database still allows a user to log in after the password expires. A password change notification is displayed every time the user logs in to the database until the password is changed.
- - Password change
- During database installation, an OS user named the same as the initial user is created. The password of the OS user needs to be changed periodically to ensure account security.
-
Change the password as prompted.
- - Both system administrators and common users need to periodically change their passwords to prevent the accounts from being stolen.
For example, to change the password of the user user1, connect to the database as the administrator and run the following command:
--- -1ALTER USER user1 IDENTIFIED BY "1234@abc" REPLACE "5678@def"; -
- 1234@abc and 5678@def represent the new password and the original password of the user user1, respectively. The new password must conform to the complexity rules. Otherwise, the new password is invalid.
- - An administrator can change its own password and other accounts' passwords. With the permission for changing other accounts' passwords, the administrator can resolve a login failure when a user forgets its password.
To change the password of the user joe, run the following command:
--- -1ALTER USER joe IDENTIFIED BY 'password'; -
- - System administrators are not allowed to change passwords for each other.
- When a system administrator changes the password of a common user, the original password is not required.
- However, when a system administrator changes its own password, the original password is required.
- During database installation, an OS user named the same as the initial user is created. The password of the OS user needs to be changed periodically to ensure account security.
-
- Password verification
Password verification is required when you set the user or role in the current session. If the entered password is inconsistent with the stored password of the user, an error is reported.
-To set the password of the user joe, run the following command:
--- -1SET ROLE joe PASSWORD 'password'; -
-
-Table 1 Special characters No.
-Character
-No.
-Character
-No.
-Character
-No.
-Character
-1
-~
-9
-*
-17
-|
-25
-<
-2
-!
-10
-(
-18
-[
-26
-.
-3
-@
-11
-)
-19
-{
-27
->
-4
-#
-12
--
-20
-}
-28
-/
-5
-$
-13
-_
-21
-]
-29
-?
-6
-%
-14
-=
-22
-;
--
--
-7
-^
-15
-+
-23
-:
--
--
-8
-&
-16
-\
-24
-,
--
--
--- diff --git a/docs/dws/dev/dws_04_0074.html b/docs/dws/dev/dws_04_0074.html index 6cdc50b3..3bdda0a2 100644 --- a/docs/dws/dev/dws_04_0074.html +++ b/docs/dws/dev/dws_04_0074.html @@ -1,4 +1,4 @@ - +--Parent topic: Setting Security Policies-Development and Design Proposal
@@ -14,14 +14,10 @@- SQL Compilation
-
- PL/Java Usage
+- User-defined External Function Usage (pgSQL/Java)
- PL/pgSQL Usage
- -
-Parent topic: Developer Guide-Development and Design Proposal
-This chapter describes the design specifications for database modeling and application development. Modeling compliant with these specifications fits the distributed processing architecture of GaussDB(DWS) and provides efficient SQL code.
-The meaning of "Proposal" and "Notice" in this chapter is as follows:
-- Proposal: Design rules. Services compliant with the rules can run efficiently, and those violating the rules may have low performance or logic errors.
- Notice: Details requiring attention during service development. This term identifies SQL behavior that complies with SQL standards but users may have misconceptions about, and default behavior that users may be unaware of in a program.
This chapter describes the design specifications for database modeling and application development. Modeling compliant with these specifications fits the distributed processing architecture of GaussDB(DWS) and provides efficient SQL code.
+The meaning of "Proposal" and "Notice" in this chapter is as follows:
+- Proposal: Design rules. Services compliant with the rules can run efficiently, and those violating the rules may have low performance or logic errors.
- Notice: Details requiring attention during service development. This term identifies SQL behavior that complies with SQL standards but users may have misconceptions about, and default behavior that users may be unaware of in a program.
diff --git a/docs/dws/dev/dws_04_0076.html b/docs/dws/dev/dws_04_0076.html index b491d1ad..a11a5185 100644 --- a/docs/dws/dev/dws_04_0076.html +++ b/docs/dws/dev/dws_04_0076.html @@ -1,11 +1,11 @@ - +Database Object Naming Conventions
-The name of a database object must contain 1 to 63 characters, start with a letter or underscore (_), and can contain letters, digits, underscores (_), dollar signs ($), and number signs (#).
-- [Proposal] Do not use reserved or non-reserved keywords to name database objects.
To query the keywords of GaussDB(DWS), run select * from pg_get_keywords() or refer to section "Keyword."
+The name of a database object must contain 1 to 63 characters, start with a letter or underscore (_), and can contain letters, digits, underscores (_), dollar signs ($), and number signs (#).
+- [Proposal] Do not use reserved or non-reserved keywords to name database objects.-
You can run SELECT * FROM pg_get_keywords() to query GaussDB(DWS) keywords or view the keywords in section "Keywords" in SQL Syntax Reference.
- [Proposal] Do not use a string enclosed in double quotation marks ("") to define the database object name, unless you need to specify its capitalization. Case sensitivity of database object names makes problem location difficult.
- [Proposal] Use the same naming format for database objects.
- In a system undergoing incremental development or service migration, you are advised to comply with its historical naming conventions.
- A database object name consists of letters, digits, and underscores (_); and cannot start with a digit. You are advised to use multiple words separated with hyphens (-).
- You are advised to use intelligible names and common acronyms or abbreviations for database objects. Acronyms or abbreviations that are generally understood are recommended. For example, you can use English words indicating actual business terms. The naming format should be consistent within a cluster.
- A variable name must be descriptive and meaningful. It must have a prefix indicating its type.
- [Proposal] The name of a table object should indicate its main characteristics, for example, whether it is an ordinary, temporary, or unlogged table.
- An ordinary table name should indicate the business relevant to a data set.
- Temporary tables are named in the format of tmp_Suffix.
- Unlogged tables are named in the format of ul_Suffix.
- Foreign tables are named in the format of f_Suffix.
- [Proposal] Do not use strings enclosed in double quotation marks to define database object names. In GaussDB(DWS), double quotation marks are used to specify that the enclosed database object names are case sensitive. Case sensitivity of database object names makes problem location difficult.
- [Proposal] Use the same naming format for database objects.
- In a system undergoing incremental development or service migration, you are advised to comply with its historical naming conventions.
- A database object name consists of letters, digits, and underscores (_); and cannot start with a digit. You are advised to use multiple words separated with hyphens (-).
- You are advised to use intelligible names and common acronyms or abbreviations for database objects. Acronyms or abbreviations that are generally understood are recommended. For example, you can use English words indicating actual business terms. The naming format should be consistent within a cluster.
- A variable name must be descriptive and meaningful. It must have a prefix indicating its type.
- [Proposal] The name of a table object should indicate its main characteristics, for example, whether it is an ordinary, temporary, or unlogged table.
- An ordinary table name should indicate the business relevant to a data set.
- Temporary tables are named in the format of tmp_Suffix.
- Unlogged tables are named in the format of ul_Suffix.
- Foreign tables are named in the format of f_Suffix.
diff --git a/docs/dws/dev/dws_04_0077.html b/docs/dws/dev/dws_04_0077.html index ed25866a..faa451bf 100644 --- a/docs/dws/dev/dws_04_0077.html +++ b/docs/dws/dev/dws_04_0077.html @@ -1,4 +1,4 @@ - +Database Object Design
diff --git a/docs/dws/dev/dws_04_0078.html b/docs/dws/dev/dws_04_0078.html index fb33fc70..ff8818f4 100644 --- a/docs/dws/dev/dws_04_0078.html +++ b/docs/dws/dev/dws_04_0078.html @@ -1,11 +1,11 @@ - +Database and Schema Design
-In GaussDB(DWS), services can be isolated by databases and schemas. Databases share little resources and cannot directly access each other. Connections to and permissions on them are also isolated. Schemas share more resources than databases do. User permissions on schemas and subordinate objects can be controlled using the GRANT and REVOKE syntax.
-- You are advised to use schemas to isolate services for convenience and resource sharing.
- It is recommended that system administrators create schemas and databases and then assign required permissions to users.
Database Design Suggestions
- Create databases as required. Do not use the default gaussdb database of a cluster.
- Create a maximum of three user-defined databases in a cluster.
- To make your database encoding compatible with most characters, you are advised to use the UTF-8 encoding when creating a database.
- Exercise caution when you set ENCODING and DBCOMPATIBILITY configuration items during database creation. In GaussDB(DWS), DBCOMPATIBILITY can be set to TD, Oracle, or MySQL to be compatible with Teradata, Oracle, or MySQL syntax, respectively. Syntax behavior may vary with the three modes. For details, see Syntax Compatibility Differences Among Oracle, Teradata, and MySQL.
- By default, a database owner has all permissions for all objects in the database, including the deletion permission. Exercise caution when using the deletion permission.
In GaussDB(DWS), services can be isolated by databases and schemas. Databases share little resources and cannot directly access each other. Connections to and permissions on them are also isolated. Schemas share more resources than databases do. User permissions on schemas and subordinate objects can be controlled using the GRANT and REVOKE syntax.
+- You are advised to use schemas to isolate services for convenience and resource sharing.
- It is recommended that system administrators create schemas and databases and then assign required permissions to users.
-Database Design Suggestions
- Create databases as required. Do not use the default gaussdb database of a cluster.
- Create a maximum of three user-defined databases in a cluster.
- To make your database encoding compatible with most characters, you are advised to use the UTF-8 encoding when creating a database.
- Exercise caution when you set ENCODING and DBCOMPATIBILITY configuration items during database creation. In GaussDB(DWS), DBCOMPATIBILITY can be set to TD, Oracle, or MySQL to be compatible with Teradata, Oracle, or MySQL syntax, respectively. Syntax behavior may vary with the three modes. For details, see Syntax Compatibility Differences Among Oracle, Teradata, and MySQL.
- By default, a database owner has all permissions for all objects in the database, including the deletion permission. Exercise caution when using the deletion permission.
Schema Design Suggestions
- To let a user access an object in a schema, grant the usage permission and the permissions for the object to the user, unless the user has the sysadmin permission or is the schema owner.
- To let a user create an object in the schema, grant the create permission for the schema to the user.
- By default, a schema owner has all permissions for all objects in the schema, including the deletion permission. Exercise caution when using the deletion permission.
Schema Design Suggestions
- To let a user access an object in a schema, grant the usage permission and the permissions for the object to the user, unless the user has the sysadmin permission or is the schema owner.
- To let a user create an object in the schema, grant the CREATE permission for the schema to the user.
- By default, a schema owner has all permissions for all objects in the schema, including the deletion permission. Exercise caution when using the deletion permission.
diff --git a/docs/dws/dev/dws_04_0079.html b/docs/dws/dev/dws_04_0079.html index de397c74..b8abe40e 100644 --- a/docs/dws/dev/dws_04_0079.html +++ b/docs/dws/dev/dws_04_0079.html @@ -1,51 +1,70 @@ - +Table Design
-GaussDB(DWS) uses a distributed architecture. Data is distributed on DNs. Comply with the following principles to properly design a table:
-- [Notice] Evenly distribute data on each DN to prevent data skew. If most data is stored on several DNs, the effective capacity of a cluster decreases. Select a proper distribution column to avoid data skew.
- [Notice] Evenly scan each DN when querying tables. Otherwise, DNs most frequently scanned will become the performance bottleneck. For example, when you use equivalent filter conditions on a fact table, the nodes are not evenly scanned.
- [Notice] Reduce the amount of data to be scanned. You can use the pruning mechanism of a partitioned table.
- [Notice] Minimize random I/O. By clustering or local clustering, you can sequentially store hot data, converting random I/O to sequential I/O to reduce the cost of I/O scanning.
- [Notice] Try to avoid data shuffling. To shuffle data is to physically transfer it from one node to another. This unnecessarily occupies many network resources. To reduce network pressure, locally process data, and to improve cluster performance and concurrency, you can minimize data shuffling by using proper association and grouping conditions.
Selecting a Storage Mode
[Proposal] Selecting a storage mode is the first step in defining a table. The storage mode mainly depends on the customer's service type. For details, see Table 1.
+GaussDB(DWS) uses a distributed architecture. Data is distributed on DNs. Comply with the following principles to properly design a table:
+- [Notice] Evenly distribute data on each DN to prevent data skew. If most data is stored on several DNs, the effective capacity of a cluster decreases. Select a proper distribution column to avoid data skew.
- [Notice] Evenly scan each DN when querying tables. Otherwise, DNs most frequently scanned will become the performance bottleneck. For example, when you use equivalent filter conditions on a fact table, the nodes are not evenly scanned.
- [Notice] Reduce the amount of data to be scanned. You can use the pruning mechanism of a partitioned table.
- [Notice] Minimize random I/O. By clustering or local clustering, you can sequentially store hot data, converting random I/O to sequential I/O to reduce the cost of I/O scanning.
- [Notice] Try to avoid data shuffling. To shuffle data is to physically transfer it from one node to another. This unnecessarily occupies many network resources. To reduce network pressure, locally process data, and to improve cluster performance and concurrency, you can minimize data shuffling by using proper association and grouping conditions.
Selecting a Storage Mode
[Proposal] Selecting a storage mode is the first step in defining a table. The storage mode mainly depends on the user's service type. For details, see Table 1.
-Table 1 Table storage modes and scenarios Storage Mode
+-Table 1 Table storage modes and scenarios Storage Mode
Application Scenarios
+Benefit
+Drawback
+Application Scenarios
Row storage
+Row storage
- Point queries (simple index-based queries that only return a few records)
- Scenarios requiring frequent addition, deletion, and modification
Data is stored by row. When you query a row of data, you can quickly locate the target row.
+All data in the queried row is read while only a few columns are needed.
+- The number of columns in the table is small, and most fields in the table are queried.
- Point queries (simple index–based query that returns only a few records) are performed.
- Add, Delete, Modify, and Query operations on entire rows are frequently performed.
Column storage
+Column storage
- Statistical analysis queries (requiring a large number of association and grouping operations)
- Ad hoc queries (using uncertain query conditions and unable to utilize indexes to scan row-store tables)
- Only necessary columns in a query are read.
- The homogeneity of data within a column facilitates efficient compression.
It is not suitable for INSERT or UPDATE operations on a small amount of data.
+- Query a few columns in a table that contains a large number of columns.
- Statistical analysis queries (requiring a large number of association and grouping operations)
- Ad hoc queries (using uncertain query conditions and unable to utilize indexes to scan row-store tables)
Selecting a Distribution Mode
[Proposal] Comply with the following rules to distribute table data. -Table 2 Table distribution modes and scenarios Distribution Mode
+Selecting a Distribution Mode
[Proposal] Comply with the following rules to distribute table data. +Table 2 Table distribution modes and scenarios Distribution Mode
Description
+Description
Application Scenarios
+Application Scenarios
Hash
+Hash
Table data is distributed on all DNs in a cluster by hash.
+Table data is distributed on all DNs in a cluster by hash.
Fact tables containing a large amount of data
+Fact tables containing a large amount of data
Replication
+Replication
Full data in a table is stored on every DN in a cluster.
+Full data in a table is stored on every DN in a cluster.
Dimension tables and fact tables containing a small amount of data
+Dimension tables and fact tables containing a small amount of data
+Round-robin
+Each row of the table is sent to each DN in turn. Therefore, data is evenly distributed on each DN.
+Fact tables that contain a large amount of data and cannot find a proper distribution column in hash mode
Selecting a Partitioning Mode
Comply with the following rules to partition a table containing a large amount of data:
-- [Proposal] Create partitions on columns that indicate certain ranges, such as dates and regions.
- [Proposal] A partition name should show the data characteristics of a partition. For example, its format can be Keyword+Range characteristics.
- [Proposal] Set the upper limit of a partition to MAXVALUE to prevent data overflow.
The example of a partitioned table definition is as follows:
-1 +
-Selecting a Partitioning Mode
Comply with the following rules to partition a table containing a large amount of data:
+- [Proposal] Create partitions on columns that indicate certain ranges, such as dates and regions.
- [Proposal] A partition name should show the data characteristics of a partition. For example, its format can be Keyword+Range characteristics.
- [Proposal] Set the upper limit of a partition to MAXVALUE to prevent data overflow.
The example of a partitioned table definition is as follows:
+1 2 3 4 @@ -75,32 +94,32 @@ 17 18 19 -20
CREATE TABLE staffS_p1 -( - staff_ID NUMBER(6) not null, - FIRST_NAME VARCHAR2(20), - LAST_NAME VARCHAR2(25), - EMAIL VARCHAR2(25), - PHONE_NUMBER VARCHAR2(20), - HIRE_DATE DATE, - employment_ID VARCHAR2(10), - SALARY NUMBER(8,2), - COMMISSION_PCT NUMBER(4,2), - MANAGER_ID NUMBER(6), - section_ID NUMBER(4) -) -PARTITION BY RANGE (HIRE_DATE) +20
CREATE TABLE staffS_p1 +( + staff_ID NUMBER(6) not null, + FIRST_NAME VARCHAR2(20), + LAST_NAME VARCHAR2(25), + EMAIL VARCHAR2(25), + PHONE_NUMBER VARCHAR2(20), + HIRE_DATE DATE, + employment_ID VARCHAR2(10), + SALARY NUMBER(8,2), + COMMISSION_PCT NUMBER(4,2), + MANAGER_ID NUMBER(6), + section_ID NUMBER(4) +) +PARTITION BY RANGE (HIRE_DATE) ( - PARTITION HIRE_19950501 VALUES LESS THAN ('1995-05-01 00:00:00'), - PARTITION HIRE_19950502 VALUES LESS THAN ('1995-05-02 00:00:00'), - PARTITION HIRE_maxvalue VALUES LESS THAN (MAXVALUE) -); + PARTITION HIRE_19950501 VALUES LESS THAN ('1995-05-01 00:00:00'), + PARTITION HIRE_19950502 VALUES LESS THAN ('1995-05-02 00:00:00'), + PARTITION HIRE_maxvalue VALUES LESS THAN (MAXVALUE) +);
Selecting a Distribution Key
Selecting a distribution key is important for a hash table. An improper distribution key may cause data skew. As a result, the I/O load is heavy on several DNs, affecting the overall query performance. After you select a distribution policy for a hash table, check for data skew to ensure that data is evenly distributed. Comply with the following rules to select a distribution key:
-- [Proposal] Select a column containing discrete data as the distribution key, so that data can be evenly distributed on each DN. If a single column is not discrete enough, consider using multiple columns as distribution keys. You can select the primary key of a table as the distribution key. For example, in an employee information table, select the certificate number column as the distribution key.
- [Proposal] If the first rule is met, do not select a column having constant filter conditions as the distribution key. For example, in a query on the dwcjk table, if the zqdh column contains the constant filter condition zqdh='000001', avoid selecting the zqdh column as the distribution key.
- [Proposal] If the first and second rules are met, select the join conditions in a query as distribution keys. If a join condition is used as a distribution key, the data involved in a join task is locally distributed on DNs, which greatly reduces the data flow cost among DNs.
Selecting a Distribution Key
Selecting a distribution key is important for a hash table. An improper distribution key may cause data skew. As a result, the I/O load is heavy on several DNs, affecting the overall query performance. After you select a distribution policy for a hash table, check for data skew to ensure that data is evenly distributed. Comply with the following rules to select a distribution key:
+- [Proposal] Select a column containing discrete data as the distribution key, so that data can be evenly distributed on each DN. If a single column is not discrete enough, consider using multiple columns as distribution keys. You can select the primary key of a table as the distribution key. For example, in an employee information table, select the certificate number column as the distribution key.
- [Proposal] If the first rule is met, do not select a column having constant filter conditions as the distribution key. For example, in a query on the dwcjk table, if the zqdh column contains the constant filter condition zqdh='000001', avoid selecting the zqdh column as the distribution key.
- [Proposal] If the first and second rules are met, select the join conditions in a query as distribution keys. If a join condition is used as a distribution key, the data involved in a join task is locally distributed on DNs, which greatly reduces the data flow cost among DNs.
diff --git a/docs/dws/dev/dws_04_0080.html b/docs/dws/dev/dws_04_0080.html index 25ce142d..8153f1b2 100644 --- a/docs/dws/dev/dws_04_0080.html +++ b/docs/dws/dev/dws_04_0080.html @@ -1,83 +1,83 @@ - +Column Design
-Selecting a Data Type
Comply with the following rules to improve query efficiency when you design columns:
-- [Proposal] Use the most efficient data types allowed.
If all of the following number types provide the required service precision, they are recommended in descending order of priority: integer, floating point, and numeric.
- - [Proposal] In tables that are logically related, columns having the same meaning should use the same data type.
- [Proposal] For string data, you are advised to use variable-length strings and specify the maximum length. To avoid truncation, ensure that the specified maximum length is greater than the maximum number of characters to be stored. You are not advised to use CHAR(n), BPCHAR(n), NCHAR(n), or CHARACTER(n), unless you know that the string length is fixed.
For details about string types, see Common String Types.
+-Selecting a Data Type
Comply with the following rules to improve query efficiency when you design columns:
+- [Proposal] Use the most efficient data types allowed.
If all of the following number types provide the required service precision, they are recommended in descending order of priority: integer, floating point, and numeric.
+ - [Proposal] In tables that are logically related, columns having the same meaning should use the same data type.
- [Proposal] For string data, you are advised to use variable-length strings and specify the maximum length. To avoid truncation, ensure that the specified maximum length is greater than the maximum number of characters to be stored. You are not advised to use CHAR(n), BPCHAR(n), NCHAR(n), or CHARACTER(n), unless you know that the string length is fixed.
For details about string types, see Common String Types.
Common String Types
Every column requires a data type suitable for its data characteristics. The following table lists common string types in GaussDB(DWS).
+-Common String Types
Every column requires a data type suitable for its data characteristics. The following table lists common string types in GaussDB(DWS).
Table 1 Common string types Parameter
+Table 1 Common string types Parameter
Description
+Description
Max. Storage Capacity
+Max. Storage Capacity
CHAR(n)
+CHAR(n)
Fixed-length string, where n indicates the stored bytes. If the length of an input string is smaller than n, the string is automatically padded to n bytes using NULL characters.
+Fixed-length string, where n indicates the stored bytes. If the length of an input string is smaller than n, the string is automatically padded to n bytes using NULL characters.
10 MB
+10 MB
CHARACTER(n)
+CHARACTER(n)
Fixed-length string, where n indicates the stored bytes. If the length of an input string is smaller than n, the string is automatically padded to n bytes using NULL characters.
+Fixed-length string, where n indicates the stored bytes. If the length of an input string is smaller than n, the string is automatically padded to n bytes using NULL characters.
10 MB
+10 MB
NCHAR(n)
+NCHAR(n)
Fixed-length string, where n indicates the stored bytes. If the length of an input string is smaller than n, the string is automatically padded to n bytes using NULL characters.
+Fixed-length string, where n indicates the stored bytes. If the length of an input string is smaller than n, the string is automatically padded to n bytes using NULL characters.
10 MB
+10 MB
BPCHAR(n)
+BPCHAR(n)
Fixed-length string, where n indicates the stored bytes. If the length of an input string is smaller than n, the string is automatically padded to n bytes using NULL characters.
+Fixed-length string, where n indicates the stored bytes. If the length of an input string is smaller than n, the string is automatically padded to n bytes using NULL characters.
10 MB
+10 MB
VARCHAR(n)
+VARCHAR(n)
Variable-length string, where n indicates the maximum number of bytes that can be stored.
+Variable-length string, where n indicates the maximum number of bytes that can be stored.
10 MB
+10 MB
CHARACTER VARYING(n)
+CHARACTER VARYING(n)
Variable-length string, where n indicates the maximum number of bytes that can be stored. This data type and VARCHAR(n) are different representations of the same data type.
+Variable-length string, where n indicates the maximum number of bytes that can be stored. This data type and VARCHAR(n) are different representations of the same data type.
10 MB
+10 MB
VARCHAR2(n)
+VARCHAR2(n)
Variable-length string, where n indicates the maximum number of bytes that can be stored. This data type is added to be compatible with the Oracle database, and its behavior is the same as that of VARCHAR(n).
+Variable-length string, where n indicates the maximum number of bytes that can be stored. This data type is added to be compatible with the Oracle database, and its behavior is the same as that of VARCHAR(n).
10 MB
+10 MB
NVARCHAR2(n)
+NVARCHAR2(n)
Variable-length string, where n indicates the maximum number of bytes that can be stored.
+Variable-length string, where n indicates the maximum number of bytes that can be stored.
10 MB
+10 MB
TEXT
+TEXT
Variable-length string. Its maximum length is 8203 bytes less than 1 GB.
+Variable-length string. Its maximum length is 8203 bytes less than 1 GB.
8203 bytes less than 1 GB
+8203 bytes less than 1 GB
Constraint Design
-DEFAULT and NULL Constraints
- [Proposal] If all the column values can be obtained from services, you are not advised to use the DEFAULT constraint, because doing so will generate unexpected results during data loading.
- [Proposal] Add NOT NULL constraints to columns that never have NULL values. The optimizer automatically optimizes the columns in certain scenarios.
- [Proposal] Explicitly name all constraints excluding NOT NULL and DEFAULT.
-DEFAULT and NULL Constraints
- [Proposal] If all the column values can be obtained from services, you are not advised to use the DEFAULT constraint, because doing so will generate unexpected results during data loading.
- [Proposal] Add NOT NULL constraints to columns that never have NULL values. The optimizer automatically optimizes the columns in certain scenarios.
- [Proposal] Explicitly name all constraints excluding NOT NULL and DEFAULT.
Partial Cluster Key
A partial cluster key (PCK) is a local clustering technology used for column-store tables. After creating a PCK, you can quickly filter and scan fact tables using min or max sparse indexes in GaussDB(DWS). Comply with the following rules to create a PCK:
-- [Notice] Only one PCK can be created in a table. A PCK can contain multiple columns, preferably no more than two columns.
- [Proposal] Create a PCK on simple expression filter conditions in a query. Such filter conditions are usually in the form of col op const, where col specifies a column name, op specifies an operator (such as =, >, >=, <=, and <), and const specifies a constant.
- [Proposal] If the preceding conditions are met, create a PCK on the column having the least distinct values.
-Partial Cluster Key
A partial cluster key (PCK) is a local clustering technology used for column-store tables. After creating a PCK, you can quickly filter and scan fact tables using min or max sparse indexes in GaussDB(DWS). Comply with the following rules to create a PCK:
+- [Notice] Only one PCK can be created in a table. A PCK can contain multiple columns, preferably no more than two columns.
- [Proposal] Create a PCK on simple expression filter conditions in a query. Such filter conditions are usually in the form of col op const, where col specifies a column name, op specifies an operator (such as =, >, >=, <=, and <), and const specifies a constant.
- [Proposal] If the preceding conditions are met, create a PCK on the column having the least distinct values.
Unique Constraint
- [Notice] Both row-store and column-store tables support unique constraints.
- [Proposal] The constraint name should indicate that it is a unique constraint, for example, UNIIncluded columns.
-Unique Constraint
- [Notice] Both row-store and column-store tables support unique constraints.
- [Proposal] The constraint name should indicate that it is a unique constraint, for example, UNIIncluded columns.
Primary Key Constraint
- [Notice] Both row-store and column-store tables support the primary key constraint.
- [Proposal] The constraint name should indicate that it is a primary key constraint, for example, PKIncluded columns.
-Primary Key Constraint
- [Notice] Both row-store and column-store tables support the primary key constraint.
- [Proposal] The constraint name should indicate that it is a primary key constraint, for example, PKIncluded columns.
Check Constraint
- [Notice] Check constraints can be used in row-store tables but not in column-store tables.
- [Proposal] The constraint name should indicate that it is a check constraint, for example, CKIncluded columns.
Check Constraint
- [Notice] Check constraints can be used in row-store tables but not in column-store tables.
- [Proposal] The constraint name should indicate that it is a check constraint, for example, CKIncluded columns.
diff --git a/docs/dws/dev/dws_04_0082.html b/docs/dws/dev/dws_04_0082.html index 8322f025..674779c2 100644 --- a/docs/dws/dev/dws_04_0082.html +++ b/docs/dws/dev/dws_04_0082.html @@ -1,9 +1,9 @@ - +View and Joined Table Design
-View Design
- [Proposal] Do not nest views unless they have strong dependency on each other.
- [Proposal] Try to avoid sort operations in a view definition.
-View Design
- [Proposal] Do not nest views unless they have strong dependency on each other.
- [Proposal] Try to avoid sort operations in a view definition.
Joined Table Design
- [Proposal] Minimize joined columns across tables.
- [Proposal] Joined columns should use the same data type.
- [Proposal] The names of joined columns should indicate their relationship. For example, they can use the same name.
Joined Table Design
- [Proposal] Minimize joined columns across tables.
- [Proposal] Joined columns should use the same data type.
- [Proposal] The names of associated fields should show the associations. For example, they can use the same name.
diff --git a/docs/dws/dev/dws_04_0083.html b/docs/dws/dev/dws_04_0083.html index 1b4497c1..e858123e 100644 --- a/docs/dws/dev/dws_04_0083.html +++ b/docs/dws/dev/dws_04_0083.html @@ -1,8 +1,8 @@ - +JDBC Configuration
-Currently, third-party tools are connected to GaussDB(DWS) trough JDBC. This section describes the precautions for configuring the tools.
-Connection Parameters
- [Notice] When a third-party tool connects to GaussDB(DWS) through JDBC, JDBC sends a connection request to GaussDB(DWS). By default, the following parameters are added. For details, see the implementation of the ConnectionFactoryImpl JDBC code.
params = { +Currently, third-party tools are connected to GaussDB(DWS) trough JDBC. This section describes the precautions for configuring the tools.
+-Connection Parameters
- [Notice] When a third-party tool connects to GaussDB(DWS) through JDBC, JDBC sends a connection request to GaussDB(DWS). By default, the following parameters are added. For details, see the implementation of the ConnectionFactoryImpl JDBC code.
params = { { "user", user }, { "database", database }, { "client_encoding", "UTF8" }, @@ -10,21 +10,22 @@ { "extra_float_digits", "2" }, { "TimeZone", createPostgresTimeZone() }, };-These parameters may cause the JDBC and gsql clients to display inconsistent data, for example, date data display mode, floating point precision representation, and timezone.
-If the result is not as expected, you are advised to explicitly set these parameters in the Java connection setting.
- - [Proposal] When connecting to the database through JDBC, ensure that the following two time zones are the same:
- Time zone of the host where the JDBC client is located
- Time zone of the host where the GaussDB(DWS) server is located
These parameters may cause the JDBC and gsql clients to display inconsistent data, for example, date data display mode, floating point precision representation, and timezone.
+If the result is not as expected, you are advised to explicitly set these parameters in the Java connection setting.
+ - [Proposal] When connecting to the database through JDBC, ensure that the following two time zones are the same:
- Time zone of the host where the JDBC client is located
- Time zone of the host where the GaussDB(DWS) server is located
fetchsize
[Notice] To use fetchsize in applications, disable the autocommit switch. Enabling the autocommit switch makes the fetchsize configuration invalid.
+-fetchsize
[Notice] To use fetchsize in applications, disable the autocommit switch. Enabling the autocommit switch makes the fetchsize configuration invalid.
autocommit
[Proposal] It is recommended that you enable the autocommit switch in the code for connecting to GaussDB(DWS) by the JDBC. If autocommit needs to be disabled to improve performance or for other purposes, applications need to ensure their transactions are committed. For example, explicitly commit translations after specifying service SQL statements. Particularly, ensure that all transactions are committed before the client exits.
+-autocommit
[Proposal] It is recommended that you enable the autocommit switch in the code for connecting to GaussDB(DWS) by the JDBC. If autocommit needs to be disabled to improve performance or for other purposes, applications need to ensure their transactions are committed. For example, explicitly commit translations after specifying service SQL statements. Particularly, ensure that all transactions are committed before the client exits.
Connection Releasing
[Proposal] You are advised to use connection pools to limit the number of connections from applications. Do not connect to a database every time you run an SQL statement.
-[Proposal] After an application completes its tasks, disconnect its connection to GaussDB(DWS) to release occupied resources. You are advised to set the session timeout interval in the task.
-[Proposal] Reset the session environment before releasing connections to the JDBC connection tool. Otherwise, historical session information may cause object conflicts.
-- If GUC parameters are set in the connection, before you return the connection to the connection pool, run SET SESSION AUTHORIZATION DEFAULT;RESET ALL; to clear the connection status.
- If a temporary table is used, delete it before you return the connection to the connection pool.
-Connection Releasing
[Proposal] You are advised to use connection pools to limit the number of connections from applications. Do not connect to a database every time you run an SQL statement.
+[Proposal] After an application completes its tasks, disconnect its connection to GaussDB(DWS) to release occupied resources. You are advised to set the session timeout interval in the task.
+[Proposal] Reset the session environment before releasing connections to the JDBC connection tool. Otherwise, historical session information may cause object conflicts.
+- If GUC parameters are set in the connection, before you return the connection to the connection pool, run SET SESSION AUTHORIZATION DEFAULT;RESET ALL; to clear the connection status.
- If a temporary table is used, delete it before you return the connection to the connection pool.
CopyManager
[Proposal] In the scenario where the ETL tool is not used and real-time data import is required, it is recommended that you use the CopyManger interface driven by the GaussDB(DWS) JDBC to import data in batches during application development. For details about how to use CopyManager, see CopyManager.
+CopyManager
[Proposal] In the scenario where the ETL tool is not used and real-time data import is required, it is recommended that you use the CopyManager interface driven by the GaussDB(DWS) JDBC to import data in batches during application development.
+For details about how to use CopyManager, see CopyManager.
diff --git a/docs/dws/dev/dws_04_0084.html b/docs/dws/dev/dws_04_0084.html index ba246f73..1d0b2a4b 100644 --- a/docs/dws/dev/dws_04_0084.html +++ b/docs/dws/dev/dws_04_0084.html @@ -1,40 +1,40 @@ - +SQL Compilation
-DDL
- [Proposal] In GaussDB(DWS), you are advised to execute DDL operations, such as creating table or making comments, separately from batch processing jobs to avoid performance deterioration caused by many concurrent transactions.
- [Proposal] Execute data truncation after unlogged tables are used because GaussDB(DWS) cannot ensure the security of unlogged tables in abnormal scenarios.
- [Proposal] Suggestions on the storage mode of temporary and unlogged tables are the same as those on base tables. Create temporary tables in the same storage mode as the base tables to avoid high computing costs caused by hybrid row and column correlation.
- [Proposal] The total length of an index column cannot exceed 50 bytes. Otherwise, the index size will increase greatly, resulting in large storage cost and low index performance.
- [Proposal] Do not delete objects using DROP...CASCADE, unless the dependency between objects is specified. Otherwise, the objects may be deleted by mistake.
-DDL
- [Proposal] In GaussDB(DWS), you are advised to execute DDL operations, such as creating table or making comments, separately from batch processing jobs to avoid performance deterioration caused by many concurrent transactions.
- [Proposal] Execute data truncation after unlogged tables are used because GaussDB(DWS) cannot ensure the security of unlogged tables in abnormal scenarios.
- [Proposal] Suggestions on the storage mode of temporary and unlogged tables are the same as those on base tables. Create temporary tables in the same storage mode as the base tables to avoid high computing costs caused by hybrid row and column correlation.
- [Proposal] The total length of an index column cannot exceed 50 bytes. Otherwise, the index size will increase greatly, resulting in large storage cost and low index performance.
- [Proposal] Do not use DROP... CASCADE to delete objects unless the dependencies between objects are specified. Otherwise, objects may be deleted by mistake.
Data Loading and Uninstalling
- [Proposal] Provide the inserted column list in the insert statement. Example:
1INSERT INTO task(name,id,comment) VALUES ('task1','100','100th task'); +
-Data Loading and Uninstalling
- [Proposal] Provide the inserted column list in the insert statement. Example:-
1INSERT INTO task(name,id,comment) VALUES ('task1','100','100th task');
- [Proposal] After data is imported to the database in batches or the data increment reaches the threshold, you are advised to analyze tables to prevent the execution plan from being degraded due to inaccurate statistics.
- [Proposal] To clear all data in a table, you are advised to use TRUNCATE TABLE instead of DELETE TABLE. DELETE TABLE is not efficient and cannot release disk space occupied by the deleted data.
- [Proposal] After data is imported to the database in batches or the data increment reaches the threshold, you are advised to analyze tables to prevent the execution plan from being degraded due to inaccurate statistics.
- [Proposal] To clear all data in a table, you are advised to use TRUNCATE TABLE instead of DELETE TABLE. DELETE TABLE is not efficient and cannot release disk space occupied by the deleted data.
Type conversion
- [Proposal] Perform type coercion to convert data types. If you perform implicit conversion, the result may differ from expected.
- [Proposal] During data query, explicitly specify the data type for constants, and do not attempt to perform any implicit data type conversion.
- [Notice] In Oracle compatibility mode, null strings will be automatically converted to NULL during data import. If a null string needs to be reserved, you need to create a database that is compatible with Teradata.
-Type conversion
- [Proposal] Perform type coercion to convert data types. If you perform implicit conversion, the result may differ from expected.
- [Proposal] During data query, explicitly specify the data type for constants, and do not attempt to perform any implicit data type conversion.
- [Notice] In Oracle compatibility mode, null strings will be automatically converted to NULL during data import. If a null string needs to be reserved, you need to create a database that is compatible with Teradata.
Query Operation
- [Proposal] Do not return a large number of result sets to a client except the ETL program. If a large result set is returned, consider modifying your service design.
- [Proposal] Perform DDL and DML operations encapsulated in transactions. Operations like table truncation, update, deletion, and dropping, cannot be rolled back once committed. You are advised to encapsulate such operations in transactions so that you can roll back the operations if necessary.
- [Proposal] During query compilation, you are advised to list all columns to be queried and avoid using *. Doing so reduces output lines, improves query performance, and avoids the impact of adding or deleting columns on front-end service compatibility.
- [Proposal] During table object access, add the schema prefix to the table object to avoid accessing an unexpected table due to schema switchover.
- [Proposal] The cost of joining more than three tables or views, especially full joins, is difficult to be estimated. You are advised to use the WITH TABLE AS statement to create interim tables to improve the readability of SQL statements.
- [Proposal] Do not use Cartesian products or full joins. Cartesian products and full joins will result in a sharp expansion of result sets and poor performance.
- [Notice] Only IS NULL and IS NOT NULL can be used to determine NULL value comparison results. If any other method is used, NULL is returned. For example, NULL instead of expected Boolean values is returned for NULL<>NULL, NULL=NULL, and NULL<>1.
- [Notice] Do not use count(col) instead of count(*) to count the total number of records in a table. count(*) counts the NULL value (actual rows) while count (col) does not.
- [Notice] While executing count(col), the number of NULL record rows is counted as 0. While executing sum(col), NULL is returned if all records are NULL. If not all the records are NULL, the number of NULL record rows is counted as 0.
- [Notice] To count multiple columns using count(), column names must be enclosed with parentheses. For example, count ((col1, col2, col3)). Note: When multiple columns are used to count the number of NULL record rows, a row is counted even if all the selected columns are NULL. The result is the same as that when count(*) is executed.
- [Notice] Null records are not counted when count(distinct col) is used to calculate the number of non-null columns that are not repeated.
- [Notice] If all statistical columns are NULL when count(distinct (col1,col2,...)) is used to count the number of unique values in multiple columns, Null records are also counted, and the records are considered the same.
- [Notice] When constants are used to filter data, the system searches for functions used for calculating these two data types based on the data types of the constants and matched columns. If no function is found, the system converts the data type implicitly. Then, the system searches for a function used for calculating the converted data type.
SELECT * FROM test WHERE timestamp_col = 20000101;
-In the preceding example, if timestamp_col is the timestamp type, the system first searches for the function that supports the "equal" operation of the timestamp and int types (constant numbers are considered as the int type). If no such function is found, the timestamp_col data and constant numbers are implicitly converted into the text type for calculation.
- - [Proposal] Do not use scalar subquery statements. A scalar subquery appears in the output list of a SELECT statement. In the following example, the part enclosed in parentheses is a scalar subquery statement:
1SELECT id, (SELECT COUNT(*) FROM films f WHERE f.did = s.id) FROM staffs_p1 s; +
Query Operation
- [Proposal] Do not return a large number of result sets to a client except the ETL program. If a large result set is returned, consider modifying your service design.
- [Proposal] Perform DDL and DML operations encapsulated in transactions. Operations like table truncation, update, deletion, and dropping, cannot be rolled back once committed. You are advised to encapsulate such operations in transactions so that you can roll back the operations if necessary.
- [Proposal] During query compilation, you are advised to list all columns to be queried and avoid using *. Doing so reduces output lines, improves query performance, and avoids the impact of adding or deleting columns on front-end service compatibility.
- [Proposal] During table object access, add the schema prefix to the table object to avoid accessing an unexpected table due to schema switchover.
- [Proposal] The cost of joining more than three tables or views, especially full joins, is difficult to be estimated. You are advised to use the WITH TABLE AS statement to create interim tables to improve the readability of SQL statements.
- [Proposal] Do not use Cartesian products or full joins. Cartesian products and full joins will result in a sharp expansion of result sets and poor performance.
- [Notice] Only IS NULL and IS NOT NULL can be used to determine NULL value comparison results. If any other method is used, NULL is returned. For example, NULL instead of expected Boolean values is returned for NULL<>NULL, NULL=NULL, and NULL<>1.
- [Notice] Do not use count(col) instead of count(*) to count the total number of records in a table. count(*) counts the NULL value (actual rows) while count (col) does not.
- [Notice] While executing count(col), the number of NULL record rows is counted as 0. While executing sum(col), NULL is returned if all records are NULL. If not all the records are NULL, the number of NULL record rows is counted as 0.
- [Notice] To count multiple columns using count(), column names must be enclosed with parentheses. For example, count ((col1, col2, col3)). Note: When multiple columns are used to count the number of NULL record rows, a row is counted even if all the selected columns are NULL. The result is the same as that when count(*) is executed.
- [Notice] Null records are not counted when count(distinct col) is used to calculate the number of non-null columns that are not repeated.
- [Notice] If all statistical columns are NULL when count(distinct (col1,col2,...)) is used to count the number of unique values in multiple columns, Null records are also counted, and the records are considered the same.
- [Notice] When constants are used to filter data, the system searches for functions used for calculating these two data types based on the data types of the constants and matched columns. If no function is found, the system converts the data type implicitly. Then, the system searches for a function used for calculating the converted data type.
SELECT * FROM test WHERE timestamp_col = 20000101;
+In the preceding example, if timestamp_col is the timestamp type, the system first searches for the function that supports the "equal" operation of the timestamp and int types (constant numbers are considered as the int type). If no such function is found, the timestamp_col data and constant numbers are implicitly converted into the text type for calculation.
+ - [Proposal] Do not use scalar subquery statements. A scalar subquery appears in the output list of a SELECT statement. In the following example, the part enclosed in parentheses is a scalar subquery statement:-
1SELECT id, (SELECT COUNT(*) FROM films f WHERE f.did = s.id) FROM staffs_p1 s;
Scalar subqueries often result in query performance deterioration. During application development, scalar subqueries need to be converted into equivalent table associations based on the service logic.
- - [Proposal] In WHERE clauses, the filtering conditions should be sorted. The condition that few records are selected for reading (the number of filtered records is small) is listed at the beginning.
- [Proposal] Filtering conditions in WHERE clauses should comply with unilateral rules. That is, when the column name is placed on one side of a comparison operator, the optimizer automatically performs pruning optimization in some scenarios. Filtering conditions in a WHERE clause will be displayed in col op expression format, where col indicates a table column, op indicates a comparison operator, such as = and >, and expression indicates an expression that does not contain a column name. For example:
1SELECT id, from_image_id, from_person_id, from_video_id FROM face_data WHERE current_timestamp(6) - time < '1 days'::interval; +
Scalar subqueries often result in query performance deterioration. During application development, scalar subqueries need to be converted into equivalent table associations based on the service logic.
+- [Proposal] In WHERE clauses, the filtering conditions should be sorted. The condition that few records are selected for reading (the number of filtered records is small) is listed at the beginning.
- [Proposal] The filter criteria in the WHERE clause should comply with the unilateral rule. That is, the field name is placed on one side of the comparison condition. This allows the optimizer to automatically perform pruning optimization in some scenarios. Filtering conditions in a WHERE clause will be displayed in col op expression format, where col indicates a table column, op indicates a comparison operator, such as = and >, and expression indicates an expression that does not contain a column name. For example:
-1SELECT id, from_image_id, from_person_id, from_video_id FROM face_data WHERE current_timestamp(6) - time < '1 days'::interval;
The modification is as follows:
-1SELECT id, from_image_id, from_person_id, from_video_id FROM face_data where time > current_timestamp(6) - '1 days'::interval; +
The modification is as follows:
+-1SELECT id, from_image_id, from_person_id, from_video_id FROM face_data WHERE time > current_timestamp(6) - '1 days'::interval;
- [Proposal] Do not perform unnecessary sorting operations. Sorting requires a large amount of memory and CPU. If service logic permits, ORDER BY and LIMIT can be combined to reduce resource overhead. By default, data in GaussDB(DWS) is sorted by ASC & NULL LAST.
- [Proposal] When the ORDER BY clause is used for sorting, specify sorting modes (ASC or DESC), and use NULL FIRST or NULL LAST for NULL record sorting.
- [proposal] Do not rely on only the LIMIT clause to return the result set displayed in a specific sequence. Combine ORDER BY and LIMIT clauses for some specific result sets and use offset to skip specific results if necessary.
- [Proposal] If the service logic is accurate, you are advised to use UNION ALL instead of UNION.
- [Proposal] If a filtering condition contains only an OR expression, convert the OR expression to UNION ALL to improve performance. SQL statements that use OR expressions cannot be optimized, resulting in slow execution. Example:
1 +- [Proposal] Do not perform unnecessary sorting operations. Sorting requires a large amount of memory and CPU. If service logic permits, ORDER BY and LIMIT can be combined to reduce resource overhead. By default, data in GaussDB(DWS) is sorted by ASC & NULL LAST.
- [Proposal] When the ORDER BY clause is used for sorting, specify sorting modes (ASC or DESC), and use NULL FIRST or NULL LAST for NULL record sorting.
- [proposal] Do not rely on only the LIMIT clause to return the result set displayed in a specific sequence. Combine ORDER BY and LIMIT clauses for some specific result sets and use offset to skip specific results if necessary.
- [Proposal] If the service logic is accurate, you are advised to use UNION ALL instead of UNION.
- [Proposal] If a filtering condition contains only an OR expression, convert the OR expression to UNION ALL to improve performance. SQL statements that use OR expressions cannot be optimized, resulting in slow execution. For example:
-1 2
SELECT * FROM scdc.pub_menu -WHERE (cdp= 300 AND inline=301) OR (cdp= 301 AND inline=302) OR (cdp= 302 AND inline=301); +WHERE (cdp= 300 AND inline=301) OR (cdp= 301 AND inline=302) OR (cdp= 302 AND inline=301);
Convert the statement to the following:
-1 +
Convert the statement to the following:
+
- [Proposal] Provide the inserted column list in the insert statement. Example:
- [Notice] When a third-party tool connects to GaussDB(DWS) through JDBC, JDBC sends a connection request to GaussDB(DWS). By default, the following parameters are added. For details, see the implementation of the ConnectionFactoryImpl JDBC code.
- [Proposal] Use the most efficient data types allowed.
- [Proposal] Do not use reserved or non-reserved keywords to name database objects.
