From 6873736b168707e2dc8af756b2f19ca3a1f6d01d Mon Sep 17 00:00:00 2001 From: Mireia Perez Date: Fri, 28 Aug 2026 15:41:06 +0100 Subject: [PATCH 1/3] Convert HTML tables to MD --- docs/whpg/6x/admin_guide/analytics/postGIS.md | 65 +- .../admin_guide/external/foreign/devel-fdw.md | 218 +--- .../6x/admin_guide/intro/about_statistics.md | 119 +-- .../admin_guide/manage_access/roles_privs.md | 215 +--- .../6x/admin_guide/managing/monitoring.md | 344 +----- .../admin_guide/query/functions-operators.md | 596 +---------- docs/whpg/6x/admin_guide/query/json-data.md | 466 +------- docs/whpg/6x/install_guide/install_modules.md | 37 +- .../install_guide/installation_utilities.md | 36 +- .../6x/install_guide/platform-requirements.md | 39 +- docs/whpg/6x/ref_guide/function-summary.md | 416 +------- .../whpg/6x/ref_guide/misc/feature_summary.md | 967 +++-------------- docs/whpg/7x/admin_guide/analytics/postGIS.md | 65 +- .../admin_guide/external/foreign/devel-fdw.md | 218 +--- .../7x/admin_guide/intro/about_statistics.md | 119 +-- .../admin_guide/manage_access/roles_privs.md | 215 +--- .../7x/admin_guide/managing/monitoring.md | 344 +----- .../admin_guide/query/functions-operators.md | 607 +---------- docs/whpg/7x/admin_guide/query/json-data.md | 119 +-- .../install_guide/installation_utilities.md | 35 +- docs/whpg/7x/ref_guide/function-summary.md | 906 ++-------------- .../whpg/7x/ref_guide/misc/feature_summary.md | 991 +++--------------- 22 files changed, 798 insertions(+), 6339 deletions(-) diff --git a/docs/whpg/6x/admin_guide/analytics/postGIS.md b/docs/whpg/6x/admin_guide/analytics/postGIS.md index 6fdb610..5b1911d 100644 --- a/docs/whpg/6x/admin_guide/analytics/postGIS.md +++ b/docs/whpg/6x/admin_guide/analytics/postGIS.md @@ -40,62 +40,15 @@ There are significant changes in PostGIS 2.5.4 compared with 2.1.5. For a list o This table lists the PostGIS extensions support by WarehousePG PostGIS. -
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 1. WarehousePG PostGIS Extensions
PostGIS ExtensionWarehousePG PostGIS Notes
postgis

PostGIS and PostGIS Raster - support

-		Supported. Both PostGIS and PostGIS Raster are enabled when the WarehousePG - postgis extension is enabled.
postgis_tiger_geocoder

The US TIGER geocoder

-		Supported. Installed with WarehousePG PostGIS.

Requires the - postgis and fuzzystrmatch - extensions.

-

The US TIGER geocoder converts addresses (like a street address) - to geographic coordinates.

-
address_standardizer

Rule-based address - standardizer

-		Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used - with TIGER geocoder.

-

A single line address parser that takes an input - address and normalizes it based on a set of rules stored in a table and helper - lex and gaz tables.

-
address_standardizer_data_us

Sample rules tables for US - address data

-		Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with - the address standardizer.

-

The extension contains gaz, - lex, and rules tables for US address data. If - you are using other types of tables, see PostGIS Extension Limitations.

-
fuzzystrmatch

Fuzzy string matching

-		Supported. This extension is bundled but not enabled with WarehousePG - Database.

Required for the PostGIS TIGER geocoder.

-
-
+**WarehousePG PostGIS Extensions** + +| PostGIS Extension | WarehousePG PostGIS Notes | +| --- | --- | +| `postgis`

PostGIS and PostGIS Raster support | Supported. Both PostGIS and PostGIS Raster are enabled when the WarehousePG `postgis` extension is enabled. | +| `postgis_tiger_geocoder`

The US TIGER geocoder | Supported. Installed with WarehousePG PostGIS.

Requires the `postgis` and `fuzzystrmatch` extensions.

The US TIGER geocoder converts addresses (like a street address) to geographic coordinates. | +| `address_standardizer`

Rule-based address standardizer | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with TIGER geocoder.

A single line address parser that takes an input address and normalizes it based on a set of rules stored in a table and helper `lex` and `gaz` tables. | +| `address_standardizer_data_us`

Sample rules tables for US address data | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with the address standardizer.

The extension contains `gaz`, `lex`, and `rules` tables for US address data. If you are using other types of tables, see [PostGIS Extension Limitations](#topic_wy2_rkb_3p). | +| `fuzzystrmatch`

Fuzzy string matching | Supported. This extension is bundled but not enabled with WarehousePG Database.

Required for the PostGIS TIGER geocoder. | > **Note** The PostGIS topology extension `postgis_topology` and the PostGIS 3D and geoprocessing extension `postgis_sfcgal` are not supported by WarehousePG PostGIS and are not included in the WarehousePG PostGIS extension package. diff --git a/docs/whpg/6x/admin_guide/external/foreign/devel-fdw.md b/docs/whpg/6x/admin_guide/external/foreign/devel-fdw.md index ae91f32..75598af 100644 --- a/docs/whpg/6x/admin_guide/external/foreign/devel-fdw.md +++ b/docs/whpg/6x/admin_guide/external/foreign/devel-fdw.md @@ -146,147 +146,28 @@ You must implement the scan-related functions in your foreign-data wrapper; impl Scan-related callback functions include: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Callback SignatureDescription
void
-GetForeignRelSize (PlannerInfo *root,
-                   RelOptInfo *baserel,
-                   Oid foreigntableid)
Obtain relation size estimates for a foreign table. - Called at the beginning of planning for a query on a foreign table.
void
-GetForeignPaths (PlannerInfo *root,
-                 RelOptInfo *baserel,
-                 Oid foreigntableid)
Create possible access paths for a scan on a - foreign table. Called during query planning.
Note: A WarehousePG - Database-compatible FDW must call - create_foreignscan_path() in its - GetForeignPaths() callback function.
ForeignScan *
-GetForeignPlan (PlannerInfo *root,
-                RelOptInfo *baserel,
-                Oid foreigntableid,
-                ForeignPath *best_path,
-                List *tlist,
-                List *scan_clauses)
Create a ForeignScan plan node from - the selected foreign access path. Called at the end of query planning.
void
-BeginForeignScan (ForeignScanState *node,
-                  int eflags)
Begin running a foreign scan. Called during - executor startup.
TupleTableSlot *
-IterateForeignScan (ForeignScanState *node)
Fetch one row from the foreign source, returning it - in a tuple table slot; return NULL if no more rows are available.
void
-ReScanForeignScan (ForeignScanState *node)
Restart the scan from the beginning.
void
-EndForeignScan (ForeignScanState *node)
End the scan and release resources.
+| Callback Signature | Description | +| --- | --- | +| `void`
`GetForeignRelSize (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Obtain relation size estimates for a foreign table. Called at the beginning of planning for a query on a foreign table. | +| `void`
`GetForeignPaths (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Create possible access paths for a scan on a foreign table. Called during query planning.

**Note:** A WarehousePG Database-compatible FDW must call `create_foreignscan_path()` in its `GetForeignPaths()` callback function. | +| `ForeignScan *`
`GetForeignPlan (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid,`
` ForeignPath *best_path,`
` List *tlist,`
` List *scan_clauses)` | Create a `ForeignScan` plan node from the selected foreign access path. Called at the end of query planning. | +| `void`
`BeginForeignScan (ForeignScanState *node,`
` int eflags)` | Begin running a foreign scan. Called during executor startup. | +| `TupleTableSlot *`
`IterateForeignScan (ForeignScanState *node)` | Fetch one row from the foreign source, returning it in a tuple table slot; return NULL if no more rows are available. | +| `void`
`ReScanForeignScan (ForeignScanState *node)` | Restart the scan from the beginning. | +| `void`
`EndForeignScan (ForeignScanState *node)` | End the scan and release resources. | If a foreign data wrapper supports writable foreign tables, it should provide the update-related callback functions that are required by the capabilities of the FDW. Update-related callback functions include: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Callback SignatureDescription
void
-AddForeignUpdateTargets (Query *parsetree,
-                         RangeTblEntry *target_rte,
-                         Relation target_relation)
Add additional information in the foreign - table that will be retrieved during an update or delete operation - to identify the exact row on which to operate.
List *
-PlanForeignModify (PlannerInfo *root,
-                   ModifyTable *plan,
-                   Index resultRelation,
-                   int subplan_index)
Perform additional planning actions required - for an insert, update, or delete operation on a foreign table, - and return the information generated.
void
-BeginForeignModify (ModifyTableState *mtstate,
-                    ResultRelInfo *rinfo,
-                    List *fdw_private,
-                    int subplan_index,
-                    int eflags)
Begin executing a modify operation on a - foreign table. Called during executor startup.
TupleTableSlot *
-ExecForeignInsert (EState *estate,
-                   ResultRelInfo *rinfo,
-                   TupleTableSlot *slot,
-                   TupleTableSlot *planSlot)
Insert a single tuple into the foreign table. - Return a slot containing the data that was actually inserted, or - NULL if no row was inserted.
TupleTableSlot *
-ExecForeignUpdate (EState *estate,
-                   ResultRelInfo *rinfo,
-                   TupleTableSlot *slot,
-                   TupleTableSlot *planSlot)
Update a single tuple in the foreign table. - Return a slot containing the row as it was actually updated, or - NULL if no row was updated. 
TupleTableSlot *
-ExecForeignDelete (EState *estate,
-                   ResultRelInfo *rinfo,
-                   TupleTableSlot *slot,
-                   TupleTableSlot *planSlot)
Delete a single tuple from the foreign table. - Return a slot containing the row that was deleted, or NULL if no - row was deleted.
void
-EndForeignModify (EState *estate,
-                  ResultRelInfo *rinfo)
End the update and release resources.
int
-IsForeignRelUpdatable (Relation rel)
Report the update operations supported by the - specified foreign table.
+| Callback Signature | Description | +| --- | --- | +| `void`
`AddForeignUpdateTargets (Query *parsetree,`
` RangeTblEntry *target_rte,`
` Relation target_relation)` | Add additional information in the foreign table that will be retrieved during an update or delete operation to identify the exact row on which to operate. | +| `List *`
`PlanForeignModify (PlannerInfo *root,`
` ModifyTable *plan,`
` Index resultRelation,`
` int subplan_index)` | Perform additional planning actions required for an insert, update, or delete operation on a foreign table, and return the information generated. | +| `void`
`BeginForeignModify (ModifyTableState *mtstate,`
` ResultRelInfo *rinfo,`
` List *fdw_private,`
` int subplan_index,`
` int eflags)` | Begin executing a modify operation on a foreign table. Called during executor startup. | +| `TupleTableSlot *`
`ExecForeignInsert (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Insert a single tuple into the foreign table. Return a slot containing the data that was actually inserted, or NULL if no row was inserted. | +| `TupleTableSlot *`
`ExecForeignUpdate (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Update a single tuple in the foreign table. Return a slot containing the row as it was actually updated, or NULL if no row was updated. | +| `TupleTableSlot *`
`ExecForeignDelete (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Delete a single tuple from the foreign table. Return a slot containing the row that was deleted, or NULL if no row was deleted. | +| `void`
`EndForeignModify (EState *estate,`
` ResultRelInfo *rinfo)` | End the update and release resources. | +| `int`
`IsForeignRelUpdatable (Relation rel)` | Report the update operations supported by the specified foreign table. | Refer to [Foreign Data Wrapper Callback Routines](https://www.postgresql.org/docs/9.4/fdw-callbacks.html) in the PostgreSQL documentation for detailed information about the inputs and outputs of the FDW callback functions. @@ -302,56 +183,17 @@ The FDW API exports several helper functions from the WarehousePG core server so The FDW API includes the helper functions listed in the table below. Refer to [Foreign Data Wrapper Helper Functions](https://www.postgresql.org/docs/9.4/fdw-helpers.html) in the PostgreSQL documentation for more information about these functions. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Helper SignatureDescription
ForeignDataWrapper *
-GetForeignDataWrapper(Oid fdwid);
Returns the ForeignDataWrapper - object for the foreign-data wrapper with the given OID.
ForeignDataWrapper *
-GetForeignDataWrapperByName(const char *name, bool missing_ok);
Returns the ForeignDataWrapper - object for the foreign-data wrapper with the given name.
ForeignServer *
-GetForeignServer(Oid serverid);
Returns the ForeignServer - object for the foreign server with the given OID.
ForeignServer *
-GetForeignServerByName(const char *name, bool missing_ok);
Returns the ForeignServer - object for the foreign server with the given name.
UserMapping *
-GetUserMapping(Oid userid, Oid serverid);
Returns the UserMapping - object for the user mapping of the given role on the given - server.
ForeignTable *
-GetForeignTable(Oid relid);
Returns the ForeignTable - object for the foreign table with the given OID.
List *
-GetForeignColumnOptions(Oid relid, AttrNumber attnum);
Returns the per-column FDW options for the - column with the given foreign table OID and attribute number.
+| Helper Signature | Description | +| --- | --- | +| `ForeignDataWrapper *`
`GetForeignDataWrapper(Oid fdwid);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given OID. | +| `ForeignDataWrapper *`
`GetForeignDataWrapperByName(const char *name, bool missing_ok);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given name. | +| `ForeignServer *`
`GetForeignServer(Oid serverid);` | Returns the `ForeignServer` object for the foreign server with the given OID. | +| `ForeignServer *`
`GetForeignServerByName(const char *name, bool missing_ok);` | Returns the `ForeignServer` object for the foreign server with the given name. | +| `UserMapping *`
`GetUserMapping(Oid userid, Oid serverid);` | Returns the `UserMapping` object for the user mapping of the given role on the given server. | +| `ForeignTable *`
`GetForeignTable(Oid relid);` | Returns the `ForeignTable` object for the foreign table with the given OID. | +| `List *`
`GetForeignColumnOptions(Oid relid, AttrNumber attnum);` | Returns the per-column FDW options for the column with the given foreign table OID and attribute number. | + + ## WarehousePG Considerations diff --git a/docs/whpg/6x/admin_guide/intro/about_statistics.md b/docs/whpg/6x/admin_guide/intro/about_statistics.md index fc27a63..674dc08 100644 --- a/docs/whpg/6x/admin_guide/intro/about_statistics.md +++ b/docs/whpg/6x/admin_guide/intro/about_statistics.md @@ -89,114 +89,17 @@ The statistics collected for a column vary for different data types, so the `pg_ The `stakindN` columns each contain a numeric code to describe the type of statistics stored in their slot. The `stakind` code numbers from 1 to 99 are reserved for core PostgreSQL data types. WarehousePG uses code numbers 1, 2, 3, 4, 5, and 99. A value of 0 means the slot is unused. The following table describes the kinds of statistics stored for the three codes. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 1. Contents of pg_statistic "slots"
stakind CodeDescription
1Most CommonValues (MCV) Slot -
    -
  • staop contains the object ID of the "=" operator, used to - decide whether values are the same or not.
    • -
  • stavalues contains an array of the K - most common non-null values appearing in the column.
    • -
  • stanumbers contains the frequencies (fractions of total - row count) of the values in the stavalues array.
    • -
The values are ordered in decreasing frequency. Since the arrays are - variable-size, K can be chosen by the statistics collector. - Values must occur more than once to be added to the stavalues - array; a unique column has no MCV slot.
2Histogram Slot – describes the distribution of scalar data.
    -
  • staop is the object ID of the "<" operator, which - describes the sort ordering.
    • -
  • stavalues contains M (where - M>=2) non-null values that divide - the non-null column data values into M-1 - bins of approximately equal population. The first stavalues - item is the minimum value and the last is the maximum value.
    • -
  • stanumbers is not used and should be - NULL.
    • -

If a Most Common Values slot is also provided, then the histogram - describes the data distribution after removing the values listed in the MCV - array. (It is a compressed histogram in the technical parlance). This - allows a more accurate representation of the distribution of a column with - some very common values. In a column with only a few distinct values, it is - possible that the MCV list describes the entire data population; in this case - the histogram reduces to empty and should be omitted.

3Correlation Slot – describes the correlation between the physical - order of table tuples and the ordering of data values of this column.
    -
  • staop is the object ID of the "<" operator. As with - the histogram, more than one entry could theoretically appear.
    • -
  • stavalues is not used and should be - NULL.
    • -
  • stanumbers contains a single entry, the correlation - coefficient between the sequence of data values and the sequence of their - actual tuple positions. The coefficient ranges from +1 to -1.
    • -
4Most Common Elements Slot - is similar to a Most Common Values (MCV) - Slot, except that it stores the most common non-null elements of the - column values. This is useful when the column datatype is an array or some other - type with identifiable elements (for instance, tsvector).
    -
  • staop contains the equality operator appropriate to the - element type.
    • -
  • stavalues contains the most common element values.
    • -
  • stanumbers contains common element frequencies.
    • -

Frequencies are measured as the fraction of non-null rows the element - value appears in, not the frequency of all rows. Also, the values are sorted - into the element type's default order (to support binary search for a - particular value). Since this puts the minimum and maximum frequencies at - unpredictable spots in stanumbers, there are two extra - members of stanumbers that hold copies of the minimum and - maximum frequencies. Optionally, there can be a third extra member that holds - the frequency of null elements (the frequency is expressed in the same terms: - the fraction of non-null rows that contain at least one null element). If this - member is omitted, the column is presumed to contain no NULL - elements.

-
Note: For tsvector columns, the stavalues - elements are of type text, even though their representation - within tsvector is not exactly - text.
5Distinct Elements Count Histogram Slot - describes the distribution - of the number of distinct element values present in each row of an array-type - column. Only non-null rows are considered, and only non-null elements.
    -
  • staop contains the equality operator appropriate to the - element type.
    • -
  • stavalues is not used and should be - NULL.
    • -
  • stanumbers contains information about distinct elements. - The last member of stanumbers is the average count of - distinct element values over all non-null rows. The preceding - M (where M >=2) - members form a histogram that divides the population of distinct-elements - counts into M-1 bins of approximately - equal population. The first of these is the minimum observed count, and the - last the maximum.
    • -
99Hyperloglog Slot - for child leaf partitions of a partitioned table, - stores the hyperloglog_counter created for the sampled data. - The hyperloglog_counter data structure is converted into a - bytea and stored in a stavalues5 slot of the - pg_statistic catalog table.
+**Contents of pg_statistic "slots"** + +| stakind Code | Description | +| --- | --- | +| 1 | *Most CommonValues (MCV) Slot*

- `staop` contains the object ID of the "=" operator, used to decide whether values are the same or not.
- `stavalues` contains an array of the *K* most common non-null values appearing in the column.
- `stanumbers` contains the frequencies (fractions of total row count) of the values in the `stavalues` array.

The values are ordered in decreasing frequency. Since the arrays are variable-size, *K* can be chosen by the statistics collector. Values must occur more than once to be added to the `stavalues` array; a unique column has no MCV slot. | +| 2 | *Histogram Slot* – describes the distribution of scalar data.

- `staop` is the object ID of the "<" operator, which describes the sort ordering.
- `stavalues` contains *M* (where `M>=2`) non-null values that divide the non-null column data values into `M-1` bins of approximately equal population. The first `stavalues` item is the minimum value and the last is the maximum value.
- `stanumbers` is not used and should be `NULL`.

If a Most Common Values slot is also provided, then the histogram describes the data distribution after removing the values listed in the MCV array. (It is a *compressed histogram* in the technical parlance). This allows a more accurate representation of the distribution of a column with some very common values. In a column with only a few distinct values, it is possible that the MCV list describes the entire data population; in this case the histogram reduces to empty and should be omitted. | +| 3 | *Correlation Slot* – describes the correlation between the physical order of table tuples and the ordering of data values of this column.

- `staop` is the object ID of the "<" operator. As with the histogram, more than one entry could theoretically appear.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains a single entry, the correlation coefficient between the sequence of data values and the sequence of their actual tuple positions. The coefficient ranges from +1 to -1. | +| 4 | *Most Common Elements Slot* - is similar to a Most Common Values (MCV) Slot, except that it stores the most common non-null *elements* of the column values. This is useful when the column datatype is an array or some other type with identifiable elements (for instance, `tsvector`).

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` contains the most common element values.
- `stanumbers` contains common element frequencies.

Frequencies are measured as the fraction of non-null rows the element value appears in, not the frequency of all rows. Also, the values are sorted into the element type's default order (to support binary search for a particular value). Since this puts the minimum and maximum frequencies at unpredictable spots in `stanumbers`, there are two extra members of `stanumbers` that hold copies of the minimum and maximum frequencies. Optionally, there can be a third extra member that holds the frequency of null elements (the frequency is expressed in the same terms: the fraction of non-null rows that contain at least one null element). If this member is omitted, the column is presumed to contain no `NULL` elements.

**Note:** For `tsvector` columns, the `stavalues` elements are of type `text`, even though their representation within `tsvector` is not exactly `text`. | +| 5 | *Distinct Elements Count Histogram Slot* - describes the distribution of the number of distinct element values present in each row of an array-type column. Only non-null rows are considered, and only non-null elements.

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains information about distinct elements. The last member of `stanumbers` is the average count of distinct element values over all non-null rows. The preceding *M* (where `M >=2`) members form a histogram that divides the population of distinct-elements counts into `M-1` bins of approximately equal population. The first of these is the minimum observed count, and the last the maximum. | +| 99 | *Hyperloglog Slot* - for child leaf partitions of a partitioned table, stores the `hyperloglog_counter` created for the sampled data. The `hyperloglog_counter` data structure is converted into a `bytea` and stored in a `stavalues5` slot of the `pg_statistic` catalog table. | + The `pg_stats` view presents the contents of `pg_statistic` in a friendlier format. The `pg_stats` view has the following columns: diff --git a/docs/whpg/6x/admin_guide/manage_access/roles_privs.md b/docs/whpg/6x/admin_guide/manage_access/roles_privs.md index 0a6a312..02e876a 100644 --- a/docs/whpg/6x/admin_guide/manage_access/roles_privs.md +++ b/docs/whpg/6x/admin_guide/manage_access/roles_privs.md @@ -117,204 +117,23 @@ The role attributes `LOGIN`, `SUPERUSER`, `CREATEDB`, `CREATEROLE`, `CREATEEXTTA When an object (table, view, sequence, database, function, language, schema, or tablespace) is created, it is assigned an owner. The owner is normally the role that ran the creation statement. For most kinds of objects, the initial state is that only the owner (or a superuser) can do anything with the object. To allow other roles to use it, privileges must be granted. WarehousePG supports the following privileges for each object type: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 2. Object Privileges
Object TypePrivileges
Tables, External Tables, Views -

- SELECT -

-

- INSERT -

-

- UPDATE -

-

- DELETE -

-

- REFERENCES -

-

- TRIGGER -

-

- TRUNCATE -

-

- ALL -

-
Columns -

- SELECT -

-

- INSERT -

-

- UPDATE -

-

- REFERENCES -

-

- ALL -

-
Sequences -

- USAGE -

-

- SELECT -

-

- UPDATE -

-

- ALL -

-
Databases -

- CREATE -

-

- CONNECT -

-

- TEMPORARY -

-

- TEMP -

-

- ALL -

-
Domains -

- USAGE -

-

- ALL -

-
Foreign Data Wrappers -

- USAGE -

-

- ALL -

-
Foreign Servers -

- USAGE -

-

- ALL -

-
Functions -

- EXECUTE -

-

- ALL -

-
Procedural Languages -

- USAGE -

-

- ALL -

-
Schemas -

- CREATE -

-

- USAGE -

-

- ALL -

-
Tablespaces -

- CREATE -

-

- ALL -

-
Types -

- USAGE -

-

- ALL -

-
Protocols -

- SELECT -

-

- INSERT -

-

- ALL -

-
+**Object Privileges** + +| Object Type | Privileges | +| --- | --- | +| Tables, External Tables, Views | `SELECT`

`INSERT`

`UPDATE`

`DELETE`

`REFERENCES`

`TRIGGER`

`TRUNCATE`

`ALL` | +| Columns | `SELECT`

`INSERT`

`UPDATE`

`REFERENCES`

`ALL` | +| Sequences | `USAGE`

`SELECT`

`UPDATE`

`ALL` | +| Databases | `CREATE`

`CONNECT`

`TEMPORARY`

`TEMP`

`ALL` | +| Domains | `USAGE`

`ALL` | +| Foreign Data Wrappers | `USAGE`

`ALL` | +| Foreign Servers | `USAGE`

`ALL` | +| Functions | `EXECUTE`

`ALL` | +| Procedural Languages | `USAGE`

`ALL` | +| Schemas | `CREATE`

`USAGE`

`ALL` | +| Tablespaces | `CREATE`

`ALL` | +| Types | `USAGE`

`ALL` | +| Protocols | `SELECT`

`INSERT`

`ALL` | > **Note** You must grant privileges for each object individually. For example, granting `ALL` on a database does not grant full access to the objects within that database. It only grants all of the database-level privileges (`CONNECT`, `CREATE`, `TEMPORARY`) to the database itself. diff --git a/docs/whpg/6x/admin_guide/managing/monitoring.md b/docs/whpg/6x/admin_guide/managing/monitoring.md index 8bfaea4..beddbe4 100644 --- a/docs/whpg/6x/admin_guide/managing/monitoring.md +++ b/docs/whpg/6x/admin_guide/managing/monitoring.md @@ -17,281 +17,56 @@ It is not necessary to implement all of these suggestions in every cluster; use ## Database State Monitoring Activities - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 1. Database State Monitoring Activities
ActivityProcedureCorrective Actions
List segments that are currently down. If any rows are - returned, this should generate a warning or alert.

Recommended - frequency: run every 5 to 10 minutes

Severity: - IMPORTANT

Run the following query in the postgres - database:
SELECT * FROM gp_segment_configuration
-WHERE status = 'd';
If the query returns any rows, follow these steps to correct - the problem:
    -
  1. Verify that the hosts with down segments are responsive.
    2. -
  2. If hosts are OK, check the log - files for the primaries and - mirrors of the down segments to discover the - root cause of the segments going down.
    3. -
  3. If no unexpected errors are found, run the -gprecoverseg utility to bring the - segments back online.
    4. -
Check for segments that are up and not in sync. If rows are - returned, this should generate a warning or alert.

Recommended - frequency: run every 5 to 10 minutes

 -
Execute the following query in the postgres - database:
SELECT * FROM gp_segment_configuration
-WHERE mode = 'n' and status = 'u' and content <> -1;
-		If the query returns rows then the segment might be in the process - of moving from Not In Sync to -Synchronized mode. Use gpstate - -e to track progress.
Check for segments that are not operating in their preferred role but are marked as up and Synchronized. - If any segments are found, the cluster may not be - balanced. If any rows are returned this should generate a - warning or alert.

Recommended frequency: run every 5 to 10 - minutes

Severity: IMPORTANT

 -
Execute the following query in the postgres - database: -
SELECT * FROM gp_segment_configuration 
-WHERE preferred_role <> role  and status = 'u' and mode = 's';
- -		 -

When the segments are not running in their preferred role, processing might be skewed. - Run gprecoverseg -r to bring the segments back into their preferred roles.

-
Run a distributed query to test that it runs on all segments. - One row should be returned for each primary segment. -

Recommended frequency: run every 5 to 10 - minutes

Severity: CRITICAL

 -
Execute the following query in the postgres - database:
SELECT gp_segment_id, count(*)
-FROM gp_dist_random('pg_class')
-GROUP BY 1;
-		 -

If this query fails, there is an issue dispatching to some - segments in the cluster. This is a rare event. Check the - hosts that are not able to be dispatched to ensure there is - no hardware or networking issue.

-
Test the state of coordinator mirroring on WarehousePG. If - the value is not "STREAMING", raise an alert or - warning.

Recommended frequency: run every 5 to 10 - minutes

Severity: IMPORTANT

 -
Run the following psql - command:
psql <dbname> -c 'SELECT pid, state FROM pg_stat_replication;'
-		 -

Check the log file from the coordinator - and standby coordinator for errors. If there are no unexpected - errors and the machines are up, run the -gpinitstandby utility to bring the - standby online.

-
Perform a basic check to see if the coordinator is up and - functioning.

Recommended frequency: run every 5 to 10 - minutes

Severity: CRITICAL

 -
Run the following query in the postgres - database: -
SELECT count(*) FROM gp_segment_configuration;
-		 -

If this query fails, the active coordinator may be down. Try to - start the database on the original coordinator if the server is - up and running. If that fails, try to activate the standby - coordinator as coordinator.

-
+ +**Database State Monitoring Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| List segments that are currently down. If any rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE status = 'd';` | If the query returns any rows, follow these steps to correct the problem:

1. Verify that the hosts with down segments are responsive.
2. If hosts are OK, check the log files for the primaries and mirrors of the down segments to discover the root cause of the segments going down.
3. If no unexpected errors are found, run the `gprecoverseg` utility to bring the segments back online. | +| Check for segments that are up and not in sync. If rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE mode = 'n' and status = 'u' and content <> -1;` | If the query returns rows then the segment might be in the process of moving from `Not In Sync` to `Synchronized` mode. Use `gpstate -e` to track progress. | +| Check for segments that are not operating in their preferred role but are marked as up and `Synchronized`. If any segments are found, the cluster may not be balanced. If any rows are returned this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE preferred_role <> role and status = 'u' and mode = 's';` | When the segments are not running in their preferred role, processing might be skewed. Run `gprecoverseg -r` to bring the segments back into their preferred roles. | +| Run a distributed query to test that it runs on all segments. One row should be returned for each primary segment.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Execute the following query in the `postgres` database:

`SELECT gp_segment_id, count(*) FROM gp_dist_random('pg_class') GROUP BY 1;` | If this query fails, there is an issue dispatching to some segments in the cluster. This is a rare event. Check the hosts that are not able to be dispatched to ensure there is no hardware or networking issue. | +| Test the state of coordinator mirroring on WarehousePG. If the value is not "STREAMING", raise an alert or warning.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following `psql` command:

`psql <dbname> -c 'SELECT pid, state FROM pg_stat_replication;'` | Check the log file from the coordinator and standby coordinator for errors. If there are no unexpected errors and the machines are up, run the `gpinitstandby` utility to bring the standby online. | +| Perform a basic check to see if the coordinator is up and functioning.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Run the following query in the `postgres` database:

`SELECT count(*) FROM gp_segment_configuration;` | If this query fails, the active coordinator may be down. Try to start the database on the original coordinator if the server is up and running. If that fails, try to activate the standby coordinator as coordinator. | + ## Hardware and Operating System Monitoring - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 2. Hardware and Operating System Monitoring Activities
ActivityProcedureCorrective Actions
Check disk space usage on volumes used for WarehousePG data storage and the OS. -

Recommended frequency: every 5 to 30 - minutes

Severity: CRITICAL

 -
Set up a disk space check.
    -
  • Set a threshold to raise an alert when a disk - reaches a percentage of capacity. The recommended - threshold is 75% full.
    • -
  • It is not recommended to run the system with - capacities approaching 100%.
    • -
-		Use VACUUM/VACUUM FULL on - user tables to reclaim space occupied by dead rows.
Check for errors or dropped packets on the network - interfaces.

Recommended frequency: hourly

Severity: - IMPORTANT

Set up a network interface checks. -

Work with network and OS teams to resolve errors.

-
Check for RAID errors or degraded RAID performance. -

Recommended frequency: every 5 minutes

Severity: - CRITICAL

Set up a RAID check. -
    -
  • Replace failed disks as soon as possible.
    • -
  • Work with system administration team to resolve other - RAID or controller errors as soon as possible.
    • -
-
Check for adequate I/O bandwidth and I/O skew.

Recommended - frequency: when create a cluster or when hardware issues are - suspected.

Run the WarehousePG -gpcheckperf utility. -
The cluster may be under-specified if data transfer rates are - not similar to the following:
    -
  • 2GB per second disk read
    • -
  • 1 GB per second disk write
    • -
  • 10 Gigabit per second network read and write
    • -
If transfer rates are lower than expected, consult with - your data architect regarding performance expectations.
-

If the machines on the cluster display an uneven performance - profile, work with the system administration team to fix - faulty machines.

-
+ +**Hardware and Operating System Monitoring Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| Check disk space usage on volumes used for WarehousePG data storage and the OS.

Recommended frequency: every 5 to 30 minutes

Severity: CRITICAL | Set up a disk space check.

- Set a threshold to raise an alert when a disk reaches a percentage of capacity. The recommended threshold is 75% full.
- It is not recommended to run the system with capacities approaching 100%. | Use `VACUUM`/`VACUUM FULL` on user tables to reclaim space occupied by dead rows. | +| Check for errors or dropped packets on the network interfaces.

Recommended frequency: hourly

Severity: IMPORTANT | Set up a network interface checks. | Work with network and OS teams to resolve errors. | +| Check for RAID errors or degraded RAID performance.

Recommended frequency: every 5 minutes

Severity: CRITICAL | Set up a RAID check. | - Replace failed disks as soon as possible.
- Work with system administration team to resolve other RAID or controller errors as soon as possible. | +| Check for adequate I/O bandwidth and I/O skew.

Recommended frequency: when create a cluster or when hardware issues are suspected. | Run the WarehousePG `gpcheckperf` utility. | The cluster may be under-specified if data transfer rates are not similar to the following:

- 2GB per second disk read
- 1 GB per second disk write
- 10 Gigabit per second network read and write

If transfer rates are lower than expected, consult with your data architect regarding performance expectations.

If the machines on the cluster display an uneven performance profile, work with the system administration team to fix faulty machines. | + ## Catalog Monitoring - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 3. Catalog Monitoring Activities
ActivityProcedureCorrective Actions
Run catalog consistency checks in each database to ensure the - catalog on each host in the cluster is consistent and in a good - state.

You may run this command while the database is up - and running.

Recommended frequency: - weekly

Severity: IMPORTANT

Run the WarehousePG gpcheckcat utility in each - database:
gpcheckcat -O
Note: With the --O option, gpcheckcat - runs just 10 of its usual 15 tests.
Run the repair scripts for any issues identified.
Check for pg_class entries that have no - corresponding pg_attribute entry.

Recommended - frequency: monthly

Severity: IMPORTANT

During a downtime, with no users on the system, run the - WarehousePG -gpcheckcat utility in each - database:
gpcheckcat -R pgclass
Run the repair scripts for any issues identified.
Check for leaked temporary schema and missing schema - definition.

Recommended frequency: monthly

Severity: - IMPORTANT

During a downtime, with no users on the system, run the - WarehousePG -gpcheckcat utility in each - database:
gpcheckcat -R namespace
Run the repair scripts for any issues identified.
Check constraints on randomly distributed - tables.

Recommended frequency: monthly

Severity: - IMPORTANT

During a downtime, with no users on the system, run the - WarehousePG -gpcheckcat utility in each - database:
gpcheckcat -R distribution_policy
Run the repair scripts for any issues identified.
Check for dependencies on non-existent objects.

Recommended - frequency: monthly

Severity: IMPORTANT

During a downtime, with no users on the system, run the - WarehousePG -gpcheckcat utility in each - database:
gpcheckcat -R dependency
Run the repair scripts for any issues identified.
+ +**Catalog Monitoring Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| Run catalog consistency checks in each database to ensure the catalog on each host in the cluster is consistent and in a good state.

You may run this command while the database is up and running.

Recommended frequency: weekly

Severity: IMPORTANT | Run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -O`

**Note:** With the `-O` option, `gpcheckcat` runs just 10 of its usual 15 tests. | Run the repair scripts for any issues identified. | +| Check for `pg_class` entries that have no corresponding pg_`attribute` entry.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R pgclass` | Run the repair scripts for any issues identified. | +| Check for leaked temporary schema and missing schema definition.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R namespace` | Run the repair scripts for any issues identified. | +| Check constraints on randomly distributed tables.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R distribution_policy` | Run the repair scripts for any issues identified. | +| Check for dependencies on non-existent objects.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R dependency` | Run the repair scripts for any issues identified. | + ## Data Maintenance - - - - - - - - - - - - - - - - - -
Table 4. Data Maintenance Activities
ActivityProcedureCorrective Actions
Check for missing statistics on tables. Check the gp_stats_missing view in each - database:
SELECT * FROM gp_toolkit.gp_stats_missing;
Run ANALYZE on tables that are missing - statistics.
Check for tables that have bloat (dead space) in data files - that cannot be recovered by a regular VACUUM - command.

Recommended frequency: weekly or - monthly

Severity: WARNING

Check the gp_bloat_diag view in each - database: -
SELECT * FROM gp_toolkit.gp_bloat_diag;
VACUUM FULL acquires an ACCESS - EXCLUSIVE lock on tables. Run VACUUM - FULL during a time when users and applications do - not require access to the tables, such as during a time of low - activity, or during a maintenance window.
+ +**Data Maintenance Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| Check for missing statistics on tables. | Check the `gp_stats_missing` view in each database:

`SELECT * FROM gp_toolkit.gp_stats_missing;` | Run `ANALYZE` on tables that are missing statistics. | +| Check for tables that have bloat (dead space) in data files that cannot be recovered by a regular `VACUUM` command.

Recommended frequency: weekly or monthly

Severity: WARNING | Check the `gp_bloat_diag` view in each database:

`SELECT * FROM gp_toolkit.gp_bloat_diag;` | `VACUUM FULL` acquires an `ACCESS EXCLUSIVE` lock on tables. Run `VACUUM FULL` during a time when users and applications do not require access to the tables, such as during a time of low activity, or during a maintenance window. | + ## Database Maintenance @@ -364,32 +139,11 @@ GROUP BY 1; ## Patching and Upgrading - - - - - - - - - - - - - - - - - -
Table 6. Patch and Upgrade Activities
ActivityProcedureCorrective Actions
Ensure any bug fixes or enhancements are applied to the - kernel.

Recommended frequency: at least every 6 - months

Severity: IMPORTANT

Follow the vendor's instructions to update the Linux - kernel.Keep the kernel current to include bug fixes and security - fixes, and to avoid difficult future upgrades.
Install WarehousePG minor releases, for example - 6.21.x.

Recommended frequency: - quarterly

Severity: IMPORTANT

Follow upgrade instructions in the WarehousePG -Release Notes. Always upgrade to the latest in the - series.Keep the WarehousePG software current to - incorporate bug fixes, performance enhancements, and feature - enhancements into your WarehousePG - cluster.
+ +**Patch and Upgrade Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| Ensure any bug fixes or enhancements are applied to the kernel.

Recommended frequency: at least every 6 months

Severity: IMPORTANT | Follow the vendor's instructions to update the Linux kernel. | Keep the kernel current to include bug fixes and security fixes, and to avoid difficult future upgrades. | +| Install WarehousePG minor releases, for example 6.21.*x*.

Recommended frequency: quarterly

Severity: IMPORTANT | Follow upgrade instructions in the WarehousePG *Release Notes*. Always upgrade to the latest in the series. | Keep the WarehousePG software current to incorporate bug fixes, performance enhancements, and feature enhancements into your WarehousePG cluster. | + diff --git a/docs/whpg/6x/admin_guide/query/functions-operators.md b/docs/whpg/6x/admin_guide/query/functions-operators.md index c4754f4..e622435 100644 --- a/docs/whpg/6x/admin_guide/query/functions-operators.md +++ b/docs/whpg/6x/admin_guide/query/functions-operators.md @@ -93,478 +93,55 @@ The following table lists the categories of built-in functions and operators sup WarehousePG includes JSON processing functions that manipulate values the `json` data type. For information about JSON data, see [Working with JSON Data](json-data.md). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 3. Built-in functions and operators
Operator/Function CategoryVOLATILE FunctionsSTABLE FunctionsRestrictions
- Logical Operators -
- Comparison Operators -
- - Mathematical Functions and Operators - - random

setseed

- - String Functions and Operators - - - All built-in conversion functions - convert

pg_client_encoding

- - Binary String Functions and Operators - -
- - Bit String Functions and Operators - -
- - Pattern Matching - -
- - Data Type Formatting Functions - - to_char

to_timestamp

- Date/Time Functions and Operators - timeofdayage

current_date

current_time

current_timestamp

localtime

localtimestamp

now

- Enum Support Functions -
- - Geometric Functions and Operators - -
- - Network Address Functions and Operators - -
- - Sequence Manipulation Functions - - nextval()

setval()

- - Conditional Expressions - -
- - Array Functions and Operators - - - All array functions -
- - Aggregate Functions - -
- - Subquery Expressions - -
- - Row and Array Comparisons - -
- - Set Returning Functions - - generate_series
- - System Information Functions - - - All session information functions -

- All access privilege inquiry functions -

- All schema visibility inquiry functions -

- All system catalog information functions -

- All comment information functions -

- All transaction ids and snapshots -

- - System Administration Functions - - set_config

pg_cancel_backend

pg_terminate_backend

pg_reload_conf

pg_rotate_logfile

pg_start_backup

pg_stop_backup

pg_size_pretty

pg_ls_dir

pg_read_file

pg_stat_file

current_setting

All database object size - functions

Note: The function - pg_column_size displays bytes required to store - the value, possibly with TOAST compression.
- XML Functions and function-like - expressions -

cursor_to_xml(cursor refcursor, count int, nulls boolean, - tableforest boolean, targetns text)

-

cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest - boolean, targetns text)

-

database_to_xml(nulls boolean, tableforest boolean, targetns - text)

-

database_to_xmlschema(nulls boolean, tableforest boolean, - targetns text)

-

database_to_xml_and_xmlschema( nulls boolean, tableforest - boolean, targetns text)

-

query_to_xml(query text, nulls boolean, tableforest boolean, - targetns text)

-

query_to_xmlschema(query text, nulls boolean, tableforest - boolean, targetns text)

-

query_to_xml_and_xmlschema( query text, nulls boolean, - tableforest boolean, targetns text)

-

schema_to_xml(schema name, nulls boolean, tableforest boolean, - targetns text)

-

schema_to_xmlschema( schema name, nulls boolean, tableforest - boolean, targetns text)

-

schema_to_xml_and_xmlschema( schema name, nulls boolean, - tableforest boolean, targetns text)

-

table_to_xml(tbl regclass, nulls boolean, tableforest boolean, - targetns text)

-

table_to_xmlschema( tbl regclass, nulls boolean, tableforest - boolean, targetns text)

-

table_to_xml_and_xmlschema( tbl regclass, nulls boolean, - tableforest boolean, targetns text)

-

xmlagg(xml)

-

xmlconcat(xml[, ...])

-

xmlelement(name name [, xmlattributes(value [AS attname] [, ... - ])] [, content, ...])

-

xmlexists(text, xml)

-

xmlforest(content [AS name] [, ...])

-

xml_is_well_formed(text)

-

xml_is_well_formed_document(text)

-

xml_is_well_formed_content(text)

-

xmlparse ( { DOCUMENT | CONTENT } value)

-

xpath(text, xml)

-

xpath(text, xml, text[])

-

xpath_exists(text, xml)

-

xpath_exists(text, xml, text[])

-

xmlpi(name target [, content])

-

xmlroot(xml, version text | no value [, standalone yes|no|no - value])

-

xmlserialize ( { DOCUMENT | CONTENT } value AS type )

-

xml(text)

-

text(xml)

-

xmlcomment(xml)

-

xmlconcat2(xml, xml)

-
+**Built-in functions and operators** + +| Operator/Function Category | VOLATILE Functions | STABLE Functions | Restrictions | +| --- | --- | --- | --- | +| [Logical Operators](https://www.postgresql.org/docs/9.4/functions-logical.html) | | | | +| [Comparison Operators](https://www.postgresql.org/docs/9.4/functions-comparison.html) | | | | +| [Mathematical Functions and Operators](https://www.postgresql.org/docs/9.4/functions-math.html) | random

setseed | | | +| [String Functions and Operators](https://www.postgresql.org/docs/9.4/functions-string.html) | *All built-in conversion functions* | convert

pg_client_encoding | | +| [Binary String Functions and Operators](https://www.postgresql.org/docs/9.4/functions-binarystring.html) | | | | +| [Bit String Functions and Operators](https://www.postgresql.org/docs/9.4/functions-bitstring.html) | | | | +| [Pattern Matching](https://www.postgresql.org/docs/9.4/functions-matching.html) | | | | +| [Data Type Formatting Functions](https://www.postgresql.org/docs/9.4/functions-formatting.html) | | to_char

to_timestamp | | +| [Date/Time Functions and Operators](https://www.postgresql.org/docs/9.4/functions-datetime.html) | timeofday | age

current_date

current_time

current_timestamp

localtime

localtimestamp

now | | +| [Enum Support Functions](https://www.postgresql.org/docs/9.4/functions-enum.html) | | | | +| [Geometric Functions and Operators](https://www.postgresql.org/docs/9.4/functions-geometry.html) | | | | +| [Network Address Functions and Operators](https://www.postgresql.org/docs/9.4/functions-net.html) | | | | +| [Sequence Manipulation Functions](https://www.postgresql.org/docs/9.4/functions-sequence.html) | nextval()

setval() | | | +| [Conditional Expressions](https://www.postgresql.org/docs/9.4/functions-conditional.html) | | | | +| [Array Functions and Operators](https://www.postgresql.org/docs/9.4/functions-array.html) | | *All array functions* | | +| [Aggregate Functions](https://www.postgresql.org/docs/9.4/functions-aggregate.html) | | | | +| [Subquery Expressions](https://www.postgresql.org/docs/9.4/functions-subquery.html) | | | | +| [Row and Array Comparisons](https://www.postgresql.org/docs/9.4/functions-comparisons.html) | | | | +| [Set Returning Functions](https://www.postgresql.org/docs/9.4/functions-srf.html) | generate_series | | | +| [System Information Functions](https://www.postgresql.org/docs/9.4/functions-info.html) | | *All session information functions*

*All access privilege inquiry functions*

*All schema visibility inquiry functions*

*All system catalog information functions*

*All comment information functions*

*All transaction ids and snapshots* | | +| [System Administration Functions](https://www.postgresql.org/docs/9.4/functions-admin.html) | set_config

pg_cancel_backend

pg_terminate_backend

pg_reload_conf

pg_rotate_logfile

pg_start_backup

pg_stop_backup

pg_size_pretty

pg_ls_dir

pg_read_file

pg_stat_file | current_setting

*All database object size functions* | **Note:** The function `pg_column_size` displays bytes required to store the value, possibly with TOAST compression. | +| [XML Functions](https://www.postgresql.org/docs/9.4/functions-xml.html) and function-like expressions | | cursor_to_xml(cursor refcursor, count int, nulls boolean, tableforest boolean, targetns text)

cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest boolean, targetns text)

database_to_xml(nulls boolean, tableforest boolean, targetns text)

database_to_xmlschema(nulls boolean, tableforest boolean, targetns text)

database_to_xml_and_xmlschema( nulls boolean, tableforest boolean, targetns text)

query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xmlschema(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xml_and_xmlschema( query text, nulls boolean, tableforest boolean, targetns text)

schema_to_xml(schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xml_and_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

table_to_xml(tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xml_and_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

xmlagg(xml)

xmlconcat(xml[, ...])

xmlelement(name name [, xmlattributes(value [AS attname] [, ... ])] [, content, ...])

xmlexists(text, xml)

xmlforest(content [AS name] [, ...])

xml_is_well_formed(text)

xml_is_well_formed_document(text)

xml_is_well_formed_content(text)

xmlparse ( { DOCUMENT \| CONTENT } value)

xpath(text, xml)

xpath(text, xml, text[])

xpath_exists(text, xml)

xpath_exists(text, xml, text[])

xmlpi(name target [, content])

xmlroot(xml, version text \| no value [, standalone yes\|no\|no value])

xmlserialize ( { DOCUMENT \| CONTENT } value AS type )

xml(text)

text(xml)

xmlcomment(xml)

xmlconcat2(xml, xml) | | + + ## Window Functions The following built-in window functions are WarehousePG extensions to the PostgreSQL database. All window functions are *immutable*. For more information about window functions, see [Window Expressions](defining-queries.md). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 4. Window functions
FunctionReturn TypeFull SyntaxDescription
- cume_dist() - - double precision - - CUME_DIST() OVER ( [PARTITION BY - expr - ] ORDER BY - expr - ) - Calculates the cumulative distribution of a value - in a group of values. Rows with equal values always evaluate to the - same cumulative distribution value.
- dense_rank() - - bigint - - DENSE_RANK () OVER ( [PARTITION BY - expr - ] ORDER BY - expr - ) - Computes the rank of a row in an ordered group of - rows without skipping rank values. Rows with equal values are given - the same rank value.
- first_value(expr) - same as input expr type - FIRST_VALUE( - expr - ) OVER ( [PARTITION BY - expr - ] ORDER BY - expr - [ROWS|RANGE - frame_expr - ] ) - Returns the first value in an ordered set of - values.
- lag(expr [,offset] [,default]) - same as input expr type - LAG( - expr - [, - offset - ] [, - default - ]) OVER ( [PARTITION BY - expr - ] ORDER BY - expr - ) - Provides access to more than one row of the same - table without doing a self join. Given a series of rows returned - from a query and a position of the cursor, LAG - provides access to a row at a given physical offset prior to that - position. The default offset is 1. default - sets the value that is returned if the offset goes beyond the scope - of the window. If default is not specified, the default value - is null.
- last_value(expr)same as input expr type - LAST_VALUE(expr) OVER ( [PARTITION BY expr] - ORDER BY expr [ROWS|RANGE frame_expr] ) - Returns the last value in an ordered set of - values.
- - lead(expr [,offset] - [,default]) - - same as input expr type - LEAD(expr [,offset] - [,exprdefault]) OVER ( [PARTITION BY - expr] ORDER BY expr ) - Provides access to more than one row of the same - table without doing a self join. Given a series of rows returned - from a query and a position of the cursor, lead - provides access to a row at a given physical offset after that - position. If offset is not specified, the default offset is - 1. default sets the value that is returned if the offset goes - beyond the scope of the window. If default is not specified, - the default value is null.
- ntile(expr) - bigint - NTILE(expr) OVER ( [PARTITION BY expr] ORDER - BY expr ) - Divides an ordered data set into a number of - buckets (as defined by expr) and assigns a bucket number to - each row.
percent_rank() - double precision - - PERCENT_RANK () OVER ( [PARTITION BY expr] ORDER BY - expr ) - Calculates the rank of a hypothetical row - R minus 1, divided by 1 less than the number of - rows being evaluated (within a window partition).
- rank() - bigint - RANK () OVER ( [PARTITION BY expr] ORDER BY expr - ) - Calculates the rank of a row in an ordered group - of values. Rows with equal values for the ranking criteria receive - the same rank. The number of tied rows are added to the rank number - to calculate the next rank value. Ranks may not be consecutive - numbers in this case.
row_number() - bigint - - ROW_NUMBER () OVER ( [PARTITION BY expr] ORDER BY - expr ) - Assigns a unique number to each row to which it is - applied (either each row in a window partition or each row of the - query).
+**Window functions** + +| Function | Return Type | Full Syntax | Description | +| --- | --- | --- | --- | +| `cume_dist()` | `double precision` | `CUME_DIST() OVER ( [PARTITION BY` expr `] ORDER BY` expr `)` | Calculates the cumulative distribution of a value in a group of values. Rows with equal values always evaluate to the same cumulative distribution value. | +| `dense_rank()` | `bigint` | `DENSE_RANK () OVER ( [PARTITION BY` expr `] ORDER BY` expr `)` | Computes the rank of a row in an ordered group of rows without skipping rank values. Rows with equal values are given the same rank value. | +| `first_value(*expr*)` | same as input expr type | `FIRST_VALUE(` expr `) OVER ( [PARTITION BY` expr `] ORDER BY` expr `[ROWS\|RANGE` frame_expr `] )` | Returns the first value in an ordered set of values. | +| `lag(*expr* [,*offset*] [,*default*])` | same as input *expr* type | `LAG(` *expr* `[,` *offset* `] [,` *default* `]) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` | Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `LAG` provides access to a row at a given physical offset prior to that position. The default `offset` is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. | +| `last_value(*expr*`) | same as input *expr* type | `LAST_VALUE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* [ROWS\|RANGE *frame_expr*] )` | Returns the last value in an ordered set of values. | +| `lead(*expr* [,*offset*] [,*default*])` | same as input *expr* type | `LEAD(*expr*[,*offset*] [,*expr**default*]) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` | Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `lead` provides access to a row at a given physical offset after that position. If *offset* is not specified, the default offset is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. | +| `ntile(*expr*)` | `bigint` | `NTILE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` | Divides an ordered data set into a number of buckets (as defined by *expr*) and assigns a bucket number to each row. | +| `percent_rank()` | `double precision` | `PERCENT_RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Calculates the rank of a hypothetical row `R` minus 1, divided by 1 less than the number of rows being evaluated (within a window partition). | +| `rank()` | `bigint` | `RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Calculates the rank of a row in an ordered group of values. Rows with equal values for the ranking criteria receive the same rank. The number of tied rows are added to the rank number to calculate the next rank value. Ranks may not be consecutive numbers in this case. | +| `row_number()` | `bigint` | `ROW_NUMBER () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Assigns a unique number to each row to which it is applied (either each row in a window partition or each row of the query). | + + ## Advanced Aggregate Functions @@ -572,79 +149,12 @@ The following built-in advanced aggregate functions are WarehousePG extensions o > **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [WarehousePG MADlib Extension for Analytics](../analytics/madlib.md) in the *WarehousePG Reference Guide*. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 5. Advanced Aggregate Functions
FunctionReturn TypeFull SyntaxDescription
- MEDIAN (expr) - - timestamp, timestamptz, interval, float - - MEDIAN (expression) -

- Example: - 

SELECT departmzent_id, MEDIAN(salary) 
-  FROM employees 
-GROUP BY department_id;
- 		Can take a two-dimensional array as input. Treats - such arrays as matrices.
- sum(array[]) - - smallint[], int[], bigint[], float[] - - sum(array[[1,2],[3,4]]) -

- Example: - 

CREATE TABLE mymatrix (myvalue int[]);
-INSERT INTO mymatrix 
-   VALUES (array[[1,2],[3,4]]);
-INSERT INTO mymatrix 
-   VALUES (array[[0,1],[1,0]]);
-SELECT sum(myvalue) FROM mymatrix;
- sum 
-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
- {{1,3},{4,4}}
- 		Performs matrix summation. Can take as input a - two-dimensional array that is treated as a matrix.
- pivot_sum (label[], label, expr) - - int[], bigint[], float[] - - pivot_sum( array['A1','A2'], attr, value) - A pivot aggregation using sum to resolve duplicate - entries.
- unnest (array[]) - set of anyelement - unnest( array['one', 'row', 'per', 'item']) - Transforms a one dimensional array into rows. - Returns a set of anyelement, a polymorphic pseudo-type in - PostgreSQL.
+**Advanced Aggregate Functions** + +| Function | Return Type | Full Syntax | Description | +| --- | --- | --- | --- | +| `MEDIAN (*expr*)` | `timestamp, timestamptz, interval, float` | `MEDIAN (*expression*)`

*Example:*

`SELECT departmzent_id, MEDIAN(salary)
FROM employees
GROUP BY department_id;` | Can take a two-dimensional array as input. Treats such arrays as matrices. | +| `sum(array[])` | `smallint[], int[], bigint[], float[]` | `sum(array[[1,2],[3,4]])`

*Example:*

`CREATE TABLE mymatrix (myvalue int[]);
INSERT INTO mymatrix
VALUES (array[[1,2],[3,4]]);
INSERT INTO mymatrix
VALUES (array[[0,1],[1,0]]);
SELECT sum(myvalue) FROM mymatrix;
sum
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
{{1,3},{4,4}}` | Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. | +| `pivot_sum (label[], label, expr)` | `int[], bigint[], float[]` | `pivot_sum( array['A1','A2'], attr, value)` | A pivot aggregation using sum to resolve duplicate entries. | +| `unnest (array[])` | set of `anyelement` | `unnest( array['one', 'row', 'per', 'item'])` | Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudo-type](https://www.postgresql.org/docs/9.4/datatype-pseudo.html) in PostgreSQL. | + diff --git a/docs/whpg/6x/admin_guide/query/json-data.md b/docs/whpg/6x/admin_guide/query/json-data.md index a6fc1ab..b15d308 100644 --- a/docs/whpg/6x/admin_guide/query/json-data.md +++ b/docs/whpg/6x/admin_guide/query/json-data.md @@ -375,75 +375,15 @@ This table describes the operators that are available for use with the `json` an Operators that require the `jsonb` data type as the left operand are described in the following table. Many of these operators can be indexed by `jsonb` operator classes. For a full description of `jsonb` containment and existence semantics, see [jsonb Containment and Existence](#topic_isx_2tw_mq). For information about how these operators can be used to effectively index `jsonb` data, see [jsonb Indexing](#topic_aqt_1tw_mq). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 3. jsonb Operators
OperatorRight Operand TypeDescriptionExample
- @> - - jsonb - Does the left JSON value contain within it the right value? - '{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb -
- <@ - - jsonb - Is the left JSON value contained within the right value? - '{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb -
- ? - - text - Does the key/element string exist within the JSON value? - '{"a":1, "b":2}'::jsonb ? 'b' -
- ?| - - text[] - Do any of these key/element strings exist? - '{"a":1, "b":2, "c":3}'::jsonb ?| array['b', 'c'] -
- ?& - - text[] - Do all of these key/element strings exist? - '["a", "b"]'::jsonb ?& array['a', 'b'] -
+**jsonb Operators** + +| Operator | Right Operand Type | Description | Example | +| --- | --- | --- | --- | +| `@>` | `jsonb` | Does the left JSON value contain within it the right value? | `'{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb` | +| `<@` | `jsonb` | Is the left JSON value contained within the right value? | `'{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb` | +| `?` | `text` | Does the key/element string exist within the JSON value? | `'{"a":1, "b":2}'::jsonb ? 'b'` | +| `?\|` | `text[]` | Do any of these key/element strings exist? | `'{"a":1, "b":2, "c":3}'::jsonb ?\| array['b', 'c']` | +| `?&` | `text[]` | Do all of these key/element strings exist? | `'["a", "b"]'::jsonb ?& array['a', 'b']` | The standard comparison operators in the following table are available only for the `jsonb` data type, not for the `json` data type. They follow the ordering rules for B-tree operations described in [jsonb Indexing](#topic_aqt_1tw_mq). @@ -464,118 +404,17 @@ The standard comparison operators in the following table are available only for This table describes the functions that create `json` data type values. (Currently, there are no equivalent functions for `jsonb`, but you can cast the result of one of these functions to `jsonb`.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 5. JSON Creation Functions
FunctionDescriptionExampleExample Result
- to_json(anyelement) - Returns the value as a JSON object. Arrays and composites are processed - recursively and are converted to arrays and objects. If the input contains a cast - from the type to json, the cast function is used to perform the - conversion; otherwise, a JSON scalar value is produced. For any scalar type other - than a number, a Boolean, or a null value, the text representation will be used, - properly quoted and escaped so that it is a valid JSON string. - to_json('Fred said "Hi."'::text) - - "Fred said \"Hi.\"" -
- array_to_json(anyarray [, pretty_bool]) - Returns the array as a JSON array. A multidimensional array becomes a JSON - array of arrays.

Line feeds will be added between dimension-1 elements if - pretty_bool is true.

 - array_to_json('{{1,5},{99,100}}'::int[]) - - [[1,5],[99,100]] -
- row_to_json(record [, pretty_bool]) - Returns the row as a JSON object.

Line feeds will be added between level-1 - elements if pretty_bool is true.

 - row_to_json(row(1,'foo')) - - {"f1":1,"f2":"foo"} -
json_build_array(VARIADIC "any")Builds a possibly-heterogeneously-typed JSON array out of a - VARIADIC argument list. - json_build_array(1,2,'3',4,5) - - [1, 2, "3", 4, 5] -
- json_build_object(VARIADIC "any") - Builds a JSON object out of a VARIADIC argument list. The - argument list is taken in order and converted to a set of key/value pairs. - json_build_object('foo',1,'bar',2) - - {"foo": 1, "bar": 2} -
- json_object(text[]) - Builds a JSON object out of a text array. The array must be either a one or a - two dimensional array.

The one dimensional array must have an even number of - elements. The elements are taken as key/value pairs.

For a two - dimensional array, each inner array must have exactly two elements, which are - taken as a key/value pair.

 -

- json_object('{a, 1, b, "def", c, 3.5}') -

-

- json_object('{{a, 1},{b, "def"},{c, 3.5}}') -

- 		- {"a": "1", "b": "def", "c": "3.5"} -
- json_object(keys text[], values text[]) - Builds a JSON object out of a text array. This form of - json_object takes keys and values pairwise from two separate - arrays. In all other respects it is identical to the one-argument form. - json_object('{a, b}', '{1,2}') - - {"a": "1", "b": "2"} -
+**JSON Creation Functions** + +| Function | Description | Example | Example Result | +| --- | --- | --- | --- | +| `to_json(anyelement)` | Returns the value as a JSON object. Arrays and composites are processed recursively and are converted to arrays and objects. If the input contains a cast from the type to `json`, the cast function is used to perform the conversion; otherwise, a JSON scalar value is produced. For any scalar type other than a number, a Boolean, or a null value, the text representation will be used, properly quoted and escaped so that it is a valid JSON string. | `to_json('Fred said "Hi."'::text)` | `"Fred said \"Hi.\""` | +| `array_to_json(anyarray [, pretty_bool])` | Returns the array as a JSON array. A multidimensional array becomes a JSON array of arrays.

Line feeds will be added between dimension-1 elements if `pretty_bool` is true. | `array_to_json('{{1,5},{99,100}}'::int[])` | `[[1,5],[99,100]]` | +| `row_to_json(record [, pretty_bool])` | Returns the row as a JSON object.

Line feeds will be added between level-1 elements if `pretty_bool` is true. | `row_to_json(row(1,'foo'))` | `{"f1":1,"f2":"foo"}` | +| `json_build_array(VARIADIC "any"`) | Builds a possibly-heterogeneously-typed JSON array out of a `VARIADIC` argument list. | `json_build_array(1,2,'3',4,5)` | `[1, 2, "3", 4, 5]` | +| `json_build_object(VARIADIC "any")` | Builds a JSON object out of a `VARIADIC` argument list. The argument list is taken in order and converted to a set of key/value pairs. | `json_build_object('foo',1,'bar',2)` | `{"foo": 1, "bar": 2}` | +| `json_object(text[])` | Builds a JSON object out of a text array. The array must be either a one or a two dimensional array.

The one dimensional array must have an even number of elements. The elements are taken as key/value pairs.

For a two dimensional array, each inner array must have exactly two elements, which are taken as a key/value pair. | `json_object('{a, 1, b, "def", c, 3.5}')`

`json_object('{{a, 1},{b, "def"},{c, 3.5}}')` | `{"a": "1", "b": "def", "c": "3.5"}` | +| `json_object(keys text[], values text[])` | Builds a JSON object out of a text array. This form of `json_object` takes keys and values pairwise from two separate arrays. In all other respects it is identical to the one-argument form. | `json_object('{a, b}', '{1,2}')` | `{"a": "1", "b": "2"}` | > **Note** `array_to_json` and `row_to_json` have the same behavior as `to_json` except for offering a pretty-printing option. The behavior described for `to_json` likewise applies to each individual value converted by the other JSON creation functions. @@ -600,254 +439,23 @@ This table shows the functions that are available for processing `json` and `jso Many of these processing functions and operators convert Unicode escapes in JSON strings to the appropriate single character. This is a not an issue if the input data type is `jsonb`, because the conversion was already done. However, for `json` data type input, this might result in an error being thrown. See [About JSON Data](#topic_upc_tcs_fz). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 7. JSON Processing Functions
FunctionReturn TypeDescriptionExampleExample Result
- json_array_length(json) -

- jsonb_array_length(jsonb) -

int - Returns the number of elements in the outermost JSON array.json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]') - - 5 -
json_each(json) -

jsonb_each(jsonb) -

setof key text, value json -

setof key text, value jsonb -

- 		Expands the outermost JSON object into a set of key/value pairs.select * from json_each('{"a":"foo", "b":"bar"}') - - 
 key | value
------+-------
- a   | "foo"
- b   | "bar"
-
-
json_each_text(json) -

jsonb_each_text(jsonb) -

- 		setof key text, value text - Expands the outermost JSON object into a set of key/value pairs. The returned - values will be of type text.select * from json_each_text('{"a":"foo", "b":"bar"}') - - 
 key | value
------+-------
- a   | foo
- b   | bar
-
-
json_extract_path(from_json json, VARIADIC path_elems - text[]) -

jsonb_extract_path(from_json jsonb, VARIADIC path_elems - text[]) -

-

json -

-

jsonb -

- 		Returns the JSON value pointed to by path_elems (equivalent - to #> operator).json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4') - - {"f5":99,"f6":"foo"} -
json_extract_path_text(from_json json, VARIADIC path_elems - text[]) -

jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems - text[]) -

text - Returns the JSON value pointed to by path_elems as text. - Equivalent to #>> operator.json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4', - 'f6') - - foo -
json_object_keys(json) -

jsonb_object_keys(jsonb) -

setof text - Returns set of keys in the outermost JSON object.json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}}') - - 
 json_object_keys
-------------------
- f1
- f2
-
-
json_populate_record(base anyelement, from_json - json)

jsonb_populate_record(base anyelement, from_json - jsonb) -

anyelement - Expands the object in from_json to a row whose columns match - the record type defined by base. See Note 1.select * from json_populate_record(null::myrowtype, - '{"a":1,"b":2}') - - 
 a | b
----+---
- 1 | 2
-
-
json_populate_recordset(base anyelement, from_json json) -

jsonb_populate_recordset(base anyelement, from_json jsonb) -

setof anyelement - Expands the outermost array of objects in from_json to a set - of rows whose columns match the record type defined by base. See Note 1.select * from json_populate_recordset(null::myrowtype, - '[{"a":1,"b":2},{"a":3,"b":4}]') - - 
 a | b
----+---
- 1 | 2
- 3 | 4
-
-
json_array_elements(json) -

jsonb_array_elements(jsonb)

 -

setof json -

-

setof jsonb -

- 		Expands a JSON array to a set of JSON values.select * from json_array_elements('[1,true, [2,false]]') - - 
   value
------------
- 1
- true
- [2,false]
-
-
json_array_elements_text(json) -

jsonb_array_elements_text(jsonb) -

setof text - Expands a JSON array to a set of text values.select * from json_array_elements_text('["foo", "bar"]') - - 
   value
------------
- foo
- bar
-
-
json_typeof(json)

jsonb_typeof(jsonb) -

text - Returns the type of the outermost JSON value as a text string. Possible types - are object, array, string, - number, boolean, and null. - See Note - 2.json_typeof('-123.4') - - number -
json_to_record(json)

jsonb_to_record(jsonb) -

record - Builds an arbitrary record from a JSON object. See Note 1.

As with all - functions returning record, the caller must explicitly define the structure of - the record with an AS clause.

select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a - int, b text, d text) - - 
 a |    b    | d
----+---------+---
- 1 | [1,2,3] |
-
-
json_to_recordset(json) -

jsonb_to_recordset(jsonb) -

setof record - Builds an arbitrary set of records from a JSON array of objects See Note 1.

As with all - functions returning record, the caller must explicitly define the structure of - the record with an AS clause.

select * from - json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b - text); - - 
 a |  b
----+-----
- 1 | foo
- 2 |
-
-
+**JSON Processing Functions** + +| Function | Return Type | Description | Example | Example Result | +| --- | --- | --- | --- | --- | +| `json_array_length(json)`

`jsonb_array_length(jsonb)` | `int` | Returns the number of elements in the outermost JSON array. | `json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]')` | `5` | +| `json_each(json)`

`jsonb_each(jsonb)` | `setof key text, value json`

`setof key text, value jsonb` | Expands the outermost JSON object into a set of key/value pairs. | `select * from json_each('{"a":"foo", "b":"bar"}')` | ` key \| value
-----+-------
a \| "foo"
b \| "bar"` | +| `json_each_text(json)`

`jsonb_each_text(jsonb)` | `setof key text, value text` | Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. | `select * from json_each_text('{"a":"foo", "b":"bar"}')` | ` key \| value
-----+-------
a \| foo
b \| bar` | +| `json_extract_path(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` | `json`

`jsonb` | Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). | `json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4')` | `{"f5":99,"f6":"foo"}` | +| `json_extract_path_text(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` | `text` | Returns the JSON value pointed to by `path_elems` as text. Equivalent to `#>>` operator. | `json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4', 'f6')` | `foo` | +| `json_object_keys(json)`

`jsonb_object_keys(jsonb)` | `setof text` | Returns set of keys in the outermost JSON object. | `json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}}')` | ` json_object_keys
------------------
f1
f2` | +| `json_populate_record(base anyelement, from_json json)`

`jsonb_populate_record(base anyelement, from_json jsonb)` | `anyelement` | Expands the object in `from_json` to a row whose columns match the record type defined by base. See [Note 1](#notes-on-json-examples). | `select * from json_populate_record(null::myrowtype, '{"a":1,"b":2}')` | ` a \| b
---+---
1 \| 2` | +| `json_populate_recordset(base anyelement, from_json json)`

`jsonb_populate_recordset(base anyelement, from_json jsonb)` | `setof anyelement` | Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by base. See [Note 1](#notes-on-json-examples). | `select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]')` | ` a \| b
---+---
1 \| 2
3 \| 4` | +| `json_array_elements(json)`

`jsonb_array_elements(jsonb`) | `setof json`

`setof jsonb` | Expands a JSON array to a set of JSON values. | `select * from json_array_elements('[1,true, [2,false]]')` | ` value
-----------
1
true
[2,false]` | +| `json_array_elements_text(json)`

`jsonb_array_elements_text(jsonb)` | `setof text` | Expands a JSON array to a set of `text` values. | `select * from json_array_elements_text('["foo", "bar"]')` | ` value
-----------
foo
bar` | +| `json_typeof(json)`

`jsonb_typeof(jsonb)` | `text` | Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. See [Note 2](#notes-on-json-examples). | `json_typeof('-123.4')` | `number` | +| `json_to_record(json)`

`jsonb_to_record(jsonb)` | `record` | Builds an arbitrary record from a JSON object. See [Note 1](#notes-on-json-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a int, b text, d text)` | ` a \| b \| d
---+---------+---
1 \| [1,2,3] \|` | +| `json_to_recordset(json)`

`jsonb_to_recordset(jsonb)` | `setof record` | Builds an arbitrary set of records from a JSON array of objects See [Note 1](#notes-on-json-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text);` | ` a \| b
---+-----
1 \| foo
2 \|` | #### Notes on JSON examples diff --git a/docs/whpg/6x/install_guide/install_modules.md b/docs/whpg/6x/install_guide/install_modules.md index e6e2d78..ceca731 100644 --- a/docs/whpg/6x/install_guide/install_modules.md +++ b/docs/whpg/6x/install_guide/install_modules.md @@ -21,41 +21,8 @@ $ psql -d testdb -c 'DROP EXTENSION dblink;' You can register the following modules in this manner: - - - - -
- - - -
+| - [btree_gin](../ref_guide/modules/bundled/btree_gin.md)
- [citext](../ref_guide/modules/bundled/citext.md)
- [dblink](../ref_guide/modules/bundled/dblink.md)
- [diskquota](../ref_guide/modules/extensions/diskquota.md)
- [fuzzystrmatch](../ref_guide/modules/bundled/fuzzystrmatch.md)
- [gp_array_agg](../ref_guide/modules/bundled/gp_array_agg.md)
- [gp_check_functions](../ref_guide/modules/bundled/gp_check_functions.md)
- [gp_parallel_retrieve_cursor](../ref_guide/modules/bundled/gp_parallel_retrieve_cursor.md)
- [gp_percentile_agg](../ref_guide/modules/bundled/gp_percentile_agg.md)
- [gp_sparse_vector](../ref_guide/modules/bundled/gp_sparse_vector.md)
- [greenplum_fdw](../ref_guide/modules/bundled/greenplum_fdw.md)
- [hstore](../ref_guide/modules/bundled/hstore.md) | - [ip4r](../ref_guide/modules/bundled/ip4r.md)
- [ltree](../ref_guide/modules/bundled/ltree.md)
- [orafce](../ref_guide/modules/bundled/orafce_ref.md) (WarehousePG only)
- [pageinspect](../ref_guide/modules/bundled/pageinspect.md)
- [pg_trgm](../ref_guide/modules/bundled/pg_trgm.md)
- [pgcrypto](../ref_guide/modules/bundled/pgcrypto.md)
- [postgres_fdw](../ref_guide/modules/bundled/postgres_fdw.md)
- [postgresql-hll](../ref_guide/modules/extensions/postgresql-hll.md)
- [sslinfo](../ref_guide/modules/bundled/sslinfo.md)
- [tablefunc](../ref_guide/modules/bundled/tablefunc.md)
- [timestamp9](../ref_guide/modules/bundled/timestamp9.md)
- [uuid-ossp](../ref_guide/modules/bundled/uuid-ossp.md) | +| --- | --- | For additional information about the modules supplied with WarehousePG, refer to [Additional Supplied Modules](../ref_guide/modules/index.md) in the *WarehousePG Reference Guide*. diff --git a/docs/whpg/6x/install_guide/installation_utilities.md b/docs/whpg/6x/install_guide/installation_utilities.md index 4a0a546..a404ef5 100644 --- a/docs/whpg/6x/install_guide/installation_utilities.md +++ b/docs/whpg/6x/install_guide/installation_utilities.md @@ -9,39 +9,7 @@ For a full reference of all WarehousePG utilities, see the [WarehousePG Utility The following WarehousePG management utilities are located in `$GPHOME/bin`. - - - - -
- - - -
+| - [gpactivatestandby](../ref_guide/utility_guide/reference/gpactivatestandby.md)
- [gpaddmirrors](../ref_guide/utility_guide/reference/gpaddmirrors.md)
- [gpcheckperf](../ref_guide/utility_guide/reference/gpcheckperf.md)
- [gpdeletesystem](../ref_guide/utility_guide/reference/gpdeletesystem.md)
- [gpinitstandby](../ref_guide/utility_guide/reference/gpinitstandby.md) | - [gpinitsystem](../ref_guide/utility_guide/reference/gpinitsystem.md)
- [gpscp](../ref_guide/utility_guide/reference/gpscp.md)
- [gpssh](../ref_guide/utility_guide/reference/gpssh.md)
- [gpssh-exkeys](../ref_guide/utility_guide/reference/gpssh-exkeys.md)
- [gpstart](../ref_guide/utility_guide/reference/gpstart.md)
- [gpstop](../ref_guide/utility_guide/reference/gpstop.md) | +| --- | --- | **Parent topic:** [Installing and Upgrading WarehousePG](index.md) diff --git a/docs/whpg/6x/install_guide/platform-requirements.md b/docs/whpg/6x/install_guide/platform-requirements.md index 9390e6f..c7f5624 100644 --- a/docs/whpg/6x/install_guide/platform-requirements.md +++ b/docs/whpg/6x/install_guide/platform-requirements.md @@ -115,37 +115,14 @@ WarehousePGd 6 supports these Java versions for PL/Java and PXF: This table lists the versions of the WarehousePG Extensions that are compatible with this release of WarehousePG 6. -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
WarehousePG Extensions Compatibility
ComponentPackage VersionAdditional Information
PL/Java2.0.4Supports Java 8 and 11.
PL/R3.0.3R 3.3.3
MADlib Machine Learning2.1, 2.0, 1.21, 1.20, 1.19, 1.18, 1.17, 1.16Support matrix at MADlib FAQ.
PostGIS Spatial and Geographic Objects2.5.4, 2.1.5 
-
+**WarehousePG Extensions Compatibility** + +| Component | Package Version | Additional Information | +| --- | --- | --- | +| [PL/Java](../admin_guide/analytics/procedural_languages/pl_java.md) | 2.0.4 | Supports Java 8 and 11. | +| [PL/R](../admin_guide/analytics/procedural_languages/pl_r.md) | 3.0.3 | R 3.3.3 | +| [MADlib Machine Learning](../admin_guide/analytics/madlib.md) | 2.1, 2.0, 1.21, 1.20, 1.19, 1.18, 1.17, 1.16 | Support matrix at [MADlib FAQ](https://cwiki.apache.org/confluence/display/MADLIB/FAQ#FAQ-Q1-2WhatdatabaseplatformsdoesMADlibsupportandwhatistheupgradematrix?). | +| [PostGIS Spatial and Geographic Objects](../admin_guide/analytics/postGIS.md) | 2.5.4, 2.1.5 | | For information about the Oracle Compatibility Functions, see [Oracle Compatibility Functions](../ref_guide/modules/bundled/orafce_ref.md). diff --git a/docs/whpg/6x/ref_guide/function-summary.md b/docs/whpg/6x/ref_guide/function-summary.md index f1a9f04..5424b64 100644 --- a/docs/whpg/6x/ref_guide/function-summary.md +++ b/docs/whpg/6x/ref_guide/function-summary.md @@ -201,270 +201,23 @@ This table shows the functions that are available for processing `json` and `jso Many of these processing functions and operators convert Unicode escapes in JSON strings to the appropriate single character. This is a not an issue if the input data type is `jsonb`, because the conversion was already done. However, for `json` data type input, this might result in an error being thrown. See [About JSON Data](../admin_guide/query/json-data.md#topic_upc_tcs_fz). -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 8. JSON Processing Functions
FunctionReturn TypeDescriptionExampleExample Result
-json_array_length(json) -

-jsonb_array_length(jsonb) -

-		int -Returns the number of elements in the outermost JSON array.json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]') - -5 -
json_each(json) -

jsonb_each(jsonb) -

-		setof key text, value json -

setof key text, value jsonb -

-		Expands the outermost JSON object into a set of key/value pairs.select * from json_each('{"a":"foo", "b":"bar"}') - -
 key | value
------+-------
- a   | "foo"
- b   | "bar"
-
-
json_each_text(json) -

jsonb_each_text(jsonb) -

-		setof key text, value text -Expands the outermost JSON object into a set of key/value pairs. The returned - values will be of type text.select * from json_each_text('{"a":"foo", "b":"bar"}') - -
 key | value
------+-------
- a   | foo
- b   | bar
-
-
json_extract_path(from_json json, VARIADIC path_elems - text[]) -

jsonb_extract_path(from_json jsonb, VARIADIC path_elems - text[]) -

-		 -

json -

-

jsonb -

-		Returns the JSON value pointed to by path_elems (equivalent - to #> operator).json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4') - -{"f5":99,"f6":"foo"} -
json_extract_path_text(from_json json, VARIADIC path_elems - text[]) -

jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems - text[]) -

-		text -Returns the JSON value pointed to by path_elems as text. - Equivalent to #>> operator.json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4', - 'f6') - -foo -
json_object_keys(json) -

jsonb_object_keys(jsonb) -

-		setof text -Returns set of keys in the outermost JSON object.json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}}') - -
 json_object_keys
-------------------
- f1
- f2
-
-
json_populate_record(base anyelement, from_json - json)

jsonb_populate_record(base anyelement, from_json - jsonb) -

-		anyelement -Expands the object in from_json to a row whose columns match - the record type defined by base. See Note 1.select * from json_populate_record(null::myrowtype, - '{"a":1,"b":2}') - -
 a | b
----+---
- 1 | 2
-
-
json_populate_recordset(base anyelement, from_json json) -

jsonb_populate_recordset(base anyelement, from_json jsonb) -

-		setof anyelement -Expands the outermost array of objects in from_json to a set - of rows whose columns match the record type defined by base. See Note 1.select * from json_populate_recordset(null::myrowtype, - '[{"a":1,"b":2},{"a":3,"b":4}]') - -
 a | b
----+---
- 1 | 2
- 3 | 4
-
-
json_array_elements(json) -

jsonb_array_elements(jsonb)

-		 -

setof json -

-

setof jsonb -

-		Expands a JSON array to a set of JSON values.select * from json_array_elements('[1,true, [2,false]]') - -
   value
------------
- 1
- true
- [2,false]
-
-
json_array_elements_text(json) -

jsonb_array_elements_text(jsonb) -

-		setof text -Expands a JSON array to a set of text values.select * from json_array_elements_text('["foo", "bar"]') - -
   value
------------
- foo
- bar
-
-
json_typeof(json)

jsonb_typeof(jsonb) -

-		text -Returns the type of the outermost JSON value as a text string. Possible types - are object, array, string, -number, boolean, and null. - See Notejson_typeof('-123.4') - -number -
json_to_record(json)

jsonb_to_record(jsonb) -

-		record -Builds an arbitrary record from a JSON object. See Note 1.

As with all - functions returning record, the caller must explicitly define the structure of - the record with an AS clause.

-		select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a - int, b text, d text) - -
 a |    b    | d
----+---------+---
- 1 | [1,2,3] |
-
-
json_to_recordset(json) -

jsonb_to_recordset(jsonb) -

-		setof record -Builds an arbitrary set of records from a JSON array of objects See Note 1.

As with all - functions returning record, the caller must explicitly define the structure of - the record with an AS clause.

-		select * from - json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b - text); - -
 a |  b
----+-----
- 1 | foo
- 2 |
-
-
-
+**JSON Processing Functions** + +| Function | Return Type | Description | Example | Example Result | +| --- | --- | --- | --- | --- | +| `json_array_length(json)`

`jsonb_array_length(jsonb)` | `int` | Returns the number of elements in the outermost JSON array. | `json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]')` | `5` | +| `json_each(json)`

`jsonb_each(jsonb)` | `setof key text, value json`

`setof key text, value jsonb` | Expands the outermost JSON object into a set of key/value pairs. | `select * from json_each('{"a":"foo", "b":"bar"}')` | `key \| value`
`-----+-------`
`a \| "foo"`
`b \| "bar"` | +| `json_each_text(json)`

`jsonb_each_text(jsonb)` | `setof key text, value text` | Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. | `select * from json_each_text('{"a":"foo", "b":"bar"}')` | `key \| value`
`-----+-------`
`a \| foo`
`b \| bar` | +| `json_extract_path(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` | `json`

`jsonb` | Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). | `json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4')` | `{"f5":99,"f6":"foo"}` | +| `json_extract_path_text(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` | `text` | Returns the JSON value pointed to by `path_elems` as text. Equivalent to `#>>` operator. | `json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4', 'f6')` | `foo` | +| `json_object_keys(json)`

`jsonb_object_keys(jsonb)` | `setof text` | Returns set of keys in the outermost JSON object. | `json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}}')` | `json_object_keys`
`------------------`
`f1`
`f2` | +| `json_populate_record(base anyelement, from_json json)`

`jsonb_populate_record(base anyelement, from_json jsonb)` | `anyelement` | Expands the object in `from_json` to a row whose columns match the record type defined by base. See [Note 1](#note-on-json-processing-functions-examples). | `select * from json_populate_record(null::myrowtype, '{"a":1,"b":2}')` | `a \| b`
`---+---`
`1 \| 2` | +| `json_populate_recordset(base anyelement, from_json json)`

`jsonb_populate_recordset(base anyelement, from_json jsonb)` | `setof anyelement` | Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by base. See [Note 1](#note-on-json-processing-functions-examples). | `select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]')` | `a \| b`
`---+---`
`1 \| 2`
`3 \| 4` | +| `json_array_elements(json)`

`jsonb_array_elements(jsonb)` | `setof json`

`setof jsonb` | Expands a JSON array to a set of JSON values. | `select * from json_array_elements('[1,true, [2,false]]')` | `value`
`-----------`
`1`
`true`
`[2,false]` | +| `json_array_elements_text(json)`

`jsonb_array_elements_text(jsonb)` | `setof text` | Expands a JSON array to a set of `text` values. | `select * from json_array_elements_text('["foo", "bar"]')` | `value`
`-----------`
`foo`
`bar` | +| `json_typeof(json)`

`jsonb_typeof(jsonb)` | `text` | Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. See [Note](#note-on-json-processing-functions-examples) | `json_typeof('-123.4')` | `number` | +| `json_to_record(json)`

`jsonb_to_record(jsonb)` | `record` | Builds an arbitrary record from a JSON object. See [Note 1](#note-on-json-processing-functions-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a int, b text, d text)` | `a \| b \| d`
`---+---------+---`
`1 \| [1,2,3] \|` | +| `json_to_recordset(json)`

`jsonb_to_recordset(jsonb)` | `setof record` | Builds an arbitrary set of records from a JSON array of objects See [Note 1](#note-on-json-processing-functions-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text);` | `a \| b`
`---+-----`
`1 \| foo`
`2 \|` | ### Note on JSON processing functions examples @@ -509,131 +262,16 @@ The following built-in advanced analytic functions are WarehousePG extensions of > **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [MADlib Extension for Analytics](../admin_guide/analytics/madlib.md). -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 10. Advanced Aggregate Functions
FunctionReturn TypeFull SyntaxDescription
-MEDIAN (expr) - -timestamp, timestamptz, interval, float - -MEDIAN (expression) -

-Example: -

-
SELECT department_id, MEDIAN(salary) 
-FROM employees 
-GROUP BY department_id;
-		Can take a two-dimensional array as input. Treats such arrays as - matrices.
-PERCENTILE_CONT (expr) WITHIN GROUP (ORDER BY expr - [DESC/ASC]) - -timestamp, timestamptz, interval, float - -PERCENTILE_CONT(percentage) WITHIN GROUP (ORDER BY -expression) -

-Example: -

-
SELECT department_id,
-PERCENTILE_CONT (0.5) WITHIN GROUP (ORDER BY salary DESC)
-"Median_cont"; 
-FROM employees GROUP BY department_id;
-		Performs an inverse distribution function that assumes a - continuous distribution model. It takes a percentile value and a sort specification - and returns the same datatype as the numeric datatype of the argument. This returned - value is a computed result after performing linear interpolation. Null are ignored - in this calculation.
PERCENTILE_DISC (expr) WITHIN GROUP (ORDER BY -expr [DESC/ASC]) -timestamp, timestamptz, interval, float - -PERCENTILE_DISC(percentage) WITHIN GROUP (ORDER BY -expression) -

-Example: -

-
SELECT department_id, 
-PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY salary DESC)
-"Median_desc"; 
-FROM employees GROUP BY department_id;
-		Performs an inverse distribution function that assumes a - discrete distribution model. It takes a percentile value and a sort specification. - This returned value is an element from the set. Null are ignored in this - calculation.
-sum(array[]) - -smallint[]int[], bigint[], float[] - -sum(array[[1,2],[3,4]]) -

-Example: -

-
CREATE TABLE mymatrix (myvalue int[]);
-INSERT INTO mymatrix VALUES (array[[1,2],[3,4]]);
-INSERT INTO mymatrix VALUES (array[[0,1],[1,0]]);
-SELECT sum(myvalue) FROM mymatrix;
- sum 
----------------
- {{1,3},{4,4}}
-		Performs matrix summation. Can take as input a two-dimensional - array that is treated as a matrix.
-pivot_sum (label[], label, expr) - -int[], bigint[], float[] - -pivot_sum( array['A1','A2'], attr, value) -A pivot aggregation using sum to resolve duplicate - entries.
-unnest (array[]) -set of anyelement -unnest( array['one', 'row', 'per', 'item']) -Transforms a one dimensional array into rows. Returns a set of -anyelement, a polymorphic pseudotype in PostgreSQL.
-
+**Advanced Aggregate Functions** + +| Function | Return Type | Full Syntax | Description | +| --- | --- | --- | --- | +| `MEDIAN (`*expr*`)` | `timestamp, timestamptz, interval, float` | `MEDIAN (`*expression*`)`

*Example:*

`SELECT department_id, MEDIAN(salary) `
`FROM employees `
`GROUP BY department_id;` | Can take a two-dimensional array as input. Treats such arrays as matrices. | +| `PERCENTILE_CONT (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` | `timestamp, timestamptz, interval, float` | `PERCENTILE_CONT(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)`

*Example:*

`SELECT department_id,`
`PERCENTILE_CONT (0.5) WITHIN GROUP (ORDER BY salary DESC)`
`"Median_cont"; `
`FROM employees GROUP BY department_id;` | Performs an inverse distribution function that assumes a continuous distribution model. It takes a percentile value and a sort specification and returns the same datatype as the numeric datatype of the argument. This returned value is a computed result after performing linear interpolation. Null are ignored in this calculation. | +| `PERCENTILE_DISC (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` | `timestamp, timestamptz, interval, float` | `PERCENTILE_DISC(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)`

*Example:*

`SELECT department_id, `
`PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY salary DESC)`
`"Median_desc"; `
`FROM employees GROUP BY department_id;` | Performs an inverse distribution function that assumes a discrete distribution model. It takes a percentile value and a sort specification. This returned value is an element from the set. Null are ignored in this calculation. | +| `sum(array[])` | `smallint[]int[], bigint[], float[]` | `sum(array[[1,2],[3,4]])`

*Example:*

`CREATE TABLE mymatrix (myvalue int[]);`
`INSERT INTO mymatrix VALUES (array[[1,2],[3,4]]);`
`INSERT INTO mymatrix VALUES (array[[0,1],[1,0]]);`
`SELECT sum(myvalue) FROM mymatrix;`
`sum `
`---------------`
`{{1,3},{4,4}}` | Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. | +| `pivot_sum (label[], label, expr)` | `int[], bigint[], float[]` | `pivot_sum( array['A1','A2'], attr, value)` | A pivot aggregation using sum to resolve duplicate entries. | +| `unnest (array[])` | set of `anyelement` | `unnest( array['one', 'row', 'per', 'item'])` | Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudotype in PostgreSQL](https://www.postgresql.org/docs/9.4/datatype-pseudo.html). | diff --git a/docs/whpg/6x/ref_guide/misc/feature_summary.md b/docs/whpg/6x/ref_guide/misc/feature_summary.md index a3b3f61..0a82eac 100644 --- a/docs/whpg/6x/ref_guide/misc/feature_summary.md +++ b/docs/whpg/6x/ref_guide/misc/feature_summary.md @@ -207,830 +207,143 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty > **Note** This syntax is deprecated and will be removed in a future WarehousePG release. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 1. SQL Support in WarehousePG
SQL Command -Supported in WarehousePG -Modifications, Limitations, Exceptions -
ALTER AGGREGATEYES 
ALTER CONVERSIONYES 
ALTER DATABASEYES 
ALTER DOMAINYES 
ALTER EVENT TRIGGERYES 
ALTER EXTENSIONYESChanges the definition of a WarehousePG extension - based - on PostgreSQL 9.6.
ALTER FUNCTIONYES 
ALTER GROUPYESAn alias for ALTER ROLE
ALTER INDEXYES 
ALTER LANGUAGEYES 
ALTER OPERATORYES 
ALTER OPERATOR CLASSYES 
ALTER OPERATOR FAMILYYES 
ALTER PROTOCOLYES 
ALTER RESOURCE QUEUEYESWarehousePG resource management feature - not in - PostgreSQL.
ALTER ROLEYESWarehousePG Clauses:

RESOURCE QUEUE - queue_name | none

-
ALTER SCHEMAYES 
ALTER SEQUENCEYES 
ALTER SYSTEMNO 
ALTER TABLEYESUnsupported Clauses / Options:

CLUSTER - ON

-

ENABLE/DISABLE TRIGGER

-

WarehousePG - Database Clauses:

-

ADD | DROP | RENAME | SPLIT | EXCHANGE - PARTITION | SET SUBPARTITION TEMPLATE | SET WITH - (REORGANIZE=true | false) | SET DISTRIBUTED - BY

-
ALTER TABLESPACEYES 
ALTER TRIGGERNO 
ALTER TYPEYESWarehousePG Clauses:

SET DEFAULT - ENCODING

-
ALTER USERYESAn alias for ALTER ROLE
ALTER VIEWYES 
ANALYZEYES 
BEGINYES 
CHECKPOINTYES 
CLOSEYES 
CLUSTERYES 
COMMENTYES 
COMMITYES 
COMMIT PREPAREDNO 
COPYYESModified Clauses:

ESCAPE [ AS ] - 'escape' | 'OFF'

-

WarehousePG - Clauses:

-

[LOG ERRORS] SEGMENT REJECT LIMIT - count [ROWS|PERCENT]

-
CREATE AGGREGATEYESUnsupported Clauses / Options:

[ , SORTOP = - sort_operator ]

-

WarehousePG - Clauses:

-

[ , COMBINEFUNC = combinefunc - ]

-

Limitations:

-

The functions used to implement the - aggregate must be IMMUTABLE functions.

-
CREATE CASTYES 
CREATE CONSTRAINT TRIGGERNO 
CREATE CONVERSIONYES 
CREATE DATABASEYES 
CREATE DOMAINYES 
CREATE EVENT TRIGGERYES 
CREATE EXTENSIONYESLoads a new extension into WarehousePG - based on - PostgreSQL 9.6.
CREATE EXTERNAL TABLEYESWarehousePG parallel ETL feature - not in PostgreSQL - 9.4.
CREATE FUNCTIONYESLimitations:

Functions defined as - STABLE or VOLATILE can be run in - WarehousePG provided that they are run on the coordinator only. - STABLE and VOLATILE functions cannot be used - in statements that run at the segment level.

-
CREATE GROUPYESAn alias for CREATE ROLE
CREATE INDEXYESWarehousePG Clauses:

USING - bitmap (bitmap - indexes)

-

Limitations:

-

UNIQUE indexes are - allowed only if they contain all of (or a superset of) the WarehousePG distribution - key columns. On partitioned tables, a unique index is only supported within an - individual partition - not across all - partitions.

-

CONCURRENTLY keyword not supported in - WarehousePG.

-
CREATE LANGUAGEYES 
CREATE MATERIALIZED VIEWYESBased on PostgreSQL 9.4.
CREATE OPERATORYESLimitations:

The function used to implement the - operator must be an IMMUTABLE function.

-
CREATE OPERATOR CLASSYES 
CREATE OPERATOR FAMILYYES 
CREATE PROTOCOLYES 
CREATE RESOURCE QUEUEYESWarehousePG resource management feature - not in - PostgreSQL 9.4.
CREATE ROLEYESWarehousePG Clauses:

RESOURCE QUEUE - queue_name | none

-
CREATE RULEYES 
CREATE SCHEMAYES 
CREATE SEQUENCEYESLimitations:

The lastval() and - currval() functions are not supported.

-

The - setval() function is only allowed in queries that do not - operate on distributed data.

-
CREATE TABLEYESUnsupported Clauses / Options:

[GLOBAL | - LOCAL]

-

REFERENCES

-

FOREIGN - KEY

-

[DEFERRABLE | NOT DEFERRABLE] -

-

Limited Clauses:

-

UNIQUE or - PRIMARY KEY constraints are only allowed on hash-distributed - tables (DISTRIBUTED BY), and the constraint columns must be the - same as or a superset of the distribution key columns of the table and must - include all the distribution key columns of the partitioning - key.

-

WarehousePG Clauses:

-

DISTRIBUTED BY - (column, [ ... ] ) |

-

DISTRIBUTED - RANDOMLY

-

PARTITION BY type (column [, ...]) -    ( partition_specification, [...] )

-

WITH - (appendoptimized=true      [,compresslevel=value,blocksize=value] - )

-
CREATE TABLE ASYESSee CREATE TABLE
CREATE TABLESPACEYESWarehousePG Clauses:

Specify host file system - locations for specific segment instances.

-

WITH - (contentID_1='/path/to/dir1...)

-
CREATE TRIGGERNO 
CREATE TYPEYESWarehousePG Clauses:

COMPRESSTYPE | - COMPRESSLEVEL | BLOCKSIZE

-

Limitations:

-

The functions - used to implement a new base type must be IMMUTABLE - functions.

-
CREATE USERYESAn alias for CREATE ROLE
CREATE VIEWYES 
DEALLOCATEYES 
DECLAREYESUnsupported Clauses / - Options:

SCROLL

-

FOR UPDATE [ OF column [, - ...] ]

-

Limitations:

-

Cursors cannot be - backward-scrolled. Forward scrolling is supported.

-

PL/pgSQL does not have - support for updatable cursors.

-
DELETEYES 
DISCARDYES -

Limitation: - DISCARD ALL is not supported.

-
DOYESPostgreSQL 9.0 feature
DROP AGGREGATEYES 
DROP CASTYES 
DROP CONVERSIONYES 
DROP DATABASEYES 
DROP DOMAINYES 
DROP EVENT TRIGGERYES 
DROP EXTENSIONYESRemoves an extension from WarehousePG – based on - PostgreSQL 9.6.
DROP EXTERNAL TABLEYESWarehousePG parallel ETL feature - not in PostgreSQL - 9.4.
DROP FUNCTIONYES 
DROP GROUPYESAn alias for DROP ROLE
DROP INDEXYES 
DROP LANGUAGEYES 
DROP OPERATORYES 
DROP OPERATOR CLASSYES 
DROP OPERATOR FAMILYYES 
DROP OWNEDNO 
DROP PROTOCOLYES 
DROP RESOURCE QUEUEYESWarehousePG resource management feature - not in - PostgreSQL 9.4.
DROP ROLEYES 
DROP RULEYES 
DROP SCHEMAYES 
DROP SEQUENCEYES 
DROP TABLEYES 
DROP TABLESPACEYES 
DROP TRIGGERNO 
DROP TYPEYES 
DROP USERYESAn alias for DROP ROLE
DROP VIEWYES 
ENDYES 
EXECUTEYES 
EXPLAINYES 
FETCHYESUnsupported Clauses / - Options:

LAST

-

PRIOR

-

BACKWARD

-

BACKWARD - ALL

-

Limitations:

-

Cannot fetch rows in a - nonsequential fashion; backward scan is not supported.

-
GRANTYES 
INSERTYES 
LATERAL Join TypeNO 
LISTENYES 
LOADYES 
LOCKYES 
MOVEYESSee FETCH
NOTIFYYES 
PREPAREYES 
PREPARE TRANSACTIONNO 
REASSIGN OWNEDYES 
REFRESH MATERIALIZED VIEWYESBased on PostgreSQL 9.4.
REINDEXYES 
RELEASE SAVEPOINTYES 
RESETYES 
RETRIEVEYESWarehousePG parallel retrieve cursor - not in PostgreSQL 9.4.
REVOKEYES 
ROLLBACKYES 
ROLLBACK PREPAREDNO 
ROLLBACK TO SAVEPOINTYES 
SAVEPOINTYES 
SELECTYESLimitations:

Limited use of VOLATILE - and STABLE functions in FROM or - WHERE clauses

-

Text search (Tsearch2) is - not supported

-

WarehousePG Clauses (OLAP):

-

[GROUP - BY grouping_element [, - ...]]

-

[WINDOW window_name AS - (window_specification)]

-

[FILTER - (WHERE condition)] applied to an aggregate - function in the SELECT list

-
SELECT INTOYESSee SELECT
SETYES 
SET CONSTRAINTSNOIn PostgreSQL, this only applies to foreign key constraints, - which are currently not enforced in WarehousePG.
SET ROLEYES 
SET SESSION AUTHORIZATIONYESDeprecated as of PostgreSQL 8.1 - see SET ROLE
SET TRANSACTIONYESLimitations:

DEFERRABLE clause has no - effect.

-

SET TRANSACTION SNAPSHOT command is not - supported.

-
SHOWYES 
START TRANSACTIONYES 
TRUNCATEYES 
UNLISTENYES 
UPDATEYESLimitations:

SET not allowed for - WarehousePG distribution key columns.

-
VACUUMYESLimitations:

VACUUM FULL is not - recommended in WarehousePG.

-
VALUESYES 
+**SQL Support in WarehousePG** + +| SQL Command | Supported in WarehousePG | Modifications, Limitations, Exceptions | +| --- | --- | --- | +| `ALTER AGGREGATE` | YES | | +| `ALTER CONVERSION` | YES | | +| `ALTER DATABASE` | YES | | +| `ALTER DOMAIN` | YES | | +| `ALTER EVENT TRIGGER` | YES | | +| `ALTER EXTENSION` | YES | Changes the definition of a WarehousePG extension - based on PostgreSQL 9.6. | +| `ALTER FUNCTION` | YES | | +| `ALTER GROUP` | YES | An alias for [ALTER ROLE](../sql_commands/ALTER_ROLE.md) | +| `ALTER INDEX` | YES | | +| `ALTER LANGUAGE` | YES | | +| `ALTER OPERATOR` | YES | | +| `ALTER OPERATOR CLASS` | YES | | +| `ALTER OPERATOR FAMILY` | YES | | +| `ALTER PROTOCOL` | YES | | +| `ALTER RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL. | +| `ALTER ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | +| `ALTER SCHEMA` | YES | | +| `ALTER SEQUENCE` | YES | | +| `ALTER SYSTEM` | **NO** | | +| `ALTER TABLE` | YES | **Unsupported Clauses / Options:**

`CLUSTER ON`

`ENABLE/DISABLE TRIGGER`

**WarehousePG Database Clauses:**

`ADD \| DROP \| RENAME \| SPLIT \| EXCHANGE PARTITION \| SET SUBPARTITION TEMPLATE \| SET WITH ` `(REORGANIZE=true \| false) \| SET DISTRIBUTED BY` | +| `ALTER TABLESPACE` | YES | | +| `ALTER TRIGGER` | **NO** | | +| `ALTER TYPE` | YES | **WarehousePG Clauses:**

`SET DEFAULT ENCODING` | +| `ALTER USER` | YES | An alias for [ALTER ROLE](../sql_commands/ALTER_ROLE.md) | +| `ALTER VIEW` | YES | | +| `ANALYZE` | YES | | +| `BEGIN` | YES | | +| `CHECKPOINT` | YES | | +| `CLOSE` | YES | | +| `CLUSTER` | YES | | +| `COMMENT` | YES | | +| `COMMIT` | YES | | +| `COMMIT PREPARED` | **NO** | | +| `COPY` | YES | **Modified Clauses:**

`ESCAPE [ AS ] '`*escape*`' \| 'OFF'`

**WarehousePG Clauses:**

`[LOG ERRORS] SEGMENT REJECT LIMIT `*count*` [ROWS\|PERCENT]` | +| `CREATE AGGREGATE` | YES | **Unsupported Clauses / Options:**

`[ , SORTOP = `*sort_operator*` ]`

**WarehousePG Clauses:**

`[ , COMBINEFUNC = `*combinefunc*` ]`

**Limitations:**

The functions used to implement the aggregate must be `IMMUTABLE` functions. | +| `CREATE CAST` | YES | | +| `CREATE CONSTRAINT TRIGGER` | **NO** | | +| `CREATE CONVERSION` | YES | | +| `CREATE DATABASE` | YES | | +| `CREATE DOMAIN` | YES | | +| `CREATE EVENT TRIGGER` | YES | | +| `CREATE EXTENSION` | YES | Loads a new extension into WarehousePG - based on PostgreSQL 9.6. | +| `CREATE EXTERNAL TABLE` | YES | WarehousePG parallel ETL feature - not in PostgreSQL 9.4. | +| `CREATE FUNCTION` | YES | **Limitations:**

Functions defined as `STABLE` or `VOLATILE` can be run in WarehousePG provided that they are run on the coordinator only. `STABLE` and `VOLATILE` functions cannot be used in statements that run at the segment level. | +| `CREATE GROUP` | YES | An alias for [CREATE ROLE](../sql_commands/CREATE_ROLE.md) | +| `CREATE INDEX` | YES | **WarehousePG Clauses:**

`USING bitmap` (bitmap indexes)

**Limitations:**

`UNIQUE` indexes are allowed only if they contain all of (or a superset of) the WarehousePG distribution key columns. On partitioned tables, a unique index is only supported within an individual partition - not across all partitions.

`CONCURRENTLY` keyword not supported in WarehousePG. | +| `CREATE LANGUAGE` | YES | | +| `CREATE MATERIALIZED VIEW` | YES | Based on PostgreSQL 9.4. | +| `CREATE OPERATOR` | YES | **Limitations:**

The function used to implement the operator must be an `IMMUTABLE` function. | +| `CREATE OPERATOR CLASS` | YES | | +| `CREATE OPERATOR FAMILY` | YES | | +| `CREATE PROTOCOL` | YES | | +| `CREATE RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL 9.4. | +| `CREATE ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | +| `CREATE RULE` | YES | | +| `CREATE SCHEMA` | YES | | +| `CREATE SEQUENCE` | YES | **Limitations:**

The `lastval()` and `currval()` functions are not supported.

The `setval()` function is only allowed in queries that do not operate on distributed data. | +| `CREATE TABLE` | YES | **Unsupported Clauses / Options:**

`[GLOBAL \| LOCAL]`

`REFERENCES`

`FOREIGN KEY`

`[DEFERRABLE \| NOT DEFERRABLE] `

**Limited Clauses:**

`UNIQUE` or `PRIMARY KEY `constraints are only allowed on hash-distributed tables (`DISTRIBUTED BY`), and the constraint columns must be the same as or a superset of the distribution key columns of the table and must include all the distribution key columns of the partitioning key.

**WarehousePG Clauses:**

`DISTRIBUTED BY (column, [ ... ] ) \|`

`DISTRIBUTED RANDOMLY`

`PARTITION BY type (column [, ...]) ( partition_specification, [...] )`

`WITH (appendoptimized=true [,compresslevel=value,blocksize=value] )` | +| `CREATE TABLE AS` | YES | See [CREATE TABLE](../sql_commands/CREATE_TABLE.md) | +| `CREATE TABLESPACE` | **YES** | **WarehousePG Clauses:**

Specify host file system locations for specific segment instances.

`WITH (contentID_1='/path/to/dir1...)` | +| `CREATE TRIGGER` | **NO** | | +| `CREATE TYPE` | YES | **WarehousePG Clauses:**

`COMPRESSTYPE \| COMPRESSLEVEL \| BLOCKSIZE`

**Limitations:**

The functions used to implement a new base type must be `IMMUTABLE` functions. | +| `CREATE USER` | YES | An alias for [CREATE ROLE](../sql_commands/CREATE_ROLE.md) | +| `CREATE VIEW` | YES | | +| `DEALLOCATE` | YES | | +| `DECLARE` | YES | **Unsupported Clauses / Options:**

`SCROLL`

`FOR UPDATE [ OF column [, ...] ]`

**Limitations:**

Cursors cannot be backward-scrolled. Forward scrolling is supported.

PL/pgSQL does not have support for updatable cursors. | +| `DELETE` | YES | | +| `DISCARD` | YES | **Limitation:** `DISCARD ALL` is not supported. | +| `DO` | YES | PostgreSQL 9.0 feature | +| `DROP AGGREGATE` | YES | | +| `DROP CAST` | YES | | +| `DROP CONVERSION` | YES | | +| `DROP DATABASE` | YES | | +| `DROP DOMAIN` | YES | | +| `DROP EVENT TRIGGER` | YES | | +| `DROP EXTENSION` | YES | Removes an extension from WarehousePG – based on PostgreSQL 9.6. | +| `DROP EXTERNAL TABLE` | YES | WarehousePG parallel ETL feature - not in PostgreSQL 9.4. | +| `DROP FUNCTION` | YES | | +| `DROP GROUP` | YES | An alias for [DROP ROLE](../sql_commands/DROP_ROLE.md) | +| `DROP INDEX` | YES | | +| `DROP LANGUAGE` | YES | | +| `DROP OPERATOR` | YES | | +| `DROP OPERATOR CLASS` | YES | | +| `DROP OPERATOR FAMILY` | YES | | +| `DROP OWNED` | **NO** | | +| `DROP PROTOCOL` | YES | | +| `DROP RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL 9.4. | +| `DROP ROLE` | YES | | +| `DROP RULE` | YES | | +| `DROP SCHEMA` | YES | | +| `DROP SEQUENCE` | YES | | +| `DROP TABLE` | YES | | +| `DROP TABLESPACE` | YES | | +| `DROP TRIGGER` | **NO** | | +| `DROP TYPE` | YES | | +| `DROP USER` | YES | An alias for [DROP ROLE](../sql_commands/DROP_ROLE.md) | +| `DROP VIEW` | YES | | +| `END` | YES | | +| `EXECUTE` | YES | | +| `EXPLAIN` | YES | | +| `FETCH` | YES | **Unsupported Clauses / Options:**

`LAST`

`PRIOR`

`BACKWARD`

`BACKWARD ALL`

**Limitations:**

Cannot fetch rows in a nonsequential fashion; backward scan is not supported. | +| `GRANT` | YES | | +| `INSERT` | YES | | +| `LATERAL` Join Type | **NO** | | +| `LISTEN` | **YES** | | +| `LOAD` | YES | | +| `LOCK` | YES | | +| `MOVE` | YES | See [FETCH](../sql_commands/FETCH.md) | +| `NOTIFY` | **YES** | | +| `PREPARE` | YES | | +| `PREPARE TRANSACTION` | **NO** | | +| `REASSIGN OWNED` | YES | | +| `REFRESH MATERIALIZED VIEW` | YES | Based on PostgreSQL 9.4. | +| `REINDEX` | YES | | +| `RELEASE SAVEPOINT` | YES | | +| `RESET` | YES | | +| `RETRIEVE` | YES | WarehousePG parallel retrieve cursor - not in PostgreSQL 9.4. | +| `REVOKE` | YES | | +| `ROLLBACK` | YES | | +| `ROLLBACK PREPARED` | **NO** | | +| `ROLLBACK TO SAVEPOINT` | YES | | +| `SAVEPOINT` | YES | | +| `SELECT` | YES | **Limitations:**

Limited use of `VOLATILE` and `STABLE` functions in `FROM` or `WHERE` clauses

Text search (`Tsearch2`) is not supported

**WarehousePG Clauses (OLAP):**

`[GROUP BY `*grouping_element*` [, ...]]`

`[WINDOW `*window_name*` AS (`*window_specification*`)]`

`[FILTER (WHERE `*condition*`)]` applied to an aggregate function in the `SELECT` list | +| `SELECT INTO` | YES | See [SELECT](../sql_commands/SELECT.md) | +| `SET` | YES | | +| `SET CONSTRAINTS` | **NO** | In PostgreSQL, this only applies to foreign key constraints, which are currently not enforced in WarehousePG. | +| `SET ROLE` | YES | | +| `SET SESSION AUTHORIZATION` | YES | Deprecated as of PostgreSQL 8.1 - see [SET ROLE](../sql_commands/SET_ROLE.md) | +| `SET TRANSACTION` | YES | **Limitations:**

`DEFERRABLE` clause has no effect.

`SET TRANSACTION SNAPSHOT` command is not supported. | +| `SHOW` | YES | | +| `START TRANSACTION` | YES | | +| `TRUNCATE` | YES | | +| `UNLISTEN` | **YES** | | +| `UPDATE` | YES | **Limitations:**

`SET` not allowed for WarehousePG distribution key columns. | +| `VACUUM` | YES | **Limitations:**

`VACUUM FULL` is not recommended in WarehousePG. | +| `VALUES` | YES | | diff --git a/docs/whpg/7x/admin_guide/analytics/postGIS.md b/docs/whpg/7x/admin_guide/analytics/postGIS.md index f4cd27c..914bd86 100644 --- a/docs/whpg/7x/admin_guide/analytics/postGIS.md +++ b/docs/whpg/7x/admin_guide/analytics/postGIS.md @@ -30,62 +30,15 @@ There are significant changes in PostGIS 3.3.2 compared with earlier versions. F This table lists the PostGIS extensions support by WarehousePG PostGIS. -
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 1. WarehousePG PostGIS Extensions
PostGIS ExtensionWarehousePG PostGIS Notes
postgis

PostGIS and PostGIS Raster - support

-		Supported. Both PostGIS and PostGIS Raster are enabled when the WarehousePG - postgis extension is enabled.
postgis_tiger_geocoder

The US TIGER geocoder

-		Supported. Installed with WarehousePG PostGIS.

Requires the - postgis and fuzzystrmatch - extensions.

-

The US TIGER geocoder converts addresses (like a street address) - to geographic coordinates.

-
address_standardizer

Rule-based address - standardizer

-		Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used - with TIGER geocoder.

-

A single line address parser that takes an input - address and normalizes it based on a set of rules stored in a table and helper - lex and gaz tables.

-
address_standardizer_data_us

Sample rules tables for US - address data

-		Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with - the address standardizer.

-

The extension contains gaz, - lex, and rules tables for US address data. If - you are using other types of tables, see PostGIS Extension Limitations.

-
fuzzystrmatch

Fuzzy string matching

-		Supported. This extension is bundled but not enabled with WarehousePG - Database.

Required for the PostGIS TIGER geocoder.

-
-
+**WarehousePG PostGIS Extensions** + +| PostGIS Extension | WarehousePG PostGIS Notes | +| --- | --- | +| `postgis`

PostGIS and PostGIS Raster support | Supported. Both PostGIS and PostGIS Raster are enabled when the WarehousePG `postgis` extension is enabled. | +| `postgis_tiger_geocoder`

The US TIGER geocoder | Supported. Installed with WarehousePG PostGIS.

Requires the `postgis` and `fuzzystrmatch` extensions.

The US TIGER geocoder converts addresses (like a street address) to geographic coordinates. | +| `address_standardizer`

Rule-based address standardizer | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with TIGER geocoder.

A single line address parser that takes an input address and normalizes it based on a set of rules stored in a table and helper `lex` and `gaz` tables. | +| `address_standardizer_data_us`

Sample rules tables for US address data | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with the address standardizer.

The extension contains `gaz`, `lex`, and `rules` tables for US address data. If you are using other types of tables, see [PostGIS Extension Limitations](#topic_wy2_rkb_3p). | +| `fuzzystrmatch`

Fuzzy string matching | Supported. This extension is bundled but not enabled with WarehousePG Database.

Required for the PostGIS TIGER geocoder. | > **Note** The PostGIS topology extension `postgis_topology` and the PostGIS 3D and geoprocessing extension `postgis_sfcgal` are not supported by WarehousePG PostGIS and are not included in the WarehousePG PostGIS extension package. diff --git a/docs/whpg/7x/admin_guide/external/foreign/devel-fdw.md b/docs/whpg/7x/admin_guide/external/foreign/devel-fdw.md index 1c654f6..f896527 100644 --- a/docs/whpg/7x/admin_guide/external/foreign/devel-fdw.md +++ b/docs/whpg/7x/admin_guide/external/foreign/devel-fdw.md @@ -134,147 +134,28 @@ You must implement the scan-related functions in your foreign-data wrapper; impl Scan-related callback functions include: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Callback SignatureDescription
void
-GetForeignRelSize (PlannerInfo *root,
-                   RelOptInfo *baserel,
-                   Oid foreigntableid)
Obtain relation size estimates for a foreign table. - Called at the beginning of planning for a query on a foreign table.
void
-GetForeignPaths (PlannerInfo *root,
-                 RelOptInfo *baserel,
-                 Oid foreigntableid)
Create possible access paths for a scan on a - foreign table. Called during query planning.
Note: A WarehousePG - Database-compatible FDW must call - create_foreignscan_path() in its - GetForeignPaths() callback function.
ForeignScan *
-GetForeignPlan (PlannerInfo *root,
-                RelOptInfo *baserel,
-                Oid foreigntableid,
-                ForeignPath *best_path,
-                List *tlist,
-                List *scan_clauses)
Create a ForeignScan plan node from - the selected foreign access path. Called at the end of query planning.
void
-BeginForeignScan (ForeignScanState *node,
-                  int eflags)
Begin running a foreign scan. Called during - executor startup.
TupleTableSlot *
-IterateForeignScan (ForeignScanState *node)
Fetch one row from the foreign source, returning it - in a tuple table slot; return NULL if no more rows are available.
void
-ReScanForeignScan (ForeignScanState *node)
Restart the scan from the beginning.
void
-EndForeignScan (ForeignScanState *node)
End the scan and release resources.
+| Callback Signature | Description | +| --- | --- | +| `void`
`GetForeignRelSize (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Obtain relation size estimates for a foreign table. Called at the beginning of planning for a query on a foreign table. | +| `void`
`GetForeignPaths (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Create possible access paths for a scan on a foreign table. Called during query planning.

**Note:** A WarehousePG Database-compatible FDW must call `create_foreignscan_path()` in its `GetForeignPaths()` callback function. | +| `ForeignScan *`
`GetForeignPlan (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid,`
` ForeignPath *best_path,`
` List *tlist,`
` List *scan_clauses)` | Create a `ForeignScan` plan node from the selected foreign access path. Called at the end of query planning. | +| `void`
`BeginForeignScan (ForeignScanState *node,`
` int eflags)` | Begin running a foreign scan. Called during executor startup. | +| `TupleTableSlot *`
`IterateForeignScan (ForeignScanState *node)` | Fetch one row from the foreign source, returning it in a tuple table slot; return NULL if no more rows are available. | +| `void`
`ReScanForeignScan (ForeignScanState *node)` | Restart the scan from the beginning. | +| `void`
`EndForeignScan (ForeignScanState *node)` | End the scan and release resources. | If a foreign data wrapper supports writable foreign tables, it should provide the update-related callback functions that are required by the capabilities of the FDW. Update-related callback functions include: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Callback SignatureDescription
void
-AddForeignUpdateTargets (Query *parsetree,
-                         RangeTblEntry *target_rte,
-                         Relation target_relation)
Add additional information in the foreign - table that will be retrieved during an update or delete operation - to identify the exact row on which to operate.
List *
-PlanForeignModify (PlannerInfo *root,
-                   ModifyTable *plan,
-                   Index resultRelation,
-                   int subplan_index)
Perform additional planning actions required - for an insert, update, or delete operation on a foreign table, - and return the information generated.
void
-BeginForeignModify (ModifyTableState *mtstate,
-                    ResultRelInfo *rinfo,
-                    List *fdw_private,
-                    int subplan_index,
-                    int eflags)
Begin executing a modify operation on a - foreign table. Called during executor startup.
TupleTableSlot *
-ExecForeignInsert (EState *estate,
-                   ResultRelInfo *rinfo,
-                   TupleTableSlot *slot,
-                   TupleTableSlot *planSlot)
Insert a single tuple into the foreign table. - Return a slot containing the data that was actually inserted, or - NULL if no row was inserted.
TupleTableSlot *
-ExecForeignUpdate (EState *estate,
-                   ResultRelInfo *rinfo,
-                   TupleTableSlot *slot,
-                   TupleTableSlot *planSlot)
Update a single tuple in the foreign table. - Return a slot containing the row as it was actually updated, or - NULL if no row was updated. 
TupleTableSlot *
-ExecForeignDelete (EState *estate,
-                   ResultRelInfo *rinfo,
-                   TupleTableSlot *slot,
-                   TupleTableSlot *planSlot)
Delete a single tuple from the foreign table. - Return a slot containing the row that was deleted, or NULL if no - row was deleted.
void
-EndForeignModify (EState *estate,
-                  ResultRelInfo *rinfo)
End the update and release resources.
int
-IsForeignRelUpdatable (Relation rel)
Report the update operations supported by the - specified foreign table.
+| Callback Signature | Description | +| --- | --- | +| `void`
`AddForeignUpdateTargets (Query *parsetree,`
` RangeTblEntry *target_rte,`
` Relation target_relation)` | Add additional information in the foreign table that will be retrieved during an update or delete operation to identify the exact row on which to operate. | +| `List *`
`PlanForeignModify (PlannerInfo *root,`
` ModifyTable *plan,`
` Index resultRelation,`
` int subplan_index)` | Perform additional planning actions required for an insert, update, or delete operation on a foreign table, and return the information generated. | +| `void`
`BeginForeignModify (ModifyTableState *mtstate,`
` ResultRelInfo *rinfo,`
` List *fdw_private,`
` int subplan_index,`
` int eflags)` | Begin executing a modify operation on a foreign table. Called during executor startup. | +| `TupleTableSlot *`
`ExecForeignInsert (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Insert a single tuple into the foreign table. Return a slot containing the data that was actually inserted, or NULL if no row was inserted. | +| `TupleTableSlot *`
`ExecForeignUpdate (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Update a single tuple in the foreign table. Return a slot containing the row as it was actually updated, or NULL if no row was updated. | +| `TupleTableSlot *`
`ExecForeignDelete (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Delete a single tuple from the foreign table. Return a slot containing the row that was deleted, or NULL if no row was deleted. | +| `void`
`EndForeignModify (EState *estate,`
` ResultRelInfo *rinfo)` | End the update and release resources. | +| `int`
`IsForeignRelUpdatable (Relation rel)` | Report the update operations supported by the specified foreign table. | Refer to [Foreign Data Wrapper Callback Routines](https://www.postgresql.org/docs/12/fdw-callbacks.html) in the PostgreSQL documentation for detailed information about the inputs and outputs of the FDW callback functions. @@ -290,56 +171,17 @@ The FDW API exports several helper functions from the WarehousePG core server so The FDW API includes the helper functions listed in the table below. Refer to [Foreign Data Wrapper Helper Functions](https://www.postgresql.org/docs/12/fdw-helpers.html) in the PostgreSQL documentation for more information about these functions. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Helper SignatureDescription
ForeignDataWrapper *
-GetForeignDataWrapper(Oid fdwid);
Returns the ForeignDataWrapper - object for the foreign-data wrapper with the given OID.
ForeignDataWrapper *
-GetForeignDataWrapperByName(const char *name, bool missing_ok);
Returns the ForeignDataWrapper - object for the foreign-data wrapper with the given name.
ForeignServer *
-GetForeignServer(Oid serverid);
Returns the ForeignServer - object for the foreign server with the given OID.
ForeignServer *
-GetForeignServerByName(const char *name, bool missing_ok);
Returns the ForeignServer - object for the foreign server with the given name.
UserMapping *
-GetUserMapping(Oid userid, Oid serverid);
Returns the UserMapping - object for the user mapping of the given role on the given - server.
ForeignTable *
-GetForeignTable(Oid relid);
Returns the ForeignTable - object for the foreign table with the given OID.
List *
-GetForeignColumnOptions(Oid relid, AttrNumber attnum);
Returns the per-column FDW options for the - column with the given foreign table OID and attribute number.
 +| Helper Signature | Description | +| --- | --- | +| `ForeignDataWrapper *`
`GetForeignDataWrapper(Oid fdwid);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given OID. | +| `ForeignDataWrapper *`
`GetForeignDataWrapperByName(const char *name, bool missing_ok);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given name. | +| `ForeignServer *`
`GetForeignServer(Oid serverid);` | Returns the `ForeignServer` object for the foreign server with the given OID. | +| `ForeignServer *`
`GetForeignServerByName(const char *name, bool missing_ok);` | Returns the `ForeignServer` object for the foreign server with the given name. | +| `UserMapping *`
`GetUserMapping(Oid userid, Oid serverid);` | Returns the `UserMapping` object for the user mapping of the given role on the given server. | +| `ForeignTable *`
`GetForeignTable(Oid relid);` | Returns the `ForeignTable` object for the foreign table with the given OID. | +| `List *`
`GetForeignColumnOptions(Oid relid, AttrNumber attnum);` | Returns the per-column FDW options for the column with the given foreign table OID and attribute number. | + + ## WarehousePG Considerations diff --git a/docs/whpg/7x/admin_guide/intro/about_statistics.md b/docs/whpg/7x/admin_guide/intro/about_statistics.md index 7c49666..f459a20 100644 --- a/docs/whpg/7x/admin_guide/intro/about_statistics.md +++ b/docs/whpg/7x/admin_guide/intro/about_statistics.md @@ -73,114 +73,17 @@ The statistics collected for a column vary for different data types, so the `pg_ The `stakindN` columns each contain a numeric code to describe the type of statistics stored in their slot. The `stakind` code numbers from 1 to 99 are reserved for core PostgreSQL data types. WarehousePG uses code numbers 1, 2, 3, 4, 5, and 99. A value of 0 means the slot is unused. The following table describes the kinds of statistics stored for the three codes. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 1. Contents of pg_statistic "slots"
stakind CodeDescription
1Most CommonValues (MCV) Slot -
    -
  • staop contains the object ID of the "=" operator, used to - decide whether values are the same or not.
    • -
  • stavalues contains an array of the K - most common non-null values appearing in the column.
    • -
  • stanumbers contains the frequencies (fractions of total - row count) of the values in the stavalues array.
    • -
The values are ordered in decreasing frequency. Since the arrays are - variable-size, K can be chosen by the statistics collector. - Values must occur more than once to be added to the stavalues - array; a unique column has no MCV slot.
2Histogram Slot – describes the distribution of scalar data.
    -
  • staop is the object ID of the "<" operator, which - describes the sort ordering.
    • -
  • stavalues contains M (where - M>=2) non-null values that divide - the non-null column data values into M-1 - bins of approximately equal population. The first stavalues - item is the minimum value and the last is the maximum value.
    • -
  • stanumbers is not used and should be - NULL.
    • -

If a Most Common Values slot is also provided, then the histogram - describes the data distribution after removing the values listed in the MCV - array. (It is a compressed histogram in the technical parlance). This - allows a more accurate representation of the distribution of a column with - some very common values. In a column with only a few distinct values, it is - possible that the MCV list describes the entire data population; in this case - the histogram reduces to empty and should be omitted.

3Correlation Slot – describes the correlation between the physical - order of table tuples and the ordering of data values of this column.
    -
  • staop is the object ID of the "<" operator. As with - the histogram, more than one entry could theoretically appear.
    • -
  • stavalues is not used and should be - NULL.
    • -
  • stanumbers contains a single entry, the correlation - coefficient between the sequence of data values and the sequence of their - actual tuple positions. The coefficient ranges from +1 to -1.
    • -
4Most Common Elements Slot - is similar to a Most Common Values (MCV) - Slot, except that it stores the most common non-null elements of the - column values. This is useful when the column datatype is an array or some other - type with identifiable elements (for instance, tsvector).
    -
  • staop contains the equality operator appropriate to the - element type.
    • -
  • stavalues contains the most common element values.
    • -
  • stanumbers contains common element frequencies.
    • -

Frequencies are measured as the fraction of non-null rows the element - value appears in, not the frequency of all rows. Also, the values are sorted - into the element type's default order (to support binary search for a - particular value). Since this puts the minimum and maximum frequencies at - unpredictable spots in stanumbers, there are two extra - members of stanumbers that hold copies of the minimum and - maximum frequencies. Optionally, there can be a third extra member that holds - the frequency of null elements (the frequency is expressed in the same terms: - the fraction of non-null rows that contain at least one null element). If this - member is omitted, the column is presumed to contain no NULL - elements.

-
Note: For tsvector columns, the stavalues - elements are of type text, even though their representation - within tsvector is not exactly - text.
5Distinct Elements Count Histogram Slot - describes the distribution - of the number of distinct element values present in each row of an array-type - column. Only non-null rows are considered, and only non-null elements.
    -
  • staop contains the equality operator appropriate to the - element type.
    • -
  • stavalues is not used and should be - NULL.
    • -
  • stanumbers contains information about distinct elements. - The last member of stanumbers is the average count of - distinct element values over all non-null rows. The preceding - M (where M >=2) - members form a histogram that divides the population of distinct-elements - counts into M-1 bins of approximately - equal population. The first of these is the minimum observed count, and the - last the maximum.
    • -
99Hyperloglog Slot - for leaf partitions of a partitioned table, - stores the hyperloglog_counter created for the sampled data. - The hyperloglog_counter data structure is converted into a - bytea and stored in a stavalues5 slot of the - pg_statistic catalog table.
+**Contents of pg_statistic "slots"** + +| stakind Code | Description | +| --- | --- | +| 1 | *Most CommonValues (MCV) Slot*

- `staop` contains the object ID of the "=" operator, used to decide whether values are the same or not.
- `stavalues` contains an array of the *K* most common non-null values appearing in the column.
- `stanumbers` contains the frequencies (fractions of total row count) of the values in the `stavalues` array.

The values are ordered in decreasing frequency. Since the arrays are variable-size, *K* can be chosen by the statistics collector. Values must occur more than once to be added to the `stavalues` array; a unique column has no MCV slot. | +| 2 | *Histogram Slot* – describes the distribution of scalar data.

- `staop` is the object ID of the "<" operator, which describes the sort ordering.
- `stavalues` contains *M* (where `M>=2`) non-null values that divide the non-null column data values into `M-1` bins of approximately equal population. The first `stavalues` item is the minimum value and the last is the maximum value.
- `stanumbers` is not used and should be `NULL`.

If a Most Common Values slot is also provided, then the histogram describes the data distribution after removing the values listed in the MCV array. (It is a *compressed histogram* in the technical parlance). This allows a more accurate representation of the distribution of a column with some very common values. In a column with only a few distinct values, it is possible that the MCV list describes the entire data population; in this case the histogram reduces to empty and should be omitted. | +| 3 | *Correlation Slot* – describes the correlation between the physical order of table tuples and the ordering of data values of this column.

- `staop` is the object ID of the "<" operator. As with the histogram, more than one entry could theoretically appear.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains a single entry, the correlation coefficient between the sequence of data values and the sequence of their actual tuple positions. The coefficient ranges from +1 to -1. | +| 4 | *Most Common Elements Slot* - is similar to a Most Common Values (MCV) Slot, except that it stores the most common non-null *elements* of the column values. This is useful when the column datatype is an array or some other type with identifiable elements (for instance, `tsvector`).

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` contains the most common element values.
- `stanumbers` contains common element frequencies.

Frequencies are measured as the fraction of non-null rows the element value appears in, not the frequency of all rows. Also, the values are sorted into the element type's default order (to support binary search for a particular value). Since this puts the minimum and maximum frequencies at unpredictable spots in `stanumbers`, there are two extra members of `stanumbers` that hold copies of the minimum and maximum frequencies. Optionally, there can be a third extra member that holds the frequency of null elements (the frequency is expressed in the same terms: the fraction of non-null rows that contain at least one null element). If this member is omitted, the column is presumed to contain no `NULL` elements.

**Note:** For `tsvector` columns, the `stavalues` elements are of type `text`, even though their representation within `tsvector` is not exactly `text`. | +| 5 | *Distinct Elements Count Histogram Slot* - describes the distribution of the number of distinct element values present in each row of an array-type column. Only non-null rows are considered, and only non-null elements.

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains information about distinct elements. The last member of `stanumbers` is the average count of distinct element values over all non-null rows. The preceding *M* (where `M >=2`) members form a histogram that divides the population of distinct-elements counts into `M-1` bins of approximately equal population. The first of these is the minimum observed count, and the last the maximum. | +| 99 | *Hyperloglog Slot* - for leaf partitions of a partitioned table, stores the `hyperloglog_counter` created for the sampled data. The `hyperloglog_counter` data structure is converted into a `bytea` and stored in a `stavalues5` slot of the `pg_statistic` catalog table. | + The `pg_stats` view presents the contents of `pg_statistic` in a friendlier format. The `pg_stats` view has the following columns: diff --git a/docs/whpg/7x/admin_guide/manage_access/roles_privs.md b/docs/whpg/7x/admin_guide/manage_access/roles_privs.md index 01244d1..b1a4d59 100644 --- a/docs/whpg/7x/admin_guide/manage_access/roles_privs.md +++ b/docs/whpg/7x/admin_guide/manage_access/roles_privs.md @@ -119,204 +119,23 @@ The role attributes `LOGIN`, `SUPERUSER`, `CREATEDB`, `CREATEROLE`, `CREATEEXTTA When an object (table, view, sequence, database, function, language, schema, or tablespace) is created, it is assigned an owner. The owner is normally the role that ran the creation statement. For most kinds of objects, the initial state is that only the owner (or a superuser) can do anything with the object. To allow other roles to use it, privileges must be granted. WarehousePG supports the following privileges for each object type: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 2. Object Privileges
Object TypePrivileges
Tables, External Tables, Views -

- SELECT -

-

- INSERT -

-

- UPDATE -

-

- DELETE -

-

- REFERENCES -

-

- TRIGGER -

-

- TRUNCATE -

-

- ALL -

-
Columns -

- SELECT -

-

- INSERT -

-

- UPDATE -

-

- REFERENCES -

-

- ALL -

-
Sequences -

- USAGE -

-

- SELECT -

-

- UPDATE -

-

- ALL -

-
Databases -

- CREATE -

-

- CONNECT -

-

- TEMPORARY -

-

- TEMP -

-

- ALL -

-
Domains -

- USAGE -

-

- ALL -

-
Foreign Data Wrappers -

- USAGE -

-

- ALL -

-
Foreign Servers -

- USAGE -

-

- ALL -

-
Functions -

- EXECUTE -

-

- ALL -

-
Procedural Languages -

- USAGE -

-

- ALL -

-
Schemas -

- CREATE -

-

- USAGE -

-

- ALL -

-
Tablespaces -

- CREATE -

-

- ALL -

-
Types -

- USAGE -

-

- ALL -

-
Protocols -

- SELECT -

-

- INSERT -

-

- ALL -

-
+**Object Privileges** + +| Object Type | Privileges | +| --- | --- | +| Tables, External Tables, Views | `SELECT`

`INSERT`

`UPDATE`

`DELETE`

`REFERENCES`

`TRIGGER`

`TRUNCATE`

`ALL` | +| Columns | `SELECT`

`INSERT`

`UPDATE`

`REFERENCES`

`ALL` | +| Sequences | `USAGE`

`SELECT`

`UPDATE`

`ALL` | +| Databases | `CREATE`

`CONNECT`

`TEMPORARY`

`TEMP`

`ALL` | +| Domains | `USAGE`

`ALL` | +| Foreign Data Wrappers | `USAGE`

`ALL` | +| Foreign Servers | `USAGE`

`ALL` | +| Functions | `EXECUTE`

`ALL` | +| Procedural Languages | `USAGE`

`ALL` | +| Schemas | `CREATE`

`USAGE`

`ALL` | +| Tablespaces | `CREATE`

`ALL` | +| Types | `USAGE`

`ALL` | +| Protocols | `SELECT`

`INSERT`

`ALL` | > **Note** You must grant privileges for each object individually. For example, granting `ALL` on a database does not grant full access to the objects within that database. It only grants all of the database-level privileges (`CONNECT`, `CREATE`, `TEMPORARY`) to the database itself. diff --git a/docs/whpg/7x/admin_guide/managing/monitoring.md b/docs/whpg/7x/admin_guide/managing/monitoring.md index c889005..c4b4ca7 100644 --- a/docs/whpg/7x/admin_guide/managing/monitoring.md +++ b/docs/whpg/7x/admin_guide/managing/monitoring.md @@ -17,281 +17,56 @@ It is not necessary to implement all of these suggestions in every cluster; use ## Database State Monitoring Activities - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 1. Database State Monitoring Activities
ActivityProcedureCorrective Actions
List segments that are currently down. If any rows are - returned, this should generate a warning or alert.

Recommended - frequency: run every 5 to 10 minutes

Severity: - IMPORTANT

Run the following query in the postgres - database:
SELECT * FROM gp_segment_configuration
-WHERE status = 'd';
If the query returns any rows, follow these steps to correct - the problem:
    -
  1. Verify that the hosts with down segments are responsive.
    2. -
  2. If hosts are OK, check the log - files for the primaries and - mirrors of the down segments to discover the - root cause of the segments going down.
    3. -
  3. If no unexpected errors are found, run the -gprecoverseg utility to bring the - segments back online.
    4. -
Check for segments that are up and not in sync. If rows are - returned, this should generate a warning or alert.

Recommended - frequency: run every 5 to 10 minutes

 -
Execute the following query in the postgres - database:
SELECT * FROM gp_segment_configuration
-WHERE mode = 'n' and status = 'u' and content <> -1;
-		If the query returns rows then the segment might be in the process - of moving from Not In Sync to -Synchronized mode. Use gpstate - -e to track progress.
Check for segments that are not operating in their preferred role but are marked as up and Synchronized. - If any segments are found, the cluster may not be - balanced. If any rows are returned this should generate a - warning or alert.

Recommended frequency: run every 5 to 10 - minutes

Severity: IMPORTANT

 -
Execute the following query in the postgres - database: -
SELECT * FROM gp_segment_configuration 
-WHERE preferred_role <> role  and status = 'u' and mode = 's';
- -		 -

When the segments are not running in their preferred role, processing might be skewed. - Run gprecoverseg -r to bring the segments back into their preferred roles.

-
Run a distributed query to test that it runs on all segments. - One row should be returned for each primary segment. -

Recommended frequency: run every 5 to 10 - minutes

Severity: CRITICAL

 -
Execute the following query in the postgres - database:
SELECT gp_segment_id, count(*)
-FROM gp_dist_random('pg_class')
-GROUP BY 1;
-		 -

If this query fails, there is an issue dispatching to some - segments in the cluster. This is a rare event. Check the - hosts that are not able to be dispatched to ensure there is - no hardware or networking issue.

-
Test the state of coordinator mirroring on WarehousePG. If - the value is not "STREAMING", raise an alert or - warning.

Recommended frequency: run every 5 to 10 - minutes

Severity: IMPORTANT

 -
Run the following psql - command:
psql <dbname> -c 'SELECT pid, state FROM pg_stat_replication;'
-		 -

Check the log file from the coordinator - and standby coordinator for errors. If there are no unexpected - errors and the machines are up, run the -gpinitstandby utility to bring the - standby online.

-
Perform a basic check to see if the coordinator is up and - functioning.

Recommended frequency: run every 5 to 10 - minutes

Severity: CRITICAL

 -
Run the following query in the postgres - database: -
SELECT count(*) FROM gp_segment_configuration;
-		 -

If this query fails, the active coordinator may be down. Try to - start the database on the original coordinator if the server is - up and running. If that fails, try to activate the standby - coordinator as coordinator.

-
+ +**Database State Monitoring Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| List segments that are currently down. If any rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE status = 'd';` | If the query returns any rows, follow these steps to correct the problem:

1. Verify that the hosts with down segments are responsive.
2. If hosts are OK, check the log files for the primaries and mirrors of the down segments to discover the root cause of the segments going down.
3. If no unexpected errors are found, run the `gprecoverseg` utility to bring the segments back online. | +| Check for segments that are up and not in sync. If rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE mode = 'n' and status = 'u' and content <> -1;` | If the query returns rows then the segment might be in the process of moving from `Not In Sync` to `Synchronized` mode. Use `gpstate -e` to track progress. | +| Check for segments that are not operating in their preferred role but are marked as up and `Synchronized`. If any segments are found, the cluster may not be balanced. If any rows are returned this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE preferred_role <> role and status = 'u' and mode = 's';` | When the segments are not running in their preferred role, processing might be skewed. Run `gprecoverseg -r` to bring the segments back into their preferred roles. | +| Run a distributed query to test that it runs on all segments. One row should be returned for each primary segment.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Execute the following query in the `postgres` database:

`SELECT gp_segment_id, count(*) FROM gp_dist_random('pg_class') GROUP BY 1;` | If this query fails, there is an issue dispatching to some segments in the cluster. This is a rare event. Check the hosts that are not able to be dispatched to ensure there is no hardware or networking issue. | +| Test the state of coordinator mirroring on WarehousePG. If the value is not "STREAMING", raise an alert or warning.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following `psql` command:

`psql <dbname> -c 'SELECT pid, state FROM pg_stat_replication;'` | Check the log file from the coordinator and standby coordinator for errors. If there are no unexpected errors and the machines are up, run the `gpinitstandby` utility to bring the standby online. | +| Perform a basic check to see if the coordinator is up and functioning.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Run the following query in the `postgres` database:

`SELECT count(*) FROM gp_segment_configuration;` | If this query fails, the active coordinator may be down. Try to start the database on the original coordinator if the server is up and running. If that fails, try to activate the standby coordinator as coordinator. | + ## Hardware and Operating System Monitoring - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 2. Hardware and Operating System Monitoring Activities
ActivityProcedureCorrective Actions
Check disk space usage on volumes used for WarehousePG data storage and the OS. -

Recommended frequency: every 5 to 30 - minutes

Severity: CRITICAL

 -
Set up a disk space check.
    -
  • Set a threshold to raise an alert when a disk - reaches a percentage of capacity. The recommended - threshold is 75% full.
    • -
  • It is not recommended to run the system with - capacities approaching 100%.
    • -
-		Use VACUUM/VACUUM FULL on - user tables to reclaim space occupied by dead rows.
Check for errors or dropped packets on the network - interfaces.

Recommended frequency: hourly

Severity: - IMPORTANT

Set up a network interface checks. -

Work with network and OS teams to resolve errors.

-
Check for RAID errors or degraded RAID performance. -

Recommended frequency: every 5 minutes

Severity: - CRITICAL

Set up a RAID check. -
    -
  • Replace failed disks as soon as possible.
    • -
  • Work with system administration team to resolve other - RAID or controller errors as soon as possible.
    • -
-
Check for adequate I/O bandwidth and I/O skew.

Recommended - frequency: when create a cluster or when hardware issues are - suspected.

Run the WarehousePG -gpcheckperf utility. -
The cluster may be under-specified if data transfer rates are - not similar to the following:
    -
  • 2GB per second disk read
    • -
  • 1 GB per second disk write
    • -
  • 10 Gigabit per second network read and write
    • -
If transfer rates are lower than expected, consult with - your data architect regarding performance expectations.
-

If the machines on the cluster display an uneven performance - profile, work with the system administration team to fix - faulty machines.

-
+ +**Hardware and Operating System Monitoring Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| Check disk space usage on volumes used for WarehousePG data storage and the OS.

Recommended frequency: every 5 to 30 minutes

Severity: CRITICAL | Set up a disk space check.

- Set a threshold to raise an alert when a disk reaches a percentage of capacity. The recommended threshold is 75% full.
- It is not recommended to run the system with capacities approaching 100%. | Use `VACUUM`/`VACUUM FULL` on user tables to reclaim space occupied by dead rows. | +| Check for errors or dropped packets on the network interfaces.

Recommended frequency: hourly

Severity: IMPORTANT | Set up a network interface checks. | Work with network and OS teams to resolve errors. | +| Check for RAID errors or degraded RAID performance.

Recommended frequency: every 5 minutes

Severity: CRITICAL | Set up a RAID check. | - Replace failed disks as soon as possible.
- Work with system administration team to resolve other RAID or controller errors as soon as possible. | +| Check for adequate I/O bandwidth and I/O skew.

Recommended frequency: when create a cluster or when hardware issues are suspected. | Run the WarehousePG `gpcheckperf` utility. | The cluster may be under-specified if data transfer rates are not similar to the following:

- 2GB per second disk read
- 1 GB per second disk write
- 10 Gigabit per second network read and write

If transfer rates are lower than expected, consult with your data architect regarding performance expectations.

If the machines on the cluster display an uneven performance profile, work with the system administration team to fix faulty machines. | + ## Catalog Monitoring - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 3. Catalog Monitoring Activities
ActivityProcedureCorrective Actions
Run catalog consistency checks in each database to ensure the - catalog on each host in the cluster is consistent and in a good - state.

You may run this command while the database is up - and running.

Recommended frequency: - weekly

Severity: IMPORTANT

Run the WarehousePG gpcheckcat utility in each - database:
gpcheckcat -O
Note: With the --O option, gpcheckcat - runs just 10 of its usual 15 tests.
Run the repair scripts for any issues identified.
Check for pg_class entries that have no - corresponding pg_attribute entry.

Recommended - frequency: monthly

Severity: IMPORTANT

During a downtime, with no users on the system, run the - WarehousePG -gpcheckcat utility in each - database:
gpcheckcat -R pgclass
Run the repair scripts for any issues identified.
Check for leaked temporary schema and missing schema - definition.

Recommended frequency: monthly

Severity: - IMPORTANT

During a downtime, with no users on the system, run the - WarehousePG -gpcheckcat utility in each - database:
gpcheckcat -R namespace
Run the repair scripts for any issues identified.
Check constraints on randomly distributed - tables.

Recommended frequency: monthly

Severity: - IMPORTANT

During a downtime, with no users on the system, run the - WarehousePG -gpcheckcat utility in each - database:
gpcheckcat -R distribution_policy
Run the repair scripts for any issues identified.
Check for dependencies on non-existent objects.

Recommended - frequency: monthly

Severity: IMPORTANT

During a downtime, with no users on the system, run the - WarehousePG -gpcheckcat utility in each - database:
gpcheckcat -R dependency
Run the repair scripts for any issues identified.
+ +**Catalog Monitoring Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| Run catalog consistency checks in each database to ensure the catalog on each host in the cluster is consistent and in a good state.

You may run this command while the database is up and running.

Recommended frequency: weekly

Severity: IMPORTANT | Run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -O`

**Note:** With the `-O` option, `gpcheckcat` runs just 10 of its usual 15 tests. | Run the repair scripts for any issues identified. | +| Check for `pg_class` entries that have no corresponding pg_`attribute` entry.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R pgclass` | Run the repair scripts for any issues identified. | +| Check for leaked temporary schema and missing schema definition.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R namespace` | Run the repair scripts for any issues identified. | +| Check constraints on randomly distributed tables.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R distribution_policy` | Run the repair scripts for any issues identified. | +| Check for dependencies on non-existent objects.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R dependency` | Run the repair scripts for any issues identified. | + ## Data Maintenance - - - - - - - - - - - - - - - - - -
Table 4. Data Maintenance Activities
ActivityProcedureCorrective Actions
Check for missing statistics on tables. Check the gp_stats_missing view in each - database:
SELECT * FROM gp_toolkit.gp_stats_missing;
Run ANALYZE on tables that are missing - statistics.
Check for tables that have bloat (dead space) in data files - that cannot be recovered by a regular VACUUM - command.

Recommended frequency: weekly or - monthly

Severity: WARNING

Check the gp_bloat_diag view in each - database: -
SELECT * FROM gp_toolkit.gp_bloat_diag;
VACUUM FULL acquires an ACCESS - EXCLUSIVE lock on tables. Run VACUUM - FULL during a time when users and applications do - not require access to the tables, such as during a time of low - activity, or during a maintenance window.
+ +**Data Maintenance Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| Check for missing statistics on tables. | Check the `gp_stats_missing` view in each database:

`SELECT * FROM gp_toolkit.gp_stats_missing;` | Run `ANALYZE` on tables that are missing statistics. | +| Check for tables that have bloat (dead space) in data files that cannot be recovered by a regular `VACUUM` command.

Recommended frequency: weekly or monthly

Severity: WARNING | Check the `gp_bloat_diag` view in each database:

`SELECT * FROM gp_toolkit.gp_bloat_diag;` | `VACUUM FULL` acquires an `ACCESS EXCLUSIVE` lock on tables. Run `VACUUM FULL` during a time when users and applications do not require access to the tables, such as during a time of low activity, or during a maintenance window. | + ## Database Maintenance @@ -364,32 +139,11 @@ GROUP BY 1; ## Patching and Upgrading - - - - - - - - - - - - - - - - - -
Table 6. Patch and Upgrade Activities
ActivityProcedureCorrective Actions
Ensure any bug fixes or enhancements are applied to the - kernel.

Recommended frequency: at least every 6 - months

Severity: IMPORTANT

Follow the vendor's instructions to update the Linux - kernel.Keep the kernel current to include bug fixes and security - fixes, and to avoid difficult future upgrades.
Install WarehousePG minor releases, for example - 5.0.x.

Recommended frequency: - quarterly

Severity: IMPORTANT

Follow upgrade instructions in the WarehousePG -Release Notes. Always upgrade to the latest in the - series.Keep the WarehousePG software current to - incorporate bug fixes, performance enhancements, and feature - enhancements into your WarehousePG - cluster.
+ +**Patch and Upgrade Activities** + +| Activity | Procedure | Corrective Actions | +| --- | --- | --- | +| Ensure any bug fixes or enhancements are applied to the kernel.

Recommended frequency: at least every 6 months

Severity: IMPORTANT | Follow the vendor's instructions to update the Linux kernel. | Keep the kernel current to include bug fixes and security fixes, and to avoid difficult future upgrades. | +| Install WarehousePG minor releases, for example 5.0.*x*.

Recommended frequency: quarterly

Severity: IMPORTANT | Follow upgrade instructions in the WarehousePG *Release Notes*. Always upgrade to the latest in the series. | Keep the WarehousePG software current to incorporate bug fixes, performance enhancements, and feature enhancements into your WarehousePG cluster. | + diff --git a/docs/whpg/7x/admin_guide/query/functions-operators.md b/docs/whpg/7x/admin_guide/query/functions-operators.md index 0f879c3..49aff5d 100644 --- a/docs/whpg/7x/admin_guide/query/functions-operators.md +++ b/docs/whpg/7x/admin_guide/query/functions-operators.md @@ -108,488 +108,56 @@ The following table lists the categories of built-in functions and operators sup WarehousePG includes JSON processing functions that manipulate values the `json` data type. For information about JSON data, see [Working with JSON Data](json-data.md). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 3. Built-in functions and operators
Operator/Function CategoryVOLATILE FunctionsSTABLE FunctionsRestrictions
- Logical Operators -
- Comparison Operators -
- - Mathematical Functions and Operators - - random

setseed

- - String Functions and Operators - - - All built-in conversion functions - convert

pg_client_encoding

- - Binary String Functions and Operators - -
- - Bit String Functions and Operators - -
- - Pattern Matching - -
- - Data Type Formatting Functions - - to_char

to_timestamp

- Date/Time Functions and Operators - timeofdayage

current_date

current_time

current_timestamp

localtime

localtimestamp

now

- Enum Support Functions -
- - Geometric Functions and Operators - -
- - Network Address Functions and Operators - -
- - Sequence Manipulation Functions - - nextval()

setval()

- - Conditional Expressions - -
- - Array Functions and Operators - - - All array functions -
- - Aggregate Functions - -
- - Subquery Expressions - -
- - Row and Array Comparisons - -
- - Set Returning Functions - - generate_series
- - System Information Functions - - - All session information functions -

- All access privilege inquiry functions -

- All schema visibility inquiry functions -

- All system catalog information functions -

- All comment information functions -

- All transaction ids and snapshots -

- - System Administration Functions - - set_config

pg_cancel_backend

pg_terminate_backend

pg_reload_conf

pg_rotate_logfile

pg_start_backup

pg_stop_backup

pg_size_pretty

pg_ls_dir

pg_read_file

pg_stat_file

current_setting

All database object size - functions

Note: The function - pg_column_size displays bytes required to store - the value, possibly with TOAST compression.
- XML Functions and function-like - expressions -

cursor_to_xml(cursor refcursor, count int, nulls boolean, - tableforest boolean, targetns text)

-

cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest - boolean, targetns text)

-

database_to_xml(nulls boolean, tableforest boolean, targetns - text)

-

database_to_xmlschema(nulls boolean, tableforest boolean, - targetns text)

-

database_to_xml_and_xmlschema( nulls boolean, tableforest - boolean, targetns text)

-

query_to_xml(query text, nulls boolean, tableforest boolean, - targetns text)

-

query_to_xmlschema(query text, nulls boolean, tableforest - boolean, targetns text)

-

query_to_xml_and_xmlschema( query text, nulls boolean, - tableforest boolean, targetns text)

-

schema_to_xml(schema name, nulls boolean, tableforest boolean, - targetns text)

-

schema_to_xmlschema( schema name, nulls boolean, tableforest - boolean, targetns text)

-

schema_to_xml_and_xmlschema( schema name, nulls boolean, - tableforest boolean, targetns text)

-

table_to_xml(tbl regclass, nulls boolean, tableforest boolean, - targetns text)

-

table_to_xmlschema( tbl regclass, nulls boolean, tableforest - boolean, targetns text)

-

table_to_xml_and_xmlschema( tbl regclass, nulls boolean, - tableforest boolean, targetns text)

-

xmlagg(xml)

-

xmlconcat(xml[, ...])

-

xmlelement(name name [, xmlattributes(value [AS attname] [, ... - ])] [, content, ...])

-

xmlexists(text, xml)

-

xmlforest(content [AS name] [, ...])

-

xml_is_well_formed(text)

-

xml_is_well_formed_document(text)

-

xml_is_well_formed_content(text)

-

xmlparse ( { DOCUMENT | CONTENT } value)

-

xpath(text, xml)

-

xpath(text, xml, text[])

-

xpath_exists(text, xml)

-

xpath_exists(text, xml, text[])

-

xmlpi(name target [, content])

-

xmlroot(xml, version text | no value [, standalone yes|no|no - value])

-

xmlserialize ( { DOCUMENT | CONTENT } value AS type )

-

xml(text)

-

text(xml)

-

xmlcomment(xml)

-

xmlconcat2(xml, xml)

-
+**Built-in functions and operators** + +| Operator/Function Category | VOLATILE Functions | STABLE Functions | Restrictions | +| --- | --- | --- | --- | +| [Logical Operators](https://www.postgresql.org/docs/12/functions-logical.html) | | | | +| [Comparison Operators](https://www.postgresql.org/docs/12/functions-comparison.html) | | | | +| [Mathematical Functions and Operators](https://www.postgresql.org/docs/12/functions-math.html) | random

setseed | | | +| [String Functions and Operators](https://www.postgresql.org/docs/12/functions-string.html) | *All built-in conversion functions* | convert

pg_client_encoding | | +| [Binary String Functions and Operators](https://www.postgresql.org/docs/12/functions-binarystring.html) | | | | +| [Bit String Functions and Operators](https://www.postgresql.org/docs/12/functions-bitstring.html) | | | | +| [Pattern Matching](https://www.postgresql.org/docs/12/functions-matching.html) | | | | +| [Data Type Formatting Functions](https://www.postgresql.org/docs/12/functions-formatting.html) | | to_char

to_timestamp | | +| [Date/Time Functions and Operators](https://www.postgresql.org/docs/12/functions-datetime.html) | timeofday | age

current_date

current_time

current_timestamp

localtime

localtimestamp

now | | +| [Enum Support Functions](https://www.postgresql.org/docs/12/functions-enum.html) | | | | +| [Geometric Functions and Operators](https://www.postgresql.org/docs/12/functions-geometry.html) | | | | +| [Network Address Functions and Operators](https://www.postgresql.org/docs/12/functions-net.html) | | | | +| [Sequence Manipulation Functions](https://www.postgresql.org/docs/12/functions-sequence.html) | nextval()

setval() | | | +| [Conditional Expressions](https://www.postgresql.org/docs/12/functions-conditional.html) | | | | +| [Array Functions and Operators](https://www.postgresql.org/docs/12/functions-array.html) | | *All array functions* | | +| [Aggregate Functions](https://www.postgresql.org/docs/12/functions-aggregate.html) | | | | +| [Subquery Expressions](https://www.postgresql.org/docs/12/functions-subquery.html) | | | | +| [Row and Array Comparisons](https://www.postgresql.org/docs/12/functions-comparisons.html) | | | | +| [Set Returning Functions](https://www.postgresql.org/docs/12/functions-srf.html) | generate_series | | | +| [System Information Functions](https://www.postgresql.org/docs/12/functions-info.html) | | *All session information functions*

*All access privilege inquiry functions*

*All schema visibility inquiry functions*

*All system catalog information functions*

*All comment information functions*

*All transaction ids and snapshots* | | +| [System Administration Functions](https://www.postgresql.org/docs/12/functions-admin.html) | set_config

pg_cancel_backend

pg_terminate_backend

pg_reload_conf

pg_rotate_logfile

pg_start_backup

pg_stop_backup

pg_size_pretty

pg_ls_dir

pg_read_file

pg_stat_file | current_setting

*All database object size functions* | **Note:** The function `pg_column_size` displays bytes required to store the value, possibly with TOAST compression. | +| [XML Functions](https://www.postgresql.org/docs/12/functions-xml.html) and function-like expressions | | cursor_to_xml(cursor refcursor, count int, nulls boolean, tableforest boolean, targetns text)

cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest boolean, targetns text)

database_to_xml(nulls boolean, tableforest boolean, targetns text)

database_to_xmlschema(nulls boolean, tableforest boolean, targetns text)

database_to_xml_and_xmlschema( nulls boolean, tableforest boolean, targetns text)

query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xmlschema(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xml_and_xmlschema( query text, nulls boolean, tableforest boolean, targetns text)

schema_to_xml(schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xml_and_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

table_to_xml(tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xml_and_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

xmlagg(xml)

xmlconcat(xml[, ...])

xmlelement(name name [, xmlattributes(value [AS attname] [, ... ])] [, content, ...])

xmlexists(text, xml)

xmlforest(content [AS name] [, ...])

xml_is_well_formed(text)

xml_is_well_formed_document(text)

xml_is_well_formed_content(text)

xmlparse ( { DOCUMENT \| CONTENT } value)

xpath(text, xml)

xpath(text, xml, text[])

xpath_exists(text, xml)

xpath_exists(text, xml, text[])

xmlpi(name target [, content])

xmlroot(xml, version text \| no value [, standalone yes\|no\|no value])

xmlserialize ( { DOCUMENT \| CONTENT } value AS type )

xml(text)

text(xml)

xmlcomment(xml)

xmlconcat2(xml, xml) | | + + ## Window Functions The following built-in window functions are WarehousePG extensions to the PostgreSQL database. All window functions are *immutable*. For more information about window functions, see [Window Expressions](defining-queries.md). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 4. Window functions
FunctionReturn TypeFull SyntaxDescription
- cume_dist() - - double precision - - CUME_DIST() OVER ( [PARTITION BY - expr - ] ORDER BY - expr - ) - Calculates the cumulative distribution of a value - in a group of values. Rows with equal values always evaluate to the - same cumulative distribution value.
- dense_rank() - - bigint - - DENSE_RANK () OVER ( [PARTITION BY - expr - ] ORDER BY - expr - ) - Computes the rank of a row in an ordered group of - rows without skipping rank values. Rows with equal values are given - the same rank value.
- first_value(expr) - same as input expr type - FIRST_VALUE( - expr - ) OVER ( [PARTITION BY - expr - ] ORDER BY - expr - [ROWS|RANGE|GROUPS - frame_expr - ] ) - Returns the first value in an ordered set of - values.
- lag(expr [,offset] [,default]) - same as input expr type - LAG( - expr - [, - offset - ] [, - default - ]) OVER ( [PARTITION BY - expr - ] ORDER BY - expr - ) - Provides access to more than one row of the same - table without doing a self join. Given a series of rows returned - from a query and a position of the cursor, LAG - provides access to a row at a given physical offset prior to that - position. The default offset is 1. default - sets the value that is returned if the offset goes beyond the scope - of the window. If default is not specified, the default value - is null.
- last_value(expr)same as input expr type - LAST_VALUE(expr) OVER ( [PARTITION BY expr] - ORDER BY expr [ROWS|RANGE|GROUPS frame_expr] ) - Returns the last value in an ordered set of - values.
- - lead(expr [,offset] - [,default]) - - same as input expr type - LEAD(expr [,offset] - [,exprdefault]) OVER ( [PARTITION BY - expr] ORDER BY expr ) - Provides access to more than one row of the same - table without doing a self join. Given a series of rows returned - from a query and a position of the cursor, lead - provides access to a row at a given physical offset after that - position. If offset is not specified, the default offset is - 1. default sets the value that is returned if the offset goes - beyond the scope of the window. If default is not specified, - the default value is null.
- nth_value(expr) - same as input expr type - NTH_VALUE(expr) OVER ( [PARTITION BY expr] ORDER - BY expr [ROWS|RANGE|GROUPS frame_expr] ) - Returns the value evaluated at the row that is the nth row of the window frame (counting from 1); null if no such row.
ntile(expr) - bigint - NTILE(expr) OVER ( [PARTITION BY expr] ORDER - BY expr ) - Divides an ordered data set into a number of - buckets (as defined by expr) and assigns a bucket number to - each row.
percent_rank() - double precision - - PERCENT_RANK () OVER ( [PARTITION BY expr] ORDER BY - expr ) - Calculates the rank of a hypothetical row - R minus 1, divided by 1 less than the number of - rows being evaluated (within a window partition).
- rank() - bigint - RANK () OVER ( [PARTITION BY expr] ORDER BY expr - ) - Calculates the rank of a row in an ordered group - of values. Rows with equal values for the ranking criteria receive - the same rank. The number of tied rows are added to the rank number - to calculate the next rank value. Ranks may not be consecutive - numbers in this case.
row_number() - bigint - - ROW_NUMBER () OVER ( [PARTITION BY expr] ORDER BY - expr ) - Assigns a unique number to each row to which it is - applied (either each row in a window partition or each row of the - query).
+**Window functions** + +| Function | Return Type | Full Syntax | Description | +| --- | --- | --- | --- | +| `cume_dist()` | `double precision` | `CUME_DIST() OVER ( [PARTITION BY` expr `] ORDER BY` expr `)` | Calculates the cumulative distribution of a value in a group of values. Rows with equal values always evaluate to the same cumulative distribution value. | +| `dense_rank()` | `bigint` | `DENSE_RANK () OVER ( [PARTITION BY` expr `] ORDER BY` expr `)` | Computes the rank of a row in an ordered group of rows without skipping rank values. Rows with equal values are given the same rank value. | +| `first_value(*expr*)` | same as input expr type | `FIRST_VALUE(` expr `) OVER ( [PARTITION BY` expr `] ORDER BY` expr `[ROWS\|RANGE\|GROUPS` frame_expr `] )` | Returns the first value in an ordered set of values. | +| `lag(*expr* [,*offset*] [,*default*])` | same as input *expr* type | `LAG(` *expr* `[,` *offset* `] [,` *default* `]) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` | Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `LAG` provides access to a row at a given physical offset prior to that position. The default `offset` is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. | +| `last_value(*expr*`) | same as input *expr* type | `LAST_VALUE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* [ROWS\|RANGE\|GROUPS *frame_expr*] )` | Returns the last value in an ordered set of values. | +| `lead(*expr* [,*offset*] [,*default*])` | same as input *expr* type | `LEAD(*expr*[,*offset*] [,*expr**default*]) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` | Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `lead` provides access to a row at a given physical offset after that position. If *offset* is not specified, the default offset is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. | +| `nth_value(*expr*)` | same as input *expr* type | `NTH_VALUE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* [ROWS\|RANGE\|GROUPS *frame_expr*] )` | Returns the value evaluated at the row that is the *nth* row of the window frame (counting from 1); null if no such row. | +| `ntile(*expr*)` | `bigint` | `NTILE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` | Divides an ordered data set into a number of buckets (as defined by *expr*) and assigns a bucket number to each row. | +| `percent_rank()` | `double precision` | `PERCENT_RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Calculates the rank of a hypothetical row `R` minus 1, divided by 1 less than the number of rows being evaluated (within a window partition). | +| `rank()` | `bigint` | `RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Calculates the rank of a row in an ordered group of values. Rows with equal values for the ranking criteria receive the same rank. The number of tied rows are added to the rank number to calculate the next rank value. Ranks may not be consecutive numbers in this case. | +| `row_number()` | `bigint` | `ROW_NUMBER () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Assigns a unique number to each row to which it is applied (either each row in a window partition or each row of the query). | + + ## Advanced Aggregate Functions @@ -597,79 +165,12 @@ The following built-in advanced aggregate functions are WarehousePG extensions o > **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [WarehousePG MADlib Extension for Analytics](../analytics/madlib.md) in the *WarehousePG Reference Guide*. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 5. Advanced Aggregate Functions
FunctionReturn TypeFull SyntaxDescription
- MEDIAN (expr) - - timestamp, timestamptz, interval, float - - MEDIAN (expression) -

- Example: - 

SELECT departmzent_id, MEDIAN(salary) 
-  FROM employees 
-GROUP BY department_id;
- 		Can take a two-dimensional array as input. Treats - such arrays as matrices.
- sum(array[]) - - smallint[], int[], bigint[], float[] - - sum(array[[1,2],[3,4]]) -

- Example: - 

CREATE TABLE mymatrix (myvalue int[]);
-INSERT INTO mymatrix 
-   VALUES (array[[1,2],[3,4]]);
-INSERT INTO mymatrix 
-   VALUES (array[[0,1],[1,0]]);
-SELECT sum(myvalue) FROM mymatrix;
- sum 
-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
- {1,3},{4,4}
- 		Performs matrix summation. Can take as input a - two-dimensional array that is treated as a matrix.
- pivot_sum (label[], label, expr) - - int[], bigint[], float[] - - pivot_sum( array['A1','A2'], attr, value) - A pivot aggregation using sum to resolve duplicate - entries.
- unnest (array[]) - set of anyelement - unnest( array['one', 'row', 'per', 'item']) - Transforms a one dimensional array into rows. - Returns a set of anyelement, a polymorphic pseudo-type in - PostgreSQL.
+**Advanced Aggregate Functions** + +| Function | Return Type | Full Syntax | Description | +| --- | --- | --- | --- | +| `MEDIAN (*expr*)` | `timestamp, timestamptz, interval, float` | `MEDIAN (*expression*)`

*Example:*

`SELECT departmzent_id, MEDIAN(salary)
FROM employees
GROUP BY department_id;` | Can take a two-dimensional array as input. Treats such arrays as matrices. | +| `sum(array[])` | `smallint[], int[], bigint[], float[]` | `sum(array[[1,2],[3,4]])`

*Example:*

`CREATE TABLE mymatrix (myvalue int[]);
INSERT INTO mymatrix
VALUES (array[[1,2],[3,4]]);
INSERT INTO mymatrix
VALUES (array[[0,1],[1,0]]);
SELECT sum(myvalue) FROM mymatrix;
sum
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
{1,3},{4,4}` | Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. | +| `pivot_sum (label[], label, expr)` | `int[], bigint[], float[]` | `pivot_sum( array['A1','A2'], attr, value)` | A pivot aggregation using sum to resolve duplicate entries. | +| `unnest (array[])` | set of `anyelement` | `unnest( array['one', 'row', 'per', 'item'])` | Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudo-type](https://www.postgresql.org/docs/12/datatype-pseudo.html) in PostgreSQL. | + diff --git a/docs/whpg/7x/admin_guide/query/json-data.md b/docs/whpg/7x/admin_guide/query/json-data.md index 3f14eba..ced19f9 100644 --- a/docs/whpg/7x/admin_guide/query/json-data.md +++ b/docs/whpg/7x/admin_guide/query/json-data.md @@ -399,103 +399,22 @@ A path expression consists of a sequence of path elements, which can be the foll For details on using `jsonpath` expressions with SQL/JSON query functions, see [SQL/JSON Filter Expression Elements](../../ref_guide/function-summary.md#topic_jsonpath_filtexp). -
-

jsonpath Variables

-
- - - - - - - - - - - - - - - - - - - - - - - - - -
VariableDescription
$A variable representing the JSON text to be queried (the context item).
$varnameA named variable. Its value can be set by the parameter vars of several JSON processing functions. See JSON Processing Functions and its notes for details.
@A variable representing the result of path evaluation in filter expressions.
-
-
-

jsonpath Accessors

-
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Accessor OperatorDescription
-

.key

-

."$varname"

- 		-

Member accessor that returns an object member with the specified key. If the key name is a named variable starting with $ or does not meet the JavaScript rules of an identifier, it must be enclosed in double quotes as a character string literal.

-
-

.*

- 		-

Wildcard member accessor that returns the values of all members located at the top level of the current object.

-
-

.**

- 		-

Recursive wildcard member accessor that processes all levels of the JSON hierarchy of the current object and returns all the member values, regardless of their nesting level. This is a WarehousePG extension of the SQL/JSON standard.

-
-

.**{level}

-

.**{start_level to end_level}

- 		-

Same as .**, but with a filter over nesting levels of JSON hierarchy. Nesting levels are specified as integers. Zero level corresponds to the current object. To access the lowest nesting level, you can use the last keyword. This is a WarehousePG extension of the SQL/JSON standard.

-
-

[subscript, ...]

- 		-

Array element accessor. subscript can be given in two forms: index or start_index to end_index. The first form returns a single array element by its index. The second form returns an array slice by the range of indexes, including the elements that correspond to the provided start_index and end_index.

-

The specified index can be an integer, as well as an expression returning a single numeric value, which is automatically cast to integer. Zero index corresponds to the first array element. You can also use the last keyword to denote the last array element, which is useful for handling arrays of unknown length.

-
-

[*]

- 		-

Wildcard array element accessor that returns all array elements.

-
-
-
+**`jsonpath` Variables** + +| Variable | Description | +| --- | --- | +| `$` | A variable representing the JSON text to be queried (the *context item*). | +| `$varname` | A named variable. Its value can be set by the parameter *`vars`* of several JSON processing functions. See [JSON Processing Functions](../../ref_guide/function-summary.md#json-processing-functions) and its notes for details. | +| `@` | A variable representing the result of path evaluation in filter expressions. | + +**`jsonpath` Accessors** + +| Accessor Operator | Description | +| --- | --- | +| .*`key`*

."$*`varname`*" | Member accessor that returns an object member with the specified key. If the key name is a named variable starting with `$` or does not meet the JavaScript rules of an identifier, it must be enclosed in double quotes as a character string literal. | +| `.*` | Wildcard member accessor that returns the values of all members located at the top level of the current object. | +| `.**` | Recursive wildcard member accessor that processes all levels of the JSON hierarchy of the current object and returns all the member values, regardless of their nesting level. This is a WarehousePG extension of the SQL/JSON standard. | +| .**{*`level`*}

.**{*`start_level`* to *`end_level`*} | Same as `.**`, but with a filter over nesting levels of JSON hierarchy. Nesting levels are specified as integers. Zero level corresponds to the current object. To access the lowest nesting level, you can use the `last` keyword. This is a WarehousePG extension of the SQL/JSON standard. | +| [*`subscript`*, ...] | Array element accessor. *`subscript`* can be given in two forms: *`index`* or *`start_index`* to *`end_index`*. The first form returns a single array element by its index. The second form returns an array slice by the range of indexes, including the elements that correspond to the provided *`start_index`* and *`end_index`*.

The specified *`index`* can be an integer, as well as an expression returning a single numeric value, which is automatically cast to integer. Zero index corresponds to the first array element. You can also use the `last` keyword to denote the last array element, which is useful for handling arrays of unknown length. | +| `[*]` | Wildcard array element accessor that returns all array elements. | + diff --git a/docs/whpg/7x/install_guide/installation_utilities.md b/docs/whpg/7x/install_guide/installation_utilities.md index 1dfe8ac..980618d 100644 --- a/docs/whpg/7x/install_guide/installation_utilities.md +++ b/docs/whpg/7x/install_guide/installation_utilities.md @@ -9,38 +9,7 @@ For a full reference of all WarehousePG utilities, see the [WarehousePG Utility The following WarehousePG management utilities are located in `$GPHOME/bin`. - - - - -
- - - -
+| - [gpactivatestandby](../ref_guide/utility_guide/reference/gpactivatestandby.md)
- [gpaddmirrors](../ref_guide/utility_guide/reference/gpaddmirrors.md)
- [gpcheckperf](../ref_guide/utility_guide/reference/gpcheckperf.md)
- [gpdeletesystem](../ref_guide/utility_guide/reference/gpdeletesystem.md)
- [gpinitstandby](../ref_guide/utility_guide/reference/gpinitstandby.md)
- [gpinitsystem](../ref_guide/utility_guide/reference/gpinitsystem.md) | - [gpssh](../ref_guide/utility_guide/reference/gpssh.md)
- [gpssh-exkeys](../ref_guide/utility_guide/reference/gpssh-exkeys.md)
- [gpstart](../ref_guide/utility_guide/reference/gpstart.md)
- [gpstop](../ref_guide/utility_guide/reference/gpstop.md)
- [gpsync](../ref_guide/utility_guide/reference/gpsync.md) | +| --- | --- | **Parent topic:** [Installing and Upgrading WarehousePG](index.md) diff --git a/docs/whpg/7x/ref_guide/function-summary.md b/docs/whpg/7x/ref_guide/function-summary.md index 855fd6c..6268dc9 100644 --- a/docs/whpg/7x/ref_guide/function-summary.md +++ b/docs/whpg/7x/ref_guide/function-summary.md @@ -187,409 +187,32 @@ This table shows the functions that are available for processing `json` and `jso Many of these processing functions and operators convert Unicode escapes in JSON strings to the appropriate single character. This is a not an issue if the input data type is `jsonb`, because the conversion was already done. However, for `json` data type input, this might result in an error being thrown as described in [About JSON Data](../admin_guide/query/json-data.md#topic_upc_tcs_fz). -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 8. JSON Processing Functions
FunctionReturn TypeDescriptionExampleExample Result
-json_array_length(json) -

-jsonb_array_length(jsonb) -

-		int -Returns the number of elements in the outermost JSON array.json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]') - -5 -
json_each(json) -

jsonb_each(jsonb) -

-		setof key text, value json -

setof key text, value jsonb -

-		Expands the outermost JSON object into a set of key/value pairs.select * from json_each('{"a":"foo", "b":"bar"}') - -
 key | value
------+-------
- a   | "foo"
- b   | "bar"
-
-
json_each_text(json) -

jsonb_each_text(jsonb) -

-		setof key text, value text -Expands the outermost JSON object into a set of key/value pairs. The returned - values will be of type text.select * from json_each_text('{"a":"foo", "b":"bar"}') - -
 key | value
------+-------
- a   | foo
- b   | bar
-
-
json_extract_path(from_json json, VARIADIC path_elems - text[]) -

jsonb_extract_path(from_json jsonb, VARIADIC path_elems - text[]) -

-		 -

json -

-

jsonb -

-		Returns the JSON value pointed to by path_elems (equivalent - to #> operator).json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}','f4') - -{"f5":99,"f6":"foo"} -
json_extract_path_text(from_json json, VARIADIC path_elems - text[]) -

jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems - text[]) -

-		text -Returns the JSON value pointed to by path_elems as text - (equivalent to #>> operator).json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}','f4', - 'f6') - -foo -
json_object_keys(json) -

jsonb_object_keys(jsonb) -

-		setof text -Returns set of keys in the outermost JSON object.json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}') - -
 json_object_keys
-------------------
- f1
- f2
-
-
json_populate_record(base anyelement, from_json - json)

jsonb_populate_record(base anyelement, from_json - jsonb) -

-		anyelement -Expands the object in from_json to a row whose columns match - the record type defined by base. See the Note 1.select * from json_populate_record(null::myrowtype, '{"a": 1, - "b": ["2", "a b"], "c": {"d": 4, "e": "a b c"}') - -
 a |   b       |      c
----+-----------+-------------
- 1 | {2,"a b"} | (4,"a b c")
-
-
json_populate_recordset(base anyelement, from_json json) -

jsonb_populate_recordset(base anyelement, from_json jsonb) -

-		setof anyelement -Expands the outermost array of objects in from_json to a set - of rows whose columns match the record type defined by base. See the Note 1.select * from json_populate_recordset(null::myrowtype, - '[{"a":1,"b":2},{"a":3,"b":4}]') - -
 a | b
----+---
- 1 | 2
- 3 | 4
-
-
json_array_elements(json) -

jsonb_array_elements(jsonb)

-		 -

setof json -

-

setof jsonb -

-		Expands a JSON array to a set of JSON values.select * from json_array_elements('[1,true, [2,false]]') - -
   value
------------
- 1
- true
- [2,false]
-
-
json_array_elements_text(json) -

jsonb_array_elements_text(jsonb) -

-		setof text -Expands a JSON array to a set of text values.select * from json_array_elements_text('["foo", "bar"]') - -
   value
------------
- foo
- bar
-
-
json_typeof(json)

jsonb_typeof(jsonb) -

-		text -Returns the type of the outermost JSON value as a text string. Possible types - are object, array, string, -number, boolean, and null. - json_typeof('-123.4') - -number -
json_to_record(json)

jsonb_to_record(jsonb) -

-		record -Builds an arbitrary record from a JSON object. See the Note 1.

As with all - functions returning record, the caller must explicitly define the structure of - the record with an AS clause.

-		select * from json_to_record('{"a":1,"b":[1,2,3], - "c":[1,2,3],"e":"bar","r": {"a": 123, "b": "a b c"}') - as x(a int, b text, c int[], d text, r myrowtype) - -
 a |    b    |    c    | d |       r
----+---------+---------+---+---------------
- 1 | [1,2,3] | {1,2,3} |   | (123,"a b c")
-
-
json_to_recordset(json) -

jsonb_to_recordset(jsonb) -

-		setof record -Builds an arbitrary set of records from a JSON array of objects See the Note 1.

As with all - functions returning record, the caller must explicitly define the structure of - the record with an AS clause.

-		select * from - json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b - text); - -
 a |  b
----+-----
- 1 | foo
- 2 |
-
-
-

json_strip_nulls(from_json json)

-

jsonb_strip_nulls(from_json jsonb)

- 		-

json

-

jsonb

- 		Returns from_json with all object fields that have null values omitted. Other null values are untouched.json_strip_nulls('[{"f1":1,"f2":null},2,null,3]')[{"f1":1},2,null,3]
-

jsonb_set(target jsonb, path text[], new_value jsonb [, create_missing boolean])

- 		-

jsonb

- 		Returns target with the section designated by path replaced by new_value, or with new_value added if create_missing is true (default is true) and the item designated by path does not exist. As with the path oriented operators, negative integers that appear in path count from the end of JSON arrays. -

jsonb_set('[{"f1":1,"f2":null},2,null,3]', '{0,f1}','[2,3,4]', false)

-

jsonb_set('[{"f1":1,"f2":null},2]', '{0,f3}','[2,3,4]')

- 		-

[{"f1":[2,3,4],"f2":null},2,null,3]

-

[{"f1": 1, "f2": null, "f3": [2, 3, 4]}, 2]

-
-

jsonb_insert(target jsonb, path text[], new_value jsonb [, insert_after boolean])

- 		-

jsonb

- 		Returns target with new_value inserted. If target section designated by path is in a JSONB array, new_value will be inserted before target or after if insert_after is true (default is false). If target section designated by path is in JSONB object, new_value will be inserted only if target does not exist. As with the path oriented operators, negative integers that appear in path count from the end of JSON arrays. -

jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"')

-

jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"', true)

- 		-

{"a": [0, "new_value", 1, 2]}

-

{"a": [0, 1, "new_value", 2]}

-
-

jsonb_pretty(from_json jsonb)

- 		-

text

- 		Returns from_json as indented JSON text.jsonb_pretty('[{"f1":1,"f2":null},2,null,3]') - 
[
-    {
-        "f1": 1,
-        "f2": null
-    },
-    2,
-    null,
-    3
-]
-
-
-

jsonb_path_exists(target jsonb, path jsonpath [, vars jsonb [, silent bool]])

- 		booleanChecks whether JSON path returns any item for the specified JSON value. -

jsonb_path_exists('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')

- 		-

true

-
-

jsonb_path_match(target jsonb, path jsonpath [, vars jsonb [, silent bool]])

- 		booleanReturns the result of JSON path predicate check for the specified JSON value. Only the first item of the result is taken into account. If the result is not Boolean, then null is returned. -

jsonb_path_match('{"a":[1,2,3,4,5]}', 'exists($.a[*] ? (@ >= $min && @ <= $max))', '{"min":2,"max":4}')

- 		-

true

-
-

jsonb_path_query(target jsonb, path jsonpath [, vars jsonb [, silent bool]])

- 		setof jsonbGets all JSON items returned by JSON path for the specified JSON value. -

select * from jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}');

- 		- 
 jsonb_path_query
-------------------
- 2
- 3
- 4
-
-
-

jsonb_path_query_array(target jsonb, path jsonpath [, vars jsonb [, silent bool]])

- 		jsonbGets all JSON items returned by JSON path for the specified JSON value and wraps result into an array. -

jsonb_path_query_array('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')

- 		-

[2, 3, 4]

-
-

jsonb_path_query_first(target jsonb, path jsonpath [, vars jsonb [, silent bool]])

- 		jsonbGets the first JSON item returned by JSON path for the specified JSON value. Returns NULL on no results. -

jsonb_path_query_first('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')

- 		-

2

-
-
+**JSON Processing Functions** + +| Function | Return Type | Description | Example | Example Result | +| --- | --- | --- | --- | --- | +| `json_array_length(json)`

`jsonb_array_length(jsonb)` | `int` | Returns the number of elements in the outermost JSON array. | `json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]')` | `5` | +| `json_each(json)`

`jsonb_each(jsonb)` | `setof key text, value json`

`setof key text, value jsonb` | Expands the outermost JSON object into a set of key/value pairs. | `select * from json_each('{"a":"foo", "b":"bar"}')` | `key \| value`
`-----+-------`
`a \| "foo"`
`b \| "bar"` | +| `json_each_text(json)`

`jsonb_each_text(jsonb)` | `setof key text, value text` | Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. | `select * from json_each_text('{"a":"foo", "b":"bar"}')` | `key \| value`
`-----+-------`
`a \| foo`
`b \| bar` | +| `json_extract_path(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` | `json`

`jsonb` | Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). | `json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}','f4')` | `{"f5":99,"f6":"foo"}` | +| `json_extract_path_text(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` | `text` | Returns the JSON value pointed to by `path_elems` as text (equivalent to `#>>` operator). | `json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}','f4', 'f6')` | `foo` | +| `json_object_keys(json)`

`jsonb_object_keys(jsonb)` | `setof text` | Returns set of keys in the outermost JSON object. | `json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}')` | `json_object_keys`
`------------------`
`f1`
`f2` | +| `json_populate_record(base anyelement, from_json json)`

`jsonb_populate_record(base anyelement, from_json jsonb)` | `anyelement` | Expands the object in `from_json` to a row whose columns match the record type defined by `base`. See the [Note 1](#notes-on-json-processing-functions-examples). | `select * from json_populate_record(null::myrowtype, '{"a": 1, "b": ["2", "a b"], "c": {"d": 4, "e": "a b c"}}')` | `a \| b \| c`
`---+-----------+-------------`
`1 \| {2,"a b"} \| (4,"a b c")` | +| `json_populate_recordset(base anyelement, from_json json)`

`jsonb_populate_recordset(base anyelement, from_json jsonb)` | `setof anyelement` | Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by `base`. See the [Note 1](#notes-on-json-processing-functions-examples). | `select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]')` | `a \| b`
`---+---`
`1 \| 2`
`3 \| 4` | +| `json_array_elements(json)`

`jsonb_array_elements(jsonb)` | `setof json`

`setof jsonb` | Expands a JSON array to a set of JSON values. | `select * from json_array_elements('[1,true, [2,false]]')` | `value`
`-----------`
`1`
`true`
`[2,false]` | +| `json_array_elements_text(json)`

`jsonb_array_elements_text(jsonb)` | `setof text` | Expands a JSON array to a set of `text` values. | `select * from json_array_elements_text('["foo", "bar"]')` | `value`
`-----------`
`foo`
`bar` | +| `json_typeof(json)`

`jsonb_typeof(jsonb)` | `text` | Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. | `json_typeof('-123.4')` | `number` | +| `json_to_record(json)`

`jsonb_to_record(jsonb)` | `record` | Builds an arbitrary record from a JSON object. See the [Note 1](#notes-on-json-processing-functions-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_record('{"a":1,"b":[1,2,3], "c":[1,2,3],"e":"bar","r": {"a": 123, "b": "a b c"}') as x(a int, b text, c int[], d text, r myrowtype)` | `a \| b \| c \| d \| r`
`---+---------+---------+---+---------------`
`1 \| [1,2,3] \| {1,2,3} \| \| (123,"a b c")` | +| `json_to_recordset(json)`

`jsonb_to_recordset(jsonb)` | `setof record` | Builds an arbitrary set of records from a JSON array of objects See the [Note 1](#notes-on-json-processing-functions-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text);` | `a \| b`
`---+-----`
`1 \| foo`
`2 \|` | +| `json_strip_nulls(from_json json)`

`jsonb_strip_nulls(from_json jsonb)` | `json`

`jsonb` | Returns *`from_json`* with all object fields that have null values omitted. Other null values are untouched. | `json_strip_nulls('[{"f1":1,"f2":null},2,null,3]')` | `[{"f1":1},2,null,3]` | +| `jsonb_set(target jsonb, path text[], new_value jsonb [, create_missing boolean])` | `jsonb` | Returns *`target`* with the section designated by *`path`* replaced by *`new_value`*, or with *`new_value`* added if *`create_missing`* is true (default is `true`) and the item designated by *`path`* does not exist. As with the path oriented operators, negative integers that appear in *`path`* count from the end of JSON arrays. | `jsonb_set('[{"f1":1,"f2":null},2,null,3]', '{0,f1}','[2,3,4]', false)`

`jsonb_set('[{"f1":1,"f2":null},2]', '{0,f3}','[2,3,4]')` | `[{"f1":[2,3,4],"f2":null},2,null,3]`

`[{"f1": 1, "f2": null, "f3": [2, 3, 4]}, 2]` | +| `jsonb_insert(target jsonb, path text[], new_value jsonb [, insert_after boolean])` | `jsonb` | Returns *`target`* with *`new_value`* inserted. If *`target`* section designated by *`path`* is in a JSONB array, *`new_value`* will be inserted before target or after if *`insert_after`* is true (default is `false`). If *`target`* section designated by *`path`* is in JSONB object, *`new_value`* will be inserted only if *`target`* does not exist. As with the path oriented operators, negative integers that appear in *`path`* count from the end of JSON arrays. | `jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"')`

`jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"', true)` | `{"a": [0, "new_value", 1, 2]}`

`{"a": [0, 1, "new_value", 2]}` | +| `jsonb_pretty(from_json jsonb)` | `text` | Returns *`from_json`* as indented JSON text. | `jsonb_pretty('[{"f1":1,"f2":null},2,null,3]')` | `[`
` {`
` "f1": 1,`
` "f2": null`
` },`
` 2,`
` null,`
` 3`
`]` | +| `jsonb_path_exists(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `boolean` | Checks whether JSON path returns any item for the specified JSON value. | `jsonb_path_exists('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')` | `true` | +| `jsonb_path_match(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `boolean` | Returns the result of JSON path predicate check for the specified JSON value. Only the first item of the result is taken into account. If the result is not Boolean, then `null` is returned. | `jsonb_path_match('{"a":[1,2,3,4,5]}', 'exists($.a[*] ? (@ >= $min && @ <= $max))', '{"min":2,"max":4}')` | `true` | +| `jsonb_path_query(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `setof jsonb` | Gets all JSON items returned by JSON path for the specified JSON value. | `select * from jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}');` | `jsonb_path_query`
`------------------`
`2`
`3`
`4` | +| `jsonb_path_query_array(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `jsonb` | Gets all JSON items returned by JSON path for the specified JSON value and wraps result into an array. | `jsonb_path_query_array('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')` | `[2, 3, 4]` | +| `jsonb_path_query_first(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `jsonb` | Gets the first JSON item returned by JSON path for the specified JSON value. Returns `NULL` on no results. | `jsonb_path_query_first('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')` | `2` | ##### Notes on JSON processing functions examples @@ -801,273 +424,49 @@ $.* ? (@ like_regex "^\\d+$") The following table describes the operators and methods available in `jsonpath`: -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Operator/MethodDescriptionExample JSONExample QueryResult
+ (unary)Plus operator that iterates over the SQL/JSON sequence{"x": [2.85, -14.7, -9.4]}+ $.x.floor()2, -15, -10
- (unary)Minus operator that iterates over the SQL/JSON sequence{"x": [2.85, -14.7, -9.4]}- $.x.floor()-2, 15, 10
+ (binary)Addition[2]2 + $[0]4
- (binary)Subtraction[2]4 - $[0]2
*Multiplication[4]2 * $[0]8
/Division[8]$[0] / 24
%Modulus[32]$[0] % 102
type()Type of the SQL/JSON item[1, "2", {}]$[*].type()"number", "string", "object"
size()Size of the SQL/JSON item{"m": [11, 15]}$.m.size()2
double()Approximate floating-point number converted from an SQL/JSON number or a string{"len": "1.9"}$.len.double() * 23.8
ceiling()Nearest integer greater than or equal to the SQL/JSON number{"h": 1.3}$.h.ceiling()2
floor()Nearest integer less than or equal to the SQL/JSON number{"h": 1.3}$.h.floor()1
abs()Absolute value of the SQL/JSON number{"z": -0.3}$.z.abs()0.3
keyvalue()Sequence of object's key-value pairs represented as array of items containing three fields ("key", "value", and "id"). "id" is a unique identifier of the object key-value pair belongs to.{"x": "20", "y": 32}$.keyvalue(){"key": "x", "value": "20", "id": 0}, {"key": "y", "value": 32, "id": 0}
-
+| Operator/Method | Description | Example JSON | Example Query | Result | +| --- | --- | --- | --- | --- | +| `+` (unary) | Plus operator that iterates over the SQL/JSON sequence | `{"x": [2.85, -14.7, -9.4]}` | `+ $.x.floor()` | `2, -15, -10` | +| `-` (unary) | Minus operator that iterates over the SQL/JSON sequence | `{"x": [2.85, -14.7, -9.4]}` | `- $.x.floor()` | `-2, 15, 10` | +| `+` (binary) | Addition | `[2]` | `2 + $[0]` | `4` | +| `-` (binary) | Subtraction | `[2]` | `4 - $[0]` | `2` | +| `*` | Multiplication | `[4]` | `2 * $[0]` | `8` | +| `/` | Division | `[8]` | `$[0] / 2` | `4` | +| `%` | Modulus | `[32]` | `$[0] % 10` | `2` | +| `type()` | Type of the SQL/JSON item | `[1, "2", {}]` | `$[*].type()` | `"number", "string", "object"` | +| `size()` | Size of the SQL/JSON item | `{"m": [11, 15]}` | `$.m.size()` | `2` | +| `double()` | Approximate floating-point number converted from an SQL/JSON number or a string | `{"len": "1.9"}` | `$.len.double() * 2` | `3.8` | +| `ceiling()` | Nearest integer greater than or equal to the SQL/JSON number | `{"h": 1.3}` | `$.h.ceiling()` | `2` | +| `floor()` | Nearest integer less than or equal to the SQL/JSON number | `{"h": 1.3}` | `$.h.floor()` | `1` | +| `abs()` | Absolute value of the SQL/JSON number | `{"z": -0.3}` | `$.z.abs()` | `0.3` | +| `keyvalue()` | Sequence of object's key-value pairs represented as array of items containing three fields (`"key"`, `"value"`, and `"id"`). `"id"` is a unique identifier of the object key-value pair belongs to. | `{"x": "20", "y": 32}` | `$.keyvalue()` | `{"key": "x", "value": "20", "id": 0}, {"key": "y", "value": 32, "id": 0}` | + + #### SQL/JSON Filter Expression Elements The following table describes the available filter expressions elements for `jsonpath`: -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Value/PredicateDescriptionExample JSONExample QueryResult
==Equality operator[1, 2, 1, 3]$[*] ? (@ == 1)1, 1
!=Non-equality operator[1, 2, 1, 3]$[*] ? (@ != 1)2, 3
<>Non-equality operator (same as !=)[1, 2, 1, 3]$[*] ? (@ <> 1)2, 3
<Less-than operator[1, 2, 3]$[*] ? (@ < 2)1
<=Less-than-or-equal-to operator[1, 2, 3]$[*] ? (@ <= 2)1, 2
>Greater-than operator[1, 2, 3]$[*] ? (@ > 2)3
>=Greater-than-or-equal-to operator[1, 2, 3]$[*] ? (@ >= 2)2, 3
trueValue used to perform comparison with JSON true literal[{"name": "John", "parent": false}, {"name": "Chris", "parent": true}]$[*] ? (@.parent == true){"name": "Chris", "parent": true}
falseValue used to perform comparison with JSON false literal[{"name": "John", "parent": false}, {"name": "Chris", "parent": true}]$[*] ? (@.parent == false){"name": "John", "parent": false}
nullValue used to perform comparison with JSON null value[{"name": "Mary", "job": null}, {"name": "Michael", "job": "driver"}]$[*] ? (@.job == null) .name"Mary"
&&Boolean AND[1, 3, 7]$[*] ? (@ > 1 && @ < 5)3
||Boolean OR[1, 3, 7]$[*] ? (@ < 1 || @ > 5)7
!Boolean NOT[1, 3, 7]$[*] ? (!(@ < 5))7
like_regexTests whether the first operand matches the regular expression given by the second operand, optionally with modifications described by a string of flag characters.["abc", "abd", "aBdC", "abdacb", "babc"]$[*] ? (@ like_regex "^ab.*c" flag "i")"abc", "aBdC", "abdacb"
starts withTests whether the second operand is an initial substring of the first operand["John Smith", "Mary Stone", "Bob Johnson"]$[*] ? (@ starts with "John")"John Smith"
existsTests whether a path expression matches at least one SQL/JSON item{"x": [1, 2], "y": [2, 4]}strict $.* ? (exists (@ ? (@[*] > 2)))2, 4
is unknownTests whether a Boolean condition is unknown[-1, 2, 7, "infinity"]$[*] ? ((@ > 0) is unknown)"infinity"
-
- +| Value/Predicate | Description | Example JSON | Example Query | Result | +| --- | --- | --- | --- | --- | +| `==` | Equality operator | `[1, 2, 1, 3]` | `$[*] ? (@ == 1)` | `1, 1` | +| `!=` | Non-equality operator | `[1, 2, 1, 3]` | `$[*] ? (@ != 1)` | `2, 3` | +| `<>` | Non-equality operator (same as `!=`) | `[1, 2, 1, 3]` | `$[*] ? (@ <> 1)` | `2, 3` | +| `<` | Less-than operator | `[1, 2, 3]` | `$[*] ? (@ < 2)` | `1` | +| `<=` | Less-than-or-equal-to operator | `[1, 2, 3]` | `$[*] ? (@ <= 2)` | `1, 2` | +| `>` | Greater-than operator | `[1, 2, 3]` | `$[*] ? (@ > 2)` | `3` | +| `>=` | Greater-than-or-equal-to operator | `[1, 2, 3]` | `$[*] ? (@ >= 2)` | `2, 3` | +| `true` | Value used to perform comparison with JSON `true` literal | `[{"name": "John", "parent": false}, {"name": "Chris", "parent": true}]` | `$[*] ? (@.parent == true)` | `{"name": "Chris", "parent": true}` | +| `false` | Value used to perform comparison with JSON `false` literal | `[{"name": "John", "parent": false}, {"name": "Chris", "parent": true}]` | `$[*] ? (@.parent == false)` | `{"name": "John", "parent": false}` | +| `null` | Value used to perform comparison with JSON `null` value | `[{"name": "Mary", "job": null}, {"name": "Michael", "job": "driver"}]` | `$[*] ? (@.job == null) .name` | `"Mary"` | +| `&&` | Boolean AND | `[1, 3, 7]` | `$[*] ? (@ > 1 && @ < 5)` | `3` | +| `\|\|` | Boolean OR | `[1, 3, 7]` | `$[*] ? (@ < 1 \|\| @ > 5)` | `7` | +| `!` | Boolean NOT | `[1, 3, 7]` | `$[*] ? (!(@ < 5))` | `7` | +| `like_regex` | Tests whether the first operand matches the regular expression given by the second operand, optionally with modifications described by a string of `flag` characters. | `["abc", "abd", "aBdC", "abdacb", "babc"]` | `$[*] ? (@ like_regex "^ab.*c" flag "i")` | `"abc", "aBdC", "abdacb"` | +| `starts with` | Tests whether the second operand is an initial substring of the first operand | `["John Smith", "Mary Stone", "Bob Johnson"]` | `$[*] ? (@ starts with "John")` | `"John Smith"` | +| `exists` | Tests whether a path expression matches at least one SQL/JSON item | `{"x": [1, 2], "y": [2, 4]}` | `strict $.* ? (exists (@ ? (@[*] > 2)))` | `2, 4` | +| `is unknown` | Tests whether a Boolean condition is `unknown` | `[-1, 2, 7, "infinity"]` | `$[*] ? ((@ > 0) is unknown)` | `"infinity"` | + ## Window Functions @@ -1095,169 +494,20 @@ The following built-in advanced analytic functions are WarehousePG extensions of > **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [MADlib Extension for Analytics](../admin_guide/analytics/madlib.md). -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Table 10. Advanced Aggregate Functions
FunctionReturn TypeFull SyntaxDescription
-gp_array_agg (anyarray) - -same as the argument data type - -gp_array_agg (anyarray) -

-Example: -

-
CREATE TABLE intarr_tbl (a int, arr int[]);
-INSERT INTO intarr_tbl SELECT i, array[i, i] FROM generate_series(1, 5)i;
-INSERT INTO intarr_tbl SELECT 6, '{6, NULL}'::int[];
-INSERT INTO intarr_tbl SELECT 8, '{NULL, 7}'::int[];
-SELECT gp_array_agg(arr ORDER BY arr) FROM intarr_tbl;
-		A parallel version of array_agg(anyarray). Concatenates input arrays to create an array of one higher dimension. The inputs must all have the same dimensions, and they cannot be empty or null.
-gp_array_agg (anynonarray) - -array of the argument type - -gp_array_agg (anynonarray) -

-Example: -

-
CREATE TABLE table1(a int4, b int4);
-INSERT INTO table1 VALUES (4,5), (2,1), (1,3), (3,null), (3,7);
-SELECT gp_array_agg(a ORDER BY b NULLS FIRST) FROM table1;
-		An parallel version of array_agg(anynonarray). Creates an array by concatenating input values, including nulls.
-MEDIAN (expr) - -timestamp, timestamptz, interval, float - -MEDIAN (expression) -

-Example: -

-
SELECT department_id, MEDIAN(salary) 
-FROM employees 
-GROUP BY department_id;
-		Can take a two-dimensional array as input. Treats such arrays as - matrices.
-PERCENTILE_CONT (expr) WITHIN GROUP (ORDER BY expr - [DESC/ASC]) - -timestamp, timestamptz, interval, float - -PERCENTILE_CONT(percentage) WITHIN GROUP (ORDER BY -expression) -

-Example: -

-
SELECT department_id,
-PERCENTILE_CONT (0.5) WITHIN GROUP (ORDER BY salary DESC)
-"Median_cont"; 
-FROM employees GROUP BY department_id;
-		Performs an inverse distribution function that assumes a - continuous distribution model. It takes a percentile value and a sort specification - and returns the same datatype as the numeric datatype of the argument. This returned - value is a computed result after performing linear interpolation. Null are ignored - in this calculation.
PERCENTILE_DISC (expr) WITHIN GROUP (ORDER BY -expr [DESC/ASC]) -timestamp, timestamptz, interval, float - -PERCENTILE_DISC(percentage) WITHIN GROUP (ORDER BY -expression) -

-Example: -

-
SELECT department_id, 
-PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY salary DESC)
-"Median_desc"; 
-FROM employees GROUP BY department_id;
-		Performs an inverse distribution function that assumes a - discrete distribution model. It takes a percentile value and a sort specification. - This returned value is an element from the set. Null are ignored in this - calculation.
-sum(array[]) - -smallint[]int[], bigint[], float[] - -sum(array[[1,2],[3,4]]) -

-Example: -

-
CREATE TABLE mymatrix (myvalue int[]);
-INSERT INTO mymatrix VALUES (array[[1,2],[3,4]]);
-INSERT INTO mymatrix VALUES (array[[0,1],[1,0]]);
-SELECT sum(myvalue) FROM mymatrix;
- sum 
----------------
- {1,3},{4,4}
-		Performs matrix summation. Can take as input a two-dimensional - array that is treated as a matrix.
-pivot_sum (label[], label, expr) - -int[], bigint[], float[] - -pivot_sum( array['A1','A2'], attr, value) -A pivot aggregation using sum to resolve duplicate - entries.
-unnest (array[]) -set of anyelement -unnest( array['one', 'row', 'per', 'item']) -Transforms a one dimensional array into rows. Returns a set of -anyelement, a polymorphic pseudotype in PostgreSQL.
-
+**Advanced Aggregate Functions** + +| Function | Return Type | Full Syntax | Description | +| --- | --- | --- | --- | +| `gp_array_agg (`*anyarray*`)` | same as the argument data type | `gp_array_agg (`*anyarray*`)`

*Example:*

`CREATE TABLE intarr_tbl (a int, arr int[]);`
`INSERT INTO intarr_tbl SELECT i, array[i, i] FROM generate_series(1, 5)i;`
`INSERT INTO intarr_tbl SELECT 6, '{6, NULL}'::int[];`
`INSERT INTO intarr_tbl SELECT 8, '{NULL, 7}'::int[];`
`SELECT gp_array_agg(arr ORDER BY arr) FROM intarr_tbl;` | A parallel version of `array_agg(anyarray)`. Concatenates input arrays to create an array of one higher dimension. The inputs must all have the same dimensions, and they cannot be empty or null. | +| `gp_array_agg (`*anynonarray*`)` | array of the argument type | `gp_array_agg (`*anynonarray*`)`

*Example:*

`CREATE TABLE table1(a int4, b int4);`
`INSERT INTO table1 VALUES (4,5), (2,1), (1,3), (3,null), (3,7);`
`SELECT gp_array_agg(a ORDER BY b NULLS FIRST) FROM table1;` | An parallel version of `array_agg(anynonarray)`. Creates an array by concatenating input values, including nulls. | +| `MEDIAN (`*expr*`)` | `timestamp, timestamptz, interval, float` | `MEDIAN (`*expression*`)`

*Example:*

`SELECT department_id, MEDIAN(salary) `
`FROM employees `
`GROUP BY department_id;` | Can take a two-dimensional array as input. Treats such arrays as matrices. | +| `PERCENTILE_CONT (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` | `timestamp, timestamptz, interval, float` | `PERCENTILE_CONT(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)`

*Example:*

`SELECT department_id,`
`PERCENTILE_CONT (0.5) WITHIN GROUP (ORDER BY salary DESC)`
`"Median_cont"; `
`FROM employees GROUP BY department_id;` | Performs an inverse distribution function that assumes a continuous distribution model. It takes a percentile value and a sort specification and returns the same datatype as the numeric datatype of the argument. This returned value is a computed result after performing linear interpolation. Null are ignored in this calculation. | +| `PERCENTILE_DISC (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` | `timestamp, timestamptz, interval, float` | `PERCENTILE_DISC(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)`

*Example:*

`SELECT department_id, `
`PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY salary DESC)`
`"Median_desc"; `
`FROM employees GROUP BY department_id;` | Performs an inverse distribution function that assumes a discrete distribution model. It takes a percentile value and a sort specification. This returned value is an element from the set. Null are ignored in this calculation. | +| `sum(array[])` | `smallint[]int[], bigint[], float[]` | `sum(array[[1,2],[3,4]])`

*Example:*

`CREATE TABLE mymatrix (myvalue int[]);`
`INSERT INTO mymatrix VALUES (array[[1,2],[3,4]]);`
`INSERT INTO mymatrix VALUES (array[[0,1],[1,0]]);`
`SELECT sum(myvalue) FROM mymatrix;`
`sum `
`---------------`
`{1,3},{4,4}` | Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. | +| `pivot_sum (label[], label, expr)` | `int[], bigint[], float[]` | `pivot_sum( array['A1','A2'], attr, value)` | A pivot aggregation using sum to resolve duplicate entries. | +| `unnest (array[])` | set of `anyelement` | `unnest( array['one', 'row', 'per', 'item'])` | Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudotype in PostgreSQL](https://www.postgresql.org/docs/12/datatype-pseudo.html). | + + ## Text Search Functions and Operators diff --git a/docs/whpg/7x/ref_guide/misc/feature_summary.md b/docs/whpg/7x/ref_guide/misc/feature_summary.md index 30de39f..90f1bc9 100644 --- a/docs/whpg/7x/ref_guide/misc/feature_summary.md +++ b/docs/whpg/7x/ref_guide/misc/feature_summary.md @@ -137,850 +137,147 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty ## WarehousePG and PostgreSQL Compatibility SQL Support - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
SQL CommandSupported in WarehousePGModifications, Limitations, Exceptions
ALTER AGGREGATEYES
ALTER CONVERSIONYES
ALTER DATABASEYES
ALTER DOMAINYES
ALTER EVENT TRIGGERYES
ALTER EXTENSIONYESChanges the definition of a WarehousePG extension - based on PostgreSQL 9.6.
ALTER FUNCTIONYES
ALTER GROUPYESAn alias for ALTER ROLE
ALTER INDEXYES
ALTER LANGUAGEYES
ALTER OPERATORYES
ALTER OPERATOR CLASSYES
ALTER OPERATOR FAMILYYES
ALTER PROTOCOLYES
ALTER PUBLICATIONNO
ALTER RESOURCE QUEUEYESWarehousePG resource management feature - not in PostgreSQL.
ALTER ROLEYESWarehousePG Clauses:

RESOURCE QUEUE - queue_name | none

-
ALTER SCHEMAYES
ALTER SEQUENCEYES
ALTER SUBSCRIPTIONNO
ALTER SYSTEMNO
ALTER TABLEYESUnsupported Clauses / Options: -

ENABLE/DISABLE TRIGGER

-

WarehousePG - Database Clauses:

-

ADD | DROP | RENAME | SPLIT | EXCHANGE - PARTITION | SET SUBPARTITION TEMPLATE | SET WITH - (REORGANIZE=true | false) | SET DISTRIBUTED - BY

-
ALTER TABLESPACEYES
ALTER TRIGGERNO
ALTER TYPEYESWarehousePG Clauses:

SET DEFAULT ENCODING

-
ALTER USERYESAn alias for ALTER ROLE
ALTER VIEWYES
ANALYZEYES
BEGINYES
CHECKPOINTYES
CLOSEYES
CLUSTERYES
COMMENTYES
COMMITYES
COMMIT PREPAREDNO
COPYYESModified Clauses:

ESCAPE [ AS ] - 'escape' | 'OFF'

-

WarehousePG - Clauses:

-

[LOG ERRORS] SEGMENT REJECT LIMIT - count [ROWS|PERCENT]

-
CREATE AGGREGATEYESUnsupported Clauses / Options:

[ , SORTOP = - sort_operator ]

-

WarehousePG - Clauses:

-

[ , COMBINEFUNC = combinefunc - ]

-

Limitations:

-

The functions used to implement the - aggregate must be IMMUTABLE functions.

-
CREATE CASTYES
CREATE CONSTRAINT TRIGGERNO
CREATE CONVERSIONYES
CREATE DATABASEYES
CREATE DOMAINYES
CREATE EVENT TRIGGERYES
CREATE EXTENSIONYESLoads a new extension into WarehousePG - based on - PostgreSQL 9.6.
CREATE EXTERNAL TABLEYESWarehousePG parallel ETL feature - not in PostgreSQL - 9.4.
CREATE FUNCTIONYESLimitations:

Functions defined as - STABLE or VOLATILE can be run in - WarehousePG provided that they are run on the coordinator only. - STABLE and VOLATILE functions cannot be used - in statements that run at the segment level.

-
CREATE GROUPYESAn alias for CREATE ROLE
CREATE INDEXYESWarehousePG Clauses:

USING - bitmap (bitmap - indexes)

-

Limitations:

-

UNIQUE indexes are - allowed only if they contain all of (or a superset of) the WarehousePG distribution - key columns. On partitioned tables, a unique index is only supported within an - individual partition - not across all - partitions.

-

CONCURRENTLY keyword not supported in - WarehousePG.

-
CREATE LANGUAGEYES
CREATE MATERIALIZED VIEWYESBased on PostgreSQL 9.4.
CREATE OPERATORYESLimitations:

The function used to implement the operator must be an IMMUTABLE function.

-
CREATE OPERATOR CLASSYES
CREATE OPERATOR FAMILYYES
CREATE PROTOCOLYES
CREATE PUBLICATIONNO
CREATE RESOURCE QUEUEYESWarehousePG resource management feature - not in PostgreSQL 9.4.
CREATE ROLEYESWarehousePG Clauses:

RESOURCE QUEUEqueue_name | none

-
CREATE RULEYES
CREATE SCHEMAYES
CREATE SEQUENCEYESLimitations:

The lastval() and - currval() functions are not supported.

-

The - setval() function is only allowed in queries that do not - operate on distributed data.

-
CREATE SUBSCRIPTIONNO
CREATE TABLEYESUnsupported Clauses / Options:

[GLOBAL | - LOCAL]

-

REFERENCES

-

FOREIGN - KEY

-

[DEFERRABLE | NOT DEFERRABLE] -

-

Limited Clauses:

-

UNIQUE or - PRIMARY KEY constraints are only allowed on hash-distributed - tables (DISTRIBUTED BY), and the constraint columns must be the - same as or a superset of the distribution key columns of the table and must - include all the distribution key columns of the partitioning - key.

-

WarehousePG Clauses:

-

DISTRIBUTED BY - (column, [ ... ] ) |

-

DISTRIBUTED - RANDOMLY

-

PARTITION BY type (column [, ...]) - ( partition_specification, [...] )

-

WITH - (appendoptimized=true [,compresslevel=value,blocksize=value] - )

-
CREATE TABLE ASYESSee CREATE TABLE
CREATE TABLESPACEYESWarehousePG Clauses:

Specify host file system - locations for specific segment instances.

-

WITH - (contentID_1='/path/to/dir1...)

-
CREATE TRIGGERNO
CREATE TYPEYESWarehousePG Clauses:

COMPRESSTYPE | - COMPRESSLEVEL | BLOCKSIZE

-

Limitations:

-

The functions - used to implement a new base type must be IMMUTABLE - functions.

-
CREATE USERYESAn alias for CREATE ROLE
CREATE VIEWYES
DEALLOCATEYES
DECLAREYESUnsupported Clauses / - Options:

SCROLL

-

FOR UPDATE [ OF column [, - ...] ]

-

Limitations:

-

Cursors cannot be - backward-scrolled. Forward scrolling is supported.

-

PL/pgSQL does not have - support for updatable cursors.

-
DELETEYES
DISCARDYES -

Limitation: - DISCARD ALL is not supported.

-
DOYESPostgreSQL 9.0 feature
DROP AGGREGATEYES
DROP CASTYES
DROP CONVERSIONYES
DROP DATABASEYES
DROP DOMAINYES
DROP EVENT TRIGGERYES
DROP EXTENSIONYESRemoves an extension from WarehousePG – based on - PostgreSQL 9.6.
DROP EXTERNAL TABLEYESWarehousePG parallel ETL feature - not in PostgreSQL - 9.4.
DROP FUNCTIONYES
DROP GROUPYESAn alias for DROP ROLE
DROP INDEXYES
DROP LANGUAGEYES
DROP OPERATORYES
DROP OPERATOR CLASSYES
DROP OPERATOR FAMILYYES
DROP OWNEDNO
DROP PROTOCOLYES
DROP PUBLICATIONNO
DROP RESOURCE QUEUEYESWarehousePG resource management feature - not in - PostgreSQL 9.4.
DROP ROLEYES
DROP RULEYES
DROP SCHEMAYES
DROP SEQUENCEYES
DROP SUBSCRIPTIONNO
DROP TABLEYES
DROP TABLESPACEYES
DROP TRIGGERNO
DROP TYPEYES
DROP USERYESAn alias for DROP ROLE
DROP VIEWYES
ENDYES
EXECUTEYES
EXPLAINYES
FETCHYESUnsupported Clauses / - Options:

LAST

-

PRIOR

-

BACKWARD

-

BACKWARD - ALL

-

Limitations:

-

Cannot fetch rows in a - nonsequential fashion; backward scan is not supported.

-
GRANTYES
INSERTYES
LATERAL Join TypeNO
LISTENYES
LOADYES
LOCKYES
MOVEYESSee FETCH
NOTIFYYES
PREPAREYES
PREPARE TRANSACTIONNO
REASSIGN OWNEDYES
REFRESH MATERIALIZED VIEWYESBased on PostgreSQL 9.4.
REINDEXYES
RELEASE SAVEPOINTYES
RESETYES
RETRIEVEYESWarehousePG parallel retrieve cursor - not in PostgreSQL 9.4.
REVOKEYES
ROLLBACKYES
ROLLBACK PREPAREDNO
ROLLBACK TO SAVEPOINTYES
SAVEPOINTYES
SELECTYESLimitations:

Limited use of VOLATILE - and STABLE functions in FROM or - WHERE clauses

-

Text search (Tsearch2) is - not supported

-

WarehousePG Clauses (OLAP):

-

[GROUP - BY grouping_element [, - ...]]

-

[WINDOW window_name AS - (window_specification)]

-

[FILTER - (WHERE condition)] applied to an aggregate - function in the SELECT list

-
SELECT INTOYESSee SELECT
SETYES
SET CONSTRAINTSNOIn PostgreSQL, this only applies to foreign key constraints, - which are currently not enforced in WarehousePG.
SET ROLEYES
SET SESSION AUTHORIZATIONYESDeprecated as of PostgreSQL 8.1 - see SET ROLE
SET TRANSACTIONYESLimitations:

DEFERRABLE clause has no - effect.

-
SHOWYES
START TRANSACTIONYES
TRUNCATEYES
UNLISTENYES
UPDATEYESLimitations:

SET not allowed for - WarehousePG distribution key columns.

-
VACUUMYESLimitations:

VACUUM FULL is not - recommended in WarehousePG.

-
VALUESYES
+| SQL Command | Supported in WarehousePG | Modifications, Limitations, Exceptions | +| --- | --- | --- | +| `ALTER AGGREGATE` | YES | | +| `ALTER CONVERSION` | YES | | +| `ALTER DATABASE` | YES | | +| `ALTER DOMAIN` | YES | | +| `ALTER EVENT TRIGGER` | YES | | +| `ALTER EXTENSION` | YES | Changes the definition of a WarehousePG extension - based on PostgreSQL 9.6. | +| `ALTER FUNCTION` | YES | | +| `ALTER GROUP` | YES | An alias for [ALTER ROLE](../sql_commands/ALTER_ROLE.md) | +| `ALTER INDEX` | YES | | +| `ALTER LANGUAGE` | YES | | +| `ALTER OPERATOR` | YES | | +| `ALTER OPERATOR CLASS` | YES | | +| `ALTER OPERATOR FAMILY` | YES | | +| `ALTER PROTOCOL` | YES | | +| `ALTER PUBLICATION` | NO | | +| `ALTER RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL. | +| `ALTER ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | +| `ALTER SCHEMA` | YES | | +| `ALTER SEQUENCE` | YES | | +| `ALTER SUBSCRIPTION` | NO | | +| `ALTER SYSTEM` | **NO** | | +| `ALTER TABLE` | YES | **Unsupported Clauses / Options:**

`ENABLE/DISABLE TRIGGER`

**WarehousePG Database Clauses:**

`ADD \| DROP \| RENAME \| SPLIT \| EXCHANGE PARTITION \| SET SUBPARTITION TEMPLATE \| SET WITH (REORGANIZE=true \| false) \| SET DISTRIBUTED BY` | +| `ALTER TABLESPACE` | YES | | +| `ALTER TRIGGER` | **NO** | | +| `ALTER TYPE` | YES | **WarehousePG Clauses:**

`SET DEFAULT ENCODING` | +| `ALTER USER` | YES | An alias for [ALTER ROLE](../sql_commands/ALTER_ROLE.md) | +| `ALTER VIEW` | YES | | +| `ANALYZE` | YES | | +| `BEGIN` | YES | | +| `CHECKPOINT` | YES | | +| `CLOSE` | YES | | +| `CLUSTER` | YES | | +| `COMMENT` | YES | | +| `COMMIT` | YES | | +| `COMMIT PREPARED` | **NO** | | +| `COPY` | YES | **Modified Clauses:**

`ESCAPE [ AS ] '`*escape*`' \| 'OFF'`

**WarehousePG Clauses:**

`[LOG ERRORS] SEGMENT REJECT LIMIT `*count*` [ROWS\|PERCENT]` | +| `CREATE AGGREGATE` | YES | **Unsupported Clauses / Options:**

`[ , SORTOP = `*sort_operator*` ]`

**WarehousePG Clauses:**

`[ , COMBINEFUNC = `*combinefunc*` ]`

**Limitations:**

The functions used to implement the aggregate must be `IMMUTABLE` functions. | +| `CREATE CAST` | YES | | +| `CREATE CONSTRAINT TRIGGER` | **NO** | | +| `CREATE CONVERSION` | YES | | +| `CREATE DATABASE` | YES | | +| `CREATE DOMAIN` | YES | | +| `CREATE EVENT TRIGGER` | YES | | +| `CREATE EXTENSION` | YES | Loads a new extension into WarehousePG - based on PostgreSQL 9.6. | +| `CREATE EXTERNAL TABLE` | YES | WarehousePG parallel ETL feature - not in PostgreSQL 9.4. | +| `CREATE FUNCTION` | YES | **Limitations:**

Functions defined as `STABLE` or `VOLATILE` can be run in WarehousePG provided that they are run on the coordinator only. `STABLE` and `VOLATILE` functions cannot be used in statements that run at the segment level. | +| `CREATE GROUP` | YES | An alias for [CREATE ROLE](../sql_commands/CREATE_ROLE.md) | +| `CREATE INDEX` | YES | **WarehousePG Clauses:**

`USING bitmap` (bitmap indexes)

**Limitations:**

`UNIQUE` indexes are allowed only if they contain all of (or a superset of) the WarehousePG distribution key columns. On partitioned tables, a unique index is only supported within an individual partition - not across all partitions.

`CONCURRENTLY` keyword not supported in WarehousePG. | +| `CREATE LANGUAGE` | YES | | +| `CREATE MATERIALIZED VIEW` | YES | Based on PostgreSQL 9.4. | +| `CREATE OPERATOR` | YES | **Limitations:**

The function used to implement the operator must be an `IMMUTABLE` function. | +| `CREATE OPERATOR CLASS` | YES | | +| `CREATE OPERATOR FAMILY` | YES | | +| `CREATE PROTOCOL` | YES | | +| `CREATE PUBLICATION` | NO | | +| `CREATE RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL 9.4. | +| `CREATE ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE`*queue_name*` \| none` | +| `CREATE RULE` | YES | | +| `CREATE SCHEMA` | YES | | +| `CREATE SEQUENCE` | YES | **Limitations:**

The `lastval()` and `currval()` functions are not supported.

The `setval()` function is only allowed in queries that do not operate on distributed data. | +| `CREATE SUBSCRIPTION` | NO | | +| `CREATE TABLE` | YES | **Unsupported Clauses / Options:**

`[GLOBAL \| LOCAL]`

`REFERENCES`

`FOREIGN KEY`

`[DEFERRABLE \| NOT DEFERRABLE] `

**Limited Clauses:**

`UNIQUE` or `PRIMARY KEY `constraints are only allowed on hash-distributed tables (`DISTRIBUTED BY`), and the constraint columns must be the same as or a superset of the distribution key columns of the table and must include all the distribution key columns of the partitioning key.

**WarehousePG Clauses:**

`DISTRIBUTED BY (column, [ ... ] ) \|`

`DISTRIBUTED RANDOMLY`

`PARTITION BY type (column [, ...]) ( partition_specification, [...] )`

`WITH (appendoptimized=true [,compresslevel=value,blocksize=value] )` | +| `CREATE TABLE AS` | YES | See [CREATE TABLE](../sql_commands/CREATE_TABLE.md) | +| `CREATE TABLESPACE` | **YES** | **WarehousePG Clauses:**

Specify host file system locations for specific segment instances.

`WITH (contentID_1='/path/to/dir1...)` | +| `CREATE TRIGGER` | **NO** | | +| `CREATE TYPE` | YES | **WarehousePG Clauses:**

`COMPRESSTYPE \| COMPRESSLEVEL \| BLOCKSIZE`

**Limitations:**

The functions used to implement a new base type must be `IMMUTABLE` functions. | +| `CREATE USER` | YES | An alias for [CREATE ROLE](../sql_commands/CREATE_ROLE.md) | +| `CREATE VIEW` | YES | | +| `DEALLOCATE` | YES | | +| `DECLARE` | YES | **Unsupported Clauses / Options:**

`SCROLL`

`FOR UPDATE [ OF column [, ...] ]`

**Limitations:**

Cursors cannot be backward-scrolled. Forward scrolling is supported.

PL/pgSQL does not have support for updatable cursors. | +| `DELETE` | YES | | +| `DISCARD` | YES | **Limitation:** `DISCARD ALL` is not supported. | +| `DO` | YES | PostgreSQL 9.0 feature | +| `DROP AGGREGATE` | YES | | +| `DROP CAST` | YES | | +| `DROP CONVERSION` | YES | | +| `DROP DATABASE` | YES | | +| `DROP DOMAIN` | YES | | +| `DROP EVENT TRIGGER` | YES | | +| `DROP EXTENSION` | YES | Removes an extension from WarehousePG – based on PostgreSQL 9.6. | +| `DROP EXTERNAL TABLE` | YES | WarehousePG parallel ETL feature - not in PostgreSQL 9.4. | +| `DROP FUNCTION` | YES | | +| `DROP GROUP` | YES | An alias for [DROP ROLE](../sql_commands/DROP_ROLE.md) | +| `DROP INDEX` | YES | | +| `DROP LANGUAGE` | YES | | +| `DROP OPERATOR` | YES | | +| `DROP OPERATOR CLASS` | YES | | +| `DROP OPERATOR FAMILY` | YES | | +| `DROP OWNED` | **NO** | | +| `DROP PROTOCOL` | YES | | +| `DROP PUBLICATION` | NO | | +| `DROP RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL 9.4. | +| `DROP ROLE` | YES | | +| `DROP RULE` | YES | | +| `DROP SCHEMA` | YES | | +| `DROP SEQUENCE` | YES | | +| `DROP SUBSCRIPTION` | NO | | +| `DROP TABLE` | YES | | +| `DROP TABLESPACE` | YES | | +| `DROP TRIGGER` | **NO** | | +| `DROP TYPE` | YES | | +| `DROP USER` | YES | An alias for [DROP ROLE](../sql_commands/DROP_ROLE.md) | +| `DROP VIEW` | YES | | +| `END` | YES | | +| `EXECUTE` | YES | | +| `EXPLAIN` | YES | | +| `FETCH` | YES | **Unsupported Clauses / Options:**

`LAST`

`PRIOR`

`BACKWARD`

`BACKWARD ALL`

**Limitations:**

Cannot fetch rows in a nonsequential fashion; backward scan is not supported. | +| `GRANT` | YES | | +| `INSERT` | YES | | +| `LATERAL` Join Type | **NO** | | +| `LISTEN` | **YES** | | +| `LOAD` | YES | | +| `LOCK` | YES | | +| `MOVE` | YES | See [FETCH](../sql_commands/FETCH.md) | +| `NOTIFY` | **YES** | | +| `PREPARE` | YES | | +| `PREPARE TRANSACTION` | **NO** | | +| `REASSIGN OWNED` | YES | | +| `REFRESH MATERIALIZED VIEW` | YES | Based on PostgreSQL 9.4. | +| `REINDEX` | YES | | +| `RELEASE SAVEPOINT` | YES | | +| `RESET` | YES | | +| `RETRIEVE` | YES | WarehousePG parallel retrieve cursor - not in PostgreSQL 9.4. | +| `REVOKE` | YES | | +| `ROLLBACK` | YES | | +| `ROLLBACK PREPARED` | **NO** | | +| `ROLLBACK TO SAVEPOINT` | YES | | +| `SAVEPOINT` | YES | | +| `SELECT` | YES | **Limitations:**

Limited use of `VOLATILE` and `STABLE` functions in `FROM` or `WHERE` clauses

Text search (`Tsearch2`) is not supported

**WarehousePG Clauses (OLAP):**

`[GROUP BY `*grouping_element*` [, ...]]`

`[WINDOW `*window_name*` AS (`*window_specification*`)]`

`[FILTER (WHERE `*condition*`)]` applied to an aggregate function in the `SELECT` list | +| `SELECT INTO` | YES | See [SELECT](../sql_commands/SELECT.md) | +| `SET` | YES | | +| `SET CONSTRAINTS` | **NO** | In PostgreSQL, this only applies to foreign key constraints, which are currently not enforced in WarehousePG. | +| `SET ROLE` | YES | | +| `SET SESSION AUTHORIZATION` | YES | Deprecated as of PostgreSQL 8.1 - see [SET ROLE](../sql_commands/SET_ROLE.md) | +| `SET TRANSACTION` | YES | **Limitations:**

`DEFERRABLE` clause has no effect. | +| `SHOW` | YES | | +| `START TRANSACTION` | YES | | +| `TRUNCATE` | YES | | +| `UNLISTEN` | **YES** | | +| `UPDATE` | YES | **Limitations:**

`SET` not allowed for WarehousePG distribution key columns. | +| `VACUUM` | YES | **Limitations:**

`VACUUM FULL` is not recommended in WarehousePG. | +| `VALUES` | YES | | From cafd3fa2d6a63071955a93bacd7895ceba5d8df3 Mon Sep 17 00:00:00 2001 From: Mireia Perez Date: Fri, 28 Aug 2026 15:59:32 +0100 Subject: [PATCH 2/3] Fix link --- docs/whpg/6x/admin_guide/manage_access/roles_privs.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/whpg/6x/admin_guide/manage_access/roles_privs.md b/docs/whpg/6x/admin_guide/manage_access/roles_privs.md index 02e876a..48e5880 100644 --- a/docs/whpg/6x/admin_guide/manage_access/roles_privs.md +++ b/docs/whpg/6x/admin_guide/manage_access/roles_privs.md @@ -95,7 +95,7 @@ Once the group role exists, you can add and remove members (user roles) using th ``` -For managing object privileges, you would then grant the appropriate permissions to the group-level role only (see [Table 2](#iq139925)). The member user roles then inherit the object privileges of the group role. For example: +For managing object privileges, you would then grant the appropriate permissions to the group-level role only (see [Object Privileges](#object-privileges)). The member user roles then inherit the object privileges of the group role. For example: ``` =# GRANT ALL ON TABLE mytable TO admin; @@ -117,7 +117,7 @@ The role attributes `LOGIN`, `SUPERUSER`, `CREATEDB`, `CREATEROLE`, `CREATEEXTTA When an object (table, view, sequence, database, function, language, schema, or tablespace) is created, it is assigned an owner. The owner is normally the role that ran the creation statement. For most kinds of objects, the initial state is that only the owner (or a superuser) can do anything with the object. To allow other roles to use it, privileges must be granted. WarehousePG supports the following privileges for each object type: -**Object Privileges** +### Object Privileges | Object Type | Privileges | | --- | --- | From f2dfa63a736a12066501b54791275c801d2e662e Mon Sep 17 00:00:00 2001 From: Mireia Perez Date: Thu, 3 Sep 2026 10:07:42 +0100 Subject: [PATCH 3/3] Address Josh's feedback --- docs/.gitignore | 1 + docs/whpg/6x/admin_guide/analytics/postGIS.md | 10 +- .../admin_guide/external/foreign/devel-fdw.md | 44 +- .../6x/admin_guide/intro/about_statistics.md | 10 +- .../admin_guide/manage_access/roles_privs.md | 26 +- .../6x/admin_guide/managing/monitoring.md | 38 +- .../admin_guide/query/functions-operators.md | 58 +- docs/whpg/6x/admin_guide/query/json-data.md | 261 +- docs/whpg/6x/install_guide/install_modules.md | 26 +- .../install_guide/installation_utilities.md | 13 +- .../6x/install_guide/platform-requirements.md | 6 +- docs/whpg/6x/ref_guide/function-summary.md | 1524 +++++++++-- .../whpg/6x/ref_guide/misc/feature_summary.md | 38 +- docs/whpg/7x/admin_guide/analytics/postGIS.md | 10 +- .../7x/admin_guide/ddl/ddl-index/ddl-brin.md | 4 +- .../external-tables/map_ext_to_foreign.md | 2 +- .../admin_guide/external/foreign/devel-fdw.md | 44 +- .../7x/admin_guide/intro/about_statistics.md | 10 +- .../admin_guide/manage_access/roles_privs.md | 26 +- .../7x/admin_guide/managing/monitoring.md | 38 +- .../admin_guide/query/functions-operators.md | 58 +- docs/whpg/7x/admin_guide/query/json-data.md | 6 +- .../install_guide/installation_utilities.md | 13 +- .../7x/install_guide/platform-requirements.md | 4 +- docs/whpg/7x/ref_guide/function-summary.md | 2402 ++++++++++++++--- .../whpg/7x/ref_guide/misc/feature_summary.md | 38 +- .../system_catalogs/catalog_ref-views.md | 2 +- 27 files changed, 3950 insertions(+), 762 deletions(-) diff --git a/docs/.gitignore b/docs/.gitignore index f84ae29..a8b1351 100644 --- a/docs/.gitignore +++ b/docs/.gitignore @@ -1,4 +1,5 @@ .vitepress/cache .vitepress/dist +.vitepress/.temp vitepress.code-workspace style.md \ No newline at end of file diff --git a/docs/whpg/6x/admin_guide/analytics/postGIS.md b/docs/whpg/6x/admin_guide/analytics/postGIS.md index 5b1911d..bc4c843 100644 --- a/docs/whpg/6x/admin_guide/analytics/postGIS.md +++ b/docs/whpg/6x/admin_guide/analytics/postGIS.md @@ -44,11 +44,11 @@ This table lists the PostGIS extensions support by WarehousePG PostGIS. | PostGIS Extension | WarehousePG PostGIS Notes | | --- | --- | -| `postgis`

PostGIS and PostGIS Raster support | Supported. Both PostGIS and PostGIS Raster are enabled when the WarehousePG `postgis` extension is enabled. | -| `postgis_tiger_geocoder`

The US TIGER geocoder | Supported. Installed with WarehousePG PostGIS.

Requires the `postgis` and `fuzzystrmatch` extensions.

The US TIGER geocoder converts addresses (like a street address) to geographic coordinates. | -| `address_standardizer`

Rule-based address standardizer | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with TIGER geocoder.

A single line address parser that takes an input address and normalizes it based on a set of rules stored in a table and helper `lex` and `gaz` tables. | -| `address_standardizer_data_us`

Sample rules tables for US address data | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with the address standardizer.

The extension contains `gaz`, `lex`, and `rules` tables for US address data. If you are using other types of tables, see [PostGIS Extension Limitations](#topic_wy2_rkb_3p). | -| `fuzzystrmatch`

Fuzzy string matching | Supported. This extension is bundled but not enabled with WarehousePG Database.

Required for the PostGIS TIGER geocoder. | +| `postgis`

PostGIS and PostGIS Raster support | Supported. Both PostGIS and PostGIS Raster are enabled when the WarehousePG `postgis` extension is enabled. | +| `postgis_tiger_geocoder`

The US TIGER geocoder | Supported. Installed with WarehousePG PostGIS.

Requires the `postgis` and `fuzzystrmatch` extensions.

The US TIGER geocoder converts addresses (like a street address) to geographic coordinates. | +| `address_standardizer`

Rule-based address standardizer | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with TIGER geocoder.

A single line address parser that takes an input address and normalizes it based on a set of rules stored in a table and helper `lex` and `gaz` tables. | +| `address_standardizer_data_us`

Sample rules tables for US address data | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with the address standardizer.

The extension contains `gaz`, `lex`, and `rules` tables for US address data. If you are using other types of tables, see [PostGIS Extension Limitations](#topic_wy2_rkb_3p). | +| `fuzzystrmatch`

Fuzzy string matching | Supported. This extension is bundled but not enabled with WarehousePG Database.

Required for the PostGIS TIGER geocoder. | > **Note** The PostGIS topology extension `postgis_topology` and the PostGIS 3D and geoprocessing extension `postgis_sfcgal` are not supported by WarehousePG PostGIS and are not included in the WarehousePG PostGIS extension package. diff --git a/docs/whpg/6x/admin_guide/external/foreign/devel-fdw.md b/docs/whpg/6x/admin_guide/external/foreign/devel-fdw.md index 75598af..dfac2ef 100644 --- a/docs/whpg/6x/admin_guide/external/foreign/devel-fdw.md +++ b/docs/whpg/6x/admin_guide/external/foreign/devel-fdw.md @@ -148,26 +148,26 @@ Scan-related callback functions include: | Callback Signature | Description | | --- | --- | -| `void`
`GetForeignRelSize (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Obtain relation size estimates for a foreign table. Called at the beginning of planning for a query on a foreign table. | -| `void`
`GetForeignPaths (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Create possible access paths for a scan on a foreign table. Called during query planning.

**Note:** A WarehousePG Database-compatible FDW must call `create_foreignscan_path()` in its `GetForeignPaths()` callback function. | -| `ForeignScan *`
`GetForeignPlan (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid,`
` ForeignPath *best_path,`
` List *tlist,`
` List *scan_clauses)` | Create a `ForeignScan` plan node from the selected foreign access path. Called at the end of query planning. | -| `void`
`BeginForeignScan (ForeignScanState *node,`
` int eflags)` | Begin running a foreign scan. Called during executor startup. | -| `TupleTableSlot *`
`IterateForeignScan (ForeignScanState *node)` | Fetch one row from the foreign source, returning it in a tuple table slot; return NULL if no more rows are available. | -| `void`
`ReScanForeignScan (ForeignScanState *node)` | Restart the scan from the beginning. | -| `void`
`EndForeignScan (ForeignScanState *node)` | End the scan and release resources. | +| `void`
`GetForeignRelSize (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Obtain relation size estimates for a foreign table. Called at the beginning of planning for a query on a foreign table. | +| `void`
`GetForeignPaths (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Create possible access paths for a scan on a foreign table. Called during query planning.

**Note:** A WarehousePG Database-compatible FDW must call `create_foreignscan_path()` in its `GetForeignPaths()` callback function. | +| `ForeignScan *`
`GetForeignPlan (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid,`
` ForeignPath *best_path,`
` List *tlist,`
` List *scan_clauses)` | Create a `ForeignScan` plan node from the selected foreign access path. Called at the end of query planning. | +| `void`
`BeginForeignScan (ForeignScanState *node,`
` int eflags)` | Begin running a foreign scan. Called during executor startup. | +| `TupleTableSlot *`
`IterateForeignScan (ForeignScanState *node)` | Fetch one row from the foreign source, returning it in a tuple table slot; return NULL if no more rows are available. | +| `void`
`ReScanForeignScan (ForeignScanState *node)` | Restart the scan from the beginning. | +| `void`
`EndForeignScan (ForeignScanState *node)` | End the scan and release resources. | If a foreign data wrapper supports writable foreign tables, it should provide the update-related callback functions that are required by the capabilities of the FDW. Update-related callback functions include: | Callback Signature | Description | | --- | --- | -| `void`
`AddForeignUpdateTargets (Query *parsetree,`
` RangeTblEntry *target_rte,`
` Relation target_relation)` | Add additional information in the foreign table that will be retrieved during an update or delete operation to identify the exact row on which to operate. | -| `List *`
`PlanForeignModify (PlannerInfo *root,`
` ModifyTable *plan,`
` Index resultRelation,`
` int subplan_index)` | Perform additional planning actions required for an insert, update, or delete operation on a foreign table, and return the information generated. | -| `void`
`BeginForeignModify (ModifyTableState *mtstate,`
` ResultRelInfo *rinfo,`
` List *fdw_private,`
` int subplan_index,`
` int eflags)` | Begin executing a modify operation on a foreign table. Called during executor startup. | -| `TupleTableSlot *`
`ExecForeignInsert (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Insert a single tuple into the foreign table. Return a slot containing the data that was actually inserted, or NULL if no row was inserted. | -| `TupleTableSlot *`
`ExecForeignUpdate (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Update a single tuple in the foreign table. Return a slot containing the row as it was actually updated, or NULL if no row was updated. | -| `TupleTableSlot *`
`ExecForeignDelete (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Delete a single tuple from the foreign table. Return a slot containing the row that was deleted, or NULL if no row was deleted. | -| `void`
`EndForeignModify (EState *estate,`
` ResultRelInfo *rinfo)` | End the update and release resources. | -| `int`
`IsForeignRelUpdatable (Relation rel)` | Report the update operations supported by the specified foreign table. | +| `void`
`AddForeignUpdateTargets (Query *parsetree,`
` RangeTblEntry *target_rte,`
` Relation target_relation)` | Add additional information in the foreign table that will be retrieved during an update or delete operation to identify the exact row on which to operate. | +| `List *`
`PlanForeignModify (PlannerInfo *root,`
` ModifyTable *plan,`
` Index resultRelation,`
` int subplan_index)` | Perform additional planning actions required for an insert, update, or delete operation on a foreign table, and return the information generated. | +| `void`
`BeginForeignModify (ModifyTableState *mtstate,`
` ResultRelInfo *rinfo,`
` List *fdw_private,`
` int subplan_index,`
` int eflags)` | Begin executing a modify operation on a foreign table. Called during executor startup. | +| `TupleTableSlot *`
`ExecForeignInsert (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Insert a single tuple into the foreign table. Return a slot containing the data that was actually inserted, or NULL if no row was inserted. | +| `TupleTableSlot *`
`ExecForeignUpdate (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Update a single tuple in the foreign table. Return a slot containing the row as it was actually updated, or NULL if no row was updated. | +| `TupleTableSlot *`
`ExecForeignDelete (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Delete a single tuple from the foreign table. Return a slot containing the row that was deleted, or NULL if no row was deleted. | +| `void`
`EndForeignModify (EState *estate,`
` ResultRelInfo *rinfo)` | End the update and release resources. | +| `int`
`IsForeignRelUpdatable (Relation rel)` | Report the update operations supported by the specified foreign table. | Refer to [Foreign Data Wrapper Callback Routines](https://www.postgresql.org/docs/9.4/fdw-callbacks.html) in the PostgreSQL documentation for detailed information about the inputs and outputs of the FDW callback functions. @@ -185,13 +185,13 @@ The FDW API includes the helper functions listed in the table below. Refer to [F | Helper Signature | Description | | --- | --- | -| `ForeignDataWrapper *`
`GetForeignDataWrapper(Oid fdwid);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given OID. | -| `ForeignDataWrapper *`
`GetForeignDataWrapperByName(const char *name, bool missing_ok);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given name. | -| `ForeignServer *`
`GetForeignServer(Oid serverid);` | Returns the `ForeignServer` object for the foreign server with the given OID. | -| `ForeignServer *`
`GetForeignServerByName(const char *name, bool missing_ok);` | Returns the `ForeignServer` object for the foreign server with the given name. | -| `UserMapping *`
`GetUserMapping(Oid userid, Oid serverid);` | Returns the `UserMapping` object for the user mapping of the given role on the given server. | -| `ForeignTable *`
`GetForeignTable(Oid relid);` | Returns the `ForeignTable` object for the foreign table with the given OID. | -| `List *`
`GetForeignColumnOptions(Oid relid, AttrNumber attnum);` | Returns the per-column FDW options for the column with the given foreign table OID and attribute number. | +| `ForeignDataWrapper *`
`GetForeignDataWrapper(Oid fdwid);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given OID. | +| `ForeignDataWrapper *`
`GetForeignDataWrapperByName(const char *name, bool missing_ok);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given name. | +| `ForeignServer *`
`GetForeignServer(Oid serverid);` | Returns the `ForeignServer` object for the foreign server with the given OID. | +| `ForeignServer *`
`GetForeignServerByName(const char *name, bool missing_ok);` | Returns the `ForeignServer` object for the foreign server with the given name. | +| `UserMapping *`
`GetUserMapping(Oid userid, Oid serverid);` | Returns the `UserMapping` object for the user mapping of the given role on the given server. | +| `ForeignTable *`
`GetForeignTable(Oid relid);` | Returns the `ForeignTable` object for the foreign table with the given OID. | +| `List *`
`GetForeignColumnOptions(Oid relid, AttrNumber attnum);` | Returns the per-column FDW options for the column with the given foreign table OID and attribute number. | diff --git a/docs/whpg/6x/admin_guide/intro/about_statistics.md b/docs/whpg/6x/admin_guide/intro/about_statistics.md index 674dc08..ad42b05 100644 --- a/docs/whpg/6x/admin_guide/intro/about_statistics.md +++ b/docs/whpg/6x/admin_guide/intro/about_statistics.md @@ -93,11 +93,11 @@ The `stakindN` columns each contain a numeric code to describe the type of stati | stakind Code | Description | | --- | --- | -| 1 | *Most CommonValues (MCV) Slot*

- `staop` contains the object ID of the "=" operator, used to decide whether values are the same or not.
- `stavalues` contains an array of the *K* most common non-null values appearing in the column.
- `stanumbers` contains the frequencies (fractions of total row count) of the values in the `stavalues` array.

The values are ordered in decreasing frequency. Since the arrays are variable-size, *K* can be chosen by the statistics collector. Values must occur more than once to be added to the `stavalues` array; a unique column has no MCV slot. | -| 2 | *Histogram Slot* – describes the distribution of scalar data.

- `staop` is the object ID of the "<" operator, which describes the sort ordering.
- `stavalues` contains *M* (where `M>=2`) non-null values that divide the non-null column data values into `M-1` bins of approximately equal population. The first `stavalues` item is the minimum value and the last is the maximum value.
- `stanumbers` is not used and should be `NULL`.

If a Most Common Values slot is also provided, then the histogram describes the data distribution after removing the values listed in the MCV array. (It is a *compressed histogram* in the technical parlance). This allows a more accurate representation of the distribution of a column with some very common values. In a column with only a few distinct values, it is possible that the MCV list describes the entire data population; in this case the histogram reduces to empty and should be omitted. | -| 3 | *Correlation Slot* – describes the correlation between the physical order of table tuples and the ordering of data values of this column.

- `staop` is the object ID of the "<" operator. As with the histogram, more than one entry could theoretically appear.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains a single entry, the correlation coefficient between the sequence of data values and the sequence of their actual tuple positions. The coefficient ranges from +1 to -1. | -| 4 | *Most Common Elements Slot* - is similar to a Most Common Values (MCV) Slot, except that it stores the most common non-null *elements* of the column values. This is useful when the column datatype is an array or some other type with identifiable elements (for instance, `tsvector`).

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` contains the most common element values.
- `stanumbers` contains common element frequencies.

Frequencies are measured as the fraction of non-null rows the element value appears in, not the frequency of all rows. Also, the values are sorted into the element type's default order (to support binary search for a particular value). Since this puts the minimum and maximum frequencies at unpredictable spots in `stanumbers`, there are two extra members of `stanumbers` that hold copies of the minimum and maximum frequencies. Optionally, there can be a third extra member that holds the frequency of null elements (the frequency is expressed in the same terms: the fraction of non-null rows that contain at least one null element). If this member is omitted, the column is presumed to contain no `NULL` elements.

**Note:** For `tsvector` columns, the `stavalues` elements are of type `text`, even though their representation within `tsvector` is not exactly `text`. | -| 5 | *Distinct Elements Count Histogram Slot* - describes the distribution of the number of distinct element values present in each row of an array-type column. Only non-null rows are considered, and only non-null elements.

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains information about distinct elements. The last member of `stanumbers` is the average count of distinct element values over all non-null rows. The preceding *M* (where `M >=2`) members form a histogram that divides the population of distinct-elements counts into `M-1` bins of approximately equal population. The first of these is the minimum observed count, and the last the maximum. | +| 1 | *Most CommonValues (MCV) Slot*

- `staop` contains the object ID of the "=" operator, used to decide whether values are the same or not.
- `stavalues` contains an array of the *K* most common non-null values appearing in the column.
- `stanumbers` contains the frequencies (fractions of total row count) of the values in the `stavalues` array.

The values are ordered in decreasing frequency. Since the arrays are variable-size, *K* can be chosen by the statistics collector. Values must occur more than once to be added to the `stavalues` array; a unique column has no MCV slot. | +| 2 | *Histogram Slot* – describes the distribution of scalar data.

- `staop` is the object ID of the "<" operator, which describes the sort ordering.
- `stavalues` contains *M* (where `M>=2`) non-null values that divide the non-null column data values into `M-1` bins of approximately equal population. The first `stavalues` item is the minimum value and the last is the maximum value.
- `stanumbers` is not used and should be `NULL`.

If a Most Common Values slot is also provided, then the histogram describes the data distribution after removing the values listed in the MCV array. (It is a *compressed histogram* in the technical parlance). This allows a more accurate representation of the distribution of a column with some very common values. In a column with only a few distinct values, it is possible that the MCV list describes the entire data population; in this case the histogram reduces to empty and should be omitted. | +| 3 | *Correlation Slot* – describes the correlation between the physical order of table tuples and the ordering of data values of this column.

- `staop` is the object ID of the "<" operator. As with the histogram, more than one entry could theoretically appear.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains a single entry, the correlation coefficient between the sequence of data values and the sequence of their actual tuple positions. The coefficient ranges from +1 to -1. | +| 4 | *Most Common Elements Slot* - is similar to a Most Common Values (MCV) Slot, except that it stores the most common non-null *elements* of the column values. This is useful when the column datatype is an array or some other type with identifiable elements (for instance, `tsvector`).

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` contains the most common element values.
- `stanumbers` contains common element frequencies.

Frequencies are measured as the fraction of non-null rows the element value appears in, not the frequency of all rows. Also, the values are sorted into the element type's default order (to support binary search for a particular value). Since this puts the minimum and maximum frequencies at unpredictable spots in `stanumbers`, there are two extra members of `stanumbers` that hold copies of the minimum and maximum frequencies. Optionally, there can be a third extra member that holds the frequency of null elements (the frequency is expressed in the same terms: the fraction of non-null rows that contain at least one null element). If this member is omitted, the column is presumed to contain no `NULL` elements.

**Note:** For `tsvector` columns, the `stavalues` elements are of type `text`, even though their representation within `tsvector` is not exactly `text`. | +| 5 | *Distinct Elements Count Histogram Slot* - describes the distribution of the number of distinct element values present in each row of an array-type column. Only non-null rows are considered, and only non-null elements.

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains information about distinct elements. The last member of `stanumbers` is the average count of distinct element values over all non-null rows. The preceding *M* (where `M >=2`) members form a histogram that divides the population of distinct-elements counts into `M-1` bins of approximately equal population. The first of these is the minimum observed count, and the last the maximum. | | 99 | *Hyperloglog Slot* - for child leaf partitions of a partitioned table, stores the `hyperloglog_counter` created for the sampled data. The `hyperloglog_counter` data structure is converted into a `bytea` and stored in a `stavalues5` slot of the `pg_statistic` catalog table. | diff --git a/docs/whpg/6x/admin_guide/manage_access/roles_privs.md b/docs/whpg/6x/admin_guide/manage_access/roles_privs.md index 48e5880..8aa31a0 100644 --- a/docs/whpg/6x/admin_guide/manage_access/roles_privs.md +++ b/docs/whpg/6x/admin_guide/manage_access/roles_privs.md @@ -121,19 +121,19 @@ When an object (table, view, sequence, database, function, language, schema, or | Object Type | Privileges | | --- | --- | -| Tables, External Tables, Views | `SELECT`

`INSERT`

`UPDATE`

`DELETE`

`REFERENCES`

`TRIGGER`

`TRUNCATE`

`ALL` | -| Columns | `SELECT`

`INSERT`

`UPDATE`

`REFERENCES`

`ALL` | -| Sequences | `USAGE`

`SELECT`

`UPDATE`

`ALL` | -| Databases | `CREATE`

`CONNECT`

`TEMPORARY`

`TEMP`

`ALL` | -| Domains | `USAGE`

`ALL` | -| Foreign Data Wrappers | `USAGE`

`ALL` | -| Foreign Servers | `USAGE`

`ALL` | -| Functions | `EXECUTE`

`ALL` | -| Procedural Languages | `USAGE`

`ALL` | -| Schemas | `CREATE`

`USAGE`

`ALL` | -| Tablespaces | `CREATE`

`ALL` | -| Types | `USAGE`

`ALL` | -| Protocols | `SELECT`

`INSERT`

`ALL` | +| Tables, External Tables, Views | `SELECT`

`INSERT`

`UPDATE`

`DELETE`

`REFERENCES`

`TRIGGER`

`TRUNCATE`

`ALL` | +| Columns | `SELECT`

`INSERT`

`UPDATE`

`REFERENCES`

`ALL` | +| Sequences | `USAGE`

`SELECT`

`UPDATE`

`ALL` | +| Databases | `CREATE`

`CONNECT`

`TEMPORARY`

`TEMP`

`ALL` | +| Domains | `USAGE`

`ALL` | +| Foreign Data Wrappers | `USAGE`

`ALL` | +| Foreign Servers | `USAGE`

`ALL` | +| Functions | `EXECUTE`

`ALL` | +| Procedural Languages | `USAGE`

`ALL` | +| Schemas | `CREATE`

`USAGE`

`ALL` | +| Tablespaces | `CREATE`

`ALL` | +| Types | `USAGE`

`ALL` | +| Protocols | `SELECT`

`INSERT`

`ALL` | > **Note** You must grant privileges for each object individually. For example, granting `ALL` on a database does not grant full access to the objects within that database. It only grants all of the database-level privileges (`CONNECT`, `CREATE`, `TEMPORARY`) to the database itself. diff --git a/docs/whpg/6x/admin_guide/managing/monitoring.md b/docs/whpg/6x/admin_guide/managing/monitoring.md index beddbe4..954ab92 100644 --- a/docs/whpg/6x/admin_guide/managing/monitoring.md +++ b/docs/whpg/6x/admin_guide/managing/monitoring.md @@ -22,12 +22,12 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| List segments that are currently down. If any rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE status = 'd';` | If the query returns any rows, follow these steps to correct the problem:

1. Verify that the hosts with down segments are responsive.
2. If hosts are OK, check the log files for the primaries and mirrors of the down segments to discover the root cause of the segments going down.
3. If no unexpected errors are found, run the `gprecoverseg` utility to bring the segments back online. | -| Check for segments that are up and not in sync. If rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE mode = 'n' and status = 'u' and content <> -1;` | If the query returns rows then the segment might be in the process of moving from `Not In Sync` to `Synchronized` mode. Use `gpstate -e` to track progress. | -| Check for segments that are not operating in their preferred role but are marked as up and `Synchronized`. If any segments are found, the cluster may not be balanced. If any rows are returned this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE preferred_role <> role and status = 'u' and mode = 's';` | When the segments are not running in their preferred role, processing might be skewed. Run `gprecoverseg -r` to bring the segments back into their preferred roles. | -| Run a distributed query to test that it runs on all segments. One row should be returned for each primary segment.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Execute the following query in the `postgres` database:

`SELECT gp_segment_id, count(*) FROM gp_dist_random('pg_class') GROUP BY 1;` | If this query fails, there is an issue dispatching to some segments in the cluster. This is a rare event. Check the hosts that are not able to be dispatched to ensure there is no hardware or networking issue. | -| Test the state of coordinator mirroring on WarehousePG. If the value is not "STREAMING", raise an alert or warning.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following `psql` command:

`psql <dbname> -c 'SELECT pid, state FROM pg_stat_replication;'` | Check the log file from the coordinator and standby coordinator for errors. If there are no unexpected errors and the machines are up, run the `gpinitstandby` utility to bring the standby online. | -| Perform a basic check to see if the coordinator is up and functioning.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Run the following query in the `postgres` database:

`SELECT count(*) FROM gp_segment_configuration;` | If this query fails, the active coordinator may be down. Try to start the database on the original coordinator if the server is up and running. If that fails, try to activate the standby coordinator as coordinator. | +| List segments that are currently down. If any rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE status = 'd';` | If the query returns any rows, follow these steps to correct the problem:

1. Verify that the hosts with down segments are responsive.
2. If hosts are OK, check the log files for the primaries and mirrors of the down segments to discover the root cause of the segments going down.
3. If no unexpected errors are found, run the `gprecoverseg` utility to bring the segments back online. | +| Check for segments that are up and not in sync. If rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE mode = 'n' and status = 'u' and content <> -1;` | If the query returns rows then the segment might be in the process of moving from `Not In Sync` to `Synchronized` mode. Use `gpstate -e` to track progress. | +| Check for segments that are not operating in their preferred role but are marked as up and `Synchronized`. If any segments are found, the cluster may not be balanced. If any rows are returned this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE preferred_role <> role and status = 'u' and mode = 's';` | When the segments are not running in their preferred role, processing might be skewed. Run `gprecoverseg -r` to bring the segments back into their preferred roles. | +| Run a distributed query to test that it runs on all segments. One row should be returned for each primary segment.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Execute the following query in the `postgres` database:

`SELECT gp_segment_id, count(*) FROM gp_dist_random('pg_class') GROUP BY 1;` | If this query fails, there is an issue dispatching to some segments in the cluster. This is a rare event. Check the hosts that are not able to be dispatched to ensure there is no hardware or networking issue. | +| Test the state of coordinator mirroring on WarehousePG. If the value is not "STREAMING", raise an alert or warning.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following `psql` command:

`psql <dbname> -c 'SELECT pid, state FROM pg_stat_replication;'` | Check the log file from the coordinator and standby coordinator for errors. If there are no unexpected errors and the machines are up, run the `gpinitstandby` utility to bring the standby online. | +| Perform a basic check to see if the coordinator is up and functioning.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Run the following query in the `postgres` database:

`SELECT count(*) FROM gp_segment_configuration;` | If this query fails, the active coordinator may be down. Try to start the database on the original coordinator if the server is up and running. If that fails, try to activate the standby coordinator as coordinator. | ## Hardware and Operating System Monitoring @@ -37,10 +37,10 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| Check disk space usage on volumes used for WarehousePG data storage and the OS.

Recommended frequency: every 5 to 30 minutes

Severity: CRITICAL | Set up a disk space check.

- Set a threshold to raise an alert when a disk reaches a percentage of capacity. The recommended threshold is 75% full.
- It is not recommended to run the system with capacities approaching 100%. | Use `VACUUM`/`VACUUM FULL` on user tables to reclaim space occupied by dead rows. | -| Check for errors or dropped packets on the network interfaces.

Recommended frequency: hourly

Severity: IMPORTANT | Set up a network interface checks. | Work with network and OS teams to resolve errors. | -| Check for RAID errors or degraded RAID performance.

Recommended frequency: every 5 minutes

Severity: CRITICAL | Set up a RAID check. | - Replace failed disks as soon as possible.
- Work with system administration team to resolve other RAID or controller errors as soon as possible. | -| Check for adequate I/O bandwidth and I/O skew.

Recommended frequency: when create a cluster or when hardware issues are suspected. | Run the WarehousePG `gpcheckperf` utility. | The cluster may be under-specified if data transfer rates are not similar to the following:

- 2GB per second disk read
- 1 GB per second disk write
- 10 Gigabit per second network read and write

If transfer rates are lower than expected, consult with your data architect regarding performance expectations.

If the machines on the cluster display an uneven performance profile, work with the system administration team to fix faulty machines. | +| Check disk space usage on volumes used for WarehousePG data storage and the OS.

Recommended frequency: every 5 to 30 minutes

Severity: CRITICAL | Set up a disk space check.

- Set a threshold to raise an alert when a disk reaches a percentage of capacity. The recommended threshold is 75% full.
- It is not recommended to run the system with capacities approaching 100%. | Use `VACUUM`/`VACUUM FULL` on user tables to reclaim space occupied by dead rows. | +| Check for errors or dropped packets on the network interfaces.

Recommended frequency: hourly

Severity: IMPORTANT | Set up a network interface checks. | Work with network and OS teams to resolve errors. | +| Check for RAID errors or degraded RAID performance.

Recommended frequency: every 5 minutes

Severity: CRITICAL | Set up a RAID check. | - Replace failed disks as soon as possible.
- Work with system administration team to resolve other RAID or controller errors as soon as possible. | +| Check for adequate I/O bandwidth and I/O skew.

Recommended frequency: when create a cluster or when hardware issues are suspected. | Run the WarehousePG `gpcheckperf` utility. | The cluster may be under-specified if data transfer rates are not similar to the following:

- 2GB per second disk read
- 1 GB per second disk write
- 10 Gigabit per second network read and write

If transfer rates are lower than expected, consult with your data architect regarding performance expectations.

If the machines on the cluster display an uneven performance profile, work with the system administration team to fix faulty machines. | ## Catalog Monitoring @@ -50,11 +50,11 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| Run catalog consistency checks in each database to ensure the catalog on each host in the cluster is consistent and in a good state.

You may run this command while the database is up and running.

Recommended frequency: weekly

Severity: IMPORTANT | Run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -O`

**Note:** With the `-O` option, `gpcheckcat` runs just 10 of its usual 15 tests. | Run the repair scripts for any issues identified. | -| Check for `pg_class` entries that have no corresponding pg_`attribute` entry.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R pgclass` | Run the repair scripts for any issues identified. | -| Check for leaked temporary schema and missing schema definition.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R namespace` | Run the repair scripts for any issues identified. | -| Check constraints on randomly distributed tables.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R distribution_policy` | Run the repair scripts for any issues identified. | -| Check for dependencies on non-existent objects.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R dependency` | Run the repair scripts for any issues identified. | +| Run catalog consistency checks in each database to ensure the catalog on each host in the cluster is consistent and in a good state.

You may run this command while the database is up and running.

Recommended frequency: weekly

Severity: IMPORTANT | Run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -O`

**Note:** With the `-O` option, `gpcheckcat` runs just 10 of its usual 15 tests. | Run the repair scripts for any issues identified. | +| Check for `pg_class` entries that have no corresponding pg_`attribute` entry.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R pgclass` | Run the repair scripts for any issues identified. | +| Check for leaked temporary schema and missing schema definition.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R namespace` | Run the repair scripts for any issues identified. | +| Check constraints on randomly distributed tables.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R distribution_policy` | Run the repair scripts for any issues identified. | +| Check for dependencies on non-existent objects.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R dependency` | Run the repair scripts for any issues identified. | ## Data Maintenance @@ -64,8 +64,8 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| Check for missing statistics on tables. | Check the `gp_stats_missing` view in each database:

`SELECT * FROM gp_toolkit.gp_stats_missing;` | Run `ANALYZE` on tables that are missing statistics. | -| Check for tables that have bloat (dead space) in data files that cannot be recovered by a regular `VACUUM` command.

Recommended frequency: weekly or monthly

Severity: WARNING | Check the `gp_bloat_diag` view in each database:

`SELECT * FROM gp_toolkit.gp_bloat_diag;` | `VACUUM FULL` acquires an `ACCESS EXCLUSIVE` lock on tables. Run `VACUUM FULL` during a time when users and applications do not require access to the tables, such as during a time of low activity, or during a maintenance window. | +| Check for missing statistics on tables. | Check the `gp_stats_missing` view in each database:

`SELECT * FROM gp_toolkit.gp_stats_missing;` | Run `ANALYZE` on tables that are missing statistics. | +| Check for tables that have bloat (dead space) in data files that cannot be recovered by a regular `VACUUM` command.

Recommended frequency: weekly or monthly

Severity: WARNING | Check the `gp_bloat_diag` view in each database:

`SELECT * FROM gp_toolkit.gp_bloat_diag;` | `VACUUM FULL` acquires an `ACCESS EXCLUSIVE` lock on tables. Run `VACUUM FULL` during a time when users and applications do not require access to the tables, such as during a time of low activity, or during a maintenance window. | ## Database Maintenance @@ -144,6 +144,6 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| Ensure any bug fixes or enhancements are applied to the kernel.

Recommended frequency: at least every 6 months

Severity: IMPORTANT | Follow the vendor's instructions to update the Linux kernel. | Keep the kernel current to include bug fixes and security fixes, and to avoid difficult future upgrades. | -| Install WarehousePG minor releases, for example 6.21.*x*.

Recommended frequency: quarterly

Severity: IMPORTANT | Follow upgrade instructions in the WarehousePG *Release Notes*. Always upgrade to the latest in the series. | Keep the WarehousePG software current to incorporate bug fixes, performance enhancements, and feature enhancements into your WarehousePG cluster. | +| Ensure any bug fixes or enhancements are applied to the kernel.

Recommended frequency: at least every 6 months

Severity: IMPORTANT | Follow the vendor's instructions to update the Linux kernel. | Keep the kernel current to include bug fixes and security fixes, and to avoid difficult future upgrades. | +| Install WarehousePG minor releases, for example 6.21.*x*.

Recommended frequency: quarterly

Severity: IMPORTANT | Follow upgrade instructions in the WarehousePG *Release Notes*. Always upgrade to the latest in the series. | Keep the WarehousePG software current to incorporate bug fixes, performance enhancements, and feature enhancements into your WarehousePG cluster. | diff --git a/docs/whpg/6x/admin_guide/query/functions-operators.md b/docs/whpg/6x/admin_guide/query/functions-operators.md index e622435..00465f5 100644 --- a/docs/whpg/6x/admin_guide/query/functions-operators.md +++ b/docs/whpg/6x/admin_guide/query/functions-operators.md @@ -99,26 +99,26 @@ WarehousePG includes JSON processing functions that manipulate values the `json` | --- | --- | --- | --- | | [Logical Operators](https://www.postgresql.org/docs/9.4/functions-logical.html) | | | | | [Comparison Operators](https://www.postgresql.org/docs/9.4/functions-comparison.html) | | | | -| [Mathematical Functions and Operators](https://www.postgresql.org/docs/9.4/functions-math.html) | random

setseed | | | -| [String Functions and Operators](https://www.postgresql.org/docs/9.4/functions-string.html) | *All built-in conversion functions* | convert

pg_client_encoding | | +| [Mathematical Functions and Operators](https://www.postgresql.org/docs/9.4/functions-math.html) | random

setseed | | | +| [String Functions and Operators](https://www.postgresql.org/docs/9.4/functions-string.html) | *All built-in conversion functions* | convert

pg_client_encoding | | | [Binary String Functions and Operators](https://www.postgresql.org/docs/9.4/functions-binarystring.html) | | | | | [Bit String Functions and Operators](https://www.postgresql.org/docs/9.4/functions-bitstring.html) | | | | | [Pattern Matching](https://www.postgresql.org/docs/9.4/functions-matching.html) | | | | -| [Data Type Formatting Functions](https://www.postgresql.org/docs/9.4/functions-formatting.html) | | to_char

to_timestamp | | -| [Date/Time Functions and Operators](https://www.postgresql.org/docs/9.4/functions-datetime.html) | timeofday | age

current_date

current_time

current_timestamp

localtime

localtimestamp

now | | +| [Data Type Formatting Functions](https://www.postgresql.org/docs/9.4/functions-formatting.html) | | to_char

to_timestamp | | +| [Date/Time Functions and Operators](https://www.postgresql.org/docs/9.4/functions-datetime.html) | timeofday | age

current_date

current_time

current_timestamp

localtime

localtimestamp

now | | | [Enum Support Functions](https://www.postgresql.org/docs/9.4/functions-enum.html) | | | | | [Geometric Functions and Operators](https://www.postgresql.org/docs/9.4/functions-geometry.html) | | | | | [Network Address Functions and Operators](https://www.postgresql.org/docs/9.4/functions-net.html) | | | | -| [Sequence Manipulation Functions](https://www.postgresql.org/docs/9.4/functions-sequence.html) | nextval()

setval() | | | +| [Sequence Manipulation Functions](https://www.postgresql.org/docs/9.4/functions-sequence.html) | nextval()

setval() | | | | [Conditional Expressions](https://www.postgresql.org/docs/9.4/functions-conditional.html) | | | | | [Array Functions and Operators](https://www.postgresql.org/docs/9.4/functions-array.html) | | *All array functions* | | | [Aggregate Functions](https://www.postgresql.org/docs/9.4/functions-aggregate.html) | | | | | [Subquery Expressions](https://www.postgresql.org/docs/9.4/functions-subquery.html) | | | | | [Row and Array Comparisons](https://www.postgresql.org/docs/9.4/functions-comparisons.html) | | | | | [Set Returning Functions](https://www.postgresql.org/docs/9.4/functions-srf.html) | generate_series | | | -| [System Information Functions](https://www.postgresql.org/docs/9.4/functions-info.html) | | *All session information functions*

*All access privilege inquiry functions*

*All schema visibility inquiry functions*

*All system catalog information functions*

*All comment information functions*

*All transaction ids and snapshots* | | -| [System Administration Functions](https://www.postgresql.org/docs/9.4/functions-admin.html) | set_config

pg_cancel_backend

pg_terminate_backend

pg_reload_conf

pg_rotate_logfile

pg_start_backup

pg_stop_backup

pg_size_pretty

pg_ls_dir

pg_read_file

pg_stat_file | current_setting

*All database object size functions* | **Note:** The function `pg_column_size` displays bytes required to store the value, possibly with TOAST compression. | -| [XML Functions](https://www.postgresql.org/docs/9.4/functions-xml.html) and function-like expressions | | cursor_to_xml(cursor refcursor, count int, nulls boolean, tableforest boolean, targetns text)

cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest boolean, targetns text)

database_to_xml(nulls boolean, tableforest boolean, targetns text)

database_to_xmlschema(nulls boolean, tableforest boolean, targetns text)

database_to_xml_and_xmlschema( nulls boolean, tableforest boolean, targetns text)

query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xmlschema(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xml_and_xmlschema( query text, nulls boolean, tableforest boolean, targetns text)

schema_to_xml(schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xml_and_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

table_to_xml(tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xml_and_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

xmlagg(xml)

xmlconcat(xml[, ...])

xmlelement(name name [, xmlattributes(value [AS attname] [, ... ])] [, content, ...])

xmlexists(text, xml)

xmlforest(content [AS name] [, ...])

xml_is_well_formed(text)

xml_is_well_formed_document(text)

xml_is_well_formed_content(text)

xmlparse ( { DOCUMENT \| CONTENT } value)

xpath(text, xml)

xpath(text, xml, text[])

xpath_exists(text, xml)

xpath_exists(text, xml, text[])

xmlpi(name target [, content])

xmlroot(xml, version text \| no value [, standalone yes\|no\|no value])

xmlserialize ( { DOCUMENT \| CONTENT } value AS type )

xml(text)

text(xml)

xmlcomment(xml)

xmlconcat2(xml, xml) | | +| [System Information Functions](https://www.postgresql.org/docs/9.4/functions-info.html) | | *All session information functions*

*All access privilege inquiry functions*

*All schema visibility inquiry functions*

*All system catalog information functions*

*All comment information functions*

*All transaction ids and snapshots* | | +| [System Administration Functions](https://www.postgresql.org/docs/9.4/functions-admin.html) | set_config

pg_cancel_backend

pg_terminate_backend

pg_reload_conf

pg_rotate_logfile

pg_start_backup

pg_stop_backup

pg_size_pretty

pg_ls_dir

pg_read_file

pg_stat_file | current_setting

*All database object size functions* | **Note:** The function `pg_column_size` displays bytes required to store the value, possibly with TOAST compression. | +| [XML Functions](https://www.postgresql.org/docs/9.4/functions-xml.html) and function-like expressions | | cursor_to_xml(cursor refcursor, count int, nulls boolean, tableforest boolean, targetns text)

cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest boolean, targetns text)

database_to_xml(nulls boolean, tableforest boolean, targetns text)

database_to_xmlschema(nulls boolean, tableforest boolean, targetns text)

database_to_xml_and_xmlschema( nulls boolean, tableforest boolean, targetns text)

query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xmlschema(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xml_and_xmlschema( query text, nulls boolean, tableforest boolean, targetns text)

schema_to_xml(schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xml_and_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

table_to_xml(tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xml_and_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

xmlagg(xml)

xmlconcat(xml[, ...])

xmlelement(name name [, xmlattributes(value [AS attname] [, ... ])] [, content, ...])

xmlexists(text, xml)

xmlforest(content [AS name] [, ...])

xml_is_well_formed(text)

xml_is_well_formed_document(text)

xml_is_well_formed_content(text)

xmlparse ( { DOCUMENT \| CONTENT } value)

xpath(text, xml)

xpath(text, xml, text[])

xpath_exists(text, xml)

xpath_exists(text, xml, text[])

xmlpi(name target [, content])

xmlroot(xml, version text \| no value [, standalone yes\|no\|no value])

xmlserialize ( { DOCUMENT \| CONTENT } value AS type )

xml(text)

text(xml)

xmlcomment(xml)

xmlconcat2(xml, xml) | | @@ -149,12 +149,46 @@ The following built-in advanced aggregate functions are WarehousePG extensions o > **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [WarehousePG MADlib Extension for Analytics](../analytics/madlib.md) in the *WarehousePG Reference Guide*. -**Advanced Aggregate Functions** - | Function | Return Type | Full Syntax | Description | | --- | --- | --- | --- | -| `MEDIAN (*expr*)` | `timestamp, timestamptz, interval, float` | `MEDIAN (*expression*)`

*Example:*

`SELECT departmzent_id, MEDIAN(salary)
FROM employees
GROUP BY department_id;` | Can take a two-dimensional array as input. Treats such arrays as matrices. | -| `sum(array[])` | `smallint[], int[], bigint[], float[]` | `sum(array[[1,2],[3,4]])`

*Example:*

`CREATE TABLE mymatrix (myvalue int[]);
INSERT INTO mymatrix
VALUES (array[[1,2],[3,4]]);
INSERT INTO mymatrix
VALUES (array[[0,1],[1,0]]);
SELECT sum(myvalue) FROM mymatrix;
sum
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
{{1,3},{4,4}}` | Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. | | `pivot_sum (label[], label, expr)` | `int[], bigint[], float[]` | `pivot_sum( array['A1','A2'], attr, value)` | A pivot aggregation using sum to resolve duplicate entries. | | `unnest (array[])` | set of `anyelement` | `unnest( array['one', 'row', 'per', 'item'])` | Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudo-type](https://www.postgresql.org/docs/9.4/datatype-pseudo.html) in PostgreSQL. | +### MEDIAN (*expr*) + +Return type: `timestamp, timestamptz, interval, float` + +Full syntax: `MEDIAN (*expression*)` + +Can take a two-dimensional array as input. Treats such arrays as matrices. + +Example: + +``` +SELECT departmzent_id, MEDIAN(salary) + FROM employees +GROUP BY department_id; +``` + +### sum(array[]) + +Return type: `smallint[], int[], bigint[], float[]` + +Full syntax: `sum(array[[1,2],[3,4]])` + +Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. + +Example: + +``` +CREATE TABLE mymatrix (myvalue int[]); +INSERT INTO mymatrix + VALUES (array[[1,2],[3,4]]); +INSERT INTO mymatrix + VALUES (array[[0,1],[1,0]]); +SELECT sum(myvalue) FROM mymatrix; + sum +------------- + {{1,3},{4,4}} +``` + diff --git a/docs/whpg/6x/admin_guide/query/json-data.md b/docs/whpg/6x/admin_guide/query/json-data.md index b15d308..2f63cf2 100644 --- a/docs/whpg/6x/admin_guide/query/json-data.md +++ b/docs/whpg/6x/admin_guide/query/json-data.md @@ -379,10 +379,10 @@ Operators that require the `jsonb` data type as the left operand are described i | Operator | Right Operand Type | Description | Example | | --- | --- | --- | --- | -| `@>` | `jsonb` | Does the left JSON value contain within it the right value? | `'{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb` | -| `<@` | `jsonb` | Is the left JSON value contained within the right value? | `'{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb` | -| `?` | `text` | Does the key/element string exist within the JSON value? | `'{"a":1, "b":2}'::jsonb ? 'b'` | -| `?\|` | `text[]` | Do any of these key/element strings exist? | `'{"a":1, "b":2, "c":3}'::jsonb ?\| array['b', 'c']` | +| `@>` | `jsonb` | Does the left JSON value contain within it the right value? | `'{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb` | +| `<@` | `jsonb` | Is the left JSON value contained within the right value? | `'{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb` | +| `?` | `text` | Does the key/element string exist within the JSON value? | `'{"a":1, "b":2}'::jsonb ? 'b'` | +| `?\|` | `text[]` | Do any of these key/element strings exist? | `'{"a":1, "b":2, "c":3}'::jsonb ?\| array['b', 'c']` | | `?&` | `text[]` | Do all of these key/element strings exist? | `'["a", "b"]'::jsonb ?& array['a', 'b']` | The standard comparison operators in the following table are available only for the `jsonb` data type, not for the `json` data type. They follow the ordering rules for B-tree operations described in [jsonb Indexing](#topic_aqt_1tw_mq). @@ -409,12 +409,12 @@ This table describes the functions that create `json` data type values. (Current | Function | Description | Example | Example Result | | --- | --- | --- | --- | | `to_json(anyelement)` | Returns the value as a JSON object. Arrays and composites are processed recursively and are converted to arrays and objects. If the input contains a cast from the type to `json`, the cast function is used to perform the conversion; otherwise, a JSON scalar value is produced. For any scalar type other than a number, a Boolean, or a null value, the text representation will be used, properly quoted and escaped so that it is a valid JSON string. | `to_json('Fred said "Hi."'::text)` | `"Fred said \"Hi.\""` | -| `array_to_json(anyarray [, pretty_bool])` | Returns the array as a JSON array. A multidimensional array becomes a JSON array of arrays.

Line feeds will be added between dimension-1 elements if `pretty_bool` is true. | `array_to_json('{{1,5},{99,100}}'::int[])` | `[[1,5],[99,100]]` | -| `row_to_json(record [, pretty_bool])` | Returns the row as a JSON object.

Line feeds will be added between level-1 elements if `pretty_bool` is true. | `row_to_json(row(1,'foo'))` | `{"f1":1,"f2":"foo"}` | +| `array_to_json(anyarray [, pretty_bool])` | Returns the array as a JSON array. A multidimensional array becomes a JSON array of arrays.

Line feeds will be added between dimension-1 elements if `pretty_bool` is true. | `array_to_json('{​{1,5},{99,100}}'::int[])` | `[[1,5],[99,100]]` | +| `row_to_json(record [, pretty_bool])` | Returns the row as a JSON object.

Line feeds will be added between level-1 elements if `pretty_bool` is true. | `row_to_json(row(1,'foo'))` | `{"f1":1,"f2":"foo"}` | | `json_build_array(VARIADIC "any"`) | Builds a possibly-heterogeneously-typed JSON array out of a `VARIADIC` argument list. | `json_build_array(1,2,'3',4,5)` | `[1, 2, "3", 4, 5]` | -| `json_build_object(VARIADIC "any")` | Builds a JSON object out of a `VARIADIC` argument list. The argument list is taken in order and converted to a set of key/value pairs. | `json_build_object('foo',1,'bar',2)` | `{"foo": 1, "bar": 2}` | -| `json_object(text[])` | Builds a JSON object out of a text array. The array must be either a one or a two dimensional array.

The one dimensional array must have an even number of elements. The elements are taken as key/value pairs.

For a two dimensional array, each inner array must have exactly two elements, which are taken as a key/value pair. | `json_object('{a, 1, b, "def", c, 3.5}')`

`json_object('{{a, 1},{b, "def"},{c, 3.5}}')` | `{"a": "1", "b": "def", "c": "3.5"}` | -| `json_object(keys text[], values text[])` | Builds a JSON object out of a text array. This form of `json_object` takes keys and values pairwise from two separate arrays. In all other respects it is identical to the one-argument form. | `json_object('{a, b}', '{1,2}')` | `{"a": "1", "b": "2"}` | +| `json_build_object(VARIADIC "any")` | Builds a JSON object out of a `VARIADIC` argument list. The argument list is taken in order and converted to a set of key/value pairs. | `json_build_object('foo',1,'bar',2)` | `{"foo": 1, "bar": 2}` | +| `json_object(text[])` | Builds a JSON object out of a text array. The array must be either a one or a two dimensional array.

The one dimensional array must have an even number of elements. The elements are taken as key/value pairs.

For a two dimensional array, each inner array must have exactly two elements, which are taken as a key/value pair. | `json_object('{a, 1, b, "def", c, 3.5}')`

`json_object('{​{a, 1},{b, "def"},{c, 3.5}}')` | `{"a": "1", "b": "def", "c": "3.5"}` | +| `json_object(keys text[], values text[])` | Builds a JSON object out of a text array. This form of `json_object` takes keys and values pairwise from two separate arrays. In all other respects it is identical to the one-argument form. | `json_object('{a, b}', '{1,2}')` | `{"a": "1", "b": "2"}` | > **Note** `array_to_json` and `row_to_json` have the same behavior as `to_json` except for offering a pretty-printing option. The behavior described for `to_json` likewise applies to each individual value converted by the other JSON creation functions. @@ -435,27 +435,234 @@ This table shows the functions aggregate records to an array of JSON objects and ### JSON Processing Functions -This table shows the functions that are available for processing `json` and `jsonb` values. +This section describes the functions that are available for processing `json` and `jsonb` values. Many of these processing functions and operators convert Unicode escapes in JSON strings to the appropriate single character. This is a not an issue if the input data type is `jsonb`, because the conversion was already done. However, for `json` data type input, this might result in an error being thrown. See [About JSON Data](#topic_upc_tcs_fz). -**JSON Processing Functions** - -| Function | Return Type | Description | Example | Example Result | -| --- | --- | --- | --- | --- | -| `json_array_length(json)`

`jsonb_array_length(jsonb)` | `int` | Returns the number of elements in the outermost JSON array. | `json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]')` | `5` | -| `json_each(json)`

`jsonb_each(jsonb)` | `setof key text, value json`

`setof key text, value jsonb` | Expands the outermost JSON object into a set of key/value pairs. | `select * from json_each('{"a":"foo", "b":"bar"}')` | ` key \| value
-----+-------
a \| "foo"
b \| "bar"` | -| `json_each_text(json)`

`jsonb_each_text(jsonb)` | `setof key text, value text` | Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. | `select * from json_each_text('{"a":"foo", "b":"bar"}')` | ` key \| value
-----+-------
a \| foo
b \| bar` | -| `json_extract_path(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` | `json`

`jsonb` | Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). | `json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4')` | `{"f5":99,"f6":"foo"}` | -| `json_extract_path_text(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` | `text` | Returns the JSON value pointed to by `path_elems` as text. Equivalent to `#>>` operator. | `json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4', 'f6')` | `foo` | -| `json_object_keys(json)`

`jsonb_object_keys(jsonb)` | `setof text` | Returns set of keys in the outermost JSON object. | `json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}}')` | ` json_object_keys
------------------
f1
f2` | -| `json_populate_record(base anyelement, from_json json)`

`jsonb_populate_record(base anyelement, from_json jsonb)` | `anyelement` | Expands the object in `from_json` to a row whose columns match the record type defined by base. See [Note 1](#notes-on-json-examples). | `select * from json_populate_record(null::myrowtype, '{"a":1,"b":2}')` | ` a \| b
---+---
1 \| 2` | -| `json_populate_recordset(base anyelement, from_json json)`

`jsonb_populate_recordset(base anyelement, from_json jsonb)` | `setof anyelement` | Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by base. See [Note 1](#notes-on-json-examples). | `select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]')` | ` a \| b
---+---
1 \| 2
3 \| 4` | -| `json_array_elements(json)`

`jsonb_array_elements(jsonb`) | `setof json`

`setof jsonb` | Expands a JSON array to a set of JSON values. | `select * from json_array_elements('[1,true, [2,false]]')` | ` value
-----------
1
true
[2,false]` | -| `json_array_elements_text(json)`

`jsonb_array_elements_text(jsonb)` | `setof text` | Expands a JSON array to a set of `text` values. | `select * from json_array_elements_text('["foo", "bar"]')` | ` value
-----------
foo
bar` | -| `json_typeof(json)`

`jsonb_typeof(jsonb)` | `text` | Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. See [Note 2](#notes-on-json-examples). | `json_typeof('-123.4')` | `number` | -| `json_to_record(json)`

`jsonb_to_record(jsonb)` | `record` | Builds an arbitrary record from a JSON object. See [Note 1](#notes-on-json-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a int, b text, d text)` | ` a \| b \| d
---+---------+---
1 \| [1,2,3] \|` | -| `json_to_recordset(json)`

`jsonb_to_recordset(jsonb)` | `setof record` | Builds an arbitrary set of records from a JSON array of objects See [Note 1](#notes-on-json-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text);` | ` a \| b
---+-----
1 \| foo
2 \|` | +#### json_array_length() / jsonb_array_length() + +Syntax: `json_array_length(json)` / `jsonb_array_length(jsonb)` + +Return type: `int` + +Returns the number of elements in the outermost JSON array. + +Example: + +``` +json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]') +5 +``` + +#### json_each() / jsonb_each() + +Syntax: `json_each(json)` / `jsonb_each(jsonb)` + +Return type: `setof key text, value json` / `setof key text, value jsonb` + +Expands the outermost JSON object into a set of key/value pairs. + +Example: + +``` +select * from json_each('{"a":"foo", "b":"bar"}') + key | value +-----+------- + a | "foo" + b | "bar" +``` + +#### json_each_text() / jsonb_each_text() + +Syntax: `json_each_text(json)` / `jsonb_each_text(jsonb)` + +Return type: `setof key text, value text` + +Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. + +Example: + +``` +select * from json_each_text('{"a":"foo", "b":"bar"}') + key | value +-----+------- + a | foo + b | bar +``` + +#### json_extract_path() / jsonb_extract_path() + +Syntax: `json_extract_path(from_json json, VARIADIC path_elems text[])` / `jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` + +Return type: `json` / `jsonb` + +Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). + +Example: + +``` +json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4') +{"f5":99,"f6":"foo"} +``` + +#### json_extract_path_text() / jsonb_extract_path_text() + +Syntax: `json_extract_path_text(from_json json, VARIADIC path_elems text[])` / `jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` + +Return type: `text` + +Returns the JSON value pointed to by `path_elems` as text. Equivalent to `#>>` operator. + +Example: + +``` +json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4', 'f6') +foo +``` + +#### json_object_keys() / jsonb_object_keys() + +Syntax: `json_object_keys(json)` / `jsonb_object_keys(jsonb)` + +Return type: `setof text` + +Returns set of keys in the outermost JSON object. + +Example: + +``` +json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}}') + json_object_keys +------------------ + f1 + f2 +``` + +#### json_populate_record() / jsonb_populate_record() + +Syntax: `json_populate_record(base anyelement, from_json json)` / `jsonb_populate_record(base anyelement, from_json jsonb)` + +Return type: `anyelement` + +Expands the object in `from_json` to a row whose columns match the record type defined by base. See [Note 1](#notes-on-json-examples). + +Example: + +``` +select * from json_populate_record(null::myrowtype, '{"a":1,"b":2}') + a | b +---+--- + 1 | 2 +``` + +#### json_populate_recordset() / jsonb_populate_recordset() + +Syntax: `json_populate_recordset(base anyelement, from_json json)` / `jsonb_populate_recordset(base anyelement, from_json jsonb)` + +Return type: `setof anyelement` + +Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by base. See [Note 1](#notes-on-json-examples). + +Example: + +``` +select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]') + a | b +---+--- + 1 | 2 + 3 | 4 +``` + +#### json_array_elements() / jsonb_array_elements() + +Syntax: `json_array_elements(json)` / `jsonb_array_elements(jsonb)` + +Return type: `setof json` / `setof jsonb` + +Expands a JSON array to a set of JSON values. + +Example: + +``` +select * from json_array_elements('[1,true, [2,false]]') + value +----------- + 1 + true + [2,false] +``` + +#### json_array_elements_text() / jsonb_array_elements_text() + +Syntax: `json_array_elements_text(json)` / `jsonb_array_elements_text(jsonb)` + +Return type: `setof text` + +Expands a JSON array to a set of `text` values. + +Example: + +``` +select * from json_array_elements_text('["foo", "bar"]') + value +----------- + foo + bar +``` + +#### json_typeof() / jsonb_typeof() + +Syntax: `json_typeof(json)` / `jsonb_typeof(jsonb)` + +Return type: `text` + +Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. See [Note 2](#notes-on-json-examples). + +Example: + +``` +json_typeof('-123.4') +number +``` + +#### json_to_record() / jsonb_to_record() + +Syntax: `json_to_record(json)` / `jsonb_to_record(jsonb)` + +Return type: `record` + +Builds an arbitrary record from a JSON object. See [Note 1](#notes-on-json-examples). + +As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. + +Example: + +``` +select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a int, b text, d text) + a | b | d +---+---------+--- + 1 | [1,2,3] | +``` + +#### json_to_recordset() / jsonb_to_recordset() + +Syntax: `json_to_recordset(json)` / `jsonb_to_recordset(jsonb)` + +Return type: `setof record` + +Builds an arbitrary set of records from a JSON array of objects See [Note 1](#notes-on-json-examples). + +As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. + +Example: + +``` +select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text); + a | b +---+----- + 1 | foo + 2 | +``` #### Notes on JSON examples diff --git a/docs/whpg/6x/install_guide/install_modules.md b/docs/whpg/6x/install_guide/install_modules.md index ceca731..dfe156d 100644 --- a/docs/whpg/6x/install_guide/install_modules.md +++ b/docs/whpg/6x/install_guide/install_modules.md @@ -21,8 +21,30 @@ $ psql -d testdb -c 'DROP EXTENSION dblink;' You can register the following modules in this manner: -| - [btree_gin](../ref_guide/modules/bundled/btree_gin.md)
- [citext](../ref_guide/modules/bundled/citext.md)
- [dblink](../ref_guide/modules/bundled/dblink.md)
- [diskquota](../ref_guide/modules/extensions/diskquota.md)
- [fuzzystrmatch](../ref_guide/modules/bundled/fuzzystrmatch.md)
- [gp_array_agg](../ref_guide/modules/bundled/gp_array_agg.md)
- [gp_check_functions](../ref_guide/modules/bundled/gp_check_functions.md)
- [gp_parallel_retrieve_cursor](../ref_guide/modules/bundled/gp_parallel_retrieve_cursor.md)
- [gp_percentile_agg](../ref_guide/modules/bundled/gp_percentile_agg.md)
- [gp_sparse_vector](../ref_guide/modules/bundled/gp_sparse_vector.md)
- [greenplum_fdw](../ref_guide/modules/bundled/greenplum_fdw.md)
- [hstore](../ref_guide/modules/bundled/hstore.md) | - [ip4r](../ref_guide/modules/bundled/ip4r.md)
- [ltree](../ref_guide/modules/bundled/ltree.md)
- [orafce](../ref_guide/modules/bundled/orafce_ref.md) (WarehousePG only)
- [pageinspect](../ref_guide/modules/bundled/pageinspect.md)
- [pg_trgm](../ref_guide/modules/bundled/pg_trgm.md)
- [pgcrypto](../ref_guide/modules/bundled/pgcrypto.md)
- [postgres_fdw](../ref_guide/modules/bundled/postgres_fdw.md)
- [postgresql-hll](../ref_guide/modules/extensions/postgresql-hll.md)
- [sslinfo](../ref_guide/modules/bundled/sslinfo.md)
- [tablefunc](../ref_guide/modules/bundled/tablefunc.md)
- [timestamp9](../ref_guide/modules/bundled/timestamp9.md)
- [uuid-ossp](../ref_guide/modules/bundled/uuid-ossp.md) | -| --- | --- | +- [btree_gin](../ref_guide/modules/bundled/btree_gin.md) +- [citext](../ref_guide/modules/bundled/citext.md) +- [dblink](../ref_guide/modules/bundled/dblink.md) +- [diskquota](../ref_guide/modules/extensions/diskquota.md) +- [fuzzystrmatch](../ref_guide/modules/bundled/fuzzystrmatch.md) +- [gp_array_agg](../ref_guide/modules/bundled/gp_array_agg.md) +- [gp_check_functions](../ref_guide/modules/bundled/gp_check_functions.md) +- [gp_parallel_retrieve_cursor](../ref_guide/modules/bundled/gp_parallel_retrieve_cursor.md) +- [gp_percentile_agg](../ref_guide/modules/bundled/gp_percentile_agg.md) +- [gp_sparse_vector](../ref_guide/modules/bundled/gp_sparse_vector.md) +- [greenplum_fdw](../ref_guide/modules/bundled/greenplum_fdw.md) +- [hstore](../ref_guide/modules/bundled/hstore.md) +- [ip4r](../ref_guide/modules/bundled/ip4r.md) +- [ltree](../ref_guide/modules/bundled/ltree.md) +- [orafce](../ref_guide/modules/bundled/orafce_ref.md) (WarehousePG only) +- [pageinspect](../ref_guide/modules/bundled/pageinspect.md) +- [pg_trgm](../ref_guide/modules/bundled/pg_trgm.md) +- [pgcrypto](../ref_guide/modules/bundled/pgcrypto.md) +- [postgres_fdw](../ref_guide/modules/bundled/postgres_fdw.md) +- [postgresql-hll](../ref_guide/modules/extensions/postgresql-hll.md) +- [sslinfo](../ref_guide/modules/bundled/sslinfo.md) +- [tablefunc](../ref_guide/modules/bundled/tablefunc.md) +- [timestamp9](../ref_guide/modules/bundled/timestamp9.md) +- [uuid-ossp](../ref_guide/modules/bundled/uuid-ossp.md) For additional information about the modules supplied with WarehousePG, refer to [Additional Supplied Modules](../ref_guide/modules/index.md) in the *WarehousePG Reference Guide*. diff --git a/docs/whpg/6x/install_guide/installation_utilities.md b/docs/whpg/6x/install_guide/installation_utilities.md index a404ef5..c0014ee 100644 --- a/docs/whpg/6x/install_guide/installation_utilities.md +++ b/docs/whpg/6x/install_guide/installation_utilities.md @@ -9,7 +9,16 @@ For a full reference of all WarehousePG utilities, see the [WarehousePG Utility The following WarehousePG management utilities are located in `$GPHOME/bin`. -| - [gpactivatestandby](../ref_guide/utility_guide/reference/gpactivatestandby.md)
- [gpaddmirrors](../ref_guide/utility_guide/reference/gpaddmirrors.md)
- [gpcheckperf](../ref_guide/utility_guide/reference/gpcheckperf.md)
- [gpdeletesystem](../ref_guide/utility_guide/reference/gpdeletesystem.md)
- [gpinitstandby](../ref_guide/utility_guide/reference/gpinitstandby.md) | - [gpinitsystem](../ref_guide/utility_guide/reference/gpinitsystem.md)
- [gpscp](../ref_guide/utility_guide/reference/gpscp.md)
- [gpssh](../ref_guide/utility_guide/reference/gpssh.md)
- [gpssh-exkeys](../ref_guide/utility_guide/reference/gpssh-exkeys.md)
- [gpstart](../ref_guide/utility_guide/reference/gpstart.md)
- [gpstop](../ref_guide/utility_guide/reference/gpstop.md) | -| --- | --- | +- [gpactivatestandby](../ref_guide/utility_guide/reference/gpactivatestandby.md) +- [gpaddmirrors](../ref_guide/utility_guide/reference/gpaddmirrors.md) +- [gpcheckperf](../ref_guide/utility_guide/reference/gpcheckperf.md) +- [gpdeletesystem](../ref_guide/utility_guide/reference/gpdeletesystem.md) +- [gpinitstandby](../ref_guide/utility_guide/reference/gpinitstandby.md) +- [gpinitsystem](../ref_guide/utility_guide/reference/gpinitsystem.md) +- [gpscp](../ref_guide/utility_guide/reference/gpscp.md) +- [gpssh](../ref_guide/utility_guide/reference/gpssh.md) +- [gpssh-exkeys](../ref_guide/utility_guide/reference/gpssh-exkeys.md) +- [gpstart](../ref_guide/utility_guide/reference/gpstart.md) +- [gpstop](../ref_guide/utility_guide/reference/gpstop.md) **Parent topic:** [Installing and Upgrading WarehousePG](index.md) diff --git a/docs/whpg/6x/install_guide/platform-requirements.md b/docs/whpg/6x/install_guide/platform-requirements.md index c7f5624..459c0ff 100644 --- a/docs/whpg/6x/install_guide/platform-requirements.md +++ b/docs/whpg/6x/install_guide/platform-requirements.md @@ -158,8 +158,10 @@ The following table lists minimum recommended specifications for hardware server |---|---| | Minimum CPU | Any x86_64 compatible CPU | | Minimum Memory | 16 GB RAM per server | -| Disk Space Requirements |
  • 150MB per host for WarehousePG installation
  • Approximately 300MB per segment instance for metadata
  • Cap disk capacity at 70% full to accommodate temporary files and prevent performance degradation
| -| Network Requirements |
  • 10 Gigabit Ethernet within the array
  • NIC bonding is recommended when multiple interfaces are present
  • WarehousePG can use either IPV4 or IPV6 protocols.
| +| Disk Space Requirements | - 150MB per host for WarehousePG installation
- Approximately 300MB per segment instance for metadata
- Cap disk capacity at 70% full to accommodate temporary files and prevent performance degradation | +| Network Requirements | - 10 Gigabit Ethernet within the array
- NIC bonding is recommended when multiple interfaces are present
- WarehousePG can use either IPV4 or IPV6 protocols. | + + ## Storage diff --git a/docs/whpg/6x/ref_guide/function-summary.md b/docs/whpg/6x/ref_guide/function-summary.md index 5424b64..6b74c64 100644 --- a/docs/whpg/6x/ref_guide/function-summary.md +++ b/docs/whpg/6x/ref_guide/function-summary.md @@ -27,13 +27,29 @@ WarehousePG supports built-in functions and operators including analytic functio ## WarehousePG Function Types -WarehousePG evaluates functions and operators used in SQL expressions. Some functions and operators are only allowed to run on the coordinator since they could lead to inconsistencies in WarehousePG segment instances. This table describes the WarehousePG Function Types. +WarehousePG evaluates functions and operators used in SQL expressions. Some functions and operators are only allowed to run on the coordinator since they could lead to inconsistencies in WarehousePG segment instances. The following describes the WarehousePG function types. -| Function Type | WarehousePG Support | Description | Comments | -| ------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| IMMUTABLE | Yes | Relies only on information directly in its argument list. Given the same argument values, always returns the same result. |   | -| STABLE | Yes, in most cases | Within a single table scan, returns the same result for same argument values, but results change across SQL statements. | Results depend on database lookups or parameter values. `current_timestamp` family of functions is `STABLE`; values do not change within an execution. | -| VOLATILE | Restricted | Function values can change within a single table scan. For example: `random()`, `timeofday()`. | Any function with side effects is volatile, even if its result is predictable. For example: `setval()`. | +### `IMMUTABLE` + +WarehousePG support: Yes + +Relies only on information directly in its argument list. Given the same argument values, always returns the same result. + +### `STABLE` + +WarehousePG support: Yes, in most cases + +Within a single table scan, returns the same result for same argument values, but results change across SQL statements. + +Results depend on database lookups or parameter values. The `current_timestamp` family of functions is `STABLE`, and values don't change within an execution. + +### `VOLATILE` + +WarehousePG support: Restricted + +Function values can change within a single table scan. For example, `random()` and `timeofday()`. + +Any function with side effects is volatile, even if its result is predictable. For example, `setval()`. In WarehousePG, data is divided up across segments — each segment is a distinct PostgreSQL database. To prevent inconsistent or unexpected results, do not run functions classified as `VOLATILE` at the segment level if they contain SQL commands or modify the database in any way. For example, functions such as `setval()` are not allowed to run on distributed data in WarehousePG because they can cause inconsistent data between segment instances. @@ -114,39 +130,158 @@ WarehousePG includes built-in functions and operators that create and manipulate ### JSON Operators -This table describes the operators that are available for use with the `json` and `jsonb` data types. +The following operators are available for use with the `json` and `jsonb` data types. + +#### `->` (int) + +Right operand type: `int` + +Get the JSON array element (indexed from zero). + +Example: + +``` +'[{"a":"foo"},{"b":"bar"},{"c":"baz"}]'::json->2 +{"c":"baz"} +``` + +#### `->` (text) + +Right operand type: `text` + +Get the JSON object field by key. + +Example: + +``` +'{"a": {"b":"foo"}}'::json->'a' +{"b":"foo"} +``` + +#### `->>` (int) + +Right operand type: `int` + +Get the JSON array element as `text`. + +Example: + +``` +'[1,2,3]'::json->>2 +3 +``` + +#### `->>` (text) -| Operator | Right Operand Type | Description | Example | Example Result | -| -------- | ------------------ | ------------------------------------------------ | -------------------------------------------------- | -------------- | -| `->` | `int` | Get the JSON array element (indexed from zero). | `'[{"a":"foo"},{"b":"bar"},{"c":"baz"}]'::json->2` | `{"c":"baz"}` | -| `->` | `text` | Get the JSON object field by key. | `'{"a": {"b":"foo"}}'::json->'a'` | `{"b":"foo"}` | -| `->>` | `int` | Get the JSON array element as `text`. | `'[1,2,3]'::json->>2` | `3` | -| `->>` | `text` | Get the JSON object field as `text`. | `'{"a":1,"b":2}'::json->>'b'` | `2` | -| `#>` | `text[]` | Get the JSON object at specified path. | `'{"a": {"b":{"c": "foo"}}}'::json#>'{a,b}`' | `{"c": "foo"}` | -| `#>>` | `text[]` | Get the JSON object at specified path as `text`. | `'{"a":[1,2,3],"b":[4,5,6]}'::json#>>'{a,2}'` | `3` | +Right operand type: `text` + +Get the JSON object field as `text`. + +Example: + +``` +'{"a":1,"b":2}'::json->>'b' +2 +``` + +#### `#>` + +Right operand type: `text[]` + +Get the JSON object at specified path. + +Example: + +``` +'{"a": {"b":{"c": "foo"}}}'::json#>'{a,b}' +{"c": "foo"} +``` + +#### `#>>` + +Right operand type: `text[]` + +Get the JSON object at specified path as `text`. + +Example: + +``` +'{"a":[1,2,3],"b":[4,5,6]}'::json#>>'{a,2}' +3 +``` > **Note** There are parallel variants of these operators for both the `json` and `jsonb` data types. The field, element, and path extraction operators return the same data type as their left-hand input (either `json` or `jsonb`), except for those specified as returning `text`, which coerce the value to `text`. The field, element, and path extraction operators return `NULL`, rather than failing, if the JSON input does not have the right structure to match the request; for example if no such element exists. -Operators that require the `jsonb` data type as the left operand are described in the following table. Many of these operators can be indexed by `jsonb` operator classes. For a full description of `jsonb` containment and existence semantics, see [jsonb Containment and Existence](../admin_guide/query/json-data.md#topic_isx_2tw_mq)in the *WarehousePG Administrator Guide*. For information about how these operators can be used to effectively index `jsonb` data, see [jsonb Indexing](../admin_guide/query/json-data.md#topic_aqt_1tw_mq)in the *WarehousePG Administrator Guide*. +Operators that require the `jsonb` data type as the left operand are described next. Many of these operators can be indexed by `jsonb` operator classes. For a full description of `jsonb` containment and existence semantics, see [jsonb Containment and Existence](../admin_guide/query/json-data.md#topic_isx_2tw_mq)in the *WarehousePG Administrator Guide*. For information about how these operators can be used to effectively index `jsonb` data, see [jsonb Indexing](../admin_guide/query/json-data.md#topic_aqt_1tw_mq)in the *WarehousePG Administrator Guide*. + +#### `@>` + +Right operand type: `jsonb` + +Does the left JSON value contain within it the right value? + +Example: + +``` +'{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb +``` + +#### `<@` + +Right operand type: `jsonb` + +Is the left JSON value contained within the right value? + +Example: + +``` +'{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb +``` + +#### `?` + +Right operand type: `text` + +Does the key/element string exist within the JSON value? + +Example: + +``` +'{"a":1, "b":2}'::jsonb ? 'b' +``` + +#### `?|` + +Right operand type: `text[]` + +Do any of these key/element strings exist? + +Example: + +``` +'{"a":1, "b":2, "c":3}'::jsonb ?| array['b', 'c'] +``` + +#### `?&` -| Operator | Right Operand Type | Description | Example | | | -| -------- | ------------------ | ----------------------------------------------------------- | --------------------------------------------- | ---------------------------------- | ----------------- | -| `@>` | `jsonb` | Does the left JSON value contain within it the right value? | `'{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb` | | | -| `<@` | `jsonb` | Is the left JSON value contained within the right value? | `'{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb` | | | -| `?` | `text` | Does the key/element string exist within the JSON value? | `'{"a":1, "b":2}'::jsonb ? 'b'` | | | -| \`? | \` | `text[]` | Do any of these key/element strings exist? | \`'{"a":1, "b":2, "c":3}'::jsonb ? | array['b', 'c']\` | -| `?&` | `text[]` | Do all of these key/element strings exist? | `'["a", "b"]'::jsonb ?& array['a', 'b']` | | | +Right operand type: `text[]` -The standard comparison operators in the following table are available only for the `jsonb` data type, not for the `json` data type. They follow the ordering rules for B-tree operations described in [jsonb Indexing](../admin_guide/query/json-data.md#topic_aqt_1tw_mq)in the *WarehousePG Administrator Guide*. +Do all of these key/element strings exist? -| Operator | Description | -| ------------ | ------------------------ | -| `<` | less than | -| `>` | greater than | -| `<=` | less than or equal to | -| `>=` | greater than or equal to | -| `=` | equal | -| `<>` or `!=` | not equal | +Example: + +``` +'["a", "b"]'::jsonb ?& array['a', 'b'] +``` + +The following standard comparison operators are available only for the `jsonb` data type, not for the `json` data type. They follow the ordering rules for B-tree operations described in [jsonb Indexing](../admin_guide/query/json-data.md#topic_aqt_1tw_mq)in the *WarehousePG Administrator Guide*. + +- `<` (less than) +- `>` (greater than) +- `<=` (less than or equal to) +- `>=` (greater than or equal to) +- `=` (equal) +- `<>` or `!=` (not equal) > **Note** The `!=` operator is converted to `<>` in the parser stage. It is not possible to implement `!=` and `<>` operators that do different things. @@ -158,17 +293,89 @@ The standard comparison operators in the following table are available only for ### JSON Creation Functions -This table describes the functions that create `json` data type values. (Currently, there are no equivalent functions for `jsonb`, but you can cast the result of one of these functions to `jsonb`.) +The following functions create `json` data type values. (Currently, there are no equivalent functions for `jsonb`, but you can cast the result of one of these functions to `jsonb`.) + +#### `to_json(anyelement)` + +Returns the value as a JSON object. Arrays and composites are processed recursively and are converted to arrays and objects. If the input contains a cast from the type to `json`, the cast function is used to perform the conversion. Otherwise, a JSON scalar value is produced. For any scalar type other than a number, a Boolean, or a null value, the text representation is used, properly quoted and escaped so that it is a valid JSON string. + +Example: + +``` +to_json('Fred said "Hi."'::text) +"Fred said \"Hi.\"" +``` + +#### `array_to_json(anyarray [, pretty_bool])` + +Returns the array as a JSON array. A multidimensional array becomes a JSON array of arrays. Line feeds are added between dimension-1 elements if `pretty_bool` is true. + +Example: + +``` +array_to_json('{ {1,5},{99,100}}'::int[]) +[[1,5],[99,100]] +``` + +#### `row_to_json(record [, pretty_bool])` + +Returns the row as a JSON object. Line feeds are added between level-1 elements if `pretty_bool` is true. + +Example: + +``` +row_to_json(row(1,'foo')) +{"f1":1,"f2":"foo"} +``` + +#### `json_build_array(VARIADIC "any")` + +Builds a possibly heterogeneously typed JSON array out of a `VARIADIC` argument list. + +Example: + +``` +json_build_array(1,2,'3',4,5) +[1, 2, "3", 4, 5] +``` + +#### `json_build_object(VARIADIC "any")` + +Builds a JSON object out of a `VARIADIC` argument list. The argument list is taken in order and converted to a set of key/value pairs. + +Example: -| Function | Description | Example | Example Result | -| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ------------------------------------ | -| `to_json(anyelement)` | Returns the value as a JSON object. Arrays and composites are processed recursively and are converted to arrays and objects. If the input contains a cast from the type to `json`, the cast function is used to perform the conversion; otherwise, a JSON scalar value is produced. For any scalar type other than a number, a Boolean, or a null value, the text representation will be used, properly quoted and escaped so that it is a valid JSON string. | `to_json('Fred said "Hi."'::text)` | `"Fred said \"Hi.\""` | -| `array_to_json(anyarray [, pretty_bool])` | Returns the array as a JSON array. A multidimensional array becomes a JSON array of arrays. Line feeds will be added between dimension-1 elements if `pretty_bool` is true. | `array_to_json('{ {1,5},{99,100}}'::int[])` | `[[1,5],[99,100]]` | -| `row_to_json(record [, pretty_bool])` | Returns the row as a JSON object. Line feeds will be added between level-1 elements if `pretty_bool` is true. | `row_to_json(row(1,'foo'))` | `{"f1":1,"f2":"foo"}` | -| `json_build_array(VARIADIC "any")` | Builds a possibly-heterogeneously-typed JSON array out of a `VARIADIC` argument list. | `json_build_array(1,2,'3',4,5)` | `[1, 2, "3", 4, 5]` | -| `json_build_object(VARIADIC "any")` | Builds a JSON object out of a `VARIADIC` argument list. The argument list is taken in order and converted to a set of key/value pairs. | `json_build_object('foo',1,'bar',2)` | `{"foo": 1, "bar": 2}` | -| `json_object(text[])` | Builds a JSON object out of a text array. The array must be either a one or a two dimensional array.

The one dimensional array must have an even number of elements. The elements are taken as key/value pairs.

For a two dimensional array, each inner array must have exactly two elements, which are taken as a key/value pair. | `json_object('{a, 1, b, "def", c, 3.5}')`

`json_object('{ {a, 1},{b, "def"},{c, 3.5}}')` | `{"a": "1", "b": "def", "c": "3.5"}` | -| `json_object(keys text[], values text[])` | Builds a JSON object out of a text array. This form of `json_object` takes keys and values pairwise from two separate arrays. In all other respects it is identical to the one-argument form. | `json_object('{a, b}', '{1,2}')` | `{"a": "1", "b": "2"}` | +``` +json_build_object('foo',1,'bar',2) +{"foo": 1, "bar": 2} +``` + +#### `json_object(text[])` + +Builds a JSON object out of a text array. The array must be either a one or a two dimensional array. + +The one dimensional array must have an even number of elements. The elements are taken as key/value pairs. + +For a two dimensional array, each inner array must have exactly two elements, which are taken as a key/value pair. + +Example: + +``` +json_object('{a, 1, b, "def", c, 3.5}') +json_object('{ {a, 1},{b, "def"},{c, 3.5}}') +{"a": "1", "b": "def", "c": "3.5"} +``` + +#### `json_object(keys text[], values text[])` + +Builds a JSON object out of a text array. This form of `json_object` takes keys and values pairwise from two separate arrays. In all other respects it is identical to the one-argument form. + +Example: + +``` +json_object('{a, b}', '{1,2}') +{"a": "1", "b": "2"} +``` > **Note** `array_to_json` and `row_to_json` have the same behavior as `to_json` except for offering a pretty-printing option. The behavior described for `to_json` likewise applies to each individual value converted by the other JSON creation functions. @@ -182,12 +389,23 @@ This table describes the functions that create `json` data type values. (Current ### JSON Aggregate Functions -This table shows the functions aggregate records to an array of JSON objects and pairs of values to a JSON object +The following functions aggregate records to an array of JSON objects and pairs of values to a JSON object. -| Function | Argument Types | Return Type | Description | -| ------------------------------ | ---------------- | ----------- | ---------------------------------------------- | -| `json_agg(record)` | `record` | `json` | Aggregates records as a JSON array of objects. | -| `json_object_agg(name, value)` | `("any", "any")` | `json` | Aggregates name/value pairs as a JSON object. | +#### `json_agg(record)` + +Argument types: `record` + +Return type: `json` + +Aggregates records as a JSON array of objects. + +#### `json_object_agg(name, value)` + +Argument types: `("any", "any")` + +Return type: `json` + +Aggregates name/value pairs as a JSON object. @@ -197,27 +415,234 @@ This table shows the functions aggregate records to an array of JSON objects and ### JSON Processing Functions -This table shows the functions that are available for processing `json` and `jsonb` values. +This section describes the functions that are available for processing `json` and `jsonb` values. Many of these processing functions and operators convert Unicode escapes in JSON strings to the appropriate single character. This is a not an issue if the input data type is `jsonb`, because the conversion was already done. However, for `json` data type input, this might result in an error being thrown. See [About JSON Data](../admin_guide/query/json-data.md#topic_upc_tcs_fz). -**JSON Processing Functions** - -| Function | Return Type | Description | Example | Example Result | -| --- | --- | --- | --- | --- | -| `json_array_length(json)`

`jsonb_array_length(jsonb)` | `int` | Returns the number of elements in the outermost JSON array. | `json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]')` | `5` | -| `json_each(json)`

`jsonb_each(jsonb)` | `setof key text, value json`

`setof key text, value jsonb` | Expands the outermost JSON object into a set of key/value pairs. | `select * from json_each('{"a":"foo", "b":"bar"}')` | `key \| value`
`-----+-------`
`a \| "foo"`
`b \| "bar"` | -| `json_each_text(json)`

`jsonb_each_text(jsonb)` | `setof key text, value text` | Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. | `select * from json_each_text('{"a":"foo", "b":"bar"}')` | `key \| value`
`-----+-------`
`a \| foo`
`b \| bar` | -| `json_extract_path(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` | `json`

`jsonb` | Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). | `json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4')` | `{"f5":99,"f6":"foo"}` | -| `json_extract_path_text(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` | `text` | Returns the JSON value pointed to by `path_elems` as text. Equivalent to `#>>` operator. | `json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4', 'f6')` | `foo` | -| `json_object_keys(json)`

`jsonb_object_keys(jsonb)` | `setof text` | Returns set of keys in the outermost JSON object. | `json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}}')` | `json_object_keys`
`------------------`
`f1`
`f2` | -| `json_populate_record(base anyelement, from_json json)`

`jsonb_populate_record(base anyelement, from_json jsonb)` | `anyelement` | Expands the object in `from_json` to a row whose columns match the record type defined by base. See [Note 1](#note-on-json-processing-functions-examples). | `select * from json_populate_record(null::myrowtype, '{"a":1,"b":2}')` | `a \| b`
`---+---`
`1 \| 2` | -| `json_populate_recordset(base anyelement, from_json json)`

`jsonb_populate_recordset(base anyelement, from_json jsonb)` | `setof anyelement` | Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by base. See [Note 1](#note-on-json-processing-functions-examples). | `select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]')` | `a \| b`
`---+---`
`1 \| 2`
`3 \| 4` | -| `json_array_elements(json)`

`jsonb_array_elements(jsonb)` | `setof json`

`setof jsonb` | Expands a JSON array to a set of JSON values. | `select * from json_array_elements('[1,true, [2,false]]')` | `value`
`-----------`
`1`
`true`
`[2,false]` | -| `json_array_elements_text(json)`

`jsonb_array_elements_text(jsonb)` | `setof text` | Expands a JSON array to a set of `text` values. | `select * from json_array_elements_text('["foo", "bar"]')` | `value`
`-----------`
`foo`
`bar` | -| `json_typeof(json)`

`jsonb_typeof(jsonb)` | `text` | Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. See [Note](#note-on-json-processing-functions-examples) | `json_typeof('-123.4')` | `number` | -| `json_to_record(json)`

`jsonb_to_record(jsonb)` | `record` | Builds an arbitrary record from a JSON object. See [Note 1](#note-on-json-processing-functions-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a int, b text, d text)` | `a \| b \| d`
`---+---------+---`
`1 \| [1,2,3] \|` | -| `json_to_recordset(json)`

`jsonb_to_recordset(jsonb)` | `setof record` | Builds an arbitrary set of records from a JSON array of objects See [Note 1](#note-on-json-processing-functions-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text);` | `a \| b`
`---+-----`
`1 \| foo`
`2 \|` | +#### `json_array_length() / jsonb_array_length()` + +Syntax: `json_array_length(json)` / `jsonb_array_length(jsonb)` + +Return type: `int` + +Returns the number of elements in the outermost JSON array. + +Example: + +``` +json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]') +5 +``` + +#### `json_each() / jsonb_each()` + +Syntax: `json_each(json)` / `jsonb_each(jsonb)` + +Return type: `setof key text, value json` / `setof key text, value jsonb` + +Expands the outermost JSON object into a set of key/value pairs. + +Example: + +``` +select * from json_each('{"a":"foo", "b":"bar"}') +key | value +-----+------- +a | "foo" +b | "bar" +``` + +#### `json_each_text() / jsonb_each_text()` + +Syntax: `json_each_text(json)` / `jsonb_each_text(jsonb)` + +Return type: `setof key text, value text` + +Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. + +Example: + +``` +select * from json_each_text('{"a":"foo", "b":"bar"}') +key | value +-----+------- +a | foo +b | bar +``` + +#### `json_extract_path() / jsonb_extract_path()` + +Syntax: `json_extract_path(from_json json, VARIADIC path_elems text[])` / `jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` + +Return type: `json` / `jsonb` + +Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). + +Example: + +``` +json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4') +{"f5":99,"f6":"foo"} +``` + +#### `json_extract_path_text() / jsonb_extract_path_text()` + +Syntax: `json_extract_path_text(from_json json, VARIADIC path_elems text[])` / `jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` + +Return type: `text` + +Returns the JSON value pointed to by `path_elems` as text. Equivalent to `#>>` operator. + +Example: + +``` +json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}}','f4', 'f6') +foo +``` + +#### `json_object_keys() / jsonb_object_keys()` + +Syntax: `json_object_keys(json)` / `jsonb_object_keys(jsonb)` + +Return type: `setof text` + +Returns set of keys in the outermost JSON object. + +Example: + +``` +json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}}') +json_object_keys +------------------ +f1 +f2 +``` + +#### `json_populate_record() / jsonb_populate_record()` + +Syntax: `json_populate_record(base anyelement, from_json json)` / `jsonb_populate_record(base anyelement, from_json jsonb)` + +Return type: `anyelement` + +Expands the object in `from_json` to a row whose columns match the record type defined by base. See [Note 1](#note-on-json-processing-functions-examples). + +Example: + +``` +select * from json_populate_record(null::myrowtype, '{"a":1,"b":2}') +a | b +---+--- +1 | 2 +``` + +#### `json_populate_recordset() / jsonb_populate_recordset()` + +Syntax: `json_populate_recordset(base anyelement, from_json json)` / `jsonb_populate_recordset(base anyelement, from_json jsonb)` + +Return type: `setof anyelement` + +Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by base. See [Note 1](#note-on-json-processing-functions-examples). + +Example: + +``` +select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]') +a | b +---+--- +1 | 2 +3 | 4 +``` + +#### `json_array_elements() / jsonb_array_elements()` + +Syntax: `json_array_elements(json)` / `jsonb_array_elements(jsonb)` + +Return type: `setof json` / `setof jsonb` + +Expands a JSON array to a set of JSON values. + +Example: + +``` +select * from json_array_elements('[1,true, [2,false]]') +value +----------- +1 +true +[2,false] +``` + +#### `json_array_elements_text() / jsonb_array_elements_text()` + +Syntax: `json_array_elements_text(json)` / `jsonb_array_elements_text(jsonb)` + +Return type: `setof text` + +Expands a JSON array to a set of `text` values. + +Example: + +``` +select * from json_array_elements_text('["foo", "bar"]') +value +----------- +foo +bar +``` + +#### `json_typeof() / jsonb_typeof()` + +Syntax: `json_typeof(json)` / `jsonb_typeof(jsonb)` + +Return type: `text` + +Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. See [Note](#note-on-json-processing-functions-examples). + +Example: + +``` +json_typeof('-123.4') +number +``` + +#### `json_to_record() / jsonb_to_record()` + +Syntax: `json_to_record(json)` / `jsonb_to_record(jsonb)` + +Return type: `record` + +Builds an arbitrary record from a JSON object. See [Note 1](#note-on-json-processing-functions-examples). + +As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. + +Example: + +``` +select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a int, b text, d text) +a | b | d +---+---------+--- +1 | [1,2,3] | +``` + +#### `json_to_recordset() / jsonb_to_recordset()` + +Syntax: `json_to_recordset(json)` / `jsonb_to_recordset(jsonb)` + +Return type: `setof record` + +Builds an arbitrary set of records from a JSON array of objects See [Note 1](#note-on-json-processing-functions-examples). + +As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. + +Example: + +``` +select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text); +a | b +---+----- +1 | foo +2 | +``` ### Note on JSON processing functions examples @@ -237,133 +662,783 @@ Many of these processing functions and operators convert Unicode escapes in JSON The following are WarehousePG built-in window functions. All window functions are *immutable*. For more information about window functions, see "Window Expressions" in the *WarehousePG Administrator Guide*. -| Function | Return Type | Full Syntax | Description | | -| --------------------------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | -| `cume_dist()` | `double precision` | `CUME_DIST() OVER ( [PARTITION BY` expr `] ORDER BY` expr `)` | Calculates the cumulative distribution of a value in a group of values. Rows with equal values always evaluate to the same cumulative distribution value. | | -| `dense_rank()` | `bigint` | `DENSE_RANK () OVER ( [PARTITION BY` expr `] ORDER BY` expr `)` | Computes the rank of a row in an ordered group of rows without skipping rank values. Rows with equal values are given the same rank value. | | -| `first_value(*expr*)` | same as input expr type | `FIRST_VALUE(` expr `) OVER ( [PARTITION BY` expr `] ORDER BY` expr \`\[ROWS | RANGE`frame\_expr`] )\` | Returns the first value in an ordered set of values. | -| `lag(*expr* [,*offset*] [,*default*])` | same as input *expr* type | `LAG(` *expr* `[,` *offset* `] [,` *default* `]) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` | Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `LAG` provides access to a row at a given physical offset prior to that position. The default `offset` is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. | | -| `last_value(*expr*`) | same as input *expr* type | \`LAST_VALUE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* \[ROWS | RANGE *frame_expr*] )\` | Returns the last value in an ordered set of values. | -| `lead(*expr* [,*offset*] [,*default*])` | same as input *expr* type | `LEAD(*expr*[,*offset*] [,*expr**default*]) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` | Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `lead` provides access to a row at a given physical offset after that position. If *offset* is not specified, the default offset is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. | | -| `ntile(*expr*)` | `bigint` | `NTILE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` | Divides an ordered data set into a number of buckets (as defined by *expr*) and assigns a bucket number to each row. | | -| `percent_rank()` | `double precision` | `PERCENT_RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Calculates the rank of a hypothetical row `R` minus 1, divided by 1 less than the number of rows being evaluated (within a window partition). | | -| `rank()` | `bigint` | `RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Calculates the rank of a row in an ordered group of values. Rows with equal values for the ranking criteria receive the same rank. The number of tied rows are added to the rank number to calculate the next rank value. Ranks may not be consecutive numbers in this case. | | -| `row_number()` | `bigint` | `ROW_NUMBER () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Assigns a unique number to each row to which it is applied (either each row in a window partition or each row of the query). | | +### `cume_dist()` - +Return type: `double precision` - +Full syntax: `CUME_DIST() OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` - +Calculates the cumulative distribution of a value in a group of values. Rows with equal values always evaluate to the same cumulative distribution value. -## Advanced Aggregate Functions +### `dense_rank()` -The following built-in advanced analytic functions are WarehousePG extensions of the PostgreSQL database. Analytic functions are *immutable*. +Return type: `bigint` -> **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [MADlib Extension for Analytics](../admin_guide/analytics/madlib.md). +Full syntax: `DENSE_RANK () OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` -**Advanced Aggregate Functions** +Computes the rank of a row in an ordered group of rows without skipping rank values. Rows with equal values are given the same rank value. -| Function | Return Type | Full Syntax | Description | -| --- | --- | --- | --- | -| `MEDIAN (`*expr*`)` | `timestamp, timestamptz, interval, float` | `MEDIAN (`*expression*`)`

*Example:*

`SELECT department_id, MEDIAN(salary) `
`FROM employees `
`GROUP BY department_id;` | Can take a two-dimensional array as input. Treats such arrays as matrices. | -| `PERCENTILE_CONT (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` | `timestamp, timestamptz, interval, float` | `PERCENTILE_CONT(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)`

*Example:*

`SELECT department_id,`
`PERCENTILE_CONT (0.5) WITHIN GROUP (ORDER BY salary DESC)`
`"Median_cont"; `
`FROM employees GROUP BY department_id;` | Performs an inverse distribution function that assumes a continuous distribution model. It takes a percentile value and a sort specification and returns the same datatype as the numeric datatype of the argument. This returned value is a computed result after performing linear interpolation. Null are ignored in this calculation. | -| `PERCENTILE_DISC (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` | `timestamp, timestamptz, interval, float` | `PERCENTILE_DISC(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)`

*Example:*

`SELECT department_id, `
`PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY salary DESC)`
`"Median_desc"; `
`FROM employees GROUP BY department_id;` | Performs an inverse distribution function that assumes a discrete distribution model. It takes a percentile value and a sort specification. This returned value is an element from the set. Null are ignored in this calculation. | -| `sum(array[])` | `smallint[]int[], bigint[], float[]` | `sum(array[[1,2],[3,4]])`

*Example:*

`CREATE TABLE mymatrix (myvalue int[]);`
`INSERT INTO mymatrix VALUES (array[[1,2],[3,4]]);`
`INSERT INTO mymatrix VALUES (array[[0,1],[1,0]]);`
`SELECT sum(myvalue) FROM mymatrix;`
`sum `
`---------------`
`{{1,3},{4,4}}` | Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. | -| `pivot_sum (label[], label, expr)` | `int[], bigint[], float[]` | `pivot_sum( array['A1','A2'], attr, value)` | A pivot aggregation using sum to resolve duplicate entries. | -| `unnest (array[])` | set of `anyelement` | `unnest( array['one', 'row', 'per', 'item'])` | Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudotype in PostgreSQL](https://www.postgresql.org/docs/9.4/datatype-pseudo.html). | +### `first_value(`*expr*`)` - +Return type: same as input *expr* type - +Full syntax: `FIRST_VALUE(` *expr* `) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `[ROWS | RANGE` *frame_expr* `] )` - +Returns the first value in an ordered set of values. -## Text Search Functions and Operators +### `lag(`*expr*` [,`*offset*`] [,`*default*`])` -The following tables summarize the functions and operators that are provided for full text searching. See [Using Full Text Search](../admin_guide/query/textsearch/index.md) for a detailed explanation of WarehousePG's text search facility. +Return type: same as input *expr* type -| Operator | Description | Example | Result | | | | | | | | -| -------- | ------------------------------ | -------------------------------------------------------------- | --------------------------------- | --------------------- | ----------------- | ------------------------- | --------------------------- | --------- | ------- | ------- | -| `@@` | `tsvector` matches `tsquery` ? | `to_tsvector('fat cats ate rats') @@ to_tsquery('cat & rat')` | `t` | | | | | | | | -| `@@@` | deprecated synonym for `@@` | `to_tsvector('fat cats ate rats') @@@ to_tsquery('cat & rat')` | `t` | | | | | | | | -| \` | | \` | concatenate`tsvector`s | \`'a:1 b:2'::tsvector | | 'c:1 d:2 b:3'::tsvector\` | `'a':1 'b':2,5 'c':3 'd':4` | | | | -| `&&` | AND `tsquery`s together | \`'fat | rat'::tsquery && 'cat'::tsquery\` | \`( 'fat' | 'rat' ) & 'cat'\` | | | | | | -| \` | | \` | OR `tsquery`s together | \`'fat | rat'::tsquery | | 'cat'::tsquery\` | \`( 'fat' | 'rat' ) | 'cat'\` | -| `!!` | negate a`tsquery` | `!! 'cat'::tsquery` | `!'cat'` | | | | | | | | -| `@>` | `tsquery` contains another ? | `'cat'::tsquery @> 'cat & rat'::tsquery` | `f` | | | | | | | | -| `<@` | `tsquery` is contained in ? | `'cat'::tsquery <@ 'cat & rat'::tsquery` | `t` | | | | | | | | +Full syntax: `LAG(` *expr* `[,` *offset* `] [,` *default* `]) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` -> **Note** The `tsquery` containment operators consider only the lexemes listed in the two queries, ignoring the combining operators. +Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `LAG` provides access to a row at a given physical offset prior to that position. The default `offset` is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. -In addition to the operators shown in the table, the ordinary B-tree comparison operators (=, <, etc) are defined for types `tsvector` and `tsquery`. These are not very useful for text searching but allow, for example, unique indexes to be built on columns of these types. - -| Function | Return Type | Description | Example | Result | | | -| ------------------------------------------------------------------------------------------- | ----------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------- | ----------------------------- | ------------- | ------- | -| `get_current_ts_config()` | regconfig | get default text search configuration | get_current_ts_config() | english | | | -| `length(tsvector)` | integer | number of lexemes in tsvector | length('fat:2,4 cat:3 rat:5A'::tsvector) | 3 | | | -| `numnode(tsquery)` | integer | number of lexemes plus operators in tsquery | numnode('(fat & rat) | cat'::tsquery) | 5 | | -| `plainto_tsquery([ config regconfig , ] querytext)` | tsquery | produce tsquery ignoring punctuation | plainto_tsquery('english', 'The Fat Rats') | 'fat' & 'rat' | | | -| `querytree(query tsquery)` | text | get indexable part of a tsquery | querytree('foo & ! bar'::tsquery) | 'foo' | | | -| `setweight(tsvector, "char")` | tsvector | assign weight to each element of tsvector | setweight('fat:2,4 cat:3 rat:5B'::tsvector, 'A') | 'cat':3A 'fat':2A,4A 'rat':5A | | | -| `strip(tsvector)` | tsvector | remove positions and weights from tsvector | strip('fat:2,4 cat:3 rat:5A'::tsvector) | 'cat' 'fat' 'rat' | | | -| `to_tsquery([ config regconfig , ] query text)` | tsquery | normalize words and convert to tsquery | to_tsquery('english', 'The & Fat & Rats') | 'fat' & 'rat' | | | -| `to_tsvector([ config regconfig , ] documenttext)` | tsvector | reduce document text to tsvector | to_tsvector('english', 'The Fat Rats') | 'fat':2 'rat':3 | | | -| `ts_headline([ config regconfig, ] documenttext, query tsquery [, options text ])` | text | display a query match | ts_headline('x y z', 'z'::tsquery) | x y z | | | -| `ts_rank([ weights float4[], ] vector tsvector,query tsquery [, normalization integer ])` | float4 | rank document for query | ts_rank(textsearch, query) | 0.818 | | | -| `ts_rank_cd([ weights float4[], ] vectortsvector, query tsquery [, normalizationinteger ])` | float4 | rank document for query using cover density | ts_rank_cd('{0.1, 0.2, 0.4, 1.0}', textsearch, query) | 2.01317 | | | -| `ts_rewrite(query tsquery, target tsquery,substitute tsquery)` | tsquery | replace target with substitute within query | ts_rewrite('a & b'::tsquery, 'a'::tsquery, 'foo | bar'::tsquery) | 'b' & ( 'foo' | 'bar' ) | -| `ts_rewrite(query tsquery, select text)` | tsquery | replace using targets and substitutes from a SELECTcommand | SELECT ts_rewrite('a & b'::tsquery, 'SELECT t,s FROM aliases') | 'b' & ( 'foo' | 'bar' ) | | -| `tsvector_update_trigger()` | trigger | trigger function for automatic tsvector column update | CREATE TRIGGER ... tsvector_update_trigger(tsvcol, 'pg_catalog.swedish', title, body) |   | | | -| `tsvector_update_trigger_column()` | trigger | trigger function for automatic tsvector column update | CREATE TRIGGER ... tsvector_update_trigger_column(tsvcol, configcol, title, body) |   | | | +### `last_value(`*expr*`)` -> **Note** All the text search functions that accept an optional `regconfig` argument will use the configuration specified by [default_text_search_config](config_params/guc-list.md) when that argument is omitted. +Return type: same as input *expr* type -The functions in the following table are listed separately because they are not usually used in everyday text searching operations. They are helpful for development and debugging of new text search configurations. +Full syntax: `LAST_VALUE(` *expr* `) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `[ROWS | RANGE` *frame_expr* `] )` -| Function | Return Type | Description | Example | Result | -| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | ---------------------------------- | -------------------------------------------------- | ---------------------------------------------------------------------- | -| `ts_debug([ *config* regconfig, ] *document* text, OUT *alias* text, OUT *description* text, OUT *token* text, OUT *dictionaries* regdictionary[], OUT *dictionary* regdictionary, OUT *lexemes* text[])` | `setof record` | test a configuration | `ts_debug('english', 'The Brightest supernovaes')` | `(asciiword,"Word, all ASCII",The,{english_stem},english_stem,{}) ...` | -| `ts_lexize(*dict* regdictionary, *token* text)` | `text[]` | test a dictionary | `ts_lexize('english_stem', 'stars')` | `{star}` | -| `ts_parse(*parser\_name* text, *document* text, OUT *tokid* integer, OUT *token* text)` | `setof record` | test a parser | `ts_parse('default', 'foo - bar')` | (`1,foo) ...` | -| `ts_parse(*parser\_oid* oid, *document* text, OUT *tokid* integer, OUT *token* text)` | `setof record` | test a parser | `ts_parse(3722, 'foo - bar')` | `(1,foo) ...` | -| `ts_token_type(*parser\_name* text, OUT *tokid* integer, OUT *alias* text, OUT description text)` | `setof record` | get token types defined by parser | `ts_token_type('default')` | `(1,asciiword,"Word, all ASCII") ...` | -| `ts_token_type(*parser\_oid* oid, OUT *tokid* integer, OUT *alias* text, OUT *description* text)` | `setof record` | get token types defined by parser | `ts_token_type(3722)` | `(1,asciiword,"Word, all ASCII") ...` | -| `ts_stat(*sqlquery* text, [ *weights* text, ] OUT *word* text, OUT *ndocinteger*, OUT *nentry* integer)` | `setof record` | get statistics of a tsvectorcolumn | `ts_stat('SELECT vector from apod')` | `(foo,10,15) ...` | +Returns the last value in an ordered set of values. - +### `lead(`*expr*` [,`*offset*`] [,`*default*`])` - +Return type: same as input *expr* type - +Full syntax: `LEAD(` *expr* `[,` *offset* `] [,` *default* `]) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` -## Range Functions and Operators +Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `lead` provides access to a row at a given physical offset after that position. If *offset* is not specified, the default offset is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. + +### `ntile(`*expr*`)` + +Return type: `bigint` + +Full syntax: `NTILE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` + +Divides an ordered data set into a number of buckets (as defined by *expr*) and assigns a bucket number to each row. + +### `percent_rank()` + +Return type: `double precision` + +Full syntax: `PERCENT_RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` + +Calculates the rank of a hypothetical row `R` minus 1, divided by 1 less than the number of rows being evaluated (within a window partition). + +### `rank()` + +Return type: `bigint` + +Full syntax: `RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` + +Calculates the rank of a row in an ordered group of values. Rows with equal values for the ranking criteria receive the same rank. The number of tied rows are added to the rank number to calculate the next rank value. Ranks may not be consecutive numbers in this case. + +### `row_number()` + +Return type: `bigint` + +Full syntax: `ROW_NUMBER () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` + +Assigns a unique number to each row to which it is applied (either each row in a window partition or each row of the query). + + + + + + + +## Advanced Aggregate Functions + +The following built-in advanced analytic functions are WarehousePG extensions of the PostgreSQL database. Analytic functions are *immutable*. + +> **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [MADlib Extension for Analytics](../admin_guide/analytics/madlib.md). + +### `pivot_sum (label[], label, expr)` + +Return type: `int[], bigint[], float[]` + +Full syntax: `pivot_sum( array['A1','A2'], attr, value)` + +A pivot aggregation using sum to resolve duplicate entries. + +### `unnest (array[])` + +Return type: set of `anyelement` + +Full syntax: `unnest( array['one', 'row', 'per', 'item'])` + +Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudotype in PostgreSQL](https://www.postgresql.org/docs/9.4/datatype-pseudo.html). + +### `MEDIAN (`*expr*`)` + +Return type: `timestamp, timestamptz, interval, float` + +Full syntax: `MEDIAN (*expression*)` + +Can take a two-dimensional array as input. Treats such arrays as matrices. + +Example: + +``` +SELECT department_id, MEDIAN(salary) +FROM employees +GROUP BY department_id; +``` + +### `PERCENTILE_CONT (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` + +Return type: `timestamp, timestamptz, interval, float` + +Full syntax: `PERCENTILE_CONT(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)` + +Performs an inverse distribution function that assumes a continuous distribution model. It takes a percentile value and a sort specification and returns the same datatype as the numeric datatype of the argument. This returned value is a computed result after performing linear interpolation. Null are ignored in this calculation. + +Example: + +``` +SELECT department_id, +PERCENTILE_CONT (0.5) WITHIN GROUP (ORDER BY salary DESC) +"Median_cont"; +FROM employees GROUP BY department_id; +``` + +### `PERCENTILE_DISC (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` + +Return type: `timestamp, timestamptz, interval, float` + +Full syntax: `PERCENTILE_DISC(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)` + +Performs an inverse distribution function that assumes a discrete distribution model. It takes a percentile value and a sort specification. This returned value is an element from the set. Null are ignored in this calculation. + +Example: + +``` +SELECT department_id, +PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY salary DESC) +"Median_desc"; +FROM employees GROUP BY department_id; +``` + +### `sum(array[])` + +Return type: `smallint[]int[], bigint[], float[]` + +Full syntax: `sum(array[[1,2],[3,4]])` + +Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. + +Example: + +``` +CREATE TABLE mymatrix (myvalue int[]); +INSERT INTO mymatrix VALUES (array[[1,2],[3,4]]); +INSERT INTO mymatrix VALUES (array[[0,1],[1,0]]); +SELECT sum(myvalue) FROM mymatrix; +sum +--------------- +{{1,3},{4,4}} +``` + + + + + + + +## Text Search Functions and Operators + +The following sections summarize the functions and operators that are provided for full text searching. See [Using Full Text Search](../admin_guide/query/textsearch/index.md) for a detailed explanation of WarehousePG's text search facility. + +### `@@` + +`tsvector` matches `tsquery`? + +Example: + +``` +to_tsvector('fat cats ate rats') @@ to_tsquery('cat & rat') +t +``` + +### `@@@` + +Deprecated synonym for `@@`. + +Example: + +``` +to_tsvector('fat cats ate rats') @@@ to_tsquery('cat & rat') +t +``` + +### `||` (tsvector) + +Concatenates `tsvector`s. + +Example: + +``` +'a:1 b:2'::tsvector || 'c:1 d:2 b:3'::tsvector +'a':1 'b':2,5 'c':3 'd':4 +``` + +### `&&` + +ANDs `tsquery`s together. + +Example: + +``` +'fat | rat'::tsquery && 'cat'::tsquery +( 'fat' | 'rat' ) & 'cat' +``` + +### `||` (tsquery) + +ORs `tsquery`s together. + +Example: + +``` +'fat | rat'::tsquery || 'cat'::tsquery +( 'fat' | 'rat' ) | 'cat' +``` + +### `!!` + +Negates a `tsquery`. + +Example: + +``` +!! 'cat'::tsquery +!'cat' +``` + +### `@>` (tsquery) + +Does one `tsquery` contain another? + +Example: + +``` +'cat'::tsquery @> 'cat & rat'::tsquery +f +``` + +### `<@` (tsquery) + +Is one `tsquery` contained in another? + +Example: + +``` +'cat'::tsquery <@ 'cat & rat'::tsquery +t +``` + +> **Note** The `tsquery` containment operators consider only the lexemes listed in the two queries, ignoring the combining operators. + +In addition to the operators described above, the ordinary B-tree comparison operators (=, <, etc) are defined for types `tsvector` and `tsquery`. These are not very useful for text searching but allow, for example, unique indexes to be built on columns of these types. + +### `get_current_ts_config()` + +Return type: `regconfig` + +Get default text search configuration. + +Example: + +``` +get_current_ts_config() +english +``` + +### `length(tsvector)` + +Return type: `integer` + +Number of lexemes in tsvector. + +Example: + +``` +length('fat:2,4 cat:3 rat:5A'::tsvector) +3 +``` + +### `numnode(tsquery)` + +Return type: `integer` + +Number of lexemes plus operators in tsquery. + +Example: + +``` +numnode('(fat & rat) | cat'::tsquery) +5 +``` + +### `plainto_tsquery([ config regconfig , ] querytext)` + +Return type: `tsquery` + +Produce tsquery ignoring punctuation. + +Example: + +``` +plainto_tsquery('english', 'The Fat Rats') +'fat' & 'rat' +``` + +### `querytree(query tsquery)` + +Return type: `text` + +Get indexable part of a tsquery. + +Example: + +``` +querytree('foo & ! bar'::tsquery) +'foo' +``` + +### `setweight(tsvector, "char")` + +Return type: `tsvector` + +Assign weight to each element of tsvector. + +Example: + +``` +setweight('fat:2,4 cat:3 rat:5B'::tsvector, 'A') +'cat':3A 'fat':2A,4A 'rat':5A +``` + +### `strip(tsvector)` + +Return type: `tsvector` + +Remove positions and weights from tsvector. + +Example: + +``` +strip('fat:2,4 cat:3 rat:5A'::tsvector) +'cat' 'fat' 'rat' +``` + +### `to_tsquery([ config regconfig , ] query text)` + +Return type: `tsquery` + +Normalize words and convert to tsquery. + +Example: + +``` +to_tsquery('english', 'The & Fat & Rats') +'fat' & 'rat' +``` + +### `to_tsvector([ config regconfig , ] documenttext)` + +Return type: `tsvector` + +Reduce document text to tsvector. + +Example: + +``` +to_tsvector('english', 'The Fat Rats') +'fat':2 'rat':3 +``` + +### `ts_headline([ config regconfig, ] documenttext, query tsquery [, options text ])` + +Return type: `text` + +Display a query match. + +Example: + +``` +ts_headline('x y z', 'z'::tsquery) +x y z +``` + +### `ts_rank([ weights float4[], ] vector tsvector,query tsquery [, normalization integer ])` + +Return type: `float4` + +Rank document for query. + +Example: + +``` +ts_rank(textsearch, query) +0.818 +``` + +### `ts_rank_cd([ weights float4[], ] vectortsvector, query tsquery [, normalizationinteger ])` + +Return type: `float4` + +Rank document for query using cover density. + +Example: + +``` +ts_rank_cd('{0.1, 0.2, 0.4, 1.0}', textsearch, query) +2.01317 +``` + +### `ts_rewrite(query tsquery, target tsquery,substitute tsquery)` + +Return type: `tsquery` + +Replace target with substitute within query. + +Example: + +``` +ts_rewrite('a & b'::tsquery, 'a'::tsquery, 'foo | bar'::tsquery) +'b' & ( 'foo' | 'bar' ) +``` + +### `ts_rewrite(query tsquery, select text)` + +Return type: `tsquery` + +Replace using targets and substitutes from a SELECT command. + +Example: + +``` +SELECT ts_rewrite('a & b'::tsquery, 'SELECT t,s FROM aliases') +'b' & ( 'foo' | 'bar' ) +``` + +### `tsvector_update_trigger()` + +Return type: `trigger` + +Trigger function for automatic tsvector column update. + +Example: + +``` +CREATE TRIGGER ... tsvector_update_trigger(tsvcol, 'pg_catalog.swedish', title, body) +``` + +### `tsvector_update_trigger_column()` + +Return type: `trigger` + +Trigger function for automatic tsvector column update. + +Example: + +``` +CREATE TRIGGER ... tsvector_update_trigger_column(tsvcol, configcol, title, body) +``` + +> **Note** All the text search functions that accept an optional `regconfig` argument will use the configuration specified by [default_text_search_config](config_params/guc-list.md) when that argument is omitted. + +The following functions are listed separately because they are not usually used in everyday text searching operations. They are helpful for development and debugging of new text search configurations. + +### `ts_debug([ `*config*` regconfig, ] `*document*` text, OUT `*alias*` text, OUT `*description*` text, OUT `*token*` text, OUT `*dictionaries*` regdictionary[], OUT `*dictionary*` regdictionary, OUT `*lexemes*` text[])` + +Return type: `setof record` + +Test a configuration. + +Example: + +``` +ts_debug('english', 'The Brightest supernovaes') +(asciiword,"Word, all ASCII",The,{english_stem},english_stem,{}) ... +``` + +### `ts_lexize(`*dict*` regdictionary, `*token*` text)` + +Return type: `text[]` + +Test a dictionary. + +Example: + +``` +ts_lexize('english_stem', 'stars') +{star} +``` + +### `ts_parse(`*parser_name*` text, `*document*` text, OUT `*tokid*` integer, OUT `*token*` text)` + +Return type: `setof record` + +Test a parser. + +Example: + +``` +ts_parse('default', 'foo - bar') +(1,foo) ... +``` + +### `ts_parse(`*parser_oid*` oid, `*document*` text, OUT `*tokid*` integer, OUT `*token*` text)` + +Return type: `setof record` + +Test a parser. + +Example: + +``` +ts_parse(3722, 'foo - bar') +(1,foo) ... +``` + +### `ts_token_type(`*parser_name*` text, OUT `*tokid*` integer, OUT `*alias*` text, OUT description text)` + +Return type: `setof record` + +Get token types defined by parser. + +Example: + +``` +ts_token_type('default') +(1,asciiword,"Word, all ASCII") ... +``` + +### `ts_token_type(`*parser_oid*` oid, OUT `*tokid*` integer, OUT `*alias*` text, OUT `*description*` text)` + +Return type: `setof record` + +Get token types defined by parser. + +Example: + +``` +ts_token_type(3722) +(1,asciiword,"Word, all ASCII") ... +``` + +### `ts_stat(`*sqlquery*` text, [ `*weights*` text, ] OUT `*word*` text, OUT `*ndocinteger*`, OUT `*nentry*` integer)` + +Return type: `setof record` + +Get statistics of a tsvector column. + +Example: + +``` +ts_stat('SELECT vector from apod') +(foo,10,15) ... +``` + + + + + + + +## Range Functions and Operators See [Range Types](data_types/datatype-range.md) for an overview of range types. -The following table shows the operators available for range types. - -| Operator | Description | Example | Result | | | -| -------- | ------------------------------- | --------------------------------------------------------------- | --------------------- | --------------------- | --- | -| `=` | equal | `int4range(1,5) = '[1,4]'::int4range` | `t` | | | -| `<>` | not equal | `numrange(1.1,2.2) <> numrange(1.1,2.3)` | `t` | | | -| `<` | less than | `int4range(1,10) < int4range(2,3)` | `t` | | | -| `>` | greater than | `int4range(1,10) > int4range(1,5)` | `t` | | | -| `<=` | less than or equal | `numrange(1.1,2.2) <= numrange(1.1,2.2)` | `t` | | | -| `>=` | greater than or equal | `numrange(1.1,2.2) >= numrange(1.1,2.0)` | `t` | | | -| `@>` | contains range | `int4range(2,4) @> int4range(2,3)` | `t` | | | -| `@>` | contains element | `'[2011-01-01,2011-03-01)'::tsrange @> '2011-01-10'::timestamp` | `t` | | | -| `<@` | range is contained by | `int4range(2,4) <@ int4range(1,7)` | `t` | | | -| `<@` | element is contained by | `42 <@ int4range(1,7)` | `f` | | | -| `&&` | overlap (have points in common) | `int8range(3,7) && int8range(4,12)` | `t` | | | -| `<<` | strictly left of | `int8range(1,10) << int8range(100,110)` | `t` | | | -| `>>` | strictly right of | `int8range(50,60) >> int8range(20,30)` | `t` | | | -| `&<` | does not extend to the right of | `int8range(1,20) &< int8range(18,20)` | `t` | | | -| `&>` | does not extend to the left of | `int8range(7,20) &> int8range(5,10)` | `t` | | | -| \`- | -\` | is adjacent to | \`numrange(1.1,2.2) - | - numrange(2.2,3.3)\` | `t` | -| `+` | union | `numrange(5,15) + numrange(10,20)` | `[5,20)` | | | -| `*` | intersection | `int8range(5,15) * int8range(10,20)` | `[10,15)` | | | -| `-` | difference | `int8range(5,15) - int8range(10,20)` | `[5,10)` | | | +The following operators are available for range types. + +### `=` + +Equal. + +Example: + +``` +int4range(1,5) = '[1,4]'::int4range +t +``` + +### `<>` + +Not equal. + +Example: + +``` +numrange(1.1,2.2) <> numrange(1.1,2.3) +t +``` + +### `<` + +Less than. + +Example: + +``` +int4range(1,10) < int4range(2,3) +t +``` + +### `>` + +Greater than. + +Example: + +``` +int4range(1,10) > int4range(1,5) +t +``` + +### `<=` + +Less than or equal. + +Example: + +``` +numrange(1.1,2.2) <= numrange(1.1,2.2) +t +``` + +### `>=` + +Greater than or equal. + +Example: + +``` +numrange(1.1,2.2) >= numrange(1.1,2.0) +t +``` + +### `@>` (contains range) + +Example: + +``` +int4range(2,4) @> int4range(2,3) +t +``` + +### `@>` (contains element) + +Example: + +``` +'[2011-01-01,2011-03-01)'::tsrange @> '2011-01-10'::timestamp +t +``` + +### `<@` (range is contained by) + +Example: + +``` +int4range(2,4) <@ int4range(1,7) +t +``` + +### `<@` (element is contained by) + +Example: + +``` +42 <@ int4range(1,7) +f +``` + +### `&&` (overlap) + +Overlap, meaning the ranges have points in common. + +Example: + +``` +int8range(3,7) && int8range(4,12) +t +``` + +### `<<` (strictly left of) + +Example: + +``` +int8range(1,10) << int8range(100,110) +t +``` + +### `>>` (strictly right of) + +Example: + +``` +int8range(50,60) >> int8range(20,30) +t +``` + +### `&<` (does not extend to the right of) + +Example: + +``` +int8range(1,20) &< int8range(18,20) +t +``` + +### `&>` (does not extend to the left of) + +Example: + +``` +int8range(7,20) &> int8range(5,10) +t +``` + +### `-|-` (is adjacent to) + +Example: + +``` +numrange(1.1,2.2) -|- numrange(2.2,3.3) +t +``` + +### `+` (union) + +Example: + +``` +numrange(5,15) + numrange(10,20) +[5,20) +``` + +### `*` (intersection) + +Example: + +``` +int8range(5,15) * int8range(10,20) +[10,15) +``` + +### `-` (difference) + +Example: + +``` +int8range(5,15) - int8range(10,20) +[5,10) +``` The simple comparison operators `<`, `>`, `<=`, and `>=` compare the lower bounds first, and only if those are equal, compare the upper bounds. These comparisons are not usually very useful for ranges, but are provided to allow B-tree indexes to be constructed on ranges. @@ -371,17 +1446,110 @@ The left-of/right-of/adjacent operators always return false when an empty range The union and difference operators will fail if the resulting range would need to contain two disjoint sub-ranges, as such a range cannot be represented. -The following table shows the functions available for use with range types. - -| Function | Return Type | Description | Example | Result | -| --------------------------------- | -------------------- | ---------------------------------------------------------- | ----------------------------------------------------- | ------- | -| `lower(anyrange)` | range's element type | lower bound of range | `lower(numrange(1.1,2.2))` | `1.1` | -| `upper(anyrange)` | range's element type | upper bound of range | `upper(numrange(1.1,2.2))` | `2.2` | -| `isempty(anyrange)` | `boolean` | is the range empty? | `isempty(numrange(1.1,2.2))` | `false` | -| `lower_inc(anyrange)` | `boolean` | is the lower bound inclusive? | `lower_inc(numrange(1.1,2.2))` | `true` | -| `upper_inc(anyrange)` | `boolean` | is the upper bound inclusive? | `upper_inc(numrange(1.1,2.2))` | `false` | -| `lower_inf(anyrange)` | `boolean` | is the lower bound infinite? | `lower_inf('(,)'::daterange)` | `true` | -| `upper_inf(anyrange)` | `boolean` | is the upper bound infinite? | `upper_inf('(,)'::daterange)` | `true` | -| `range_merge(anyrange, anyrange)` | `anyrange` | the smallest range which includes both of the given ranges | `range_merge('[1,2)'::int4range, '[3,4)'::int4range)` | `[1,4)` | +The following functions are available for use with range types. + +### `lower(anyrange)` + +Return type: range's element type + +Lower bound of range. + +Example: + +``` +lower(numrange(1.1,2.2)) +1.1 +``` + +### `upper(anyrange)` + +Return type: range's element type + +Upper bound of range. + +Example: + +``` +upper(numrange(1.1,2.2)) +2.2 +``` + +### `isempty(anyrange)` + +Return type: `boolean` + +Is the range empty? + +Example: + +``` +isempty(numrange(1.1,2.2)) +false +``` + +### `lower_inc(anyrange)` + +Return type: `boolean` + +Is the lower bound inclusive? + +Example: + +``` +lower_inc(numrange(1.1,2.2)) +true +``` + +### `upper_inc(anyrange)` + +Return type: `boolean` + +Is the upper bound inclusive? + +Example: + +``` +upper_inc(numrange(1.1,2.2)) +false +``` + +### `lower_inf(anyrange)` + +Return type: `boolean` + +Is the lower bound infinite? + +Example: + +``` +lower_inf('(,)'::daterange) +true +``` + +### `upper_inf(anyrange)` + +Return type: `boolean` + +Is the upper bound infinite? + +Example: + +``` +upper_inf('(,)'::daterange) +true +``` + +### `range_merge(anyrange, anyrange)` + +Return type: `anyrange` + +The smallest range which includes both of the given ranges. + +Example: + +``` +range_merge('[1,2)'::int4range, '[3,4)'::int4range) +[1,4) +``` The `lower` and `upper` functions return null if the range is empty or the requested bound is infinite. The `lower_inc`, `upper_inc`, `lower_inf`, and `upper_inf` functions all return false for an empty range. diff --git a/docs/whpg/6x/ref_guide/misc/feature_summary.md b/docs/whpg/6x/ref_guide/misc/feature_summary.md index 0a82eac..779ad3d 100644 --- a/docs/whpg/6x/ref_guide/misc/feature_summary.md +++ b/docs/whpg/6x/ref_guide/misc/feature_summary.md @@ -226,14 +226,14 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `ALTER OPERATOR FAMILY` | YES | | | `ALTER PROTOCOL` | YES | | | `ALTER RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL. | -| `ALTER ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | +| `ALTER ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | | `ALTER SCHEMA` | YES | | | `ALTER SEQUENCE` | YES | | | `ALTER SYSTEM` | **NO** | | -| `ALTER TABLE` | YES | **Unsupported Clauses / Options:**

`CLUSTER ON`

`ENABLE/DISABLE TRIGGER`

**WarehousePG Database Clauses:**

`ADD \| DROP \| RENAME \| SPLIT \| EXCHANGE PARTITION \| SET SUBPARTITION TEMPLATE \| SET WITH ` `(REORGANIZE=true \| false) \| SET DISTRIBUTED BY` | +| `ALTER TABLE` | YES | **Unsupported Clauses / Options:**

`CLUSTER ON`

`ENABLE/DISABLE TRIGGER`

**WarehousePG Database Clauses:**

`ADD \| DROP \| RENAME \| SPLIT \| EXCHANGE PARTITION \| SET SUBPARTITION TEMPLATE \| SET WITH ` `(REORGANIZE=true \| false) \| SET DISTRIBUTED BY` | | `ALTER TABLESPACE` | YES | | | `ALTER TRIGGER` | **NO** | | -| `ALTER TYPE` | YES | **WarehousePG Clauses:**

`SET DEFAULT ENCODING` | +| `ALTER TYPE` | YES | **WarehousePG Clauses:**

`SET DEFAULT ENCODING` | | `ALTER USER` | YES | An alias for [ALTER ROLE](../sql_commands/ALTER_ROLE.md) | | `ALTER VIEW` | YES | | | `ANALYZE` | YES | | @@ -244,8 +244,8 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `COMMENT` | YES | | | `COMMIT` | YES | | | `COMMIT PREPARED` | **NO** | | -| `COPY` | YES | **Modified Clauses:**

`ESCAPE [ AS ] '`*escape*`' \| 'OFF'`

**WarehousePG Clauses:**

`[LOG ERRORS] SEGMENT REJECT LIMIT `*count*` [ROWS\|PERCENT]` | -| `CREATE AGGREGATE` | YES | **Unsupported Clauses / Options:**

`[ , SORTOP = `*sort_operator*` ]`

**WarehousePG Clauses:**

`[ , COMBINEFUNC = `*combinefunc*` ]`

**Limitations:**

The functions used to implement the aggregate must be `IMMUTABLE` functions. | +| `COPY` | YES | **Modified Clauses:**

`ESCAPE [ AS ] '`*escape*`' \| 'OFF'`

**WarehousePG Clauses:**

`[LOG ERRORS] SEGMENT REJECT LIMIT `*count*` [ROWS\|PERCENT]` | +| `CREATE AGGREGATE` | YES | **Unsupported Clauses / Options:**

`[ , SORTOP = `*sort_operator*` ]`

**WarehousePG Clauses:**

`[ , COMBINEFUNC = `*combinefunc*` ]`

**Limitations:**

The functions used to implement the aggregate must be `IMMUTABLE` functions. | | `CREATE CAST` | YES | | | `CREATE CONSTRAINT TRIGGER` | **NO** | | | `CREATE CONVERSION` | YES | | @@ -254,29 +254,29 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `CREATE EVENT TRIGGER` | YES | | | `CREATE EXTENSION` | YES | Loads a new extension into WarehousePG - based on PostgreSQL 9.6. | | `CREATE EXTERNAL TABLE` | YES | WarehousePG parallel ETL feature - not in PostgreSQL 9.4. | -| `CREATE FUNCTION` | YES | **Limitations:**

Functions defined as `STABLE` or `VOLATILE` can be run in WarehousePG provided that they are run on the coordinator only. `STABLE` and `VOLATILE` functions cannot be used in statements that run at the segment level. | +| `CREATE FUNCTION` | YES | **Limitations:**

Functions defined as `STABLE` or `VOLATILE` can be run in WarehousePG provided that they are run on the coordinator only. `STABLE` and `VOLATILE` functions cannot be used in statements that run at the segment level. | | `CREATE GROUP` | YES | An alias for [CREATE ROLE](../sql_commands/CREATE_ROLE.md) | -| `CREATE INDEX` | YES | **WarehousePG Clauses:**

`USING bitmap` (bitmap indexes)

**Limitations:**

`UNIQUE` indexes are allowed only if they contain all of (or a superset of) the WarehousePG distribution key columns. On partitioned tables, a unique index is only supported within an individual partition - not across all partitions.

`CONCURRENTLY` keyword not supported in WarehousePG. | +| `CREATE INDEX` | YES | **WarehousePG Clauses:**

`USING bitmap` (bitmap indexes)

**Limitations:**

`UNIQUE` indexes are allowed only if they contain all of (or a superset of) the WarehousePG distribution key columns. On partitioned tables, a unique index is only supported within an individual partition - not across all partitions.

`CONCURRENTLY` keyword not supported in WarehousePG. | | `CREATE LANGUAGE` | YES | | | `CREATE MATERIALIZED VIEW` | YES | Based on PostgreSQL 9.4. | -| `CREATE OPERATOR` | YES | **Limitations:**

The function used to implement the operator must be an `IMMUTABLE` function. | +| `CREATE OPERATOR` | YES | **Limitations:**

The function used to implement the operator must be an `IMMUTABLE` function. | | `CREATE OPERATOR CLASS` | YES | | | `CREATE OPERATOR FAMILY` | YES | | | `CREATE PROTOCOL` | YES | | | `CREATE RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL 9.4. | -| `CREATE ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | +| `CREATE ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | | `CREATE RULE` | YES | | | `CREATE SCHEMA` | YES | | -| `CREATE SEQUENCE` | YES | **Limitations:**

The `lastval()` and `currval()` functions are not supported.

The `setval()` function is only allowed in queries that do not operate on distributed data. | -| `CREATE TABLE` | YES | **Unsupported Clauses / Options:**

`[GLOBAL \| LOCAL]`

`REFERENCES`

`FOREIGN KEY`

`[DEFERRABLE \| NOT DEFERRABLE] `

**Limited Clauses:**

`UNIQUE` or `PRIMARY KEY `constraints are only allowed on hash-distributed tables (`DISTRIBUTED BY`), and the constraint columns must be the same as or a superset of the distribution key columns of the table and must include all the distribution key columns of the partitioning key.

**WarehousePG Clauses:**

`DISTRIBUTED BY (column, [ ... ] ) \|`

`DISTRIBUTED RANDOMLY`

`PARTITION BY type (column [, ...]) ( partition_specification, [...] )`

`WITH (appendoptimized=true [,compresslevel=value,blocksize=value] )` | +| `CREATE SEQUENCE` | YES | **Limitations:**

The `lastval()` and `currval()` functions are not supported.

The `setval()` function is only allowed in queries that do not operate on distributed data. | +| `CREATE TABLE` | YES | **Unsupported Clauses / Options:**

`[GLOBAL \| LOCAL]`

`REFERENCES`

`FOREIGN KEY`

`[DEFERRABLE \| NOT DEFERRABLE] `

**Limited Clauses:**

`UNIQUE` or `PRIMARY KEY `constraints are only allowed on hash-distributed tables (`DISTRIBUTED BY`), and the constraint columns must be the same as or a superset of the distribution key columns of the table and must include all the distribution key columns of the partitioning key.

**WarehousePG Clauses:**

`DISTRIBUTED BY (column, [ ... ] ) \|`

`DISTRIBUTED RANDOMLY`

`PARTITION BY type (column [, ...]) ( partition_specification, [...] )`

`WITH (appendoptimized=true [,compresslevel=value,blocksize=value] )` | | `CREATE TABLE AS` | YES | See [CREATE TABLE](../sql_commands/CREATE_TABLE.md) | -| `CREATE TABLESPACE` | **YES** | **WarehousePG Clauses:**

Specify host file system locations for specific segment instances.

`WITH (contentID_1='/path/to/dir1...)` | +| `CREATE TABLESPACE` | **YES** | **WarehousePG Clauses:**

Specify host file system locations for specific segment instances.

`WITH (contentID_1='/path/to/dir1...)` | | `CREATE TRIGGER` | **NO** | | -| `CREATE TYPE` | YES | **WarehousePG Clauses:**

`COMPRESSTYPE \| COMPRESSLEVEL \| BLOCKSIZE`

**Limitations:**

The functions used to implement a new base type must be `IMMUTABLE` functions. | +| `CREATE TYPE` | YES | **WarehousePG Clauses:**

`COMPRESSTYPE \| COMPRESSLEVEL \| BLOCKSIZE`

**Limitations:**

The functions used to implement a new base type must be `IMMUTABLE` functions. | | `CREATE USER` | YES | An alias for [CREATE ROLE](../sql_commands/CREATE_ROLE.md) | | `CREATE VIEW` | YES | | | `DEALLOCATE` | YES | | -| `DECLARE` | YES | **Unsupported Clauses / Options:**

`SCROLL`

`FOR UPDATE [ OF column [, ...] ]`

**Limitations:**

Cursors cannot be backward-scrolled. Forward scrolling is supported.

PL/pgSQL does not have support for updatable cursors. | +| `DECLARE` | YES | **Unsupported Clauses / Options:**

`SCROLL`

`FOR UPDATE [ OF column [, ...] ]`

**Limitations:**

Cursors cannot be backward-scrolled. Forward scrolling is supported.

PL/pgSQL does not have support for updatable cursors. | | `DELETE` | YES | | | `DISCARD` | YES | **Limitation:** `DISCARD ALL` is not supported. | | `DO` | YES | PostgreSQL 9.0 feature | @@ -311,7 +311,7 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `END` | YES | | | `EXECUTE` | YES | | | `EXPLAIN` | YES | | -| `FETCH` | YES | **Unsupported Clauses / Options:**

`LAST`

`PRIOR`

`BACKWARD`

`BACKWARD ALL`

**Limitations:**

Cannot fetch rows in a nonsequential fashion; backward scan is not supported. | +| `FETCH` | YES | **Unsupported Clauses / Options:**

`LAST`

`PRIOR`

`BACKWARD`

`BACKWARD ALL`

**Limitations:**

Cannot fetch rows in a nonsequential fashion; backward scan is not supported. | | `GRANT` | YES | | | `INSERT` | YES | | | `LATERAL` Join Type | **NO** | | @@ -333,17 +333,17 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `ROLLBACK PREPARED` | **NO** | | | `ROLLBACK TO SAVEPOINT` | YES | | | `SAVEPOINT` | YES | | -| `SELECT` | YES | **Limitations:**

Limited use of `VOLATILE` and `STABLE` functions in `FROM` or `WHERE` clauses

Text search (`Tsearch2`) is not supported

**WarehousePG Clauses (OLAP):**

`[GROUP BY `*grouping_element*` [, ...]]`

`[WINDOW `*window_name*` AS (`*window_specification*`)]`

`[FILTER (WHERE `*condition*`)]` applied to an aggregate function in the `SELECT` list | +| `SELECT` | YES | **Limitations:**

Limited use of `VOLATILE` and `STABLE` functions in `FROM` or `WHERE` clauses

Text search (`Tsearch2`) is not supported

**WarehousePG Clauses (OLAP):**

`[GROUP BY `*grouping_element*` [, ...]]`

`[WINDOW `*window_name*` AS (`*window_specification*`)]`

`[FILTER (WHERE `*condition*`)]` applied to an aggregate function in the `SELECT` list | | `SELECT INTO` | YES | See [SELECT](../sql_commands/SELECT.md) | | `SET` | YES | | | `SET CONSTRAINTS` | **NO** | In PostgreSQL, this only applies to foreign key constraints, which are currently not enforced in WarehousePG. | | `SET ROLE` | YES | | | `SET SESSION AUTHORIZATION` | YES | Deprecated as of PostgreSQL 8.1 - see [SET ROLE](../sql_commands/SET_ROLE.md) | -| `SET TRANSACTION` | YES | **Limitations:**

`DEFERRABLE` clause has no effect.

`SET TRANSACTION SNAPSHOT` command is not supported. | +| `SET TRANSACTION` | YES | **Limitations:**

`DEFERRABLE` clause has no effect.

`SET TRANSACTION SNAPSHOT` command is not supported. | | `SHOW` | YES | | | `START TRANSACTION` | YES | | | `TRUNCATE` | YES | | | `UNLISTEN` | **YES** | | -| `UPDATE` | YES | **Limitations:**

`SET` not allowed for WarehousePG distribution key columns. | -| `VACUUM` | YES | **Limitations:**

`VACUUM FULL` is not recommended in WarehousePG. | +| `UPDATE` | YES | **Limitations:**

`SET` not allowed for WarehousePG distribution key columns. | +| `VACUUM` | YES | **Limitations:**

`VACUUM FULL` is not recommended in WarehousePG. | | `VALUES` | YES | | diff --git a/docs/whpg/7x/admin_guide/analytics/postGIS.md b/docs/whpg/7x/admin_guide/analytics/postGIS.md index 914bd86..2187502 100644 --- a/docs/whpg/7x/admin_guide/analytics/postGIS.md +++ b/docs/whpg/7x/admin_guide/analytics/postGIS.md @@ -34,11 +34,11 @@ This table lists the PostGIS extensions support by WarehousePG PostGIS. | PostGIS Extension | WarehousePG PostGIS Notes | | --- | --- | -| `postgis`

PostGIS and PostGIS Raster support | Supported. Both PostGIS and PostGIS Raster are enabled when the WarehousePG `postgis` extension is enabled. | -| `postgis_tiger_geocoder`

The US TIGER geocoder | Supported. Installed with WarehousePG PostGIS.

Requires the `postgis` and `fuzzystrmatch` extensions.

The US TIGER geocoder converts addresses (like a street address) to geographic coordinates. | -| `address_standardizer`

Rule-based address standardizer | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with TIGER geocoder.

A single line address parser that takes an input address and normalizes it based on a set of rules stored in a table and helper `lex` and `gaz` tables. | -| `address_standardizer_data_us`

Sample rules tables for US address data | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with the address standardizer.

The extension contains `gaz`, `lex`, and `rules` tables for US address data. If you are using other types of tables, see [PostGIS Extension Limitations](#topic_wy2_rkb_3p). | -| `fuzzystrmatch`

Fuzzy string matching | Supported. This extension is bundled but not enabled with WarehousePG Database.

Required for the PostGIS TIGER geocoder. | +| `postgis`

PostGIS and PostGIS Raster support | Supported. Both PostGIS and PostGIS Raster are enabled when the WarehousePG `postgis` extension is enabled. | +| `postgis_tiger_geocoder`

The US TIGER geocoder | Supported. Installed with WarehousePG PostGIS.

Requires the `postgis` and `fuzzystrmatch` extensions.

The US TIGER geocoder converts addresses (like a street address) to geographic coordinates. | +| `address_standardizer`

Rule-based address standardizer | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with TIGER geocoder.

A single line address parser that takes an input address and normalizes it based on a set of rules stored in a table and helper `lex` and `gaz` tables. | +| `address_standardizer_data_us`

Sample rules tables for US address data | Supported. Installed but not enabled with WarehousePG PostGIS.

Can be used with the address standardizer.

The extension contains `gaz`, `lex`, and `rules` tables for US address data. If you are using other types of tables, see [PostGIS Extension Limitations](#topic_wy2_rkb_3p). | +| `fuzzystrmatch`

Fuzzy string matching | Supported. This extension is bundled but not enabled with WarehousePG Database.

Required for the PostGIS TIGER geocoder. | > **Note** The PostGIS topology extension `postgis_topology` and the PostGIS 3D and geoprocessing extension `postgis_sfcgal` are not supported by WarehousePG PostGIS and are not included in the WarehousePG PostGIS extension package. diff --git a/docs/whpg/7x/admin_guide/ddl/ddl-index/ddl-brin.md b/docs/whpg/7x/admin_guide/ddl/ddl-index/ddl-brin.md index 39e3b8f..c8c11e8 100644 --- a/docs/whpg/7x/admin_guide/ddl/ddl-index/ddl-brin.md +++ b/docs/whpg/7x/admin_guide/ddl/ddl-index/ddl-brin.md @@ -51,7 +51,7 @@ The `minmax` operator classes store the minimum and the maximum values appearing | int8_minmax_ops | bigint | `< <= = >= >` | | | | | | bit_minmax_ops | bit | `< <= = >= >` | | | | | | varbit_minmax_ops | bit varying | `< <= = >= >` | | | | | -| box_inclusion_ops | box | \`<< &< && <> >> ~= @> <@ &< | << | | >> | &>\` | +| box_inclusion_ops | box | `<< &< && <> >> ~= @> <@ &<\| <<\| \|>> \|&>` | | | | | | bytea_minmax_ops | bytea | `< <= = >= >` | | | | | | bpchar_minmax_ops | character | `< <= = >= >` | | | | | | char_minmax_ops | "char" | `< <= = >= >` | | | | | @@ -67,7 +67,7 @@ The `minmax` operator classes store the minimum and the maximum values appearing | numeric_minmax_ops | numeric | `< <= = >= >` | | | | | | pg_lsn_minmax_ops | pg_lsn | `< <= = >= >` | | | | | | oid_minmax_ops | oid | `< <= = >= >` | | | | | -| range_inclusion_ops | any range type | \`<< &< && <> >> @> <@ - | - = < <= = > >=\` | | | | +| range_inclusion_ops | any range type | `<< &< && <> >> @> <@ -\|- = < <= = > >=` | | | | | | float4_minmax_ops | real | `< <= = >= >` | | | | | | int2_minmax_ops | smallint | `< <= = >= >` | | | | | | text_minmax_ops | text | `< <= = >= >` | | | | | diff --git a/docs/whpg/7x/admin_guide/external/external-tables/map_ext_to_foreign.md b/docs/whpg/7x/admin_guide/external/external-tables/map_ext_to_foreign.md index 03c6eca..253774c 100644 --- a/docs/whpg/7x/admin_guide/external/external-tables/map_ext_to_foreign.md +++ b/docs/whpg/7x/admin_guide/external/external-tables/map_ext_to_foreign.md @@ -23,7 +23,7 @@ The external table clause to foreign table option mapping follows: | ----------------------------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- | | CREATE EXTERNAL TABLE | `is_writable 'false'` | Readable external tables are not writable. | | | CREATE WRITABLE EXTERNAL TABLE | `is_writable 'true'` | Writable external tables are writable and alterable. | | -| `LOCATION ('' [, ...])` | `location_uris ''` | The location of the external data. `gp_exttable_fdw` uses a pipe (\` | \`) character to separate locations when you provide more than one. | +| `LOCATION ('' [, ...])` | `location_uris ''` | The location of the external data. `gp_exttable_fdw` uses a pipe (`\|`) character to separate locations when you provide more than one. | | | `FORMAT 'TEXT'` | `format 'text'` | The external data is text format. | | | `FORMAT 'CSV'` | `format 'csv'` | The external data is comma-separated value format. | | | `FORMAT 'CUSTOM'` | `format 'custom' formatter ''` | The external data is of a custom format, and WarehousePG uses the specified formatter to parse the data. | | diff --git a/docs/whpg/7x/admin_guide/external/foreign/devel-fdw.md b/docs/whpg/7x/admin_guide/external/foreign/devel-fdw.md index f896527..820c9b2 100644 --- a/docs/whpg/7x/admin_guide/external/foreign/devel-fdw.md +++ b/docs/whpg/7x/admin_guide/external/foreign/devel-fdw.md @@ -136,26 +136,26 @@ Scan-related callback functions include: | Callback Signature | Description | | --- | --- | -| `void`
`GetForeignRelSize (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Obtain relation size estimates for a foreign table. Called at the beginning of planning for a query on a foreign table. | -| `void`
`GetForeignPaths (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Create possible access paths for a scan on a foreign table. Called during query planning.

**Note:** A WarehousePG Database-compatible FDW must call `create_foreignscan_path()` in its `GetForeignPaths()` callback function. | -| `ForeignScan *`
`GetForeignPlan (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid,`
` ForeignPath *best_path,`
` List *tlist,`
` List *scan_clauses)` | Create a `ForeignScan` plan node from the selected foreign access path. Called at the end of query planning. | -| `void`
`BeginForeignScan (ForeignScanState *node,`
` int eflags)` | Begin running a foreign scan. Called during executor startup. | -| `TupleTableSlot *`
`IterateForeignScan (ForeignScanState *node)` | Fetch one row from the foreign source, returning it in a tuple table slot; return NULL if no more rows are available. | -| `void`
`ReScanForeignScan (ForeignScanState *node)` | Restart the scan from the beginning. | -| `void`
`EndForeignScan (ForeignScanState *node)` | End the scan and release resources. | +| `void`
`GetForeignRelSize (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Obtain relation size estimates for a foreign table. Called at the beginning of planning for a query on a foreign table. | +| `void`
`GetForeignPaths (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid)` | Create possible access paths for a scan on a foreign table. Called during query planning.

**Note:** A WarehousePG Database-compatible FDW must call `create_foreignscan_path()` in its `GetForeignPaths()` callback function. | +| `ForeignScan *`
`GetForeignPlan (PlannerInfo *root,`
` RelOptInfo *baserel,`
` Oid foreigntableid,`
` ForeignPath *best_path,`
` List *tlist,`
` List *scan_clauses)` | Create a `ForeignScan` plan node from the selected foreign access path. Called at the end of query planning. | +| `void`
`BeginForeignScan (ForeignScanState *node,`
` int eflags)` | Begin running a foreign scan. Called during executor startup. | +| `TupleTableSlot *`
`IterateForeignScan (ForeignScanState *node)` | Fetch one row from the foreign source, returning it in a tuple table slot; return NULL if no more rows are available. | +| `void`
`ReScanForeignScan (ForeignScanState *node)` | Restart the scan from the beginning. | +| `void`
`EndForeignScan (ForeignScanState *node)` | End the scan and release resources. | If a foreign data wrapper supports writable foreign tables, it should provide the update-related callback functions that are required by the capabilities of the FDW. Update-related callback functions include: | Callback Signature | Description | | --- | --- | -| `void`
`AddForeignUpdateTargets (Query *parsetree,`
` RangeTblEntry *target_rte,`
` Relation target_relation)` | Add additional information in the foreign table that will be retrieved during an update or delete operation to identify the exact row on which to operate. | -| `List *`
`PlanForeignModify (PlannerInfo *root,`
` ModifyTable *plan,`
` Index resultRelation,`
` int subplan_index)` | Perform additional planning actions required for an insert, update, or delete operation on a foreign table, and return the information generated. | -| `void`
`BeginForeignModify (ModifyTableState *mtstate,`
` ResultRelInfo *rinfo,`
` List *fdw_private,`
` int subplan_index,`
` int eflags)` | Begin executing a modify operation on a foreign table. Called during executor startup. | -| `TupleTableSlot *`
`ExecForeignInsert (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Insert a single tuple into the foreign table. Return a slot containing the data that was actually inserted, or NULL if no row was inserted. | -| `TupleTableSlot *`
`ExecForeignUpdate (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Update a single tuple in the foreign table. Return a slot containing the row as it was actually updated, or NULL if no row was updated. | -| `TupleTableSlot *`
`ExecForeignDelete (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Delete a single tuple from the foreign table. Return a slot containing the row that was deleted, or NULL if no row was deleted. | -| `void`
`EndForeignModify (EState *estate,`
` ResultRelInfo *rinfo)` | End the update and release resources. | -| `int`
`IsForeignRelUpdatable (Relation rel)` | Report the update operations supported by the specified foreign table. | +| `void`
`AddForeignUpdateTargets (Query *parsetree,`
` RangeTblEntry *target_rte,`
` Relation target_relation)` | Add additional information in the foreign table that will be retrieved during an update or delete operation to identify the exact row on which to operate. | +| `List *`
`PlanForeignModify (PlannerInfo *root,`
` ModifyTable *plan,`
` Index resultRelation,`
` int subplan_index)` | Perform additional planning actions required for an insert, update, or delete operation on a foreign table, and return the information generated. | +| `void`
`BeginForeignModify (ModifyTableState *mtstate,`
` ResultRelInfo *rinfo,`
` List *fdw_private,`
` int subplan_index,`
` int eflags)` | Begin executing a modify operation on a foreign table. Called during executor startup. | +| `TupleTableSlot *`
`ExecForeignInsert (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Insert a single tuple into the foreign table. Return a slot containing the data that was actually inserted, or NULL if no row was inserted. | +| `TupleTableSlot *`
`ExecForeignUpdate (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Update a single tuple in the foreign table. Return a slot containing the row as it was actually updated, or NULL if no row was updated. | +| `TupleTableSlot *`
`ExecForeignDelete (EState *estate,`
` ResultRelInfo *rinfo,`
` TupleTableSlot *slot,`
` TupleTableSlot *planSlot)` | Delete a single tuple from the foreign table. Return a slot containing the row that was deleted, or NULL if no row was deleted. | +| `void`
`EndForeignModify (EState *estate,`
` ResultRelInfo *rinfo)` | End the update and release resources. | +| `int`
`IsForeignRelUpdatable (Relation rel)` | Report the update operations supported by the specified foreign table. | Refer to [Foreign Data Wrapper Callback Routines](https://www.postgresql.org/docs/12/fdw-callbacks.html) in the PostgreSQL documentation for detailed information about the inputs and outputs of the FDW callback functions. @@ -173,13 +173,13 @@ The FDW API includes the helper functions listed in the table below. Refer to [F | Helper Signature | Description | | --- | --- | -| `ForeignDataWrapper *`
`GetForeignDataWrapper(Oid fdwid);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given OID. | -| `ForeignDataWrapper *`
`GetForeignDataWrapperByName(const char *name, bool missing_ok);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given name. | -| `ForeignServer *`
`GetForeignServer(Oid serverid);` | Returns the `ForeignServer` object for the foreign server with the given OID. | -| `ForeignServer *`
`GetForeignServerByName(const char *name, bool missing_ok);` | Returns the `ForeignServer` object for the foreign server with the given name. | -| `UserMapping *`
`GetUserMapping(Oid userid, Oid serverid);` | Returns the `UserMapping` object for the user mapping of the given role on the given server. | -| `ForeignTable *`
`GetForeignTable(Oid relid);` | Returns the `ForeignTable` object for the foreign table with the given OID. | -| `List *`
`GetForeignColumnOptions(Oid relid, AttrNumber attnum);` | Returns the per-column FDW options for the column with the given foreign table OID and attribute number. | +| `ForeignDataWrapper *`
`GetForeignDataWrapper(Oid fdwid);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given OID. | +| `ForeignDataWrapper *`
`GetForeignDataWrapperByName(const char *name, bool missing_ok);` | Returns the `ForeignDataWrapper` object for the foreign-data wrapper with the given name. | +| `ForeignServer *`
`GetForeignServer(Oid serverid);` | Returns the `ForeignServer` object for the foreign server with the given OID. | +| `ForeignServer *`
`GetForeignServerByName(const char *name, bool missing_ok);` | Returns the `ForeignServer` object for the foreign server with the given name. | +| `UserMapping *`
`GetUserMapping(Oid userid, Oid serverid);` | Returns the `UserMapping` object for the user mapping of the given role on the given server. | +| `ForeignTable *`
`GetForeignTable(Oid relid);` | Returns the `ForeignTable` object for the foreign table with the given OID. | +| `List *`
`GetForeignColumnOptions(Oid relid, AttrNumber attnum);` | Returns the per-column FDW options for the column with the given foreign table OID and attribute number. | diff --git a/docs/whpg/7x/admin_guide/intro/about_statistics.md b/docs/whpg/7x/admin_guide/intro/about_statistics.md index f459a20..03dc371 100644 --- a/docs/whpg/7x/admin_guide/intro/about_statistics.md +++ b/docs/whpg/7x/admin_guide/intro/about_statistics.md @@ -77,11 +77,11 @@ The `stakindN` columns each contain a numeric code to describe the type of stati | stakind Code | Description | | --- | --- | -| 1 | *Most CommonValues (MCV) Slot*

- `staop` contains the object ID of the "=" operator, used to decide whether values are the same or not.
- `stavalues` contains an array of the *K* most common non-null values appearing in the column.
- `stanumbers` contains the frequencies (fractions of total row count) of the values in the `stavalues` array.

The values are ordered in decreasing frequency. Since the arrays are variable-size, *K* can be chosen by the statistics collector. Values must occur more than once to be added to the `stavalues` array; a unique column has no MCV slot. | -| 2 | *Histogram Slot* – describes the distribution of scalar data.

- `staop` is the object ID of the "<" operator, which describes the sort ordering.
- `stavalues` contains *M* (where `M>=2`) non-null values that divide the non-null column data values into `M-1` bins of approximately equal population. The first `stavalues` item is the minimum value and the last is the maximum value.
- `stanumbers` is not used and should be `NULL`.

If a Most Common Values slot is also provided, then the histogram describes the data distribution after removing the values listed in the MCV array. (It is a *compressed histogram* in the technical parlance). This allows a more accurate representation of the distribution of a column with some very common values. In a column with only a few distinct values, it is possible that the MCV list describes the entire data population; in this case the histogram reduces to empty and should be omitted. | -| 3 | *Correlation Slot* – describes the correlation between the physical order of table tuples and the ordering of data values of this column.

- `staop` is the object ID of the "<" operator. As with the histogram, more than one entry could theoretically appear.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains a single entry, the correlation coefficient between the sequence of data values and the sequence of their actual tuple positions. The coefficient ranges from +1 to -1. | -| 4 | *Most Common Elements Slot* - is similar to a Most Common Values (MCV) Slot, except that it stores the most common non-null *elements* of the column values. This is useful when the column datatype is an array or some other type with identifiable elements (for instance, `tsvector`).

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` contains the most common element values.
- `stanumbers` contains common element frequencies.

Frequencies are measured as the fraction of non-null rows the element value appears in, not the frequency of all rows. Also, the values are sorted into the element type's default order (to support binary search for a particular value). Since this puts the minimum and maximum frequencies at unpredictable spots in `stanumbers`, there are two extra members of `stanumbers` that hold copies of the minimum and maximum frequencies. Optionally, there can be a third extra member that holds the frequency of null elements (the frequency is expressed in the same terms: the fraction of non-null rows that contain at least one null element). If this member is omitted, the column is presumed to contain no `NULL` elements.

**Note:** For `tsvector` columns, the `stavalues` elements are of type `text`, even though their representation within `tsvector` is not exactly `text`. | -| 5 | *Distinct Elements Count Histogram Slot* - describes the distribution of the number of distinct element values present in each row of an array-type column. Only non-null rows are considered, and only non-null elements.

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains information about distinct elements. The last member of `stanumbers` is the average count of distinct element values over all non-null rows. The preceding *M* (where `M >=2`) members form a histogram that divides the population of distinct-elements counts into `M-1` bins of approximately equal population. The first of these is the minimum observed count, and the last the maximum. | +| 1 | *Most CommonValues (MCV) Slot*

- `staop` contains the object ID of the "=" operator, used to decide whether values are the same or not.
- `stavalues` contains an array of the *K* most common non-null values appearing in the column.
- `stanumbers` contains the frequencies (fractions of total row count) of the values in the `stavalues` array.

The values are ordered in decreasing frequency. Since the arrays are variable-size, *K* can be chosen by the statistics collector. Values must occur more than once to be added to the `stavalues` array; a unique column has no MCV slot. | +| 2 | *Histogram Slot* – describes the distribution of scalar data.

- `staop` is the object ID of the "<" operator, which describes the sort ordering.
- `stavalues` contains *M* (where `M>=2`) non-null values that divide the non-null column data values into `M-1` bins of approximately equal population. The first `stavalues` item is the minimum value and the last is the maximum value.
- `stanumbers` is not used and should be `NULL`.

If a Most Common Values slot is also provided, then the histogram describes the data distribution after removing the values listed in the MCV array. (It is a *compressed histogram* in the technical parlance). This allows a more accurate representation of the distribution of a column with some very common values. In a column with only a few distinct values, it is possible that the MCV list describes the entire data population; in this case the histogram reduces to empty and should be omitted. | +| 3 | *Correlation Slot* – describes the correlation between the physical order of table tuples and the ordering of data values of this column.

- `staop` is the object ID of the "<" operator. As with the histogram, more than one entry could theoretically appear.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains a single entry, the correlation coefficient between the sequence of data values and the sequence of their actual tuple positions. The coefficient ranges from +1 to -1. | +| 4 | *Most Common Elements Slot* - is similar to a Most Common Values (MCV) Slot, except that it stores the most common non-null *elements* of the column values. This is useful when the column datatype is an array or some other type with identifiable elements (for instance, `tsvector`).

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` contains the most common element values.
- `stanumbers` contains common element frequencies.

Frequencies are measured as the fraction of non-null rows the element value appears in, not the frequency of all rows. Also, the values are sorted into the element type's default order (to support binary search for a particular value). Since this puts the minimum and maximum frequencies at unpredictable spots in `stanumbers`, there are two extra members of `stanumbers` that hold copies of the minimum and maximum frequencies. Optionally, there can be a third extra member that holds the frequency of null elements (the frequency is expressed in the same terms: the fraction of non-null rows that contain at least one null element). If this member is omitted, the column is presumed to contain no `NULL` elements.

**Note:** For `tsvector` columns, the `stavalues` elements are of type `text`, even though their representation within `tsvector` is not exactly `text`. | +| 5 | *Distinct Elements Count Histogram Slot* - describes the distribution of the number of distinct element values present in each row of an array-type column. Only non-null rows are considered, and only non-null elements.

- `staop` contains the equality operator appropriate to the element type.
- `stavalues` is not used and should be `NULL`.
- `stanumbers` contains information about distinct elements. The last member of `stanumbers` is the average count of distinct element values over all non-null rows. The preceding *M* (where `M >=2`) members form a histogram that divides the population of distinct-elements counts into `M-1` bins of approximately equal population. The first of these is the minimum observed count, and the last the maximum. | | 99 | *Hyperloglog Slot* - for leaf partitions of a partitioned table, stores the `hyperloglog_counter` created for the sampled data. The `hyperloglog_counter` data structure is converted into a `bytea` and stored in a `stavalues5` slot of the `pg_statistic` catalog table. | diff --git a/docs/whpg/7x/admin_guide/manage_access/roles_privs.md b/docs/whpg/7x/admin_guide/manage_access/roles_privs.md index b1a4d59..62f7f46 100644 --- a/docs/whpg/7x/admin_guide/manage_access/roles_privs.md +++ b/docs/whpg/7x/admin_guide/manage_access/roles_privs.md @@ -123,19 +123,19 @@ When an object (table, view, sequence, database, function, language, schema, or | Object Type | Privileges | | --- | --- | -| Tables, External Tables, Views | `SELECT`

`INSERT`

`UPDATE`

`DELETE`

`REFERENCES`

`TRIGGER`

`TRUNCATE`

`ALL` | -| Columns | `SELECT`

`INSERT`

`UPDATE`

`REFERENCES`

`ALL` | -| Sequences | `USAGE`

`SELECT`

`UPDATE`

`ALL` | -| Databases | `CREATE`

`CONNECT`

`TEMPORARY`

`TEMP`

`ALL` | -| Domains | `USAGE`

`ALL` | -| Foreign Data Wrappers | `USAGE`

`ALL` | -| Foreign Servers | `USAGE`

`ALL` | -| Functions | `EXECUTE`

`ALL` | -| Procedural Languages | `USAGE`

`ALL` | -| Schemas | `CREATE`

`USAGE`

`ALL` | -| Tablespaces | `CREATE`

`ALL` | -| Types | `USAGE`

`ALL` | -| Protocols | `SELECT`

`INSERT`

`ALL` | +| Tables, External Tables, Views | `SELECT`

`INSERT`

`UPDATE`

`DELETE`

`REFERENCES`

`TRIGGER`

`TRUNCATE`

`ALL` | +| Columns | `SELECT`

`INSERT`

`UPDATE`

`REFERENCES`

`ALL` | +| Sequences | `USAGE`

`SELECT`

`UPDATE`

`ALL` | +| Databases | `CREATE`

`CONNECT`

`TEMPORARY`

`TEMP`

`ALL` | +| Domains | `USAGE`

`ALL` | +| Foreign Data Wrappers | `USAGE`

`ALL` | +| Foreign Servers | `USAGE`

`ALL` | +| Functions | `EXECUTE`

`ALL` | +| Procedural Languages | `USAGE`

`ALL` | +| Schemas | `CREATE`

`USAGE`

`ALL` | +| Tablespaces | `CREATE`

`ALL` | +| Types | `USAGE`

`ALL` | +| Protocols | `SELECT`

`INSERT`

`ALL` | > **Note** You must grant privileges for each object individually. For example, granting `ALL` on a database does not grant full access to the objects within that database. It only grants all of the database-level privileges (`CONNECT`, `CREATE`, `TEMPORARY`) to the database itself. diff --git a/docs/whpg/7x/admin_guide/managing/monitoring.md b/docs/whpg/7x/admin_guide/managing/monitoring.md index c4b4ca7..2b61137 100644 --- a/docs/whpg/7x/admin_guide/managing/monitoring.md +++ b/docs/whpg/7x/admin_guide/managing/monitoring.md @@ -22,12 +22,12 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| List segments that are currently down. If any rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE status = 'd';` | If the query returns any rows, follow these steps to correct the problem:

1. Verify that the hosts with down segments are responsive.
2. If hosts are OK, check the log files for the primaries and mirrors of the down segments to discover the root cause of the segments going down.
3. If no unexpected errors are found, run the `gprecoverseg` utility to bring the segments back online. | -| Check for segments that are up and not in sync. If rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE mode = 'n' and status = 'u' and content <> -1;` | If the query returns rows then the segment might be in the process of moving from `Not In Sync` to `Synchronized` mode. Use `gpstate -e` to track progress. | -| Check for segments that are not operating in their preferred role but are marked as up and `Synchronized`. If any segments are found, the cluster may not be balanced. If any rows are returned this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE preferred_role <> role and status = 'u' and mode = 's';` | When the segments are not running in their preferred role, processing might be skewed. Run `gprecoverseg -r` to bring the segments back into their preferred roles. | -| Run a distributed query to test that it runs on all segments. One row should be returned for each primary segment.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Execute the following query in the `postgres` database:

`SELECT gp_segment_id, count(*) FROM gp_dist_random('pg_class') GROUP BY 1;` | If this query fails, there is an issue dispatching to some segments in the cluster. This is a rare event. Check the hosts that are not able to be dispatched to ensure there is no hardware or networking issue. | -| Test the state of coordinator mirroring on WarehousePG. If the value is not "STREAMING", raise an alert or warning.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following `psql` command:

`psql <dbname> -c 'SELECT pid, state FROM pg_stat_replication;'` | Check the log file from the coordinator and standby coordinator for errors. If there are no unexpected errors and the machines are up, run the `gpinitstandby` utility to bring the standby online. | -| Perform a basic check to see if the coordinator is up and functioning.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Run the following query in the `postgres` database:

`SELECT count(*) FROM gp_segment_configuration;` | If this query fails, the active coordinator may be down. Try to start the database on the original coordinator if the server is up and running. If that fails, try to activate the standby coordinator as coordinator. | +| List segments that are currently down. If any rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE status = 'd';` | If the query returns any rows, follow these steps to correct the problem:

1. Verify that the hosts with down segments are responsive.
2. If hosts are OK, check the log files for the primaries and mirrors of the down segments to discover the root cause of the segments going down.
3. If no unexpected errors are found, run the `gprecoverseg` utility to bring the segments back online. | +| Check for segments that are up and not in sync. If rows are returned, this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE mode = 'n' and status = 'u' and content <> -1;` | If the query returns rows then the segment might be in the process of moving from `Not In Sync` to `Synchronized` mode. Use `gpstate -e` to track progress. | +| Check for segments that are not operating in their preferred role but are marked as up and `Synchronized`. If any segments are found, the cluster may not be balanced. If any rows are returned this should generate a warning or alert.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Execute the following query in the `postgres` database:

`SELECT * FROM gp_segment_configuration WHERE preferred_role <> role and status = 'u' and mode = 's';` | When the segments are not running in their preferred role, processing might be skewed. Run `gprecoverseg -r` to bring the segments back into their preferred roles. | +| Run a distributed query to test that it runs on all segments. One row should be returned for each primary segment.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Execute the following query in the `postgres` database:

`SELECT gp_segment_id, count(*) FROM gp_dist_random('pg_class') GROUP BY 1;` | If this query fails, there is an issue dispatching to some segments in the cluster. This is a rare event. Check the hosts that are not able to be dispatched to ensure there is no hardware or networking issue. | +| Test the state of coordinator mirroring on WarehousePG. If the value is not "STREAMING", raise an alert or warning.

Recommended frequency: run every 5 to 10 minutes

Severity: IMPORTANT | Run the following `psql` command:

`psql <dbname> -c 'SELECT pid, state FROM pg_stat_replication;'` | Check the log file from the coordinator and standby coordinator for errors. If there are no unexpected errors and the machines are up, run the `gpinitstandby` utility to bring the standby online. | +| Perform a basic check to see if the coordinator is up and functioning.

Recommended frequency: run every 5 to 10 minutes

Severity: CRITICAL | Run the following query in the `postgres` database:

`SELECT count(*) FROM gp_segment_configuration;` | If this query fails, the active coordinator may be down. Try to start the database on the original coordinator if the server is up and running. If that fails, try to activate the standby coordinator as coordinator. | ## Hardware and Operating System Monitoring @@ -37,10 +37,10 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| Check disk space usage on volumes used for WarehousePG data storage and the OS.

Recommended frequency: every 5 to 30 minutes

Severity: CRITICAL | Set up a disk space check.

- Set a threshold to raise an alert when a disk reaches a percentage of capacity. The recommended threshold is 75% full.
- It is not recommended to run the system with capacities approaching 100%. | Use `VACUUM`/`VACUUM FULL` on user tables to reclaim space occupied by dead rows. | -| Check for errors or dropped packets on the network interfaces.

Recommended frequency: hourly

Severity: IMPORTANT | Set up a network interface checks. | Work with network and OS teams to resolve errors. | -| Check for RAID errors or degraded RAID performance.

Recommended frequency: every 5 minutes

Severity: CRITICAL | Set up a RAID check. | - Replace failed disks as soon as possible.
- Work with system administration team to resolve other RAID or controller errors as soon as possible. | -| Check for adequate I/O bandwidth and I/O skew.

Recommended frequency: when create a cluster or when hardware issues are suspected. | Run the WarehousePG `gpcheckperf` utility. | The cluster may be under-specified if data transfer rates are not similar to the following:

- 2GB per second disk read
- 1 GB per second disk write
- 10 Gigabit per second network read and write

If transfer rates are lower than expected, consult with your data architect regarding performance expectations.

If the machines on the cluster display an uneven performance profile, work with the system administration team to fix faulty machines. | +| Check disk space usage on volumes used for WarehousePG data storage and the OS.

Recommended frequency: every 5 to 30 minutes

Severity: CRITICAL | Set up a disk space check.

- Set a threshold to raise an alert when a disk reaches a percentage of capacity. The recommended threshold is 75% full.
- It is not recommended to run the system with capacities approaching 100%. | Use `VACUUM`/`VACUUM FULL` on user tables to reclaim space occupied by dead rows. | +| Check for errors or dropped packets on the network interfaces.

Recommended frequency: hourly

Severity: IMPORTANT | Set up a network interface checks. | Work with network and OS teams to resolve errors. | +| Check for RAID errors or degraded RAID performance.

Recommended frequency: every 5 minutes

Severity: CRITICAL | Set up a RAID check. | - Replace failed disks as soon as possible.
- Work with system administration team to resolve other RAID or controller errors as soon as possible. | +| Check for adequate I/O bandwidth and I/O skew.

Recommended frequency: when create a cluster or when hardware issues are suspected. | Run the WarehousePG `gpcheckperf` utility. | The cluster may be under-specified if data transfer rates are not similar to the following:

- 2GB per second disk read
- 1 GB per second disk write
- 10 Gigabit per second network read and write

If transfer rates are lower than expected, consult with your data architect regarding performance expectations.

If the machines on the cluster display an uneven performance profile, work with the system administration team to fix faulty machines. | ## Catalog Monitoring @@ -50,11 +50,11 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| Run catalog consistency checks in each database to ensure the catalog on each host in the cluster is consistent and in a good state.

You may run this command while the database is up and running.

Recommended frequency: weekly

Severity: IMPORTANT | Run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -O`

**Note:** With the `-O` option, `gpcheckcat` runs just 10 of its usual 15 tests. | Run the repair scripts for any issues identified. | -| Check for `pg_class` entries that have no corresponding pg_`attribute` entry.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R pgclass` | Run the repair scripts for any issues identified. | -| Check for leaked temporary schema and missing schema definition.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R namespace` | Run the repair scripts for any issues identified. | -| Check constraints on randomly distributed tables.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R distribution_policy` | Run the repair scripts for any issues identified. | -| Check for dependencies on non-existent objects.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R dependency` | Run the repair scripts for any issues identified. | +| Run catalog consistency checks in each database to ensure the catalog on each host in the cluster is consistent and in a good state.

You may run this command while the database is up and running.

Recommended frequency: weekly

Severity: IMPORTANT | Run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -O`

**Note:** With the `-O` option, `gpcheckcat` runs just 10 of its usual 15 tests. | Run the repair scripts for any issues identified. | +| Check for `pg_class` entries that have no corresponding pg_`attribute` entry.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R pgclass` | Run the repair scripts for any issues identified. | +| Check for leaked temporary schema and missing schema definition.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R namespace` | Run the repair scripts for any issues identified. | +| Check constraints on randomly distributed tables.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R distribution_policy` | Run the repair scripts for any issues identified. | +| Check for dependencies on non-existent objects.

Recommended frequency: monthly

Severity: IMPORTANT | During a downtime, with no users on the system, run the WarehousePG `gpcheckcat` utility in each database:

`gpcheckcat -R dependency` | Run the repair scripts for any issues identified. | ## Data Maintenance @@ -64,8 +64,8 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| Check for missing statistics on tables. | Check the `gp_stats_missing` view in each database:

`SELECT * FROM gp_toolkit.gp_stats_missing;` | Run `ANALYZE` on tables that are missing statistics. | -| Check for tables that have bloat (dead space) in data files that cannot be recovered by a regular `VACUUM` command.

Recommended frequency: weekly or monthly

Severity: WARNING | Check the `gp_bloat_diag` view in each database:

`SELECT * FROM gp_toolkit.gp_bloat_diag;` | `VACUUM FULL` acquires an `ACCESS EXCLUSIVE` lock on tables. Run `VACUUM FULL` during a time when users and applications do not require access to the tables, such as during a time of low activity, or during a maintenance window. | +| Check for missing statistics on tables. | Check the `gp_stats_missing` view in each database:

`SELECT * FROM gp_toolkit.gp_stats_missing;` | Run `ANALYZE` on tables that are missing statistics. | +| Check for tables that have bloat (dead space) in data files that cannot be recovered by a regular `VACUUM` command.

Recommended frequency: weekly or monthly

Severity: WARNING | Check the `gp_bloat_diag` view in each database:

`SELECT * FROM gp_toolkit.gp_bloat_diag;` | `VACUUM FULL` acquires an `ACCESS EXCLUSIVE` lock on tables. Run `VACUUM FULL` during a time when users and applications do not require access to the tables, such as during a time of low activity, or during a maintenance window. | ## Database Maintenance @@ -144,6 +144,6 @@ It is not necessary to implement all of these suggestions in every cluster; use | Activity | Procedure | Corrective Actions | | --- | --- | --- | -| Ensure any bug fixes or enhancements are applied to the kernel.

Recommended frequency: at least every 6 months

Severity: IMPORTANT | Follow the vendor's instructions to update the Linux kernel. | Keep the kernel current to include bug fixes and security fixes, and to avoid difficult future upgrades. | -| Install WarehousePG minor releases, for example 5.0.*x*.

Recommended frequency: quarterly

Severity: IMPORTANT | Follow upgrade instructions in the WarehousePG *Release Notes*. Always upgrade to the latest in the series. | Keep the WarehousePG software current to incorporate bug fixes, performance enhancements, and feature enhancements into your WarehousePG cluster. | +| Ensure any bug fixes or enhancements are applied to the kernel.

Recommended frequency: at least every 6 months

Severity: IMPORTANT | Follow the vendor's instructions to update the Linux kernel. | Keep the kernel current to include bug fixes and security fixes, and to avoid difficult future upgrades. | +| Install WarehousePG minor releases, for example 5.0.*x*.

Recommended frequency: quarterly

Severity: IMPORTANT | Follow upgrade instructions in the WarehousePG *Release Notes*. Always upgrade to the latest in the series. | Keep the WarehousePG software current to incorporate bug fixes, performance enhancements, and feature enhancements into your WarehousePG cluster. | diff --git a/docs/whpg/7x/admin_guide/query/functions-operators.md b/docs/whpg/7x/admin_guide/query/functions-operators.md index 49aff5d..c5e93b7 100644 --- a/docs/whpg/7x/admin_guide/query/functions-operators.md +++ b/docs/whpg/7x/admin_guide/query/functions-operators.md @@ -114,26 +114,26 @@ WarehousePG includes JSON processing functions that manipulate values the `json` | --- | --- | --- | --- | | [Logical Operators](https://www.postgresql.org/docs/12/functions-logical.html) | | | | | [Comparison Operators](https://www.postgresql.org/docs/12/functions-comparison.html) | | | | -| [Mathematical Functions and Operators](https://www.postgresql.org/docs/12/functions-math.html) | random

setseed | | | -| [String Functions and Operators](https://www.postgresql.org/docs/12/functions-string.html) | *All built-in conversion functions* | convert

pg_client_encoding | | +| [Mathematical Functions and Operators](https://www.postgresql.org/docs/12/functions-math.html) | random

setseed | | | +| [String Functions and Operators](https://www.postgresql.org/docs/12/functions-string.html) | *All built-in conversion functions* | convert

pg_client_encoding | | | [Binary String Functions and Operators](https://www.postgresql.org/docs/12/functions-binarystring.html) | | | | | [Bit String Functions and Operators](https://www.postgresql.org/docs/12/functions-bitstring.html) | | | | | [Pattern Matching](https://www.postgresql.org/docs/12/functions-matching.html) | | | | -| [Data Type Formatting Functions](https://www.postgresql.org/docs/12/functions-formatting.html) | | to_char

to_timestamp | | -| [Date/Time Functions and Operators](https://www.postgresql.org/docs/12/functions-datetime.html) | timeofday | age

current_date

current_time

current_timestamp

localtime

localtimestamp

now | | +| [Data Type Formatting Functions](https://www.postgresql.org/docs/12/functions-formatting.html) | | to_char

to_timestamp | | +| [Date/Time Functions and Operators](https://www.postgresql.org/docs/12/functions-datetime.html) | timeofday | age

current_date

current_time

current_timestamp

localtime

localtimestamp

now | | | [Enum Support Functions](https://www.postgresql.org/docs/12/functions-enum.html) | | | | | [Geometric Functions and Operators](https://www.postgresql.org/docs/12/functions-geometry.html) | | | | | [Network Address Functions and Operators](https://www.postgresql.org/docs/12/functions-net.html) | | | | -| [Sequence Manipulation Functions](https://www.postgresql.org/docs/12/functions-sequence.html) | nextval()

setval() | | | +| [Sequence Manipulation Functions](https://www.postgresql.org/docs/12/functions-sequence.html) | nextval()

setval() | | | | [Conditional Expressions](https://www.postgresql.org/docs/12/functions-conditional.html) | | | | | [Array Functions and Operators](https://www.postgresql.org/docs/12/functions-array.html) | | *All array functions* | | | [Aggregate Functions](https://www.postgresql.org/docs/12/functions-aggregate.html) | | | | | [Subquery Expressions](https://www.postgresql.org/docs/12/functions-subquery.html) | | | | | [Row and Array Comparisons](https://www.postgresql.org/docs/12/functions-comparisons.html) | | | | | [Set Returning Functions](https://www.postgresql.org/docs/12/functions-srf.html) | generate_series | | | -| [System Information Functions](https://www.postgresql.org/docs/12/functions-info.html) | | *All session information functions*

*All access privilege inquiry functions*

*All schema visibility inquiry functions*

*All system catalog information functions*

*All comment information functions*

*All transaction ids and snapshots* | | -| [System Administration Functions](https://www.postgresql.org/docs/12/functions-admin.html) | set_config

pg_cancel_backend

pg_terminate_backend

pg_reload_conf

pg_rotate_logfile

pg_start_backup

pg_stop_backup

pg_size_pretty

pg_ls_dir

pg_read_file

pg_stat_file | current_setting

*All database object size functions* | **Note:** The function `pg_column_size` displays bytes required to store the value, possibly with TOAST compression. | -| [XML Functions](https://www.postgresql.org/docs/12/functions-xml.html) and function-like expressions | | cursor_to_xml(cursor refcursor, count int, nulls boolean, tableforest boolean, targetns text)

cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest boolean, targetns text)

database_to_xml(nulls boolean, tableforest boolean, targetns text)

database_to_xmlschema(nulls boolean, tableforest boolean, targetns text)

database_to_xml_and_xmlschema( nulls boolean, tableforest boolean, targetns text)

query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xmlschema(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xml_and_xmlschema( query text, nulls boolean, tableforest boolean, targetns text)

schema_to_xml(schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xml_and_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

table_to_xml(tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xml_and_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

xmlagg(xml)

xmlconcat(xml[, ...])

xmlelement(name name [, xmlattributes(value [AS attname] [, ... ])] [, content, ...])

xmlexists(text, xml)

xmlforest(content [AS name] [, ...])

xml_is_well_formed(text)

xml_is_well_formed_document(text)

xml_is_well_formed_content(text)

xmlparse ( { DOCUMENT \| CONTENT } value)

xpath(text, xml)

xpath(text, xml, text[])

xpath_exists(text, xml)

xpath_exists(text, xml, text[])

xmlpi(name target [, content])

xmlroot(xml, version text \| no value [, standalone yes\|no\|no value])

xmlserialize ( { DOCUMENT \| CONTENT } value AS type )

xml(text)

text(xml)

xmlcomment(xml)

xmlconcat2(xml, xml) | | +| [System Information Functions](https://www.postgresql.org/docs/12/functions-info.html) | | *All session information functions*

*All access privilege inquiry functions*

*All schema visibility inquiry functions*

*All system catalog information functions*

*All comment information functions*

*All transaction ids and snapshots* | | +| [System Administration Functions](https://www.postgresql.org/docs/12/functions-admin.html) | set_config

pg_cancel_backend

pg_terminate_backend

pg_reload_conf

pg_rotate_logfile

pg_start_backup

pg_stop_backup

pg_size_pretty

pg_ls_dir

pg_read_file

pg_stat_file | current_setting

*All database object size functions* | **Note:** The function `pg_column_size` displays bytes required to store the value, possibly with TOAST compression. | +| [XML Functions](https://www.postgresql.org/docs/12/functions-xml.html) and function-like expressions | | cursor_to_xml(cursor refcursor, count int, nulls boolean, tableforest boolean, targetns text)

cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest boolean, targetns text)

database_to_xml(nulls boolean, tableforest boolean, targetns text)

database_to_xmlschema(nulls boolean, tableforest boolean, targetns text)

database_to_xml_and_xmlschema( nulls boolean, tableforest boolean, targetns text)

query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xmlschema(query text, nulls boolean, tableforest boolean, targetns text)

query_to_xml_and_xmlschema( query text, nulls boolean, tableforest boolean, targetns text)

schema_to_xml(schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

schema_to_xml_and_xmlschema( schema name, nulls boolean, tableforest boolean, targetns text)

table_to_xml(tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

table_to_xml_and_xmlschema( tbl regclass, nulls boolean, tableforest boolean, targetns text)

xmlagg(xml)

xmlconcat(xml[, ...])

xmlelement(name name [, xmlattributes(value [AS attname] [, ... ])] [, content, ...])

xmlexists(text, xml)

xmlforest(content [AS name] [, ...])

xml_is_well_formed(text)

xml_is_well_formed_document(text)

xml_is_well_formed_content(text)

xmlparse ( { DOCUMENT \| CONTENT } value)

xpath(text, xml)

xpath(text, xml, text[])

xpath_exists(text, xml)

xpath_exists(text, xml, text[])

xmlpi(name target [, content])

xmlroot(xml, version text \| no value [, standalone yes\|no\|no value])

xmlserialize ( { DOCUMENT \| CONTENT } value AS type )

xml(text)

text(xml)

xmlcomment(xml)

xmlconcat2(xml, xml) | | @@ -165,12 +165,46 @@ The following built-in advanced aggregate functions are WarehousePG extensions o > **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [WarehousePG MADlib Extension for Analytics](../analytics/madlib.md) in the *WarehousePG Reference Guide*. -**Advanced Aggregate Functions** - | Function | Return Type | Full Syntax | Description | | --- | --- | --- | --- | -| `MEDIAN (*expr*)` | `timestamp, timestamptz, interval, float` | `MEDIAN (*expression*)`

*Example:*

`SELECT departmzent_id, MEDIAN(salary)
FROM employees
GROUP BY department_id;` | Can take a two-dimensional array as input. Treats such arrays as matrices. | -| `sum(array[])` | `smallint[], int[], bigint[], float[]` | `sum(array[[1,2],[3,4]])`

*Example:*

`CREATE TABLE mymatrix (myvalue int[]);
INSERT INTO mymatrix
VALUES (array[[1,2],[3,4]]);
INSERT INTO mymatrix
VALUES (array[[0,1],[1,0]]);
SELECT sum(myvalue) FROM mymatrix;
sum
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
{1,3},{4,4}` | Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. | | `pivot_sum (label[], label, expr)` | `int[], bigint[], float[]` | `pivot_sum( array['A1','A2'], attr, value)` | A pivot aggregation using sum to resolve duplicate entries. | | `unnest (array[])` | set of `anyelement` | `unnest( array['one', 'row', 'per', 'item'])` | Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudo-type](https://www.postgresql.org/docs/12/datatype-pseudo.html) in PostgreSQL. | +### MEDIAN (*expr*) + +Return type: `timestamp, timestamptz, interval, float` + +Full syntax: `MEDIAN (*expression*)` + +Can take a two-dimensional array as input. Treats such arrays as matrices. + +Example: + +``` +SELECT departmzent_id, MEDIAN(salary) + FROM employees +GROUP BY department_id; +``` + +### sum(array[]) + +Return type: `smallint[], int[], bigint[], float[]` + +Full syntax: `sum(array[[1,2],[3,4]])` + +Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. + +Example: + +``` +CREATE TABLE mymatrix (myvalue int[]); +INSERT INTO mymatrix + VALUES (array[[1,2],[3,4]]); +INSERT INTO mymatrix + VALUES (array[[0,1],[1,0]]); +SELECT sum(myvalue) FROM mymatrix; + sum +------------- + {1,3},{4,4} +``` + diff --git a/docs/whpg/7x/admin_guide/query/json-data.md b/docs/whpg/7x/admin_guide/query/json-data.md index ced19f9..214a06c 100644 --- a/docs/whpg/7x/admin_guide/query/json-data.md +++ b/docs/whpg/7x/admin_guide/query/json-data.md @@ -411,10 +411,10 @@ For details on using `jsonpath` expressions with SQL/JSON query functions, see [ | Accessor Operator | Description | | --- | --- | -| .*`key`*

."$*`varname`*" | Member accessor that returns an object member with the specified key. If the key name is a named variable starting with `$` or does not meet the JavaScript rules of an identifier, it must be enclosed in double quotes as a character string literal. | +| .*`key`*

."$*`varname`*" | Member accessor that returns an object member with the specified key. If the key name is a named variable starting with `$` or does not meet the JavaScript rules of an identifier, it must be enclosed in double quotes as a character string literal. | | `.*` | Wildcard member accessor that returns the values of all members located at the top level of the current object. | | `.**` | Recursive wildcard member accessor that processes all levels of the JSON hierarchy of the current object and returns all the member values, regardless of their nesting level. This is a WarehousePG extension of the SQL/JSON standard. | -| .**{*`level`*}

.**{*`start_level`* to *`end_level`*} | Same as `.**`, but with a filter over nesting levels of JSON hierarchy. Nesting levels are specified as integers. Zero level corresponds to the current object. To access the lowest nesting level, you can use the `last` keyword. This is a WarehousePG extension of the SQL/JSON standard. | -| [*`subscript`*, ...] | Array element accessor. *`subscript`* can be given in two forms: *`index`* or *`start_index`* to *`end_index`*. The first form returns a single array element by its index. The second form returns an array slice by the range of indexes, including the elements that correspond to the provided *`start_index`* and *`end_index`*.

The specified *`index`* can be an integer, as well as an expression returning a single numeric value, which is automatically cast to integer. Zero index corresponds to the first array element. You can also use the `last` keyword to denote the last array element, which is useful for handling arrays of unknown length. | +| .**{*`level`*}

.**{*`start_level`* to *`end_level`*} | Same as `.**`, but with a filter over nesting levels of JSON hierarchy. Nesting levels are specified as integers. Zero level corresponds to the current object. To access the lowest nesting level, you can use the `last` keyword. This is a WarehousePG extension of the SQL/JSON standard. | +| [*`subscript`*, ...] | Array element accessor. *`subscript`* can be given in two forms: *`index`* or *`start_index`* to *`end_index`*. The first form returns a single array element by its index. The second form returns an array slice by the range of indexes, including the elements that correspond to the provided *`start_index`* and *`end_index`*.

The specified *`index`* can be an integer, as well as an expression returning a single numeric value, which is automatically cast to integer. Zero index corresponds to the first array element. You can also use the `last` keyword to denote the last array element, which is useful for handling arrays of unknown length. | | `[*]` | Wildcard array element accessor that returns all array elements. | diff --git a/docs/whpg/7x/install_guide/installation_utilities.md b/docs/whpg/7x/install_guide/installation_utilities.md index 980618d..baf8f88 100644 --- a/docs/whpg/7x/install_guide/installation_utilities.md +++ b/docs/whpg/7x/install_guide/installation_utilities.md @@ -9,7 +9,16 @@ For a full reference of all WarehousePG utilities, see the [WarehousePG Utility The following WarehousePG management utilities are located in `$GPHOME/bin`. -| - [gpactivatestandby](../ref_guide/utility_guide/reference/gpactivatestandby.md)
- [gpaddmirrors](../ref_guide/utility_guide/reference/gpaddmirrors.md)
- [gpcheckperf](../ref_guide/utility_guide/reference/gpcheckperf.md)
- [gpdeletesystem](../ref_guide/utility_guide/reference/gpdeletesystem.md)
- [gpinitstandby](../ref_guide/utility_guide/reference/gpinitstandby.md)
- [gpinitsystem](../ref_guide/utility_guide/reference/gpinitsystem.md) | - [gpssh](../ref_guide/utility_guide/reference/gpssh.md)
- [gpssh-exkeys](../ref_guide/utility_guide/reference/gpssh-exkeys.md)
- [gpstart](../ref_guide/utility_guide/reference/gpstart.md)
- [gpstop](../ref_guide/utility_guide/reference/gpstop.md)
- [gpsync](../ref_guide/utility_guide/reference/gpsync.md) | -| --- | --- | +- [gpactivatestandby](../ref_guide/utility_guide/reference/gpactivatestandby.md) +- [gpaddmirrors](../ref_guide/utility_guide/reference/gpaddmirrors.md) +- [gpcheckperf](../ref_guide/utility_guide/reference/gpcheckperf.md) +- [gpdeletesystem](../ref_guide/utility_guide/reference/gpdeletesystem.md) +- [gpinitstandby](../ref_guide/utility_guide/reference/gpinitstandby.md) +- [gpinitsystem](../ref_guide/utility_guide/reference/gpinitsystem.md) +- [gpssh](../ref_guide/utility_guide/reference/gpssh.md) +- [gpssh-exkeys](../ref_guide/utility_guide/reference/gpssh-exkeys.md) +- [gpstart](../ref_guide/utility_guide/reference/gpstart.md) +- [gpstop](../ref_guide/utility_guide/reference/gpstop.md) +- [gpsync](../ref_guide/utility_guide/reference/gpsync.md) **Parent topic:** [Installing and Upgrading WarehousePG](index.md) diff --git a/docs/whpg/7x/install_guide/platform-requirements.md b/docs/whpg/7x/install_guide/platform-requirements.md index c271c06..ae8625a 100644 --- a/docs/whpg/7x/install_guide/platform-requirements.md +++ b/docs/whpg/7x/install_guide/platform-requirements.md @@ -157,8 +157,8 @@ The following table lists minimum recommended specifications for hardware server |---|---| | Minimum CPU | Any x86_64 compatible CPU | | Minimum Memory | 16 GB RAM per server | -| Disk Space Requirements |
  • 150MB per host for WarehousePG installation
  • Approximately 300MB per segment instance for metadata
  • Cap disk capacity at 70% full to accommodate temporary files and prevent performance degradation
| -| Network Requirements |
  • 10 Gigabit Ethernet within the array
  • NIC bonding is recommended when multiple interfaces are present
  • WarehousePG can use either IPV4 or IPV6 protocols.
| +| Disk Space Requirements | - 150MB per host for WarehousePG installation
- Approximately 300MB per segment instance for metadata
- Cap disk capacity at 70% full to accommodate temporary files and prevent performance degradation | +| Network Requirements | - 10 Gigabit Ethernet within the array
- NIC bonding is recommended when multiple interfaces are present
- WarehousePG can use either IPV4 or IPV6 protocols. | ### Hyperthreading diff --git a/docs/whpg/7x/ref_guide/function-summary.md b/docs/whpg/7x/ref_guide/function-summary.md index 6268dc9..24b4a50 100644 --- a/docs/whpg/7x/ref_guide/function-summary.md +++ b/docs/whpg/7x/ref_guide/function-summary.md @@ -19,13 +19,29 @@ WarehousePG supports built-in functions and operators including analytic functio ## WarehousePG Function Types -WarehousePG evaluates functions and operators used in SQL expressions. Some functions and operators are only allowed to run on the coordinator since they could lead to inconsistencies in WarehousePG segment instances. This table describes the WarehousePG Function Types. +WarehousePG evaluates functions and operators used in SQL expressions. Some functions and operators are only allowed to run on the coordinator since they could lead to inconsistencies in WarehousePG segment instances. The following describes the WarehousePG function types. -| Function Type | WarehousePG Support | Description | Comments | -| ------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| IMMUTABLE | Yes | Relies only on information directly in its argument list. Given the same argument values, always returns the same result. |   | -| STABLE | Yes, in most cases | Within a single table scan, returns the same result for same argument values, but results change across SQL statements. | Results depend on database lookups or parameter values. `current_timestamp` family of functions is `STABLE`; values do not change within an execution. | -| VOLATILE | Restricted | Function values can change within a single table scan. For example: `random()`, `timeofday()`. | Any function with side effects is volatile, even if its result is predictable. For example: `setval()`. | +### `IMMUTABLE` + +WarehousePG support: Yes + +Relies only on information directly in its argument list. Given the same argument values, always returns the same result. + +### `STABLE` + +WarehousePG support: Yes, in most cases + +Within a single table scan, returns the same result for same argument values, but results change across SQL statements. + +Results depend on database lookups or parameter values. The `current_timestamp` family of functions is `STABLE`, and values don't change within an execution. + +### `VOLATILE` + +WarehousePG support: Restricted + +Function values can change within a single table scan. For example, `random()` and `timeofday()`. + +Any function with side effects is volatile, even if its result is predictable. For example, `setval()`. In WarehousePG, data is divided up across segments — each segment is a distinct PostgreSQL database. To prevent inconsistent or unexpected results, do not run functions classified as `VOLATILE` at the segment level if they contain SQL commands or modify the database in any way. For example, functions such as `setval()` are not allowed to run on distributed data in WarehousePG because they can cause inconsistent data between segment instances. @@ -101,513 +117,2199 @@ WarehousePG includes built-in functions and operators that create and manipulate #### JSON Operators -This table describes the operators that are available for use with the `json` and `jsonb` data types. +The following operators are available for use with the `json` and `jsonb` data types. -| Operator | Right Operand Type | Return Type | Description | Example | Example Result | -| -------- | ------------------ | ----------------- | ------------------------------------------------------------------------------------- | -------------------------------------------------- | -------------- | -| `->` | `int` | `json` or `jsonb` | Get the JSON array element (indexed from zero, negative integers count from the end). | `'[{"a":"foo"},{"b":"bar"},{"c":"baz"}]'::json->2` | `{"c":"baz"}` | -| `->` | `text` | `json` or `jsonb` | Get the JSON object field by key. | `'{"a": {"b":"foo"}'::json->'a'` | `{"b":"foo"}` | -| `->>` | `int` | `text` | Get the JSON array element as `text`. | `'[1,2,3]'::json->>2` | `3` | -| `->>` | `text` | `text` | Get the JSON object field as `text`. | `'{"a":1,"b":2}'::json->>'b'` | `2` | -| `#>` | `text[]` | `json` or `jsonb` | Get the JSON object at the specified path. | `'{"a": {"b":{"c": "foo"}}'::json#>'{a,b}`' | `{"c": "foo"}` | -| `#>>` | `text[]` | `text` | Get the JSON object at the specified path as `text`. | `'{"a":[1,2,3],"b":[4,5,6]}'::json#>>'{a,2}'` | `3` | +##### `->` (int) -> **Note** There are parallel variants of these operators for both the `json` and `jsonb` data types. The field/element/path extraction operators return the same data type as their left-hand input (either `json` or `jsonb`), except for those specified as returning `text`, which coerce the value to `text`. The field/element/path extraction operators return `NULL`, rather than failing, if the JSON input does not have the right structure to match the request; for example if no such element exists. The field/element/path extraction operators that accept integer JSON array subscripts all support negative subscripting from the end of arrays. +Right operand type: `int` -These standard comparison operators are available for `jsonb`, but not for `json.` They follow the ordering rules for B-tree operations outlined at [jsonb Indexing](../admin_guide/query/json-data.md#topic_aqt_1tw_mq). +Return type: `json` or `jsonb` -| Operator | Description | -| ------------ | ------------------------ | -| `<` | less than | -| `>` | greater than | -| `<=` | less than or equal to | -| `>=` | greater than or equal to | -| `=` | equal | -| `<>` or `!=` | not equal | +Get the JSON array element (indexed from zero, negative integers count from the end). -> **Note** The `!=` operator is converted to `<>` in the parser stage. It is not possible to implement `!=` and `<>` operators that do different things. +Example: -Operators that require the `jsonb` data type as the left operand are described in the following table. Many of these operators can be indexed by `jsonb` operator classes. For a full description of `jsonb` containment and existence semantics, refer to [jsonb Containment and Existence](../admin_guide/query/json-data.md#topic_isx_2tw_mq). [jsonb Indexing](../admin_guide/query/json-data.md#topic_aqt_1tw_mq) describes how these operators can be used to effectively index `jsonb` data. - -| Operator | Right Operand Type | Description | Example | | | | | -| -------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- | -------------------------------------------------------- | --------------------- | - | --------------------- | -| `@>` | `jsonb` | Does the left JSON value contain the right JSON path/value entries at the top level? | `'{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb` | | | | | -| `<@` | `jsonb` | Are the left JSON path/value enries contained at the top level within the right JSON value? | `'{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb` | | | | | -| `?` | `text` | Does the *string* exist as a top-level key within the JSON value? | `'{"a":1, "b":2}'::jsonb ? 'b'` | | | | | -| \`? | \` | `text[]` | Do any of these array *strings* exist as a top-level key? | \`'{"a":1, "b":2, "c":3}'::jsonb ? | array['b', 'c']\` | | | -| `?&` | `text[]` | Do all of these array *strings* exist as top-level keys? | `'["a", "b"]'::jsonb ?& array['a', 'b']` | | | | | -| \` | | \` | `jsonb` | Concatenate two `jsonb` values into a new `jsonb` value. | \`'["a", "b"]'::jsonb | | '["c", "d"]'::jsonb\` | -| `-` | `text` | Delete key/value pair or *string* elements from left operand. Key/value pairs are matched based on their key value. | `'{"a": "b"}'::jsonb - 'a'` | | | | | -| `-` | `text[]` | Delete multiple key/value pairs or *string* elements from left operand. Key/value pairs are matched based on their key value. | `'{"a": "b", "c": "d"}'::jsonb - '{a,c}'::text[]` | | | | | -| `-` | `integer` | Delete the array element with specified index (Negative integers count from the end). Throws an error if top level container is not an array. | `'["a", "b"]'::jsonb - 1` | | | | | -| `#-` | `text[]` | Delete the field or element with specified path (for JSON arrays, negative integers count from the end) | `'["a", {"b":1}]'::jsonb #- '{1,b}'` | | | | | -| `@?` | `jsonpath` | Does JSON path return any item for the specified JSON value? | `'{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ > 2)'` | | | | | -| `@@` | `jsonpath` | Returns the result of JSON path predicate check for the specified JSON value. Only the first item of the result is taken into account. If the result is not Boolean, then `null` is returned. | `'{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] > 2'` | | | | | +``` +'[{"a":"foo"},{"b":"bar"},{"c":"baz"}]'::json->2 +{"c":"baz"} +``` -> **Note** The `||` operator concatenates two JSON objects by generating an object containing the union of their keys, taking the second object's value when there are duplicate keys. All other cases produce a JSON array: first, any non-array input is converted into a single-element array, and then the two arrays are concatenated. It does not operate recursively; only the top-level array or object structure is merged. +##### `->` (text) -> **Note** The `@?` and `@@` operators suppress the following errors: lacking object field or array element, unexpected JSON item type, and numeric errors. This behavior might be helpful while searching over JSON document collections of varying structure. +Right operand type: `text` - +Return type: `json` or `jsonb` -#### JSON Creation Functions +Get the JSON object field by key. -This table describes the functions that create `json` and `jsonb` data type values. (There are no equivalent functions for `jsonb` for `row_to_json()` and `array_to_json()`. However, the `to_jsonb()` function supplies much the same functionality as these functions would.) +Example: -| Function | Description | Example | Example Result | -| ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | ------------------------------------ | -| `to_json(anyelement)`
`to_jsonb(anyelement)` | Returns the value as a `json` or `jsonb` object. Arrays and composites are converted (recursively) to arrays and objects; otherwise, if the input contains a cast from the type to `json`, the cast function is used to perform the conversion; otherwise, a scalar value is produced. For any scalar type other than a number, a Boolean, or a null value, the text representation will be used, in such a fashion that it is a valid `json` or `jsonb` value. | `to_json('Fred said "Hi."'::text)` | `"Fred said \"Hi.\""` | -| `array_to_json(anyarray [, pretty_bool])` | Returns the array as a JSON array. A multidimensional array becomes a JSON array of arrays. Line feeds will be added between dimension-1 elements if `pretty_bool` is true. | `array_to_json('{1,5},{99,100}'::int[])` | `[[1,5],[99,100]]` | -| `row_to_json(record [, pretty_bool])` | Returns the row as a JSON object. Line feeds will be added between level-1 elements if `pretty_bool` is true. | `row_to_json(row(1,'foo'))` | `{"f1":1,"f2":"foo"}` | -| `json_build_array(VARIADIC "any")` | Builds a possibly-heterogeneously-typed JSON array out of a `VARIADIC` argument list. | `json_build_array(1,2,'3',4,5)` | `[1, 2, "3", 4, 5]` | -| `json_build_object(VARIADIC "any")` | Builds a JSON object out of a `VARIADIC` argument list. The argument list is taken in order and converted to a set of key/value pairs. | `json_build_object('foo',1,'bar',2)` | `{"foo": 1, "bar": 2}` | -| `json_object(text[])` | Builds a JSON object out of a text array. The array must have either exactly one dimension with an even number of members, in which case they are taken as alternating key/value pairs, or two dimensions such that each inner array has exactly two elements, which are taken as a key/value pair. | `json_object('{a, 1, b, "def", c, 3.5}')`

`json_object('{a, 1},{b, "def"},{c, 3.5}')` | `{"a": "1", "b": "def", "c": "3.5"}` | -| `json_object(keys text[], values text[])` | Builds a JSON object out of a text array. This form of `json_object` takes keys and values pairwise from two separate arrays. In all other respects it is identical to the one-argument form. | `json_object('{a, b}', '{1,2}')` | `{"a": "1", "b": "2"}` | +``` +'{"a": {"b":"foo"}}'::json->'a' +{"b":"foo"} +``` -> **Note** `array_to_json()` and `row_to_json()` have the same behavior as `to_json()` except for offering a pretty-printing option. The behavior described for `to_json()` likewise applies to each individual value converted by the other JSON creation functions. +##### `->>` (int) -> **Note** The [hstore](modules/bundled/hstore.md) extension has a cast from `hstore` to `json`, so that `hstore` values converted via the JSON creation functions will be represented as JSON objects, not as primitive string values. +Right operand type: `int` - +Return type: `text` -#### JSON Aggregate Functions +Get the JSON array element as `text`. -This table shows the functions that aggregate records to an array of JSON objects and pairs of values to a JSON object +Example: -| Function | Argument Types | Return Type | Description | -| ------------------------------------------------------------------- | ---------------- | ----------- | ---------------------------------------------- | -| `json_agg(record)`
`jsonb_agg(record)` | `record` | `json` | Aggregates records as a JSON array of objects. | -| `json_object_agg(name, value)`
`jsonb_object_agg(name, value)` | `("any", "any")` | `json` | Aggregates name/value pairs as a JSON object. | +``` +'[1,2,3]'::json->>2 +3 +``` - +##### `->>` (text) -#### JSON Processing Functions +Right operand type: `text` -This table shows the functions that are available for processing `json` and `jsonb` values. +Return type: `text` -Many of these processing functions and operators convert Unicode escapes in JSON strings to the appropriate single character. This is a not an issue if the input data type is `jsonb`, because the conversion was already done. However, for `json` data type input, this might result in an error being thrown as described in [About JSON Data](../admin_guide/query/json-data.md#topic_upc_tcs_fz). +Get the JSON object field as `text`. -**JSON Processing Functions** - -| Function | Return Type | Description | Example | Example Result | -| --- | --- | --- | --- | --- | -| `json_array_length(json)`

`jsonb_array_length(jsonb)` | `int` | Returns the number of elements in the outermost JSON array. | `json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]')` | `5` | -| `json_each(json)`

`jsonb_each(jsonb)` | `setof key text, value json`

`setof key text, value jsonb` | Expands the outermost JSON object into a set of key/value pairs. | `select * from json_each('{"a":"foo", "b":"bar"}')` | `key \| value`
`-----+-------`
`a \| "foo"`
`b \| "bar"` | -| `json_each_text(json)`

`jsonb_each_text(jsonb)` | `setof key text, value text` | Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. | `select * from json_each_text('{"a":"foo", "b":"bar"}')` | `key \| value`
`-----+-------`
`a \| foo`
`b \| bar` | -| `json_extract_path(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` | `json`

`jsonb` | Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). | `json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}','f4')` | `{"f5":99,"f6":"foo"}` | -| `json_extract_path_text(from_json json, VARIADIC path_elems text[])`

`jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` | `text` | Returns the JSON value pointed to by `path_elems` as text (equivalent to `#>>` operator). | `json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}','f4', 'f6')` | `foo` | -| `json_object_keys(json)`

`jsonb_object_keys(jsonb)` | `setof text` | Returns set of keys in the outermost JSON object. | `json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}')` | `json_object_keys`
`------------------`
`f1`
`f2` | -| `json_populate_record(base anyelement, from_json json)`

`jsonb_populate_record(base anyelement, from_json jsonb)` | `anyelement` | Expands the object in `from_json` to a row whose columns match the record type defined by `base`. See the [Note 1](#notes-on-json-processing-functions-examples). | `select * from json_populate_record(null::myrowtype, '{"a": 1, "b": ["2", "a b"], "c": {"d": 4, "e": "a b c"}}')` | `a \| b \| c`
`---+-----------+-------------`
`1 \| {2,"a b"} \| (4,"a b c")` | -| `json_populate_recordset(base anyelement, from_json json)`

`jsonb_populate_recordset(base anyelement, from_json jsonb)` | `setof anyelement` | Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by `base`. See the [Note 1](#notes-on-json-processing-functions-examples). | `select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]')` | `a \| b`
`---+---`
`1 \| 2`
`3 \| 4` | -| `json_array_elements(json)`

`jsonb_array_elements(jsonb)` | `setof json`

`setof jsonb` | Expands a JSON array to a set of JSON values. | `select * from json_array_elements('[1,true, [2,false]]')` | `value`
`-----------`
`1`
`true`
`[2,false]` | -| `json_array_elements_text(json)`

`jsonb_array_elements_text(jsonb)` | `setof text` | Expands a JSON array to a set of `text` values. | `select * from json_array_elements_text('["foo", "bar"]')` | `value`
`-----------`
`foo`
`bar` | -| `json_typeof(json)`

`jsonb_typeof(jsonb)` | `text` | Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. | `json_typeof('-123.4')` | `number` | -| `json_to_record(json)`

`jsonb_to_record(jsonb)` | `record` | Builds an arbitrary record from a JSON object. See the [Note 1](#notes-on-json-processing-functions-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_record('{"a":1,"b":[1,2,3], "c":[1,2,3],"e":"bar","r": {"a": 123, "b": "a b c"}') as x(a int, b text, c int[], d text, r myrowtype)` | `a \| b \| c \| d \| r`
`---+---------+---------+---+---------------`
`1 \| [1,2,3] \| {1,2,3} \| \| (123,"a b c")` | -| `json_to_recordset(json)`

`jsonb_to_recordset(jsonb)` | `setof record` | Builds an arbitrary set of records from a JSON array of objects See the [Note 1](#notes-on-json-processing-functions-examples).

As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. | `select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text);` | `a \| b`
`---+-----`
`1 \| foo`
`2 \|` | -| `json_strip_nulls(from_json json)`

`jsonb_strip_nulls(from_json jsonb)` | `json`

`jsonb` | Returns *`from_json`* with all object fields that have null values omitted. Other null values are untouched. | `json_strip_nulls('[{"f1":1,"f2":null},2,null,3]')` | `[{"f1":1},2,null,3]` | -| `jsonb_set(target jsonb, path text[], new_value jsonb [, create_missing boolean])` | `jsonb` | Returns *`target`* with the section designated by *`path`* replaced by *`new_value`*, or with *`new_value`* added if *`create_missing`* is true (default is `true`) and the item designated by *`path`* does not exist. As with the path oriented operators, negative integers that appear in *`path`* count from the end of JSON arrays. | `jsonb_set('[{"f1":1,"f2":null},2,null,3]', '{0,f1}','[2,3,4]', false)`

`jsonb_set('[{"f1":1,"f2":null},2]', '{0,f3}','[2,3,4]')` | `[{"f1":[2,3,4],"f2":null},2,null,3]`

`[{"f1": 1, "f2": null, "f3": [2, 3, 4]}, 2]` | -| `jsonb_insert(target jsonb, path text[], new_value jsonb [, insert_after boolean])` | `jsonb` | Returns *`target`* with *`new_value`* inserted. If *`target`* section designated by *`path`* is in a JSONB array, *`new_value`* will be inserted before target or after if *`insert_after`* is true (default is `false`). If *`target`* section designated by *`path`* is in JSONB object, *`new_value`* will be inserted only if *`target`* does not exist. As with the path oriented operators, negative integers that appear in *`path`* count from the end of JSON arrays. | `jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"')`

`jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"', true)` | `{"a": [0, "new_value", 1, 2]}`

`{"a": [0, 1, "new_value", 2]}` | -| `jsonb_pretty(from_json jsonb)` | `text` | Returns *`from_json`* as indented JSON text. | `jsonb_pretty('[{"f1":1,"f2":null},2,null,3]')` | `[`
` {`
` "f1": 1,`
` "f2": null`
` },`
` 2,`
` null,`
` 3`
`]` | -| `jsonb_path_exists(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `boolean` | Checks whether JSON path returns any item for the specified JSON value. | `jsonb_path_exists('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')` | `true` | -| `jsonb_path_match(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `boolean` | Returns the result of JSON path predicate check for the specified JSON value. Only the first item of the result is taken into account. If the result is not Boolean, then `null` is returned. | `jsonb_path_match('{"a":[1,2,3,4,5]}', 'exists($.a[*] ? (@ >= $min && @ <= $max))', '{"min":2,"max":4}')` | `true` | -| `jsonb_path_query(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `setof jsonb` | Gets all JSON items returned by JSON path for the specified JSON value. | `select * from jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}');` | `jsonb_path_query`
`------------------`
`2`
`3`
`4` | -| `jsonb_path_query_array(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `jsonb` | Gets all JSON items returned by JSON path for the specified JSON value and wraps result into an array. | `jsonb_path_query_array('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')` | `[2, 3, 4]` | -| `jsonb_path_query_first(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` | `jsonb` | Gets the first JSON item returned by JSON path for the specified JSON value. Returns `NULL` on no results. | `jsonb_path_query_first('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}')` | `2` | +Example: -##### Notes on JSON processing functions examples +``` +'{"a":1,"b":2}'::json->>'b' +2 +``` -1. The functions `json[b]_populate_record()`, `json[b]_populate_recordset()`, `json[b]_to_record()` and `json[b]_to_recordset()` operate on a JSON object, or array of objects, and extract the values associated with keys whose names match column names of the output row type. Object fields that do not correspond to any output column name are ignored, and output columns that do not match any object field will be filled with nulls. To convert a JSON value to the SQL type of an output column, the following rules are applied in sequence: +##### `#>` - - A JSON null value is converted to a SQL null in all cases. - - If the output column is of type `json` or `jsonb`, the JSON value is just reproduced exactly. - - If the output column is a composite (row) type, and the JSON value is a JSON object, the fields of the object are converted to columns of the output row type by recursive application of these rules. - - Likewise, if the output column is an array type and the JSON value is a JSON array, the elements of the JSON array are converted to elements of the output array by recursive application of these rules. - - Otherwise, if the JSON value is a string literal, the contents of the string are fed to the input conversion function for the column's data type. - - Otherwise, the ordinary text representation of the JSON value is fed to the input conversion function for the column's data type. +Right operand type: `text[]` - While the examples for these functions use constants, the typical use would be to reference a table in the `FROM` clause and use one of its `json` or `jsonb` columns as an argument to the function. Extracted key values can then be referenced in other parts of the query, like `WHERE` clauses and target lists. Extracting multiple values in this way can improve performance over extracting them separately with per-key operators. +Return type: `json` or `jsonb` -2. All the items of the `path` parameter of `jsonb_set()` as well as `jsonb_insert()` except the last item must be present in the target. If `create_missing` is false, all items of the `path` parameter of `jsonb_set()` must be present. If these conditions are not met the target is returned unchanged. +Get the JSON object at the specified path. - If the last path item is an object key, it will be created if it is absent and given the new value. If the last path item is an array index, if it is positive the item to set is found by counting from the left, and if negative by counting from the right - `-1` designates the rightmost element, and so on. If the item is out of the range -array_length .. array_length -1, and create_missing is true, the new value is added at the beginning of the array if the item is negative, and at the end of the array if it is positive. +Example: +``` +'{"a": {"b":{"c": "foo"}}}'::json#>'{a,b}' +{"c": "foo"} +``` -1. The `json_typeof` function's `null` return value should not be confused with a SQL NULL. While calling `json_typeof('null'::json)` will return `null`, calling `json_typeof(NULL::json)` will return a SQL NULL. +##### `#>>` +Right operand type: `text[]` -1. If the argument to `json_strip_nulls()` contains duplicate field names in any object, the result could be semantically somewhat different, depending on the order in which they occur. This is not an issue for `jsonb_strip_nulls()` since `jsonb` values never have duplicate object field names. +Return type: `text` -2. The `jsonb_path_exists()`, `jsonb_path_match()`, `jsonb_path_query()`, `jsonb_path_query_array()`, and `jsonb_path_query_first()` functions have optional `vars` and `silent` arguments. +Get the JSON object at the specified path as `text`. - If the `vars` argument is specified, it provides an object containing named variables to be substituted into a `jsonpath` expression. +Example: - If the `silent` argument is specified and has the `true` value, these functions suppress the same errors as the `@?` and `@@` operators. +``` +'{"a":[1,2,3],"b":[4,5,6]}'::json#>>'{a,2}' +3 +``` - +> **Note** There are parallel variants of these operators for both the `json` and `jsonb` data types. The field/element/path extraction operators return the same data type as their left-hand input (either `json` or `jsonb`), except for those specified as returning `text`, which coerce the value to `text`. The field/element/path extraction operators return `NULL`, rather than failing, if the JSON input does not have the right structure to match the request; for example if no such element exists. The field/element/path extraction operators that accept integer JSON array subscripts all support negative subscripting from the end of arrays. -### The SQL/JSON Path Language +These standard comparison operators are available for `jsonb`, but not for `json.` They follow the ordering rules for B-tree operations outlined at [jsonb Indexing](../admin_guide/query/json-data.md#topic_aqt_1tw_mq). -SQL/JSON path expressions specify the items to be retrieved from the JSON data, similar to XPath expressions used for SQL access to XML. In WarehousePG, path expressions are implemented as the `jsonpath` data type and can use any elements described in [jsonpath Type](../admin_guide/query/json-data.md#topic_jsonpath). +- `<` (less than) +- `>` (greater than) +- `<=` (less than or equal to) +- `>=` (greater than or equal to) +- `=` (equal) +- `<>` or `!=` (not equal) -JSON query functions and operators pass the provided path expression to the *path engine* for evaluation. If the expression matches the queried JSON data, the corresponding SQL/JSON item is returned. Path expressions are written in the SQL/JSON path language and can also include arithmetic expressions and functions. Query functions treat the provided expression as a text string, so it must be enclosed in single quotes. +> **Note** The `!=` operator is converted to `<>` in the parser stage. It is not possible to implement `!=` and `<>` operators that do different things. -A path expression consists of a sequence of elements allowed by the `jsonpath` data type. The path expression is evaluated from left to right, but you can use parentheses to change the order of operations. If the evaluation is successful, a sequence of SQL/JSON items (*SQL/JSON sequence*) is produced, and the evaluation result is returned to the JSON query function that completes the specified computation. +Operators that require the `jsonb` data type as the left operand are described next. Many of these operators can be indexed by `jsonb` operator classes. For a full description of `jsonb` containment and existence semantics, refer to [jsonb Containment and Existence](../admin_guide/query/json-data.md#topic_isx_2tw_mq). [jsonb Indexing](../admin_guide/query/json-data.md#topic_aqt_1tw_mq) describes how these operators can be used to effectively index `jsonb` data. -To refer to the JSON data to be queried (the *context item*), use the `$` sign in the path expression. It can be followed by one or more [accessor operators](../admin_guide/query/json-data.md#topic_jsonpath), which go down the JSON structure level by level to retrieve the content of context item. Each operator that follows deals with the result of the previous evaluation step. +##### `@>` -For example, suppose you have some JSON data from a GPS tracker that you would like to parse, such as: +Right operand type: `jsonb` + +Does the left JSON value contain the right JSON path/value entries at the top level? + +Example: ``` -{ - "track": { - "segments": [ - { - "location"[ 47.763, 13.4034 ], - "start time": "2018-10-14 10:05:14", - "HR": 73 - }, - { - "location"[ 47.706, 13.2635 ], - "start time": "2018-10-14 10:39:21", - "HR": 135 - } - ] - } -} +'{"a":1, "b":2}'::jsonb @> '{"b":2}'::jsonb ``` -To retrieve the available track segments, you need to use the `.` accessor operator for all the preceding JSON objects: +##### `<@` + +Right operand type: `jsonb` + +Are the left JSON path/value entries contained at the top level within the right JSON value? + +Example: ``` -'$.track.segments' +'{"b":2}'::jsonb <@ '{"a":1, "b":2}'::jsonb ``` -If the item to retrieve is an element of an array, you have to unnest this array using the `[*]` operator. For example, the following path will return location coordinates for all the available track segments: +##### `?` + +Right operand type: `text` + +Does the string exist as a top-level key within the JSON value? + +Example: ``` -'$.track.segments[*].location' +'{"a":1, "b":2}'::jsonb ? 'b' ``` -To return the coordinates of the first segment only, you can specify the corresponding subscript in the `[]` accessor operator. Note that the SQL/JSON arrays are 0-relative: +##### `?|` + +Right operand type: `text[]` + +Do any of these array strings exist as a top-level key? + +Example: ``` -'$.track.segments[0].location' +'{"a":1, "b":2, "c":3}'::jsonb ?| array['b', 'c'] ``` -The result of each path evaluation step can be processed by one or more `jsonpath` operators and methods listed in [SQL/JSON Path Operators and Methods](#topic_jsonpath_opsmeth) below. Each method name must be preceded by a dot. For example, you can get an array size: +##### `?&` + +Right operand type: `text[]` + +Do all of these array strings exist as top-level keys? + +Example: ``` -'$.track.segments.size()' +'["a", "b"]'::jsonb ?& array['a', 'b'] ``` -For more examples of using `jsonpath` operators and methods within path expressions, see [SQL/JSON Path Operators and Methods](#topic_jsonpath_opsmeth) below. +##### `||` -When defining the path, you can also use one or more *filter expressions* that work similar to the `WHERE` clause in SQL. A filter expression begins with a question mark and provides a condition in parentheses: +Right operand type: `jsonb` + +Concatenates two `jsonb` values into a new `jsonb` value. + +Example: ``` -? (condition) +'["a", "b"]'::jsonb || '["c", "d"]'::jsonb ``` -Filter expressions must be specified right after the path evaluation step to which they are applied. The result of this step is filtered to include only those items that satisfy the provided condition. SQL/JSON defines three-valued logic, so the condition can be `true`, `false`, or `unknown`. The `unknown` value plays the same role as SQL `NULL` and can be tested for with the is `unknown` predicate. Further path evaluation steps use only those items for which filter expressions return `true`. +##### `-` (text) -Functions and operators that can be used in filter expressions are listed in [jsonpath Filter Expression Elements](#topic_jsonpath_filtexp). The path evaluation result to be filtered is denoted by the `@` variable. To refer to a JSON element stored at a lower nesting level, add one or more accessor operators after `@`. +Right operand type: `text` -Suppose you would like to retrieve all heart rate values higher than 130. You can achieve this using the following expression: +Deletes a key/value pair or string element from the left operand. Key/value pairs are matched based on their key value. + +Example: ``` -'$.track.segments[*].HR ? (@ > 130)' +'{"a": "b"}'::jsonb - 'a' ``` -To get the start time of segments with such values instead, you have to filter out irrelevant segments before returning the start time, so the filter expression is applied to the previous step, and the path used in the condition is different: +##### `-` (text[]) + +Right operand type: `text[]` + +Deletes multiple key/value pairs or string elements from the left operand. Key/value pairs are matched based on their key value. + +Example: ``` -'$.track.segments[*] ? (@.HR > 130)."start time"' +'{"a": "b", "c": "d"}'::jsonb - '{a,c}'::text[] ``` -You can use several filter expressions on the same nesting level, if required. For example, the following expression selects all segments that contain locations with relevant coordinates and high heart rate values: +##### `-` (integer) + +Right operand type: `integer` + +Deletes the array element with the specified index (negative integers count from the end). Throws an error if the top-level container isn't an array. + +Example: ``` -'$.track.segments[*] ? (@.location[1] < 13.4) ? (@.HR > 130)."start time"' +'["a", "b"]'::jsonb - 1 ``` -Using filter expressions at different nesting levels is also allowed. The following example first filters all segments by location, and then returns high heart rate values for these segments, if available: +##### `#-` + +Right operand type: `text[]` + +Deletes the field or element with the specified path. For JSON arrays, negative integers count from the end. + +Example: ``` -'$.track.segments[*] ? (@.location[1] < 13.4).HR ? (@ > 130)' +'["a", {"b":1}]'::jsonb #- '{1,b}' ``` -You can also nest filter expressions within each other: +##### `@?` + +Right operand type: `jsonpath` + +Does the JSON path return any item for the specified JSON value? + +Example: ``` -'$.track ? (exists(@.segments[*] ? (@.HR > 130))).segments.size()' +'{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ > 2)' ``` -This expression returns the size of the track if it contains any segments with high heart rate values, or an empty sequence otherwise. +##### `@@` - +Right operand type: `jsonpath` -#### Deviations from Standard +Returns the result of the JSON path predicate check for the specified JSON value. Only the first item of the result is taken into account. If the result isn't Boolean, `null` is returned. -WarehousePG's implementation of SQL/JSON path language has the following deviations from the SQL/JSON standard: +Example: -- `.datetime()` item method is not implemented yet mainly because immutable `jsonpath` functions and operators cannot reference session timezone, which is used in some datetime operations. Datetime support will be added to `jsonpath` in future versions of WarehousePG. -- A path expression can be a Boolean predicate, although the SQL/JSON standard allows predicates only in filters. This is necessary for implementation of the `@@` operator. For example, the following `jsonpath` expression is valid in WarehousePG: +``` +'{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] > 2' +``` - ``` - '$.track.segments[*].HR < 70' - ``` -- There are minor differences in the interpretation of regular expression patterns used in `like_regex` filters as described in [Regular Expressions](#topic_jsonpath_regexp). +> **Note** The `||` operator concatenates two JSON objects by generating an object containing the union of their keys, taking the second object's value when there are duplicate keys. All other cases produce a JSON array: first, any non-array input is converted into a single-element array, and then the two arrays are concatenated. It does not operate recursively; only the top-level array or object structure is merged. - +> **Note** The `@?` and `@@` operators suppress the following errors: lacking object field or array element, unexpected JSON item type, and numeric errors. This behavior might be helpful while searching over JSON document collections of varying structure. -#### Strict And Lax Modes + -When you query JSON data, the path expression may not match the actual JSON data structure. An attempt to access a non-existent member of an object or element of an array results in a structural error. SQL/JSON path expressions have two modes of handling structural errors: +#### JSON Creation Functions -- lax (default) — the path engine implicitly adapts the queried data to the specified path. Any remaining structural errors are suppressed and converted to empty SQL/JSON sequences. +The following functions create `json` and `jsonb` data type values. (There are no equivalent functions for `jsonb` for `row_to_json()` and `array_to_json()`. However, the `to_jsonb()` function supplies much the same functionality as these functions would.) -- strict — if a structural error occurs, an error is raised. +##### `to_json(anyelement) / to_jsonb(anyelement)` -The lax mode facilitates matching of a JSON document structure and path expression if the JSON data does not conform to the expected schema. If an operand does not match the requirements of a particular operation, it can be automatically wrapped as an SQL/JSON array or unwrapped by converting its elements into an SQL/JSON sequence before performing this operation. Besides, comparison operators automatically unwrap their operands in the lax mode, so you can compare SQL/JSON arrays out-of-the-box. An array of size 1 is considered equal to its sole element. Automatic unwrapping is not performed only when: +Returns the value as a `json` or `jsonb` object. Arrays and composites are converted recursively to arrays and objects. Otherwise, if the input contains a cast from the type to `json`, the cast function is used to perform the conversion. Otherwise, a scalar value is produced. For any scalar type other than a number, a Boolean, or a null value, the text representation is used, in a fashion that makes it a valid `json` or `jsonb` value. -- The path expression contains `type()` or `size()` methods that return the type and the number of elements in the array, respectively. +Example: -- The queried JSON data contain nested arrays. In this case, only the outermost array is unwrapped, while all the inner arrays remain unchanged. Thus, implicit unwrapping can only go one level down within each path evaluation step. +``` +to_json('Fred said "Hi."'::text) +"Fred said \"Hi.\"" +``` -For example, when querying the GPS data listed above, you can abstract from the fact that it stores an array of segments when using the lax mode: +##### `array_to_json(anyarray [, pretty_bool])` + +Returns the array as a JSON array. A multidimensional array becomes a JSON array of arrays. Line feeds are added between dimension-1 elements if `pretty_bool` is true. + +Example: ``` -'lax $.track.segments.location' +array_to_json('{1,5},{99,100}'::int[]) +[[1,5],[99,100]] ``` -In the strict mode, the specified path must exactly match the structure of the queried JSON document to return an SQL/JSON item, so using this path expression will cause an error. To get the same result as in the lax mode, you have to explicitly unwrap the segments array: +##### `row_to_json(record [, pretty_bool])` + +Returns the row as a JSON object. Line feeds are added between level-1 elements if `pretty_bool` is true. + +Example: ``` -'strict $.track.segments[*].location' +row_to_json(row(1,'foo')) +{"f1":1,"f2":"foo"} ``` -The `.**` accessor can lead to surprising results when using the lax mode. For instance, the following query selects every HR value twice: +##### `json_build_array(VARIADIC "any")` + +Builds a possibly heterogeneously typed JSON array out of a `VARIADIC` argument list. + +Example: ``` -lax $.**.HR +json_build_array(1,2,'3',4,5) +[1, 2, "3", 4, 5] ``` -This happens because the `.**` accessor selects both the segments array and each of its elements, while the `.HR` accessor automatically unwraps arrays when using the lax mode. To avoid surprising results, we recommend using the `.**` accessor only in the strict mode. The following query selects each HR value just once: +##### `json_build_object(VARIADIC "any")` + +Builds a JSON object out of a `VARIADIC` argument list. The argument list is taken in order and converted to a set of key/value pairs. + +Example: ``` -strict $.**.HR +json_build_object('foo',1,'bar',2) +{"foo": 1, "bar": 2} ``` - +##### `json_object(text[])` -#### Regular Expressions +Builds a JSON object out of a text array. The array must have either exactly one dimension with an even number of members, in which case they're taken as alternating key/value pairs, or two dimensions such that each inner array has exactly two elements, which are taken as a key/value pair. -SQL/JSON path expressions allow matching text to a regular expression with the `like_regex` filter. For example, the following SQL/JSON path query would case-insensitively match all strings in an array that starts with an English vowel: +Example: ``` -'$[*] ? (@ like_regex "^[aeiou]" flag "i")' +json_object('{a, 1, b, "def", c, 3.5}') +json_object('{a, 1},{b, "def"},{c, 3.5}') +{"a": "1", "b": "def", "c": "3.5"} ``` -The optional flag string may include one or more of the characters `i` for case-insensitive match, `m` to allow `^` and `$` to match at newlines, `s` to allow `.` to match a newline, and `q` to quote the whole pattern (reducing the behavior to a simple substring match). +##### `json_object(keys text[], values text[])` -The SQL/JSON standard borrows its definition for regular expressions from the `LIKE_REGEX` operator, which in turn uses the XQuery standard. WarehousePG does not currently support the `LIKE_REGEX` operator. Therefore, the `like_regex` filter is implemented using the POSIX regular expression engine as described in [POSIX Regular Expressions](https://www.postgresql.org/docs/12/functions-matching.html#FUNCTIONS-POSIX-REGEXP). This leads to various minor discrepancies from standard SQL/JSON behavior which are catalogued in [Differences From XQuery (LIKE_REGEX)](https://www.postgresql.org/docs/12/functions-matching.html#POSIX-VS-XQUERY). Note, however, that the flag-letter incompatibilities described there do not apply to SQL/JSON, as it translates the XQuery flag letters to match what the POSIX engine expects. +Builds a JSON object out of a text array. This form of `json_object` takes keys and values pairwise from two separate arrays. In all other respects it's identical to the one-argument form. -Keep in mind that the pattern argument of `like_regex` is a JSON path string literal, written according to the rules given in [jsonpath Type](../admin_guide/query/json-data.md#topic_jsonpath). This means in particular that any backslashes you want to use in the regular expression must be doubled. For example, to match string values of the root document that contain only digits: +Example: ``` -$.* ? (@ like_regex "^\\d+$") +json_object('{a, b}', '{1,2}') +{"a": "1", "b": "2"} ``` - +> **Note** `array_to_json()` and `row_to_json()` have the same behavior as `to_json()` except for offering a pretty-printing option. The behavior described for `to_json()` likewise applies to each individual value converted by the other JSON creation functions. -#### SQL/JSON Path Operators and Methods +> **Note** The [hstore](modules/bundled/hstore.md) extension has a cast from `hstore` to `json`, so that `hstore` values converted via the JSON creation functions will be represented as JSON objects, not as primitive string values. -The following table describes the operators and methods available in `jsonpath`: - -| Operator/Method | Description | Example JSON | Example Query | Result | -| --- | --- | --- | --- | --- | -| `+` (unary) | Plus operator that iterates over the SQL/JSON sequence | `{"x": [2.85, -14.7, -9.4]}` | `+ $.x.floor()` | `2, -15, -10` | -| `-` (unary) | Minus operator that iterates over the SQL/JSON sequence | `{"x": [2.85, -14.7, -9.4]}` | `- $.x.floor()` | `-2, 15, 10` | -| `+` (binary) | Addition | `[2]` | `2 + $[0]` | `4` | -| `-` (binary) | Subtraction | `[2]` | `4 - $[0]` | `2` | -| `*` | Multiplication | `[4]` | `2 * $[0]` | `8` | -| `/` | Division | `[8]` | `$[0] / 2` | `4` | -| `%` | Modulus | `[32]` | `$[0] % 10` | `2` | -| `type()` | Type of the SQL/JSON item | `[1, "2", {}]` | `$[*].type()` | `"number", "string", "object"` | -| `size()` | Size of the SQL/JSON item | `{"m": [11, 15]}` | `$.m.size()` | `2` | -| `double()` | Approximate floating-point number converted from an SQL/JSON number or a string | `{"len": "1.9"}` | `$.len.double() * 2` | `3.8` | -| `ceiling()` | Nearest integer greater than or equal to the SQL/JSON number | `{"h": 1.3}` | `$.h.ceiling()` | `2` | -| `floor()` | Nearest integer less than or equal to the SQL/JSON number | `{"h": 1.3}` | `$.h.floor()` | `1` | -| `abs()` | Absolute value of the SQL/JSON number | `{"z": -0.3}` | `$.z.abs()` | `0.3` | -| `keyvalue()` | Sequence of object's key-value pairs represented as array of items containing three fields (`"key"`, `"value"`, and `"id"`). `"id"` is a unique identifier of the object key-value pair belongs to. | `{"x": "20", "y": 32}` | `$.keyvalue()` | `{"key": "x", "value": "20", "id": 0}, {"key": "y", "value": 32, "id": 0}` | + - +#### JSON Aggregate Functions -#### SQL/JSON Filter Expression Elements +The following functions aggregate records to an array of JSON objects and pairs of values to a JSON object. -The following table describes the available filter expressions elements for `jsonpath`: - -| Value/Predicate | Description | Example JSON | Example Query | Result | -| --- | --- | --- | --- | --- | -| `==` | Equality operator | `[1, 2, 1, 3]` | `$[*] ? (@ == 1)` | `1, 1` | -| `!=` | Non-equality operator | `[1, 2, 1, 3]` | `$[*] ? (@ != 1)` | `2, 3` | -| `<>` | Non-equality operator (same as `!=`) | `[1, 2, 1, 3]` | `$[*] ? (@ <> 1)` | `2, 3` | -| `<` | Less-than operator | `[1, 2, 3]` | `$[*] ? (@ < 2)` | `1` | -| `<=` | Less-than-or-equal-to operator | `[1, 2, 3]` | `$[*] ? (@ <= 2)` | `1, 2` | -| `>` | Greater-than operator | `[1, 2, 3]` | `$[*] ? (@ > 2)` | `3` | -| `>=` | Greater-than-or-equal-to operator | `[1, 2, 3]` | `$[*] ? (@ >= 2)` | `2, 3` | -| `true` | Value used to perform comparison with JSON `true` literal | `[{"name": "John", "parent": false}, {"name": "Chris", "parent": true}]` | `$[*] ? (@.parent == true)` | `{"name": "Chris", "parent": true}` | -| `false` | Value used to perform comparison with JSON `false` literal | `[{"name": "John", "parent": false}, {"name": "Chris", "parent": true}]` | `$[*] ? (@.parent == false)` | `{"name": "John", "parent": false}` | -| `null` | Value used to perform comparison with JSON `null` value | `[{"name": "Mary", "job": null}, {"name": "Michael", "job": "driver"}]` | `$[*] ? (@.job == null) .name` | `"Mary"` | -| `&&` | Boolean AND | `[1, 3, 7]` | `$[*] ? (@ > 1 && @ < 5)` | `3` | -| `\|\|` | Boolean OR | `[1, 3, 7]` | `$[*] ? (@ < 1 \|\| @ > 5)` | `7` | -| `!` | Boolean NOT | `[1, 3, 7]` | `$[*] ? (!(@ < 5))` | `7` | -| `like_regex` | Tests whether the first operand matches the regular expression given by the second operand, optionally with modifications described by a string of `flag` characters. | `["abc", "abd", "aBdC", "abdacb", "babc"]` | `$[*] ? (@ like_regex "^ab.*c" flag "i")` | `"abc", "aBdC", "abdacb"` | -| `starts with` | Tests whether the second operand is an initial substring of the first operand | `["John Smith", "Mary Stone", "Bob Johnson"]` | `$[*] ? (@ starts with "John")` | `"John Smith"` | -| `exists` | Tests whether a path expression matches at least one SQL/JSON item | `{"x": [1, 2], "y": [2, 4]}` | `strict $.* ? (exists (@ ? (@[*] > 2)))` | `2, 4` | -| `is unknown` | Tests whether a Boolean condition is `unknown` | `[-1, 2, 7, "infinity"]` | `$[*] ? ((@ > 0) is unknown)` | `"infinity"` | +##### `json_agg(record) / jsonb_agg(record)` - +Argument types: `record` -## Window Functions +Return type: `json` -The following are WarehousePG built-in window functions. All window functions are *immutable*. For more information about window functions, see "Window Expressions" in the *WarehousePG Administrator Guide*. +Aggregates records as a JSON array of objects. -| Function | Return Type | Full Syntax | Description | | -| --------------------------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | -| `cume_dist()` | `double precision` | `CUME_DIST() OVER ( [PARTITION BY` expr `] ORDER BY` expr `)` | Calculates the cumulative distribution of a value in a group of values. Rows with equal values always evaluate to the same cumulative distribution value. | | -| `dense_rank()` | `bigint` | `DENSE_RANK () OVER ( [PARTITION BY` expr `] ORDER BY` expr `)` | Computes the rank of a row in an ordered group of rows without skipping rank values. Rows with equal values are given the same rank value. | | -| `first_value(*expr*)` | same as input expr type | `FIRST_VALUE(` expr `) OVER ( [PARTITION BY` expr `] ORDER BY` expr \`\[ROWS | RANGE`frame\_expr`] )\` | Returns the first value in an ordered set of values. | -| `lag(*expr* [,*offset*] [,*default*])` | same as input *expr* type | `LAG(` *expr* `[,` *offset* `] [,` *default* `]) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` | Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `LAG` provides access to a row at a given physical offset prior to that position. The default `offset` is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. | | -| `last_value(*expr*`) | same as input *expr* type | \`LAST_VALUE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* \[ROWS | RANGE *frame_expr*] )\` | Returns the last value in an ordered set of values. | -| `lead(*expr* [,*offset*] [,*default*])` | same as input *expr* type | `LEAD(*expr*[,*offset*] [,*expr**default*]) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` | Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `lead` provides access to a row at a given physical offset after that position. If *offset* is not specified, the default offset is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. | | -| `ntile(*expr*)` | `bigint` | `NTILE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` | Divides an ordered data set into a number of buckets (as defined by *expr*) and assigns a bucket number to each row. | | -| `percent_rank()` | `double precision` | `PERCENT_RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Calculates the rank of a hypothetical row `R` minus 1, divided by 1 less than the number of rows being evaluated (within a window partition). | | -| `rank()` | `bigint` | `RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Calculates the rank of a row in an ordered group of values. Rows with equal values for the ranking criteria receive the same rank. The number of tied rows are added to the rank number to calculate the next rank value. Ranks may not be consecutive numbers in this case. | | -| `row_number()` | `bigint` | `ROW_NUMBER () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` | Assigns a unique number to each row to which it is applied (either each row in a window partition or each row of the query). | | +##### `json_object_agg(name, value) / jsonb_object_agg(name, value)` - +Argument types: `("any", "any")` -## Advanced Aggregate Functions +Return type: `json` -The following built-in advanced analytic functions are WarehousePG extensions of the PostgreSQL database. Analytic functions are *immutable*. +Aggregates name/value pairs as a JSON object. -> **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [MADlib Extension for Analytics](../admin_guide/analytics/madlib.md). + -**Advanced Aggregate Functions** +#### JSON Processing Functions -| Function | Return Type | Full Syntax | Description | -| --- | --- | --- | --- | -| `gp_array_agg (`*anyarray*`)` | same as the argument data type | `gp_array_agg (`*anyarray*`)`

*Example:*

`CREATE TABLE intarr_tbl (a int, arr int[]);`
`INSERT INTO intarr_tbl SELECT i, array[i, i] FROM generate_series(1, 5)i;`
`INSERT INTO intarr_tbl SELECT 6, '{6, NULL}'::int[];`
`INSERT INTO intarr_tbl SELECT 8, '{NULL, 7}'::int[];`
`SELECT gp_array_agg(arr ORDER BY arr) FROM intarr_tbl;` | A parallel version of `array_agg(anyarray)`. Concatenates input arrays to create an array of one higher dimension. The inputs must all have the same dimensions, and they cannot be empty or null. | -| `gp_array_agg (`*anynonarray*`)` | array of the argument type | `gp_array_agg (`*anynonarray*`)`

*Example:*

`CREATE TABLE table1(a int4, b int4);`
`INSERT INTO table1 VALUES (4,5), (2,1), (1,3), (3,null), (3,7);`
`SELECT gp_array_agg(a ORDER BY b NULLS FIRST) FROM table1;` | An parallel version of `array_agg(anynonarray)`. Creates an array by concatenating input values, including nulls. | -| `MEDIAN (`*expr*`)` | `timestamp, timestamptz, interval, float` | `MEDIAN (`*expression*`)`

*Example:*

`SELECT department_id, MEDIAN(salary) `
`FROM employees `
`GROUP BY department_id;` | Can take a two-dimensional array as input. Treats such arrays as matrices. | -| `PERCENTILE_CONT (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` | `timestamp, timestamptz, interval, float` | `PERCENTILE_CONT(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)`

*Example:*

`SELECT department_id,`
`PERCENTILE_CONT (0.5) WITHIN GROUP (ORDER BY salary DESC)`
`"Median_cont"; `
`FROM employees GROUP BY department_id;` | Performs an inverse distribution function that assumes a continuous distribution model. It takes a percentile value and a sort specification and returns the same datatype as the numeric datatype of the argument. This returned value is a computed result after performing linear interpolation. Null are ignored in this calculation. | -| `PERCENTILE_DISC (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` | `timestamp, timestamptz, interval, float` | `PERCENTILE_DISC(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)`

*Example:*

`SELECT department_id, `
`PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY salary DESC)`
`"Median_desc"; `
`FROM employees GROUP BY department_id;` | Performs an inverse distribution function that assumes a discrete distribution model. It takes a percentile value and a sort specification. This returned value is an element from the set. Null are ignored in this calculation. | -| `sum(array[])` | `smallint[]int[], bigint[], float[]` | `sum(array[[1,2],[3,4]])`

*Example:*

`CREATE TABLE mymatrix (myvalue int[]);`
`INSERT INTO mymatrix VALUES (array[[1,2],[3,4]]);`
`INSERT INTO mymatrix VALUES (array[[0,1],[1,0]]);`
`SELECT sum(myvalue) FROM mymatrix;`
`sum `
`---------------`
`{1,3},{4,4}` | Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. | -| `pivot_sum (label[], label, expr)` | `int[], bigint[], float[]` | `pivot_sum( array['A1','A2'], attr, value)` | A pivot aggregation using sum to resolve duplicate entries. | -| `unnest (array[])` | set of `anyelement` | `unnest( array['one', 'row', 'per', 'item'])` | Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudotype in PostgreSQL](https://www.postgresql.org/docs/12/datatype-pseudo.html). | +This section describes the functions that are available for processing `json` and `jsonb` values. - +Many of these processing functions and operators convert Unicode escapes in JSON strings to the appropriate single character. This is a not an issue if the input data type is `jsonb`, because the conversion was already done. However, for `json` data type input, this might result in an error being thrown as described in [About JSON Data](../admin_guide/query/json-data.md#topic_upc_tcs_fz). -## Text Search Functions and Operators +##### `json_array_length() / jsonb_array_length()` -The following tables summarize the functions and operators that are provided for full text searching. See [Using Full Text Search](../admin_guide/query/textsearch/index.md) for a detailed explanation of WarehousePG's text search facility. +Syntax: `json_array_length(json)` / `jsonb_array_length(jsonb)` -| Operator | Description | Example | Result | | | | | | | | -| -------- | ------------------------------ | -------------------------------------------------------------- | --------------------------------- | --------------------- | ----------------- | ------------------------- | --------------------------- | --------- | ------- | ------- | -| `@@` | `tsvector` matches `tsquery` ? | `to_tsvector('fat cats ate rats') @@ to_tsquery('cat & rat')` | `t` | | | | | | | | -| `@@@` | deprecated synonym for `@@` | `to_tsvector('fat cats ate rats') @@@ to_tsquery('cat & rat')` | `t` | | | | | | | | -| \` | | \` | concatenate`tsvector`s | \`'a:1 b:2'::tsvector | | 'c:1 d:2 b:3'::tsvector\` | `'a':1 'b':2,5 'c':3 'd':4` | | | | -| `&&` | AND `tsquery`s together | \`'fat | rat'::tsquery && 'cat'::tsquery\` | \`( 'fat' | 'rat' ) & 'cat'\` | | | | | | -| \` | | \` | OR `tsquery`s together | \`'fat | rat'::tsquery | | 'cat'::tsquery\` | \`( 'fat' | 'rat' ) | 'cat'\` | -| `!!` | negate a`tsquery` | `!! 'cat'::tsquery` | `!'cat'` | | | | | | | | -| `@>` | `tsquery` contains another ? | `'cat'::tsquery @> 'cat & rat'::tsquery` | `f` | | | | | | | | -| `<@` | `tsquery` is contained in ? | `'cat'::tsquery <@ 'cat & rat'::tsquery` | `t` | | | | | | | | +Return type: `int` -> **Note** The `tsquery` containment operators consider only the lexemes listed in the two queries, ignoring the combining operators. +Returns the number of elements in the outermost JSON array. -In addition to the operators shown in the table, the ordinary B-tree comparison operators (=, <, etc) are defined for types `tsvector` and `tsquery`. These are not very useful for text searching but allow, for example, unique indexes to be built on columns of these types. - -| Function | Return Type | Description | Example | Result | -| ------------------------------------------------------------------------------------------- | ----------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------- | ----------------------------- | -| `get_current_ts_config()` | regconfig | get default text search configuration | get_current_ts_config() | english | -| `length(tsvector)` | integer | number of lexemes in tsvector | length('fat:2,4 cat:3 rat:5A'::tsvector) | 3 | -| `numnode(tsquery)` | integer | number of lexemes plus operators in tsquery | numnode('(fat & rat) \| cat'::tsquery) | 5 | -| `plainto_tsquery([ config regconfig , ] querytext)` | tsquery | produce tsquery ignoring punctuation | plainto_tsquery('english', 'The Fat Rats') | 'fat' & 'rat' | -| `querytree(query tsquery)` | text | get indexable part of a tsquery | querytree('foo & ! bar'::tsquery) | 'foo' | -| `setweight(tsvector, "char")` | tsvector | assign weight to each element of tsvector | setweight('fat:2,4 cat:3 rat:5B'::tsvector, 'A') | 'cat':3A 'fat':2A,4A 'rat':5A | -| `strip(tsvector)` | tsvector | remove positions and weights from tsvector | strip('fat:2,4 cat:3 rat:5A'::tsvector) | 'cat' 'fat' 'rat' | -| `to_tsquery([ config regconfig , ] query text)` | tsquery | normalize words and convert to tsquery | to_tsquery('english', 'The & Fat & Rats') | 'fat' & 'rat' | -| `to_tsvector([ config regconfig , ] documenttext)` | tsvector | reduce document text to tsvector | to_tsvector('english', 'The Fat Rats') | 'fat':2 'rat':3 | -| `ts_headline([ config regconfig, ] documenttext, query tsquery [, options text ])` | text | display a query match | ts_headline('x y z', 'z'::tsquery) | x y <b>z</b> | -| `ts_rank([ weights float4[], ] vector tsvector,query tsquery [, normalization integer ])` | float4 | rank document for query | ts_rank(textsearch, query) | 0.818 | -| `ts_rank_cd([ weights float4[], ] vectortsvector, query tsquery [, normalizationinteger ])` | float4 | rank document for query using cover density | ts_rank_cd('{0.1, 0.2, 0.4, 1.0}', textsearch, query) | 2.01317 | -| `ts_rewrite(query tsquery, target tsquery,substitute tsquery)` | tsquery | replace target with substitute within query | ts_rewrite('a & b'::tsquery, 'a'::tsquery, 'foo\|bar'::tsquery) | 'b' & ( 'foo' \| 'bar' ) | -| `ts_rewrite(query tsquery, select text)` | tsquery | replace using targets and substitutes from a SELECTcommand | SELECT ts_rewrite('a & b'::tsquery, 'SELECT t,s FROM aliases') | 'b' & ( 'foo' \| 'bar' ) | -| `tsvector_update_trigger()` | trigger | trigger function for automatic tsvector column update | CREATE TRIGGER ... tsvector_update_trigger(tsvcol, 'pg_catalog.swedish', title, body) |   | -| `tsvector_update_trigger_column()` | trigger | trigger function for automatic tsvector column update | CREATE TRIGGER ... tsvector_update_trigger_column(tsvcol, configcol, title, body) |   | +Example: -> **Note** All the text search functions that accept an optional `regconfig` argument will use the configuration specified by [default_text_search_config](config_params/guc-list.md) when that argument is omitted. +``` +json_array_length('[1,2,3,{"f1":1,"f2":[5,6]},4]') +5 +``` -The functions in the following table are listed separately because they are not usually used in everyday text searching operations. They are helpful for development and debugging of new text search configurations. +##### `json_each() / jsonb_each()` -| Function | Return Type | Description | Example | Result | -| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | ---------------------------------- | -------------------------------------------------- | ---------------------------------------------------------------------- | -| `ts_debug([ *config* regconfig, ] *document* text, OUT *alias* text, OUT *description* text, OUT *token* text, OUT *dictionaries* regdictionary[], OUT *dictionary* regdictionary, OUT *lexemes* text[])` | `setof record` | test a configuration | `ts_debug('english', 'The Brightest supernovaes')` | `(asciiword,"Word, all ASCII",The,{english_stem},english_stem,{}) ...` | -| `ts_lexize(*dict* regdictionary, *token* text)` | `text[]` | test a dictionary | `ts_lexize('english_stem', 'stars')` | `{star}` | -| `ts_parse(*parser\_name* text, *document* text, OUT *tokid* integer, OUT *token* text)` | `setof record` | test a parser | `ts_parse('default', 'foo - bar')` | (`1,foo) ...` | -| `ts_parse(*parser\_oid* oid, *document* text, OUT *tokid* integer, OUT *token* text)` | `setof record` | test a parser | `ts_parse(3722, 'foo - bar')` | `(1,foo) ...` | -| `ts_token_type(*parser\_name* text, OUT *tokid* integer, OUT *alias* text, OUT description text)` | `setof record` | get token types defined by parser | `ts_token_type('default')` | `(1,asciiword,"Word, all ASCII") ...` | -| `ts_token_type(*parser\_oid* oid, OUT *tokid* integer, OUT *alias* text, OUT *description* text)` | `setof record` | get token types defined by parser | `ts_token_type(3722)` | `(1,asciiword,"Word, all ASCII") ...` | -| `ts_stat(*sqlquery* text, [ *weights* text, ] OUT *word* text, OUT *ndocinteger*, OUT *nentry* integer)` | `setof record` | get statistics of a tsvectorcolumn | `ts_stat('SELECT vector from apod')` | `(foo,10,15) ...` | +Syntax: `json_each(json)` / `jsonb_each(jsonb)` - +Return type: `setof key text, value json` / `setof key text, value jsonb` -## Range Functions and Operators +Expands the outermost JSON object into a set of key/value pairs. -See [Range Types](data_types/datatype-range.md) for an overview of range types. +Example: + +``` +select * from json_each('{"a":"foo", "b":"bar"}') +key | value +-----+------- +a | "foo" +b | "bar" +``` -The following table shows the operators available for range types. - -| Operator | Description | Example | Result | | | -| -------- | ------------------------------- | --------------------------------------------------------------- | --------------------- | --------------------- | --- | -| `=` | equal | `int4range(1,5) = '[1,4]'::int4range` | `t` | | | -| `<>` | not equal | `numrange(1.1,2.2) <> numrange(1.1,2.3)` | `t` | | | -| `<` | less than | `int4range(1,10) < int4range(2,3)` | `t` | | | -| `>` | greater than | `int4range(1,10) > int4range(1,5)` | `t` | | | -| `<=` | less than or equal | `numrange(1.1,2.2) <= numrange(1.1,2.2)` | `t` | | | -| `>=` | greater than or equal | `numrange(1.1,2.2) >= numrange(1.1,2.0)` | `t` | | | -| `@>` | contains range | `int4range(2,4) @> int4range(2,3)` | `t` | | | -| `@>` | contains element | `'[2011-01-01,2011-03-01)'::tsrange @> '2011-01-10'::timestamp` | `t` | | | -| `<@` | range is contained by | `int4range(2,4) <@ int4range(1,7)` | `t` | | | -| `<@` | element is contained by | `42 <@ int4range(1,7)` | `f` | | | -| `&&` | overlap (have points in common) | `int8range(3,7) && int8range(4,12)` | `t` | | | -| `<<` | strictly left of | `int8range(1,10) << int8range(100,110)` | `t` | | | -| `>>` | strictly right of | `int8range(50,60) >> int8range(20,30)` | `t` | | | -| `&<` | does not extend to the right of | `int8range(1,20) &< int8range(18,20)` | `t` | | | -| `&>` | does not extend to the left of | `int8range(7,20) &> int8range(5,10)` | `t` | | | -| \`- | -\` | is adjacent to | \`numrange(1.1,2.2) - | - numrange(2.2,3.3)\` | `t` | -| `+` | union | `numrange(5,15) + numrange(10,20)` | `[5,20)` | | | -| `*` | intersection | `int8range(5,15) * int8range(10,20)` | `[10,15)` | | | -| `-` | difference | `int8range(5,15) - int8range(10,20)` | `[5,10)` | | | +##### `json_each_text() / jsonb_each_text()` -The simple comparison operators `<`, `>`, `<=`, and `>=` compare the lower bounds first, and only if those are equal, compare the upper bounds. These comparisons are not usually very useful for ranges, but are provided to allow B-tree indexes to be constructed on ranges. +Syntax: `json_each_text(json)` / `jsonb_each_text(jsonb)` -The left-of/right-of/adjacent operators always return false when an empty range is involved; that is, an empty range is not considered to be either before or after any other range. +Return type: `setof key text, value text` -The union and difference operators will fail if the resulting range would need to contain two disjoint sub-ranges, as such a range cannot be represented. +Expands the outermost JSON object into a set of key/value pairs. The returned values will be of type `text`. + +Example: + +``` +select * from json_each_text('{"a":"foo", "b":"bar"}') +key | value +-----+------- +a | foo +b | bar +``` + +##### `json_extract_path() / jsonb_extract_path()` + +Syntax: `json_extract_path(from_json json, VARIADIC path_elems text[])` / `jsonb_extract_path(from_json jsonb, VARIADIC path_elems text[])` + +Return type: `json` / `jsonb` + +Returns the JSON value pointed to by `path_elems` (equivalent to `#>` operator). + +Example: + +``` +json_extract_path('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}','f4') +{"f5":99,"f6":"foo"} +``` + +##### `json_extract_path_text() / jsonb_extract_path_text()` + +Syntax: `json_extract_path_text(from_json json, VARIADIC path_elems text[])` / `jsonb_extract_path_text(from_json jsonb, VARIADIC path_elems text[])` + +Return type: `text` + +Returns the JSON value pointed to by `path_elems` as text (equivalent to `#>>` operator). + +Example: + +``` +json_extract_path_text('{"f2":{"f3":1},"f4":{"f5":99,"f6":"foo"}','f4', 'f6') +foo +``` + +##### `json_object_keys() / jsonb_object_keys()` + +Syntax: `json_object_keys(json)` / `jsonb_object_keys(jsonb)` + +Return type: `setof text` + +Returns set of keys in the outermost JSON object. + +Example: + +``` +json_object_keys('{"f1":"abc","f2":{"f3":"a", "f4":"b"}') +json_object_keys +------------------ +f1 +f2 +``` + +##### `json_populate_record() / jsonb_populate_record()` + +Syntax: `json_populate_record(base anyelement, from_json json)` / `jsonb_populate_record(base anyelement, from_json jsonb)` + +Return type: `anyelement` + +Expands the object in `from_json` to a row whose columns match the record type defined by `base`. See the [Note 1](#notes-on-json-processing-functions-examples). + +Example: + +``` +select * from json_populate_record(null::myrowtype, '{"a": 1, "b": ["2", "a b"], "c": {"d": 4, "e": "a b c"}}') +a | b | c +---+-----------+------------- +1 | {2,"a b"} | (4,"a b c") +``` + +##### `json_populate_recordset() / jsonb_populate_recordset()` + +Syntax: `json_populate_recordset(base anyelement, from_json json)` / `jsonb_populate_recordset(base anyelement, from_json jsonb)` + +Return type: `setof anyelement` -The following table shows the functions available for use with range types. - -| Function | Return Type | Description | Example | Result | -| --------------------------------- | -------------------- | ---------------------------------------------------------- | ----------------------------------------------------- | ------- | -| `lower(anyrange)` | range's element type | lower bound of range | `lower(numrange(1.1,2.2))` | `1.1` | -| `upper(anyrange)` | range's element type | upper bound of range | `upper(numrange(1.1,2.2))` | `2.2` | -| `isempty(anyrange)` | `boolean` | is the range empty? | `isempty(numrange(1.1,2.2))` | `false` | -| `lower_inc(anyrange)` | `boolean` | is the lower bound inclusive? | `lower_inc(numrange(1.1,2.2))` | `true` | -| `upper_inc(anyrange)` | `boolean` | is the upper bound inclusive? | `upper_inc(numrange(1.1,2.2))` | `false` | -| `lower_inf(anyrange)` | `boolean` | is the lower bound infinite? | `lower_inf('(,)'::daterange)` | `true` | -| `upper_inf(anyrange)` | `boolean` | is the upper bound infinite? | `upper_inf('(,)'::daterange)` | `true` | -| `range_merge(anyrange, anyrange)` | `anyrange` | the smallest range which includes both of the given ranges | `range_merge('[1,2)'::int4range, '[3,4)'::int4range)` | `[1,4)` | +Expands the outermost array of objects in `from_json` to a set of rows whose columns match the record type defined by `base`. See the [Note 1](#notes-on-json-processing-functions-examples). + +Example: + +``` +select * from json_populate_recordset(null::myrowtype, '[{"a":1,"b":2},{"a":3,"b":4}]') +a | b +---+--- +1 | 2 +3 | 4 +``` + +##### `json_array_elements() / jsonb_array_elements()` + +Syntax: `json_array_elements(json)` / `jsonb_array_elements(jsonb)` + +Return type: `setof json` / `setof jsonb` + +Expands a JSON array to a set of JSON values. + +Example: + +``` +select * from json_array_elements('[1,true, [2,false]]') +value +----------- +1 +true +[2,false] +``` + +##### `json_array_elements_text() / jsonb_array_elements_text()` + +Syntax: `json_array_elements_text(json)` / `jsonb_array_elements_text(jsonb)` + +Return type: `setof text` + +Expands a JSON array to a set of `text` values. + +Example: + +``` +select * from json_array_elements_text('["foo", "bar"]') +value +----------- +foo +bar +``` + +##### `json_typeof() / jsonb_typeof()` + +Syntax: `json_typeof(json)` / `jsonb_typeof(jsonb)` + +Return type: `text` + +Returns the type of the outermost JSON value as a text string. Possible types are `object`, `array`, `string`, `number`, `boolean`, and `null`. + +Example: + +``` +json_typeof('-123.4') +number +``` + +##### `json_to_record() / jsonb_to_record()` + +Syntax: `json_to_record(json)` / `jsonb_to_record(jsonb)` + +Return type: `record` + +Builds an arbitrary record from a JSON object. See the [Note 1](#notes-on-json-processing-functions-examples). + +As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. + +Example: + +``` +select * from json_to_record('{"a":1,"b":[1,2,3], "c":[1,2,3],"e":"bar","r": {"a": 123, "b": "a b c"}') as x(a int, b text, c int[], d text, r myrowtype) +a | b | c | d | r +---+---------+---------+---+--------------- +1 | [1,2,3] | {1,2,3} | | (123,"a b c") +``` + +##### `json_to_recordset() / jsonb_to_recordset()` + +Syntax: `json_to_recordset(json)` / `jsonb_to_recordset(jsonb)` + +Return type: `setof record` + +Builds an arbitrary set of records from a JSON array of objects See the [Note 1](#notes-on-json-processing-functions-examples). + +As with all functions returning record, the caller must explicitly define the structure of the record with an `AS` clause. + +Example: + +``` +select * from json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","c":"bar"}]') as x(a int, b text); +a | b +---+----- +1 | foo +2 | +``` + +##### `json_strip_nulls() / jsonb_strip_nulls()` + +Syntax: `json_strip_nulls(from_json json)` / `jsonb_strip_nulls(from_json jsonb)` + +Return type: `json` / `jsonb` + +Returns *`from_json`* with all object fields that have null values omitted. Other null values are untouched. + +Example: + +``` +json_strip_nulls('[{"f1":1,"f2":null},2,null,3]') +[{"f1":1},2,null,3] +``` + +##### `jsonb_set()` + +Syntax: `jsonb_set(target jsonb, path text[], new_value jsonb [, create_missing boolean])` + +Return type: `jsonb` + +Returns *`target`* with the section designated by *`path`* replaced by *`new_value`*, or with *`new_value`* added if *`create_missing`* is true (default is `true`) and the item designated by *`path`* does not exist. As with the path oriented operators, negative integers that appear in *`path`* count from the end of JSON arrays. + +Example: + +``` +jsonb_set('[{"f1":1,"f2":null},2,null,3]', '{0,f1}','[2,3,4]', false) +[{"f1":[2,3,4],"f2":null},2,null,3] + +jsonb_set('[{"f1":1,"f2":null},2]', '{0,f3}','[2,3,4]') +[{"f1": 1, "f2": null, "f3": [2, 3, 4]}, 2] +``` + +##### `jsonb_insert()` + +Syntax: `jsonb_insert(target jsonb, path text[], new_value jsonb [, insert_after boolean])` + +Return type: `jsonb` + +Returns *`target`* with *`new_value`* inserted. If *`target`* section designated by *`path`* is in a JSONB array, *`new_value`* will be inserted before target or after if *`insert_after`* is true (default is `false`). If *`target`* section designated by *`path`* is in JSONB object, *`new_value`* will be inserted only if *`target`* does not exist. As with the path oriented operators, negative integers that appear in *`path`* count from the end of JSON arrays. + +Example: + +``` +jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"') +{"a": [0, "new_value", 1, 2]} + +jsonb_insert('{"a": [0,1,2]}', '{a, 1}', '"new_value"', true) +{"a": [0, 1, "new_value", 2]} +``` + +##### `jsonb_pretty()` + +Syntax: `jsonb_pretty(from_json jsonb)` + +Return type: `text` + +Returns *`from_json`* as indented JSON text. + +Example: + +``` +jsonb_pretty('[{"f1":1,"f2":null},2,null,3]') +[ + { + "f1": 1, + "f2": null + }, + 2, + null, + 3 +] +``` + +##### `jsonb_path_exists()` + +Syntax: `jsonb_path_exists(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` + +Return type: `boolean` + +Checks whether JSON path returns any item for the specified JSON value. + +Example: + +``` +jsonb_path_exists('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}') +true +``` + +##### `jsonb_path_match()` + +Syntax: `jsonb_path_match(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` + +Return type: `boolean` + +Returns the result of JSON path predicate check for the specified JSON value. Only the first item of the result is taken into account. If the result is not Boolean, then `null` is returned. + +Example: + +``` +jsonb_path_match('{"a":[1,2,3,4,5]}', 'exists($.a[*] ? (@ >= $min && @ <= $max))', '{"min":2,"max":4}') +true +``` + +##### `jsonb_path_query()` + +Syntax: `jsonb_path_query(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` + +Return type: `setof jsonb` + +Gets all JSON items returned by JSON path for the specified JSON value. + +Example: + +``` +select * from jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}'); +jsonb_path_query +------------------ +2 +3 +4 +``` + +##### `jsonb_path_query_array()` + +Syntax: `jsonb_path_query_array(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` + +Return type: `jsonb` + +Gets all JSON items returned by JSON path for the specified JSON value and wraps result into an array. + +Example: + +``` +jsonb_path_query_array('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}') +[2, 3, 4] +``` + +##### `jsonb_path_query_first()` + +Syntax: `jsonb_path_query_first(target jsonb, path jsonpath [, vars jsonb [, silent bool]])` + +Return type: `jsonb` + +Gets the first JSON item returned by JSON path for the specified JSON value. Returns `NULL` on no results. + +Example: + +``` +jsonb_path_query_first('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ >= $min && @ <= $max)', '{"min":2,"max":4}') +2 +``` + +##### Notes on JSON processing functions examples + +1. The functions `json[b]_populate_record()`, `json[b]_populate_recordset()`, `json[b]_to_record()` and `json[b]_to_recordset()` operate on a JSON object, or array of objects, and extract the values associated with keys whose names match column names of the output row type. Object fields that do not correspond to any output column name are ignored, and output columns that do not match any object field will be filled with nulls. To convert a JSON value to the SQL type of an output column, the following rules are applied in sequence: + + - A JSON null value is converted to a SQL null in all cases. + - If the output column is of type `json` or `jsonb`, the JSON value is just reproduced exactly. + - If the output column is a composite (row) type, and the JSON value is a JSON object, the fields of the object are converted to columns of the output row type by recursive application of these rules. + - Likewise, if the output column is an array type and the JSON value is a JSON array, the elements of the JSON array are converted to elements of the output array by recursive application of these rules. + - Otherwise, if the JSON value is a string literal, the contents of the string are fed to the input conversion function for the column's data type. + - Otherwise, the ordinary text representation of the JSON value is fed to the input conversion function for the column's data type. + + While the examples for these functions use constants, the typical use would be to reference a table in the `FROM` clause and use one of its `json` or `jsonb` columns as an argument to the function. Extracted key values can then be referenced in other parts of the query, like `WHERE` clauses and target lists. Extracting multiple values in this way can improve performance over extracting them separately with per-key operators. + +2. All the items of the `path` parameter of `jsonb_set()` as well as `jsonb_insert()` except the last item must be present in the target. If `create_missing` is false, all items of the `path` parameter of `jsonb_set()` must be present. If these conditions are not met the target is returned unchanged. + + If the last path item is an object key, it will be created if it is absent and given the new value. If the last path item is an array index, if it is positive the item to set is found by counting from the left, and if negative by counting from the right - `-1` designates the rightmost element, and so on. If the item is out of the range -array_length .. array_length -1, and create_missing is true, the new value is added at the beginning of the array if the item is negative, and at the end of the array if it is positive. + + +1. The `json_typeof` function's `null` return value should not be confused with a SQL NULL. While calling `json_typeof('null'::json)` will return `null`, calling `json_typeof(NULL::json)` will return a SQL NULL. + + +1. If the argument to `json_strip_nulls()` contains duplicate field names in any object, the result could be semantically somewhat different, depending on the order in which they occur. This is not an issue for `jsonb_strip_nulls()` since `jsonb` values never have duplicate object field names. + +2. The `jsonb_path_exists()`, `jsonb_path_match()`, `jsonb_path_query()`, `jsonb_path_query_array()`, and `jsonb_path_query_first()` functions have optional `vars` and `silent` arguments. + + If the `vars` argument is specified, it provides an object containing named variables to be substituted into a `jsonpath` expression. + + If the `silent` argument is specified and has the `true` value, these functions suppress the same errors as the `@?` and `@@` operators. + + + +### The SQL/JSON Path Language + +SQL/JSON path expressions specify the items to be retrieved from the JSON data, similar to XPath expressions used for SQL access to XML. In WarehousePG, path expressions are implemented as the `jsonpath` data type and can use any elements described in [jsonpath Type](../admin_guide/query/json-data.md#topic_jsonpath). + +JSON query functions and operators pass the provided path expression to the *path engine* for evaluation. If the expression matches the queried JSON data, the corresponding SQL/JSON item is returned. Path expressions are written in the SQL/JSON path language and can also include arithmetic expressions and functions. Query functions treat the provided expression as a text string, so it must be enclosed in single quotes. + +A path expression consists of a sequence of elements allowed by the `jsonpath` data type. The path expression is evaluated from left to right, but you can use parentheses to change the order of operations. If the evaluation is successful, a sequence of SQL/JSON items (*SQL/JSON sequence*) is produced, and the evaluation result is returned to the JSON query function that completes the specified computation. + +To refer to the JSON data to be queried (the *context item*), use the `$` sign in the path expression. It can be followed by one or more [accessor operators](../admin_guide/query/json-data.md#topic_jsonpath), which go down the JSON structure level by level to retrieve the content of context item. Each operator that follows deals with the result of the previous evaluation step. + +For example, suppose you have some JSON data from a GPS tracker that you would like to parse, such as: + +``` +{ + "track": { + "segments": [ + { + "location"[ 47.763, 13.4034 ], + "start time": "2018-10-14 10:05:14", + "HR": 73 + }, + { + "location"[ 47.706, 13.2635 ], + "start time": "2018-10-14 10:39:21", + "HR": 135 + } + ] + } +} +``` + +To retrieve the available track segments, you need to use the `.` accessor operator for all the preceding JSON objects: + +``` +'$.track.segments' +``` + +If the item to retrieve is an element of an array, you have to unnest this array using the `[*]` operator. For example, the following path will return location coordinates for all the available track segments: + +``` +'$.track.segments[*].location' +``` + +To return the coordinates of the first segment only, you can specify the corresponding subscript in the `[]` accessor operator. Note that the SQL/JSON arrays are 0-relative: + +``` +'$.track.segments[0].location' +``` + +The result of each path evaluation step can be processed by one or more `jsonpath` operators and methods listed in [SQL/JSON Path Operators and Methods](#topic_jsonpath_opsmeth) below. Each method name must be preceded by a dot. For example, you can get an array size: + +``` +'$.track.segments.size()' +``` + +For more examples of using `jsonpath` operators and methods within path expressions, see [SQL/JSON Path Operators and Methods](#topic_jsonpath_opsmeth) below. + +When defining the path, you can also use one or more *filter expressions* that work similar to the `WHERE` clause in SQL. A filter expression begins with a question mark and provides a condition in parentheses: + +``` +? (condition) +``` + +Filter expressions must be specified right after the path evaluation step to which they are applied. The result of this step is filtered to include only those items that satisfy the provided condition. SQL/JSON defines three-valued logic, so the condition can be `true`, `false`, or `unknown`. The `unknown` value plays the same role as SQL `NULL` and can be tested for with the is `unknown` predicate. Further path evaluation steps use only those items for which filter expressions return `true`. + +Functions and operators that can be used in filter expressions are listed in [jsonpath Filter Expression Elements](#topic_jsonpath_filtexp). The path evaluation result to be filtered is denoted by the `@` variable. To refer to a JSON element stored at a lower nesting level, add one or more accessor operators after `@`. + +Suppose you would like to retrieve all heart rate values higher than 130. You can achieve this using the following expression: + +``` +'$.track.segments[*].HR ? (@ > 130)' +``` + +To get the start time of segments with such values instead, you have to filter out irrelevant segments before returning the start time, so the filter expression is applied to the previous step, and the path used in the condition is different: + +``` +'$.track.segments[*] ? (@.HR > 130)."start time"' +``` + +You can use several filter expressions on the same nesting level, if required. For example, the following expression selects all segments that contain locations with relevant coordinates and high heart rate values: + +``` +'$.track.segments[*] ? (@.location[1] < 13.4) ? (@.HR > 130)."start time"' +``` + +Using filter expressions at different nesting levels is also allowed. The following example first filters all segments by location, and then returns high heart rate values for these segments, if available: + +``` +'$.track.segments[*] ? (@.location[1] < 13.4).HR ? (@ > 130)' +``` + +You can also nest filter expressions within each other: + +``` +'$.track ? (exists(@.segments[*] ? (@.HR > 130))).segments.size()' +``` + +This expression returns the size of the track if it contains any segments with high heart rate values, or an empty sequence otherwise. + + + +#### Deviations from Standard + +WarehousePG's implementation of SQL/JSON path language has the following deviations from the SQL/JSON standard: + +- `.datetime()` item method is not implemented yet mainly because immutable `jsonpath` functions and operators cannot reference session timezone, which is used in some datetime operations. Datetime support will be added to `jsonpath` in future versions of WarehousePG. +- A path expression can be a Boolean predicate, although the SQL/JSON standard allows predicates only in filters. This is necessary for implementation of the `@@` operator. For example, the following `jsonpath` expression is valid in WarehousePG: + + ``` + '$.track.segments[*].HR < 70' + ``` +- There are minor differences in the interpretation of regular expression patterns used in `like_regex` filters as described in [Regular Expressions](#topic_jsonpath_regexp). + + + +#### Strict And Lax Modes + +When you query JSON data, the path expression may not match the actual JSON data structure. An attempt to access a non-existent member of an object or element of an array results in a structural error. SQL/JSON path expressions have two modes of handling structural errors: + +- lax (default) — the path engine implicitly adapts the queried data to the specified path. Any remaining structural errors are suppressed and converted to empty SQL/JSON sequences. + +- strict — if a structural error occurs, an error is raised. + +The lax mode facilitates matching of a JSON document structure and path expression if the JSON data does not conform to the expected schema. If an operand does not match the requirements of a particular operation, it can be automatically wrapped as an SQL/JSON array or unwrapped by converting its elements into an SQL/JSON sequence before performing this operation. Besides, comparison operators automatically unwrap their operands in the lax mode, so you can compare SQL/JSON arrays out-of-the-box. An array of size 1 is considered equal to its sole element. Automatic unwrapping is not performed only when: + +- The path expression contains `type()` or `size()` methods that return the type and the number of elements in the array, respectively. + +- The queried JSON data contain nested arrays. In this case, only the outermost array is unwrapped, while all the inner arrays remain unchanged. Thus, implicit unwrapping can only go one level down within each path evaluation step. + +For example, when querying the GPS data listed above, you can abstract from the fact that it stores an array of segments when using the lax mode: + +``` +'lax $.track.segments.location' +``` + +In the strict mode, the specified path must exactly match the structure of the queried JSON document to return an SQL/JSON item, so using this path expression will cause an error. To get the same result as in the lax mode, you have to explicitly unwrap the segments array: + +``` +'strict $.track.segments[*].location' +``` + +The `.**` accessor can lead to surprising results when using the lax mode. For instance, the following query selects every HR value twice: + +``` +lax $.**.HR +``` + +This happens because the `.**` accessor selects both the segments array and each of its elements, while the `.HR` accessor automatically unwraps arrays when using the lax mode. To avoid surprising results, we recommend using the `.**` accessor only in the strict mode. The following query selects each HR value just once: + +``` +strict $.**.HR +``` + + + +#### Regular Expressions + +SQL/JSON path expressions allow matching text to a regular expression with the `like_regex` filter. For example, the following SQL/JSON path query would case-insensitively match all strings in an array that starts with an English vowel: + +``` +'$[*] ? (@ like_regex "^[aeiou]" flag "i")' +``` + +The optional flag string may include one or more of the characters `i` for case-insensitive match, `m` to allow `^` and `$` to match at newlines, `s` to allow `.` to match a newline, and `q` to quote the whole pattern (reducing the behavior to a simple substring match). + +The SQL/JSON standard borrows its definition for regular expressions from the `LIKE_REGEX` operator, which in turn uses the XQuery standard. WarehousePG does not currently support the `LIKE_REGEX` operator. Therefore, the `like_regex` filter is implemented using the POSIX regular expression engine as described in [POSIX Regular Expressions](https://www.postgresql.org/docs/12/functions-matching.html#FUNCTIONS-POSIX-REGEXP). This leads to various minor discrepancies from standard SQL/JSON behavior which are catalogued in [Differences From XQuery (LIKE_REGEX)](https://www.postgresql.org/docs/12/functions-matching.html#POSIX-VS-XQUERY). Note, however, that the flag-letter incompatibilities described there do not apply to SQL/JSON, as it translates the XQuery flag letters to match what the POSIX engine expects. + +Keep in mind that the pattern argument of `like_regex` is a JSON path string literal, written according to the rules given in [jsonpath Type](../admin_guide/query/json-data.md#topic_jsonpath). This means in particular that any backslashes you want to use in the regular expression must be doubled. For example, to match string values of the root document that contain only digits: + +``` +$.* ? (@ like_regex "^\\d+$") +``` + + + +#### SQL/JSON Path Operators and Methods + +The following operators and methods are available in `jsonpath`: + +##### `+ (unary)` + +Plus operator that iterates over the SQL/JSON sequence. + +Example JSON: `{"x": [2.85, -14.7, -9.4]}` + +Example query: `+ $.x.floor()` + +Result: `2, -15, -10` + +##### `- (unary)` + +Minus operator that iterates over the SQL/JSON sequence. + +Example JSON: `{"x": [2.85, -14.7, -9.4]}` + +Example query: `- $.x.floor()` + +Result: `-2, 15, 10` + +##### `+ (binary)` + +Addition. + +Example JSON: `[2]` + +Example query: `2 + $[0]` + +Result: `4` + +##### `- (binary)` + +Subtraction. + +Example JSON: `[2]` + +Example query: `4 - $[0]` + +Result: `2` + +##### `*` + +Multiplication. + +Example JSON: `[4]` + +Example query: `2 * $[0]` + +Result: `8` + +##### `/` + +Division. + +Example JSON: `[8]` + +Example query: `$[0] / 2` + +Result: `4` + +##### `%` + +Modulus. + +Example JSON: `[32]` + +Example query: `$[0] % 10` + +Result: `2` + +##### `type()` + +Type of the SQL/JSON item. + +Example JSON: `[1, "2", {}]` + +Example query: `$[*].type()` + +Result: `"number", "string", "object"` + +##### `size()` + +Size of the SQL/JSON item. + +Example JSON: `{"m": [11, 15]}` + +Example query: `$.m.size()` + +Result: `2` + +##### `double()` + +Approximate floating-point number converted from an SQL/JSON number or a string. + +Example JSON: `{"len": "1.9"}` + +Example query: `$.len.double() * 2` + +Result: `3.8` + +##### `ceiling()` + +Nearest integer greater than or equal to the SQL/JSON number. + +Example JSON: `{"h": 1.3}` + +Example query: `$.h.ceiling()` + +Result: `2` + +##### `floor()` + +Nearest integer less than or equal to the SQL/JSON number. + +Example JSON: `{"h": 1.3}` + +Example query: `$.h.floor()` + +Result: `1` + +##### `abs()` + +Absolute value of the SQL/JSON number. + +Example JSON: `{"z": -0.3}` + +Example query: `$.z.abs()` + +Result: `0.3` + +##### `keyvalue()` + +Sequence of the object's key-value pairs represented as an array of items containing three fields (`"key"`, `"value"`, and `"id"`). `"id"` is a unique identifier of the object the key-value pair belongs to. + +Example JSON: `{"x": "20", "y": 32}` + +Example query: `$.keyvalue()` + +Result: `{"key": "x", "value": "20", "id": 0}, {"key": "y", "value": 32, "id": 0}` + + + +#### SQL/JSON Filter Expression Elements + +The following filter expression elements are available for `jsonpath`: + +##### `==` + +Equality operator. + +Example JSON: `[1, 2, 1, 3]` + +Example query: `$[*] ? (@ == 1)` + +Result: `1, 1` + +##### `!=` + +Non-equality operator. + +Example JSON: `[1, 2, 1, 3]` + +Example query: `$[*] ? (@ != 1)` + +Result: `2, 3` + +##### `<>` + +Non-equality operator (same as `!=`). + +Example JSON: `[1, 2, 1, 3]` + +Example query: `$[*] ? (@ <> 1)` + +Result: `2, 3` + +##### `<` + +Less-than operator. + +Example JSON: `[1, 2, 3]` + +Example query: `$[*] ? (@ < 2)` + +Result: `1` + +##### `<=` + +Less-than-or-equal-to operator. + +Example JSON: `[1, 2, 3]` + +Example query: `$[*] ? (@ <= 2)` + +Result: `1, 2` + +##### `>` + +Greater-than operator. + +Example JSON: `[1, 2, 3]` + +Example query: `$[*] ? (@ > 2)` + +Result: `3` + +##### `>=` + +Greater-than-or-equal-to operator. + +Example JSON: `[1, 2, 3]` + +Example query: `$[*] ? (@ >= 2)` + +Result: `2, 3` + +##### `true` + +Value used to perform comparison with JSON `true` literal. + +Example JSON: `[{"name": "John", "parent": false}, {"name": "Chris", "parent": true}]` + +Example query: `$[*] ? (@.parent == true)` + +Result: `{"name": "Chris", "parent": true}` + +##### `false` + +Value used to perform comparison with JSON `false` literal. + +Example JSON: `[{"name": "John", "parent": false}, {"name": "Chris", "parent": true}]` + +Example query: `$[*] ? (@.parent == false)` + +Result: `{"name": "John", "parent": false}` + +##### `null` + +Value used to perform comparison with JSON `null` value. + +Example JSON: `[{"name": "Mary", "job": null}, {"name": "Michael", "job": "driver"}]` + +Example query: `$[*] ? (@.job == null) .name` + +Result: `"Mary"` + +##### `&&` + +Boolean AND. + +Example JSON: `[1, 3, 7]` + +Example query: `$[*] ? (@ > 1 && @ < 5)` + +Result: `3` + +##### `||` + +Boolean OR. + +Example JSON: `[1, 3, 7]` + +Example query: `$[*] ? (@ < 1 || @ > 5)` + +Result: `7` + +##### `!` + +Boolean NOT. + +Example JSON: `[1, 3, 7]` + +Example query: `$[*] ? (!(@ < 5))` + +Result: `7` + +##### `like_regex` + +Tests whether the first operand matches the regular expression given by the second operand, optionally with modifications described by a string of `flag` characters. + +Example JSON: `["abc", "abd", "aBdC", "abdacb", "babc"]` + +Example query: `$[*] ? (@ like_regex "^ab.*c" flag "i")` + +Result: `"abc", "aBdC", "abdacb"` + +##### `starts with` + +Tests whether the second operand is an initial substring of the first operand. + +Example JSON: `["John Smith", "Mary Stone", "Bob Johnson"]` + +Example query: `$[*] ? (@ starts with "John")` + +Result: `"John Smith"` + +##### `exists` + +Tests whether a path expression matches at least one SQL/JSON item. + +Example JSON: `{"x": [1, 2], "y": [2, 4]}` + +Example query: `strict $.* ? (exists (@ ? (@[*] > 2)))` + +Result: `2, 4` + +##### `is unknown` + +Tests whether a Boolean condition is `unknown`. + +Example JSON: `[-1, 2, 7, "infinity"]` + +Example query: `$[*] ? ((@ > 0) is unknown)` + +Result: `"infinity"` + + + +## Window Functions + +The following are WarehousePG built-in window functions. All window functions are *immutable*. For more information about window functions, see "Window Expressions" in the *WarehousePG Administrator Guide*. + +### `cume_dist()` + +Return type: `double precision` + +Full syntax: `CUME_DIST() OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` + +Calculates the cumulative distribution of a value in a group of values. Rows with equal values always evaluate to the same cumulative distribution value. + +### `dense_rank()` + +Return type: `bigint` + +Full syntax: `DENSE_RANK () OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` + +Computes the rank of a row in an ordered group of rows without skipping rank values. Rows with equal values are given the same rank value. + +### `first_value(`*expr*`)` + +Return type: same as input *expr* type + +Full syntax: `FIRST_VALUE(` *expr* `) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `[ROWS | RANGE` *frame_expr* `] )` + +Returns the first value in an ordered set of values. + +### `lag(`*expr*` [,`*offset*`] [,`*default*`])` + +Return type: same as input *expr* type + +Full syntax: `LAG(` *expr* `[,` *offset* `] [,` *default* `]) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` + +Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `LAG` provides access to a row at a given physical offset prior to that position. The default `offset` is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. + +### `last_value(`*expr*`)` + +Return type: same as input *expr* type + +Full syntax: `LAST_VALUE(` *expr* `) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `[ROWS | RANGE` *frame_expr* `] )` + +Returns the last value in an ordered set of values. + +### `lead(`*expr*` [,`*offset*`] [,`*default*`])` + +Return type: same as input *expr* type + +Full syntax: `LEAD(` *expr* `[,` *offset* `] [,` *default* `]) OVER ( [PARTITION BY` *expr* `] ORDER BY` *expr* `)` + +Provides access to more than one row of the same table without doing a self join. Given a series of rows returned from a query and a position of the cursor, `lead` provides access to a row at a given physical offset after that position. If *offset* is not specified, the default offset is 1. *default* sets the value that is returned if the offset goes beyond the scope of the window. If *default* is not specified, the default value is null. + +### `ntile(`*expr*`)` + +Return type: `bigint` + +Full syntax: `NTILE(*expr*) OVER ( [PARTITION BY *expr*] ORDER BY *expr* )` + +Divides an ordered data set into a number of buckets (as defined by *expr*) and assigns a bucket number to each row. + +### `percent_rank()` + +Return type: `double precision` + +Full syntax: `PERCENT_RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` + +Calculates the rank of a hypothetical row `R` minus 1, divided by 1 less than the number of rows being evaluated (within a window partition). + +### `rank()` + +Return type: `bigint` + +Full syntax: `RANK () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` + +Calculates the rank of a row in an ordered group of values. Rows with equal values for the ranking criteria receive the same rank. The number of tied rows are added to the rank number to calculate the next rank value. Ranks may not be consecutive numbers in this case. + +### `row_number()` + +Return type: `bigint` + +Full syntax: `ROW_NUMBER () OVER ( [PARTITION BY *expr*] ORDER BY *expr*)` + +Assigns a unique number to each row to which it is applied (either each row in a window partition or each row of the query). + + + +## Advanced Aggregate Functions + +The following built-in advanced analytic functions are WarehousePG extensions of the PostgreSQL database. Analytic functions are *immutable*. + +> **Note** The WarehousePG MADlib Extension for Analytics provides additional advanced functions to perform statistical analysis and machine learning with WarehousePG data. See [MADlib Extension for Analytics](../admin_guide/analytics/madlib.md). + +### `pivot_sum (label[], label, expr)` + +Return type: `int[], bigint[], float[]` + +Full syntax: `pivot_sum( array['A1','A2'], attr, value)` + +A pivot aggregation using sum to resolve duplicate entries. + +### `unnest (array[])` + +Return type: set of `anyelement` + +Full syntax: `unnest( array['one', 'row', 'per', 'item'])` + +Transforms a one dimensional array into rows. Returns a set of `anyelement`, a polymorphic [pseudotype in PostgreSQL](https://www.postgresql.org/docs/12/datatype-pseudo.html). + +### `gp_array_agg (`*anyarray*`)` + +Return type: same as the argument data type + +Full syntax: `gp_array_agg (`*anyarray*`)` + +A parallel version of `array_agg(anyarray)`. Concatenates input arrays to create an array of one higher dimension. The inputs must all have the same dimensions, and they cannot be empty or null. + +Example: + +``` +CREATE TABLE intarr_tbl (a int, arr int[]); +INSERT INTO intarr_tbl SELECT i, array[i, i] FROM generate_series(1, 5)i; +INSERT INTO intarr_tbl SELECT 6, '{6, NULL}'::int[]; +INSERT INTO intarr_tbl SELECT 8, '{NULL, 7}'::int[]; +SELECT gp_array_agg(arr ORDER BY arr) FROM intarr_tbl; +``` + +### `gp_array_agg (`*anynonarray*`)` + +Return type: array of the argument type + +Full syntax: `gp_array_agg (`*anynonarray*`)` + +An parallel version of `array_agg(anynonarray)`. Creates an array by concatenating input values, including nulls. + +Example: + +``` +CREATE TABLE table1(a int4, b int4); +INSERT INTO table1 VALUES (4,5), (2,1), (1,3), (3,null), (3,7); +SELECT gp_array_agg(a ORDER BY b NULLS FIRST) FROM table1; +``` + +### `MEDIAN (`*expr*`)` + +Return type: `timestamp, timestamptz, interval, float` + +Full syntax: `MEDIAN (`*expression*`)` + +Can take a two-dimensional array as input. Treats such arrays as matrices. + +Example: + +``` +SELECT department_id, MEDIAN(salary) +FROM employees +GROUP BY department_id; +``` + +### `PERCENTILE_CONT (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` + +Return type: `timestamp, timestamptz, interval, float` + +Full syntax: `PERCENTILE_CONT(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)` + +Performs an inverse distribution function that assumes a continuous distribution model. It takes a percentile value and a sort specification and returns the same datatype as the numeric datatype of the argument. This returned value is a computed result after performing linear interpolation. Null are ignored in this calculation. + +Example: + +``` +SELECT department_id, +PERCENTILE_CONT (0.5) WITHIN GROUP (ORDER BY salary DESC) +"Median_cont"; +FROM employees GROUP BY department_id; +``` + +### `PERCENTILE_DISC (`*expr*`) WITHIN GROUP (ORDER BY `*expr*` [DESC/ASC])` + +Return type: `timestamp, timestamptz, interval, float` + +Full syntax: `PERCENTILE_DISC(`*percentage*`) WITHIN GROUP (ORDER BY `*expression*`)` + +Performs an inverse distribution function that assumes a discrete distribution model. It takes a percentile value and a sort specification. This returned value is an element from the set. Null are ignored in this calculation. + +Example: + +``` +SELECT department_id, +PERCENTILE_DISC (0.5) WITHIN GROUP (ORDER BY salary DESC) +"Median_desc"; +FROM employees GROUP BY department_id; +``` + +### `sum(array[])` + +Return type: `smallint[]int[], bigint[], float[]` + +Full syntax: `sum(array[[1,2],[3,4]])` + +Performs matrix summation. Can take as input a two-dimensional array that is treated as a matrix. + +Example: + +``` +CREATE TABLE mymatrix (myvalue int[]); +INSERT INTO mymatrix VALUES (array[[1,2],[3,4]]); +INSERT INTO mymatrix VALUES (array[[0,1],[1,0]]); +SELECT sum(myvalue) FROM mymatrix; +sum +--------------- +{1,3},{4,4} +``` + + + +## Text Search Functions and Operators + +The following sections summarize the functions and operators that are provided for full text searching. See [Using Full Text Search](../admin_guide/query/textsearch/index.md) for a detailed explanation of WarehousePG's text search facility. + +### `@@` + +`tsvector` matches `tsquery`? + +Example: + +``` +to_tsvector('fat cats ate rats') @@ to_tsquery('cat & rat') +t +``` + +### `@@@` + +Deprecated synonym for `@@`. + +Example: + +``` +to_tsvector('fat cats ate rats') @@@ to_tsquery('cat & rat') +t +``` + +### `||` (tsvector) + +Concatenates `tsvector`s. + +Example: + +``` +'a:1 b:2'::tsvector || 'c:1 d:2 b:3'::tsvector +'a':1 'b':2,5 'c':3 'd':4 +``` + +### `&&` + +ANDs `tsquery`s together. + +Example: + +``` +'fat | rat'::tsquery && 'cat'::tsquery +( 'fat' | 'rat' ) & 'cat' +``` + +### `||` (tsquery) + +ORs `tsquery`s together. + +Example: + +``` +'fat | rat'::tsquery || 'cat'::tsquery +( 'fat' | 'rat' ) | 'cat' +``` + +### `!!` + +Negates a `tsquery`. + +Example: + +``` +!! 'cat'::tsquery +!'cat' +``` + +### `@>` (tsquery) + +Does one `tsquery` contain another? + +Example: + +``` +'cat'::tsquery @> 'cat & rat'::tsquery +f +``` + +### `<@` (tsquery) + +Is one `tsquery` contained in another? + +Example: + +``` +'cat'::tsquery <@ 'cat & rat'::tsquery +t +``` + +> **Note** The `tsquery` containment operators consider only the lexemes listed in the two queries, ignoring the combining operators. + +In addition to the operators described above, the ordinary B-tree comparison operators (=, <, etc) are defined for types `tsvector` and `tsquery`. These are not very useful for text searching but allow, for example, unique indexes to be built on columns of these types. + +### `get_current_ts_config()` + +Return type: `regconfig` + +Get default text search configuration. + +Example: + +``` +get_current_ts_config() +english +``` + +### `length(tsvector)` + +Return type: `integer` + +Number of lexemes in tsvector. + +Example: + +``` +length('fat:2,4 cat:3 rat:5A'::tsvector) +3 +``` + +### `numnode(tsquery)` + +Return type: `integer` + +Number of lexemes plus operators in tsquery. + +Example: + +``` +numnode('(fat & rat) | cat'::tsquery) +5 +``` + +### `plainto_tsquery([ config regconfig , ] querytext)` + +Return type: `tsquery` + +Produce tsquery ignoring punctuation. + +Example: + +``` +plainto_tsquery('english', 'The Fat Rats') +'fat' & 'rat' +``` + +### `querytree(query tsquery)` + +Return type: `text` + +Get indexable part of a tsquery. + +Example: + +``` +querytree('foo & ! bar'::tsquery) +'foo' +``` + +### `setweight(tsvector, "char")` + +Return type: `tsvector` + +Assign weight to each element of tsvector. + +Example: + +``` +setweight('fat:2,4 cat:3 rat:5B'::tsvector, 'A') +'cat':3A 'fat':2A,4A 'rat':5A +``` + +### `strip(tsvector)` + +Return type: `tsvector` + +Remove positions and weights from tsvector. + +Example: + +``` +strip('fat:2,4 cat:3 rat:5A'::tsvector) +'cat' 'fat' 'rat' +``` + +### `to_tsquery([ config regconfig , ] query text)` + +Return type: `tsquery` + +Normalize words and convert to tsquery. + +Example: + +``` +to_tsquery('english', 'The & Fat & Rats') +'fat' & 'rat' +``` + +### `to_tsvector([ config regconfig , ] documenttext)` + +Return type: `tsvector` + +Reduce document text to tsvector. + +Example: + +``` +to_tsvector('english', 'The Fat Rats') +'fat':2 'rat':3 +``` + +### `ts_headline([ config regconfig, ] documenttext, query tsquery [, options text ])` + +Return type: `text` + +Display a query match. + +Example: + +``` +ts_headline('x y z', 'z'::tsquery) +x y z +``` + +### `ts_rank([ weights float4[], ] vector tsvector,query tsquery [, normalization integer ])` + +Return type: `float4` + +Rank document for query. + +Example: + +``` +ts_rank(textsearch, query) +0.818 +``` + +### `ts_rank_cd([ weights float4[], ] vectortsvector, query tsquery [, normalizationinteger ])` + +Return type: `float4` + +Rank document for query using cover density. + +Example: + +``` +ts_rank_cd('{0.1, 0.2, 0.4, 1.0}', textsearch, query) +2.01317 +``` + +### `ts_rewrite(query tsquery, target tsquery,substitute tsquery)` + +Return type: `tsquery` + +Replace target with substitute within query. + +Example: + +``` +ts_rewrite('a & b'::tsquery, 'a'::tsquery, 'foo|bar'::tsquery) +'b' & ( 'foo' | 'bar' ) +``` + +### `ts_rewrite(query tsquery, select text)` + +Return type: `tsquery` + +Replace using targets and substitutes from a SELECT command. + +Example: + +``` +SELECT ts_rewrite('a & b'::tsquery, 'SELECT t,s FROM aliases') +'b' & ( 'foo' | 'bar' ) +``` + +### `tsvector_update_trigger()` + +Return type: `trigger` + +Trigger function for automatic tsvector column update. + +Example: + +``` +CREATE TRIGGER ... tsvector_update_trigger(tsvcol, 'pg_catalog.swedish', title, body) +``` + +### `tsvector_update_trigger_column()` + +Return type: `trigger` + +Trigger function for automatic tsvector column update. + +Example: + +``` +CREATE TRIGGER ... tsvector_update_trigger_column(tsvcol, configcol, title, body) +``` + +> **Note** All the text search functions that accept an optional `regconfig` argument will use the configuration specified by [default_text_search_config](config_params/guc-list.md) when that argument is omitted. + +The following functions are listed separately because they are not usually used in everyday text searching operations. They are helpful for development and debugging of new text search configurations. + +### `ts_debug([ `*config*` regconfig, ] `*document*` text, OUT `*alias*` text, OUT `*description*` text, OUT `*token*` text, OUT `*dictionaries*` regdictionary[], OUT `*dictionary*` regdictionary, OUT `*lexemes*` text[])` + +Return type: `setof record` + +Test a configuration. + +Example: + +``` +ts_debug('english', 'The Brightest supernovaes') +(asciiword,"Word, all ASCII",The,{english_stem},english_stem,{}) ... +``` + +### `ts_lexize(`*dict*` regdictionary, `*token*` text)` + +Return type: `text[]` + +Test a dictionary. + +Example: + +``` +ts_lexize('english_stem', 'stars') +{star} +``` + +### `ts_parse(`*parser_name*` text, `*document*` text, OUT `*tokid*` integer, OUT `*token*` text)` + +Return type: `setof record` + +Test a parser. + +Example: + +``` +ts_parse('default', 'foo - bar') +(1,foo) ... +``` + +### `ts_parse(`*parser_oid*` oid, `*document*` text, OUT `*tokid*` integer, OUT `*token*` text)` + +Return type: `setof record` + +Test a parser. + +Example: + +``` +ts_parse(3722, 'foo - bar') +(1,foo) ... +``` + +### `ts_token_type(`*parser_name*` text, OUT `*tokid*` integer, OUT `*alias*` text, OUT description text)` + +Return type: `setof record` + +Get token types defined by parser. + +Example: + +``` +ts_token_type('default') +(1,asciiword,"Word, all ASCII") ... +``` + +### `ts_token_type(`*parser_oid*` oid, OUT `*tokid*` integer, OUT `*alias*` text, OUT `*description*` text)` + +Return type: `setof record` + +Get token types defined by parser. + +Example: + +``` +ts_token_type(3722) +(1,asciiword,"Word, all ASCII") ... +``` + +### `ts_stat(`*sqlquery*` text, [ `*weights*` text, ] OUT `*word*` text, OUT `*ndocinteger*`, OUT `*nentry*` integer)` + +Return type: `setof record` + +Get statistics of a tsvector column. + +Example: + +``` +ts_stat('SELECT vector from apod') +(foo,10,15) ... +``` + + + +## Range Functions and Operators + +See [Range Types](data_types/datatype-range.md) for an overview of range types. + +The following operators are available for range types. + +### `=` + +Equal. + +Example: + +``` +int4range(1,5) = '[1,4]'::int4range +t +``` + +### `<>` + +Not equal. + +Example: + +``` +numrange(1.1,2.2) <> numrange(1.1,2.3) +t +``` + +### `<` + +Less than. + +Example: + +``` +int4range(1,10) < int4range(2,3) +t +``` + +### `>` + +Greater than. + +Example: + +``` +int4range(1,10) > int4range(1,5) +t +``` + +### `<=` + +Less than or equal. + +Example: + +``` +numrange(1.1,2.2) <= numrange(1.1,2.2) +t +``` + +### `>=` + +Greater than or equal. + +Example: + +``` +numrange(1.1,2.2) >= numrange(1.1,2.0) +t +``` + +### `@>` (contains range) + +Example: + +``` +int4range(2,4) @> int4range(2,3) +t +``` + +### `@>` (contains element) + +Example: + +``` +'[2011-01-01,2011-03-01)'::tsrange @> '2011-01-10'::timestamp +t +``` + +### `<@` (range is contained by) + +Example: + +``` +int4range(2,4) <@ int4range(1,7) +t +``` + +### `<@` (element is contained by) + +Example: + +``` +42 <@ int4range(1,7) +f +``` + +### `&&` (overlap) + +Overlap, meaning the ranges have points in common. + +Example: + +``` +int8range(3,7) && int8range(4,12) +t +``` + +### `<<` (strictly left of) + +Example: + +``` +int8range(1,10) << int8range(100,110) +t +``` + +### `>>` (strictly right of) + +Example: + +``` +int8range(50,60) >> int8range(20,30) +t +``` + +### `&<` (does not extend to the right of) + +Example: + +``` +int8range(1,20) &< int8range(18,20) +t +``` + +### `&>` (does not extend to the left of) + +Example: + +``` +int8range(7,20) &> int8range(5,10) +t +``` + +### `-|-` (is adjacent to) + +Example: + +``` +numrange(1.1,2.2) -|- numrange(2.2,3.3) +t +``` + +### `+` (union) + +Example: + +``` +numrange(5,15) + numrange(10,20) +[5,20) +``` + +### `*` (intersection) + +Example: + +``` +int8range(5,15) * int8range(10,20) +[10,15) +``` + +### `-` (difference) + +Example: + +``` +int8range(5,15) - int8range(10,20) +[5,10) +``` + +The simple comparison operators `<`, `>`, `<=`, and `>=` compare the lower bounds first, and only if those are equal, compare the upper bounds. These comparisons are not usually very useful for ranges, but are provided to allow B-tree indexes to be constructed on ranges. + +The left-of/right-of/adjacent operators always return false when an empty range is involved; that is, an empty range is not considered to be either before or after any other range. + +The union and difference operators will fail if the resulting range would need to contain two disjoint sub-ranges, as such a range cannot be represented. + +The following functions are available for use with range types. + +### `lower(anyrange)` + +Return type: range's element type + +Lower bound of range. + +Example: + +``` +lower(numrange(1.1,2.2)) +1.1 +``` + +### `upper(anyrange)` + +Return type: range's element type + +Upper bound of range. + +Example: + +``` +upper(numrange(1.1,2.2)) +2.2 +``` + +### `isempty(anyrange)` + +Return type: `boolean` + +Is the range empty? + +Example: + +``` +isempty(numrange(1.1,2.2)) +false +``` + +### `lower_inc(anyrange)` + +Return type: `boolean` + +Is the lower bound inclusive? + +Example: + +``` +lower_inc(numrange(1.1,2.2)) +true +``` + +### `upper_inc(anyrange)` + +Return type: `boolean` + +Is the upper bound inclusive? + +Example: + +``` +upper_inc(numrange(1.1,2.2)) +false +``` + +### `lower_inf(anyrange)` + +Return type: `boolean` + +Is the lower bound infinite? + +Example: + +``` +lower_inf('(,)'::daterange) +true +``` + +### `upper_inf(anyrange)` + +Return type: `boolean` + +Is the upper bound infinite? + +Example: + +``` +upper_inf('(,)'::daterange) +true +``` + +### `range_merge(anyrange, anyrange)` + +Return type: `anyrange` + +The smallest range which includes both of the given ranges. + +Example: + +``` +range_merge('[1,2)'::int4range, '[3,4)'::int4range) +[1,4) +``` The `lower` and `upper` functions return null if the range is empty or the requested bound is infinite. The `lower_inc`, `upper_inc`, `lower_inf`, and `upper_inf` functions all return false for an empty range. diff --git a/docs/whpg/7x/ref_guide/misc/feature_summary.md b/docs/whpg/7x/ref_guide/misc/feature_summary.md index 90f1bc9..ba49f7c 100644 --- a/docs/whpg/7x/ref_guide/misc/feature_summary.md +++ b/docs/whpg/7x/ref_guide/misc/feature_summary.md @@ -155,15 +155,15 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `ALTER PROTOCOL` | YES | | | `ALTER PUBLICATION` | NO | | | `ALTER RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL. | -| `ALTER ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | +| `ALTER ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE `*queue_name*` \| none` | | `ALTER SCHEMA` | YES | | | `ALTER SEQUENCE` | YES | | | `ALTER SUBSCRIPTION` | NO | | | `ALTER SYSTEM` | **NO** | | -| `ALTER TABLE` | YES | **Unsupported Clauses / Options:**

`ENABLE/DISABLE TRIGGER`

**WarehousePG Database Clauses:**

`ADD \| DROP \| RENAME \| SPLIT \| EXCHANGE PARTITION \| SET SUBPARTITION TEMPLATE \| SET WITH (REORGANIZE=true \| false) \| SET DISTRIBUTED BY` | +| `ALTER TABLE` | YES | **Unsupported Clauses / Options:**

`ENABLE/DISABLE TRIGGER`

**WarehousePG Database Clauses:**

`ADD \| DROP \| RENAME \| SPLIT \| EXCHANGE PARTITION \| SET SUBPARTITION TEMPLATE \| SET WITH (REORGANIZE=true \| false) \| SET DISTRIBUTED BY` | | `ALTER TABLESPACE` | YES | | | `ALTER TRIGGER` | **NO** | | -| `ALTER TYPE` | YES | **WarehousePG Clauses:**

`SET DEFAULT ENCODING` | +| `ALTER TYPE` | YES | **WarehousePG Clauses:**

`SET DEFAULT ENCODING` | | `ALTER USER` | YES | An alias for [ALTER ROLE](../sql_commands/ALTER_ROLE.md) | | `ALTER VIEW` | YES | | | `ANALYZE` | YES | | @@ -174,8 +174,8 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `COMMENT` | YES | | | `COMMIT` | YES | | | `COMMIT PREPARED` | **NO** | | -| `COPY` | YES | **Modified Clauses:**

`ESCAPE [ AS ] '`*escape*`' \| 'OFF'`

**WarehousePG Clauses:**

`[LOG ERRORS] SEGMENT REJECT LIMIT `*count*` [ROWS\|PERCENT]` | -| `CREATE AGGREGATE` | YES | **Unsupported Clauses / Options:**

`[ , SORTOP = `*sort_operator*` ]`

**WarehousePG Clauses:**

`[ , COMBINEFUNC = `*combinefunc*` ]`

**Limitations:**

The functions used to implement the aggregate must be `IMMUTABLE` functions. | +| `COPY` | YES | **Modified Clauses:**

`ESCAPE [ AS ] '`*escape*`' \| 'OFF'`

**WarehousePG Clauses:**

`[LOG ERRORS] SEGMENT REJECT LIMIT `*count*` [ROWS\|PERCENT]` | +| `CREATE AGGREGATE` | YES | **Unsupported Clauses / Options:**

`[ , SORTOP = `*sort_operator*` ]`

**WarehousePG Clauses:**

`[ , COMBINEFUNC = `*combinefunc*` ]`

**Limitations:**

The functions used to implement the aggregate must be `IMMUTABLE` functions. | | `CREATE CAST` | YES | | | `CREATE CONSTRAINT TRIGGER` | **NO** | | | `CREATE CONVERSION` | YES | | @@ -184,31 +184,31 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `CREATE EVENT TRIGGER` | YES | | | `CREATE EXTENSION` | YES | Loads a new extension into WarehousePG - based on PostgreSQL 9.6. | | `CREATE EXTERNAL TABLE` | YES | WarehousePG parallel ETL feature - not in PostgreSQL 9.4. | -| `CREATE FUNCTION` | YES | **Limitations:**

Functions defined as `STABLE` or `VOLATILE` can be run in WarehousePG provided that they are run on the coordinator only. `STABLE` and `VOLATILE` functions cannot be used in statements that run at the segment level. | +| `CREATE FUNCTION` | YES | **Limitations:**

Functions defined as `STABLE` or `VOLATILE` can be run in WarehousePG provided that they are run on the coordinator only. `STABLE` and `VOLATILE` functions cannot be used in statements that run at the segment level. | | `CREATE GROUP` | YES | An alias for [CREATE ROLE](../sql_commands/CREATE_ROLE.md) | -| `CREATE INDEX` | YES | **WarehousePG Clauses:**

`USING bitmap` (bitmap indexes)

**Limitations:**

`UNIQUE` indexes are allowed only if they contain all of (or a superset of) the WarehousePG distribution key columns. On partitioned tables, a unique index is only supported within an individual partition - not across all partitions.

`CONCURRENTLY` keyword not supported in WarehousePG. | +| `CREATE INDEX` | YES | **WarehousePG Clauses:**

`USING bitmap` (bitmap indexes)

**Limitations:**

`UNIQUE` indexes are allowed only if they contain all of (or a superset of) the WarehousePG distribution key columns. On partitioned tables, a unique index is only supported within an individual partition - not across all partitions.

`CONCURRENTLY` keyword not supported in WarehousePG. | | `CREATE LANGUAGE` | YES | | | `CREATE MATERIALIZED VIEW` | YES | Based on PostgreSQL 9.4. | -| `CREATE OPERATOR` | YES | **Limitations:**

The function used to implement the operator must be an `IMMUTABLE` function. | +| `CREATE OPERATOR` | YES | **Limitations:**

The function used to implement the operator must be an `IMMUTABLE` function. | | `CREATE OPERATOR CLASS` | YES | | | `CREATE OPERATOR FAMILY` | YES | | | `CREATE PROTOCOL` | YES | | | `CREATE PUBLICATION` | NO | | | `CREATE RESOURCE QUEUE` | YES | WarehousePG resource management feature - not in PostgreSQL 9.4. | -| `CREATE ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE`*queue_name*` \| none` | +| `CREATE ROLE` | YES | **WarehousePG Clauses:**

`RESOURCE QUEUE`*queue_name*` \| none` | | `CREATE RULE` | YES | | | `CREATE SCHEMA` | YES | | -| `CREATE SEQUENCE` | YES | **Limitations:**

The `lastval()` and `currval()` functions are not supported.

The `setval()` function is only allowed in queries that do not operate on distributed data. | +| `CREATE SEQUENCE` | YES | **Limitations:**

The `lastval()` and `currval()` functions are not supported.

The `setval()` function is only allowed in queries that do not operate on distributed data. | | `CREATE SUBSCRIPTION` | NO | | -| `CREATE TABLE` | YES | **Unsupported Clauses / Options:**

`[GLOBAL \| LOCAL]`

`REFERENCES`

`FOREIGN KEY`

`[DEFERRABLE \| NOT DEFERRABLE] `

**Limited Clauses:**

`UNIQUE` or `PRIMARY KEY `constraints are only allowed on hash-distributed tables (`DISTRIBUTED BY`), and the constraint columns must be the same as or a superset of the distribution key columns of the table and must include all the distribution key columns of the partitioning key.

**WarehousePG Clauses:**

`DISTRIBUTED BY (column, [ ... ] ) \|`

`DISTRIBUTED RANDOMLY`

`PARTITION BY type (column [, ...]) ( partition_specification, [...] )`

`WITH (appendoptimized=true [,compresslevel=value,blocksize=value] )` | +| `CREATE TABLE` | YES | **Unsupported Clauses / Options:**

`[GLOBAL \| LOCAL]`

`REFERENCES`

`FOREIGN KEY`

`[DEFERRABLE \| NOT DEFERRABLE] `

**Limited Clauses:**

`UNIQUE` or `PRIMARY KEY `constraints are only allowed on hash-distributed tables (`DISTRIBUTED BY`), and the constraint columns must be the same as or a superset of the distribution key columns of the table and must include all the distribution key columns of the partitioning key.

**WarehousePG Clauses:**

`DISTRIBUTED BY (column, [ ... ] ) \|`

`DISTRIBUTED RANDOMLY`

`PARTITION BY type (column [, ...]) ( partition_specification, [...] )`

`WITH (appendoptimized=true [,compresslevel=value,blocksize=value] )` | | `CREATE TABLE AS` | YES | See [CREATE TABLE](../sql_commands/CREATE_TABLE.md) | -| `CREATE TABLESPACE` | **YES** | **WarehousePG Clauses:**

Specify host file system locations for specific segment instances.

`WITH (contentID_1='/path/to/dir1...)` | +| `CREATE TABLESPACE` | **YES** | **WarehousePG Clauses:**

Specify host file system locations for specific segment instances.

`WITH (contentID_1='/path/to/dir1...)` | | `CREATE TRIGGER` | **NO** | | -| `CREATE TYPE` | YES | **WarehousePG Clauses:**

`COMPRESSTYPE \| COMPRESSLEVEL \| BLOCKSIZE`

**Limitations:**

The functions used to implement a new base type must be `IMMUTABLE` functions. | +| `CREATE TYPE` | YES | **WarehousePG Clauses:**

`COMPRESSTYPE \| COMPRESSLEVEL \| BLOCKSIZE`

**Limitations:**

The functions used to implement a new base type must be `IMMUTABLE` functions. | | `CREATE USER` | YES | An alias for [CREATE ROLE](../sql_commands/CREATE_ROLE.md) | | `CREATE VIEW` | YES | | | `DEALLOCATE` | YES | | -| `DECLARE` | YES | **Unsupported Clauses / Options:**

`SCROLL`

`FOR UPDATE [ OF column [, ...] ]`

**Limitations:**

Cursors cannot be backward-scrolled. Forward scrolling is supported.

PL/pgSQL does not have support for updatable cursors. | +| `DECLARE` | YES | **Unsupported Clauses / Options:**

`SCROLL`

`FOR UPDATE [ OF column [, ...] ]`

**Limitations:**

Cursors cannot be backward-scrolled. Forward scrolling is supported.

PL/pgSQL does not have support for updatable cursors. | | `DELETE` | YES | | | `DISCARD` | YES | **Limitation:** `DISCARD ALL` is not supported. | | `DO` | YES | PostgreSQL 9.0 feature | @@ -245,7 +245,7 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `END` | YES | | | `EXECUTE` | YES | | | `EXPLAIN` | YES | | -| `FETCH` | YES | **Unsupported Clauses / Options:**

`LAST`

`PRIOR`

`BACKWARD`

`BACKWARD ALL`

**Limitations:**

Cannot fetch rows in a nonsequential fashion; backward scan is not supported. | +| `FETCH` | YES | **Unsupported Clauses / Options:**

`LAST`

`PRIOR`

`BACKWARD`

`BACKWARD ALL`

**Limitations:**

Cannot fetch rows in a nonsequential fashion; backward scan is not supported. | | `GRANT` | YES | | | `INSERT` | YES | | | `LATERAL` Join Type | **NO** | | @@ -267,17 +267,17 @@ WarehousePG is based on PostgreSQL 9.4. To support the distributed nature and ty | `ROLLBACK PREPARED` | **NO** | | | `ROLLBACK TO SAVEPOINT` | YES | | | `SAVEPOINT` | YES | | -| `SELECT` | YES | **Limitations:**

Limited use of `VOLATILE` and `STABLE` functions in `FROM` or `WHERE` clauses

Text search (`Tsearch2`) is not supported

**WarehousePG Clauses (OLAP):**

`[GROUP BY `*grouping_element*` [, ...]]`

`[WINDOW `*window_name*` AS (`*window_specification*`)]`

`[FILTER (WHERE `*condition*`)]` applied to an aggregate function in the `SELECT` list | +| `SELECT` | YES | **Limitations:**

Limited use of `VOLATILE` and `STABLE` functions in `FROM` or `WHERE` clauses

Text search (`Tsearch2`) is not supported

**WarehousePG Clauses (OLAP):**

`[GROUP BY `*grouping_element*` [, ...]]`

`[WINDOW `*window_name*` AS (`*window_specification*`)]`

`[FILTER (WHERE `*condition*`)]` applied to an aggregate function in the `SELECT` list | | `SELECT INTO` | YES | See [SELECT](../sql_commands/SELECT.md) | | `SET` | YES | | | `SET CONSTRAINTS` | **NO** | In PostgreSQL, this only applies to foreign key constraints, which are currently not enforced in WarehousePG. | | `SET ROLE` | YES | | | `SET SESSION AUTHORIZATION` | YES | Deprecated as of PostgreSQL 8.1 - see [SET ROLE](../sql_commands/SET_ROLE.md) | -| `SET TRANSACTION` | YES | **Limitations:**

`DEFERRABLE` clause has no effect. | +| `SET TRANSACTION` | YES | **Limitations:**

`DEFERRABLE` clause has no effect. | | `SHOW` | YES | | | `START TRANSACTION` | YES | | | `TRUNCATE` | YES | | | `UNLISTEN` | **YES** | | -| `UPDATE` | YES | **Limitations:**

`SET` not allowed for WarehousePG distribution key columns. | -| `VACUUM` | YES | **Limitations:**

`VACUUM FULL` is not recommended in WarehousePG. | +| `UPDATE` | YES | **Limitations:**

`SET` not allowed for WarehousePG distribution key columns. | +| `VACUUM` | YES | **Limitations:**

`VACUUM FULL` is not recommended in WarehousePG. | | `VALUES` | YES | | diff --git a/docs/whpg/7x/ref_guide/system_catalogs/catalog_ref-views.md b/docs/whpg/7x/ref_guide/system_catalogs/catalog_ref-views.md index 33164ae..f37f244 100644 --- a/docs/whpg/7x/ref_guide/system_catalogs/catalog_ref-views.md +++ b/docs/whpg/7x/ref_guide/system_catalogs/catalog_ref-views.md @@ -927,7 +927,7 @@ This view may be useful if a coordinator restart remains in a recovery state for | `phase` | text | Status of the recovery. The possible values are: 'initializing', 'recovering commited distributed transactions', 'gathering in-doubt transactions', 'aborting in-doubt transactions', 'gathering in-doubt orphaned transactions', and 'managing in-doubt orphaned transactions'. | | `recover_commited_dtx_total` | integer | Total number of committed transactions found to recover. | | `recover_commited_dtx_completed` | integer | Number of committed transactions that have been recovered. | -| in_doubt_tx_total\` | integer | Total number of in-doubt transaction found, used in startup and non-startup phases. | +| `in_doubt_tx_total` | integer | Total number of in-doubt transaction found, used in startup and non-startup phases. | | `in_doubt_tx_in_progress` | integer | Number of in-progress, in-doubt transactions. | | `in_doubt_tx_aborted` | integer | Number of aborted in-doubt transactions. |