Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions docs/whpg/6x/admin_guide/analytics/procedural_languages/pl_java.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,16 +118,6 @@ The commands that you use to install Java depend on the host system operating sy

For OpenJDK 8 the version is `1.8.0`, for OpenJDK 11 the version is `11`.

- RHEL 6/CentOS 6

- Java 8 - This `yum` command installs OpenJDK 8.

```
$ sudo yum install java-1.8.0-openjdk-devel
```

- Java 11 - Download the OpenJDK 11 tar file from [http://jdk.java.net/archive/](http://jdk.java.net/archive/) and install and configure the operating system to use Java 11.

1. This example `tar` command installs the OpenJDK 11 in `/usr/lib/jvm`.

```
Expand Down
4 changes: 0 additions & 4 deletions docs/whpg/6x/admin_guide/performance/wlmgmt/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,6 @@ WarehousePG provides features to help you prioritize and allocate resources to q

You can use resource management features to limit the number of concurrent queries, the amount of memory used to run a query, and the relative amount of CPU devoted to processing a query. WarehousePG provides two schemes to manage resources - Resource Queues and Resource Groups.

> **Important** Significant WarehousePG performance degradation has been observed when enabling resource group-based workload management on RedHat 6.x and CentOS 6.x. This issue is caused by a Linux cgroup kernel bug. This kernel bug has been fixed in CentOS 7.x and Red Hat 7.x/8.x systems.

If you use RedHat 6 and the performance with resource groups is acceptable for your use case, upgrade your kernel to version 2.6.32-696 or higher to benefit from other fixes to the cgroups implementation.

Either the resource queue or the resource group management scheme can be active in WarehousePG; both schemes cannot be active at the same time.

Resource queues are enabled by default when you install your WarehousePG cluster. While you can create and assign resource groups when resource queues are active, you must explicitly enable resource groups to start using that management scheme.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -287,10 +287,6 @@ Resource groups for roles track all WarehousePG memory allocated via the `palloc

## Configuring and Using Resource Groups

> **Important** Significant WarehousePG performance degradation has been observed when enabling resource group-based workload management on RedHat 6.x and CentOS 6.x systems. This issue is caused by a Linux cgroup kernel bug. This kernel bug has been fixed in CentOS 7.x and Red Hat 7.x/8.x systems.

If you use RedHat 6 and the performance with resource groups is acceptable for your use case, upgrade your kernel to version 2.6.32-696 or higher to benefit from other fixes to the cgroups implementation.

<a id="topic833"></a>

### Prerequisites
Expand Down Expand Up @@ -326,9 +322,9 @@ Complete the following tasks on each node in your WarehousePG cluster to set up

2. Create the required cgroup hierarchies on each WarehousePG node. Since the hierarchies are cleaned when the operating system rebooted, a service is applied to recreate them automatically on boot. Follow the below steps based on your operating system version.

#### Redhat/CentOS 6.x/7.x/8.x
#### Redhat/CentOS 7.x/8.x

These operating systems include the `libcgroup-tools` package (for Redhat/CentOS 7.x/8.x) or `libcgroup` (for Redhat/CentOS 6.x)
These operating systems include the `libcgroup-tools` package.

1. Locate the cgroups configuration file `/etc/cgconfig.conf`. You must be the superuser or have `sudo` access to edit this file:

Expand Down Expand Up @@ -365,35 +361,21 @@ These operating systems include the `libcgroup-tools` package (for Redhat/CentOS

3. Start the cgroups service on each WarehousePG node. You must be the superuser or have `sudo` access to run the command:

- Redhat/CentOS 7.x/8.x systems:

```
cgconfigparser -l /etc/cgconfig.conf
```
- Redhat/CentOS 6.x systems:

```
service cgconfig start
```

4. To automatically recreate WarehousePG required cgroup hierarchies and parameters when your system is restarted, configure your system to enable the Linux cgroup service daemon `cgconfig.service` (Redhat/CentOS 7.x/8.x) or `cgconfig` (Redhat/CentOS 6.x) at node start-up. To ensure the configuration is persistent after reboot, run the following commands as user root:

- Redhat/CentOS 7.x/8.x systems:
```
cgconfigparser -l /etc/cgconfig.conf
```

```
systemctl enable cgconfig.service
```
4. To automatically recreate WarehousePG required cgroup hierarchies and parameters when your system is restarted, configure your system to enable the Linux cgroup service daemon `cgconfig.service` at node start-up. To ensure the configuration is persistent after reboot, run the following commands as user root:

To start the service immediately (without having to reboot) enter:
```
systemctl enable cgconfig.service
```

```
systemctl start cgconfig.service
```
- Redhat/CentOS 6.x systems:
To start the service immediately (without having to reboot) enter:

```
chkconfig cgconfig on
```
```
systemctl start cgconfig.service
```

5. Identify the `cgroup` directory mount point for the node:

Expand Down
56 changes: 34 additions & 22 deletions docs/whpg/6x/install_guide/config_os.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ If you choose to deactivate SELinux:
SELINUX=disabled
```

::: info Note
After editing the config file, rerun `sestatus` and confirm `Mode from config file: disabled`. That field reads `/etc/selinux/config` directly and reflects your edit immediately. `SELinux status:` and `Current mode:` show the state the running kernel loaded at boot, and continue to show the previous state until you reboot.
:::

3. If the System Security Services Daemon (SSSD) is installed on your systems, edit the SSSD configuration file and set the `selinux_provider` parameter to `none` to prevent SELinux-related SSH authentication denials that could occur even with SELinux deactivated. As root, edit `/etc/sssd/sssd.conf` and add this parameter:

```
Expand All @@ -63,7 +67,7 @@ If you choose to enable SELinux in `Enforcing` mode, then WarehousePG processes

## Deactivate or Configure Firewall Software

You should also deactivate firewall software such as `iptables` (on systems such as RHEL 6.x and CentOS 6.x) or `firewalld` (on systems such as RHEL 7.x and CentOS 7.x and later). If firewall software is not deactivated, you must instead configure your software to allow required communication between WarehousePG hosts.
You should also deactivate firewall software such as `iptables` or `firewalld`. If firewall software is not deactivated, you must instead configure your software to allow required communication between WarehousePG hosts.

To deactivate `iptables`:

Expand Down Expand Up @@ -105,9 +109,13 @@ To deactivate `iptables`:

```
# systemctl stop firewalld.service
# systemctl deactivate firewalld.service
# systemctl disable firewalld.service
```

::: info Note
On some machine images, `firewalld` isn't installed. In that case, `systemctl stop firewalld.service` returns `Failed to stop firewalld.service: Unit firewalld.service not loaded`, and `systemctl disable firewalld.service` fails with its own `unit not found` message. Both errors are harmless. If `firewalld` isn't installed, it's already effectively deactivated, and no further action is needed.
:::

If you decide to enable `iptables` with WarehousePG for security purposes, see [Enabling iptables (Optional)](enable_iptables.md) for important considerations and example configurations.

See the documentation for the firewall or your operating system for additional information.
Expand All @@ -120,7 +128,7 @@ WarehousePG requires that certain Linux operating system (OS) parameters be set

In general, the following categories of system parameters need to be altered:

- **Shared Memory** - A WarehousePG instance will not work unless the shared memory segment for your kernel is properly sized. Most default OS installations have the shared memory values set too low for WarehousePG. On Linux systems, you must also deactivate the OOM (out of memory) killer. For information about WarehousePG shared memory requirements, see the WarehousePG server configuration parameter [shared_buffers](../ref_guide/config_params/guc-list.md) in the *WarehousePG Reference Guide*.
- **Shared Memory** - A WarehousePG instance will not work unless the shared memory segment for your kernel is properly sized. Most default OS installations have the shared memory values set too low for WarehousePG. On Linux systems, you must also deactivate the OOM (out of memory) killer. For information about WarehousePG shared memory requirements, see the WarehousePG server configuration parameter [shared_buffers](../ref_guide/config_params/guc-list.md#shared_buffers) in the *WarehousePG Reference Guide*.
- **Network** - On high-volume WarehousePG clusters, certain network-related tuning parameters must be set to optimize network connections made by the WarehousePG interconnect.
- **User Limits** - User limits control the resources available to processes started by a user's shell. WarehousePG requires a higher limit on the allowed number of file descriptors that a single process can have open. The default settings may cause some WarehousePG queries to fail because they will run out of file descriptors needed to process the query.

Expand Down Expand Up @@ -153,6 +161,10 @@ The `sysctl.conf` parameters listed in this topic are for performance, optimizat

Set the parameters in the `/etc/sysctl.conf` file and reload with `sysctl -p`:

:::: warning Caution
The `kernel.shmall`, `kernel.shmmax`, and `vm.overcommit_ratio` values in this example are calculated for a specific host with 1583 GB of memory. Don't copy these values as-is to a host with a different amount of memory. A `vm.overcommit_ratio` that's too high for the host's actual memory can produce the same `could not map anonymous shared memory` failure covered in [Troubleshooting Initialization Problems](init_whpg.md#topic7). Calculate `kernel.shmall` and `kernel.shmmax` for your own host as described in [Shared Memory Pages](#shared-memory-pages) below, and `vm.overcommit_ratio` as described in [Segment Host Memory](#segment-host-memory) below, then substitute your calculated values in the file.
::::

```

# kernel.shmall = _PHYS_PAGES / 2 # See Shared Memory Pages
Expand Down Expand Up @@ -206,8 +218,8 @@ kernel.shmmax = ( _PHYS_PAGES / 2) * PAGE_SIZE
To calculate the values for `kernel.shmall` and `kernel.shmmax`, run the following commands using the `getconf` command, which returns the value of an operating system variable.

```
$ echo $(expr $(getconf _PHYS_PAGES) / 2)
$ echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE))
echo $(expr $(getconf _PHYS_PAGES) / 2)
echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE))
```

As best practice, we recommend you set the following values in the `/etc/sysctl.conf` file using calculated values. For example, a host system has 1583 GB of memory installed and returns these values: \_PHYS_PAGES = 395903676 and PAGE_SIZE = 4096. These would be the `kernel.shmall` and `kernel.shmmax` values:
Expand Down Expand Up @@ -495,22 +507,15 @@ You may control the value of the MTU at various locations:

These settings are connected, in that they should always be either the same, or close to the same, value, or otherwise in the order of WarehousePG &lt; Operating System &lt; Virtual or Physical switch for MTU size.

If the interconnect network supports jumbo frames, set the MTU to 9000 and keep `gp_max_packet_size` at its default of 8192. If the network doesn't support jumbo frames and the MTU is 1500, reduce `gp_max_packet_size` to 1472 to avoid fragmented packets.

9000 is a common supported setting for switches, and is the recommended OS and rack switch MTU setting for your WarehousePG hosts.

<a id="huge_pages"></a>

### Transparent Huge Pages (THP)

Deactivate Transparent Huge Pages (THP) as it degrades WarehousePG performance. RHEL 6.0 or higher enables THP by default. One way to deactivate THP on RHEL 6.x is by adding the parameter `transparent_hugepage=never` to the kernel command in the file `/boot/grub/grub.conf`, the GRUB boot loader configuration file. This is an example kernel command from a `grub.conf` file. The command is on multiple lines for readability:

```
kernel /vmlinuz-2.6.18-274.3.1.el5 ro root=LABEL=/
elevator=deadline crashkernel=128M@16M quiet console=tty1
console=ttyS1,115200 panic=30 transparent_hugepage=never
initrd /initrd-2.6.18-274.3.1.el5.img
```

On systems that use `grub2` such as RHEL 7.x or CentOS 7.x and later, use the system utility `grubby`. This command adds the parameter when run as root.
Deactivate Transparent Huge Pages (THP) as it degrades WarehousePG performance. On systems that use `grub2`, such as RHEL 7.x or CentOS 7.x and later, use the system utility `grubby`. This command adds the parameter when run as root.

```
# grubby --update-kernel=ALL --args="transparent_hugepage=never"
Expand All @@ -521,7 +526,8 @@ After adding the parameter, reboot the system.
This cat command checks the state of THP. The output indicates that THP is deactivated.

```
$ cat /sys/kernel/mm/*transparent_hugepage/enabled
cat /sys/kernel/mm/*transparent_hugepage/enabled
__OUTPUT__
always [never]
```

Expand Down Expand Up @@ -570,10 +576,10 @@ Max Startups 10:30:200
MaxSessions 200
```

Restart the SSH daemon after you update `MaxStartups` and `MaxSessions`. For example, on a CentOS 6 system, run the following command as the `root` user:
Restart the SSH daemon after you update `MaxStartups` and `MaxSessions`. For example, run the following command as the `root` user:

```
# service sshd restart
# systemctl restart sshd
```

For detailed information about SSH configuration options, refer to the SSH documentation for your Linux distribution.
Expand Down Expand Up @@ -686,7 +692,7 @@ The `gpadmin` user must have permission to access the services and directories r

The `gpadmin` user on each WarehousePG host must have an SSH key pair installed and be able to SSH from any host in the cluster to any other host in the cluster without entering a password or passphrase (called "passwordless SSH"). If you enable passwordless SSH from the coordinator host to every other host in the cluster ("1-*n* passwordless SSH"), you can use the WarehousePG `gpssh-exkeys` command-line utility later to enable passwordless SSH from every host to every other host ("*n*-*n* passwordless SSH").

You can optionally give the `gpadmin` user sudo privilege, so that you can easily administer all hosts in the WarehousePG cluster as `gpadmin` using the `sudo`, `ssh/scp`, and `gpssh/gpscp` commands.
You can optionally give the `gpadmin` user sudo privilege, so that you can administer all hosts in the WarehousePG cluster as `gpadmin` using the `sudo`, `ssh/scp`, and `gpssh/gpscp` commands.

The following steps show how to set up the `gpadmin` user on a host, set a password, create an SSH key pair, and (optionally) enable sudo capability. These steps must be performed as root on every WarehousePG cluster host. (For a large WarehousePG cluster you will want to automate these steps using your system provisioning tools.)

Expand All @@ -702,19 +708,21 @@ The following steps show how to set up the `gpadmin` user on a host, set a passw
# groupadd gpadmin
# useradd gpadmin -r -m -g gpadmin
# passwd gpadmin
__OUTPUT__
New password: <changeme>
Retype new password: <changeme>
```

> **Note** You must have root permission to create the `gpadmin` group and user.

> **Note** Make sure the `gpadmin` user has the same user id (uid) and group id (gid) numbers on each host to prevent problems with scripts or services that use them for identity or permissions. For example, backing up WarehousePGs to some networked filesy stems or storage appliances could fail if the `gpadmin` user has different uid or gid numbers on different segment hosts. When you create the `gpadmin` group and user, you can use the `groupadd -g` option to specify a gid number and the `useradd -u` option to specify the uid number. Use the command `id gpadmin` to see the uid and gid for the `gpadmin` user on the current host.
> **Note** Make sure the `gpadmin` user has the same user id (uid) and group id (gid) numbers on each host to prevent problems with scripts or services that use them for identity or permissions. For example, backing up WarehousePGs to some networked filesystems or storage appliances could fail if the `gpadmin` user has different uid or gid numbers on different segment hosts. When you create the `gpadmin` group and user, you can use the `groupadd -g` option to specify a gid number and the `useradd -u` option to specify the uid number. Use the command `id gpadmin` to see the uid and gid for the `gpadmin` user on the current host.

2. Switch to the `gpadmin` user and generate an SSH key pair for the `gpadmin` user.

```
$ su gpadmin
$ ssh-keygen -t rsa -b 4096
su gpadmin
ssh-keygen -t rsa -b 4096
__OUTPUT__
Generating public/private rsa key pair.
Comment thread
mpfuster marked this conversation as resolved.
Enter file in which to save the key (/home/gpadmin/.ssh/id_rsa):
Created directory '/home/gpadmin/.ssh'.
Expand All @@ -727,6 +735,10 @@ The following steps show how to set up the `gpadmin` user on a host, set a passw

3. Grant sudo access to the `gpadmin` user.

::: info Note
This step is optional, but it's helpful, and it simplifies some procedures elsewhere in this guide, such as [Creating the Data Storage Areas](create_data_dirs.md), that run `sudo` commands on remote hosts through `gpssh`. If `gpadmin` doesn't have sudo access, you can't use `gpssh` or `gpscp` to run those commands. Because `gpssh` provides no terminal for a password prompt, the grant must let `gpadmin` run `sudo` without a password (`NOPASSWD`), as shown below, for those procedures to work.
:::

On Red Hat or CentOS, run `visudo` and uncomment the `%wheel` group entry.

```
Expand Down
Loading