Updating to Satellite 6.16 and RHEL9
Satellite 6.16 has been released, and it's a good time to get my lab systems updated to the latest version :-)
The process is fairly similar to my previous post for Upgrading Satellite 6.11 from RHEL7 to RHEL8
Anyhow, I found a couple of snags related to leapp (limit of file descriptors) and a Jira issue that is still to be published that will make you want to delay your RHEL9 update for a couple of weeks.
The whole process is documented in the Satellite 6.16 - Upgrading Red Hat Satellite chapter.
Satellite 6.15 to 6.16 Upgrade (RHEL8)
This process is the same as previous versions, with the one caveat that now
satellite-maintain supports three different verbs:
-
satellite-maintain self-upgradeto update the satellite-maintain tools. -
satellite-maintain update run -yto update RHEL+Satellite within the same Satellite version (6.x). -
satellite-maintain upgrade run -yto update RHEL+Satellite to the next Satellite version (6.x+1).
Now the update process itself:
root@sat.p1.lab ~ # satellite-maintain self-upgrade
Running Enables the specified version's maintenance repository and,
updates the satellite-maintain packages
================================================================================
Update package(s) rubygem-foreman_maintain, satellite-maintain: [OK]
--------------------------------------------------------------------------------
root@sat.p1.lab ~ # time satellite-maintain upgrade run -y
Checking for new version of satellite-maintain...
Security: kernel-core-4.18.0-553.27.1.el8_10.x86_64 is an installed security update
Security: kernel-core-4.18.0-553.8.1.el8_10.x86_64 is the currently running version
Nothing to update, can't find new version of satellite-maintain.
Running preparation steps required to run the next scenarios
================================================================================
Check whether all services are running: [OK]
--------------------------------------------------------------------------------
Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled:
- Checking repositories enabled on the system [OK]
--------------------------------------------------------------------------------
Running Checks before upgrading
================================================================================
Check number of fact names in database: [OK]
--------------------------------------------------------------------------------
Clean old Kernel and initramfs files from tftp-boot: [OK]
--------------------------------------------------------------------------------
Check for verifying syntax for ISP DHCP configurations: [OK]
--------------------------------------------------------------------------------
Check whether all services are running: [OK]
--------------------------------------------------------------------------------
Check whether all services are running using the ping call: [OK]
--------------------------------------------------------------------------------
Check for paused tasks: [OK]
--------------------------------------------------------------------------------
Check to verify no empty CA cert requests exist: [OK]
--------------------------------------------------------------------------------
Check whether system is self-registered or not: [OK]
--------------------------------------------------------------------------------
Check to verify if any hotfix installed on system:
| Checking for presence of hotfix(es). It may take some time to verify.
[OK]
--------------------------------------------------------------------------------
Check if TMOUT environment variable is set: [OK]
--------------------------------------------------------------------------------
Check if any upstream repositories are enabled on system:
- Checking for presence of upstream repositories [OK]
--------------------------------------------------------------------------------
Check to make sure root(/) partition has enough space: [OK]
--------------------------------------------------------------------------------
Check to make sure /var/lib/candlepin has enough space: [OK]
--------------------------------------------------------------------------------
Make sure server is running on required database version: [OK]
--------------------------------------------------------------------------------
Check for roles that have filters with multiple resources attached: [OK]
--------------------------------------------------------------------------------
Check for duplicate permissions from database: [OK]
--------------------------------------------------------------------------------
Check if system requirements match current tuning profile: [OK]
--------------------------------------------------------------------------------
Check whether reports have correct associations: [OK]
--------------------------------------------------------------------------------
Check for running tasks: [OK]
--------------------------------------------------------------------------------
Check for old tasks in paused/stopped state: [OK]
--------------------------------------------------------------------------------
Check for pending tasks which are safe to delete: [OK]
--------------------------------------------------------------------------------
Check for tasks in planning state: [OK]
--------------------------------------------------------------------------------
Check for running pulpcore tasks: [OK]
--------------------------------------------------------------------------------
Check if system has any non Red Hat RPMs installed (e.g.: Fedora): [OK]
--------------------------------------------------------------------------------
Check to validate dnf configuration before upgrade: [OK]
--------------------------------------------------------------------------------
Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled:
| Checking repositories enabled on the system [OK]
--------------------------------------------------------------------------------
Check if ipv6.disable=1 is set at kernel level: [OK]
--------------------------------------------------------------------------------
Check to make sure PostgreSQL 13 work directory has enough space for upgrade:
[OK]
--------------------------------------------------------------------------------
Check if any organizations are using entitlement mode:
| Checking organization content access modes [OK]
--------------------------------------------------------------------------------
Validate availability of repositories:
- Validating availability of repositories for 6.16 [OK]
--------------------------------------------------------------------------------
The pre-upgrade checks indicate that the system is ready for upgrade.
It's recommended to perform a backup at this stage.
Confirm to continue with the modification part of the upgrade (assuming yes)
Running preparation steps required to run the next scenarios
================================================================================
Check whether all services are running: [OK]
--------------------------------------------------------------------------------
Running Procedures before migrating
================================================================================
disable active sync plans:
| Total 0 sync plans are now disabled. [OK]
--------------------------------------------------------------------------------
Add maintenance_mode tables/chain to nftables/iptables: [OK]
--------------------------------------------------------------------------------
Stop cron service:
Stopping the following service(s):
crond
\ All services stopped [OK]
--------------------------------------------------------------------------------
Running Migration scripts
================================================================================
Setup repositories:
/ Configuring repositories for 6.16 [OK]
--------------------------------------------------------------------------------
Switch the given stream modules: [OK]
--------------------------------------------------------------------------------
Enable the given stream modules: [OK]
--------------------------------------------------------------------------------
Download package(s) : [OK]
--------------------------------------------------------------------------------
Stop applicable services:
Stopping the following service(s):
redis, mosquitto, postgresql, pulpcore-api, pulpcore-content, pulpcore-api.socket, pulpcore-content.socket, pulpcore-worker@1.service, pulpcore-worker@2.service, tomcat, dynflow-sidekiq@orchestrator, foreman, httpd, foreman.socket, dynflow-sidekiq@worker-1, dynflow-sidekiq@worker-hosts-queue-1, foreman-proxy, foreman-cockpit
| All services stopped [OK]
--------------------------------------------------------------------------------
Update package(s) : [OK]
--------------------------------------------------------------------------------
Running satellite-installer : [OK]
--------------------------------------------------------------------------------
Execute upgrade:run rake task: [OK]
--------------------------------------------------------------------------------
Running preparation steps required to run the next scenarios
================================================================================
Check whether all services are running: [OK]
--------------------------------------------------------------------------------
Running Procedures after migrating
================================================================================
Refresh detected features: [OK]
--------------------------------------------------------------------------------
Start applicable services:
Starting the following service(s):
redis, mosquitto, postgresql, pulpcore-api, pulpcore-content, pulpcore-worker@1.service, pulpcore-worker@2.service, tomcat, dynflow-sidekiq@orchestrator, foreman, httpd, dynflow-sidekiq@worker-1, dynflow-sidekiq@worker-hosts-queue-1, foreman-proxy, foreman-cockpit
| All services started [OK]
--------------------------------------------------------------------------------
Start cron service:
Starting the following service(s):
crond
- All services started [OK]
--------------------------------------------------------------------------------
re-enable sync plans:
\ Total 0 sync plans are now enabled. [OK]
--------------------------------------------------------------------------------
Remove maintenance mode table/chain from nftables/iptables: [OK]
--------------------------------------------------------------------------------
Running preparation steps required to run the next scenarios
================================================================================
Check whether all services are running: [OK]
--------------------------------------------------------------------------------
Running Checks after upgrading
================================================================================
Check number of fact names in database: [OK]
--------------------------------------------------------------------------------
Clean old Kernel and initramfs files from tftp-boot: [OK]
--------------------------------------------------------------------------------
Check for verifying syntax for ISP DHCP configurations: [OK]
--------------------------------------------------------------------------------
Check whether all services are running: [OK]
--------------------------------------------------------------------------------
Check whether all services are running using the ping call: [OK]
--------------------------------------------------------------------------------
Check for paused tasks: [OK]
--------------------------------------------------------------------------------
Check to verify no empty CA cert requests exist: [OK]
--------------------------------------------------------------------------------
Check whether system is self-registered or not: [OK]
--------------------------------------------------------------------------------
Check if system needs reboot: [WARNING]
Updating Subscription Management repositories.
Core libraries or services have been updated since boot-up:
* glibc
* kernel
* linux-firmware
* microcode_ctl
* systemd
Reboot is required to fully utilize these updates.
More information: https://access.redhat.com/solutions/27943
--------------------------------------------------------------------------------
Initialize and expose container image metadata in the pulpcore db:
\ Adding image metadata to pulp. You can continue using the system normally while the task runs in the background.
[OK]
--------------------------------------------------------------------------------
Import container manifest metadata:
\ Adding image metadata. You can continue using the system normally while the task runs in the background.kground.
[OK]
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Upgrade finished.
real 16m15.508s
user 6m37.830s
sys 0m59.038s
Note that the satellite-maintain upgrade process waits until the data migration
tasks mentioned above are done. Once it finishes, it's safe to start/stop the
Satellite services or reboot the system (eg, to prepare for Leapp).
Leapp upgrade to RHEL9
This process is documented in the Satellite 6.16 - Upgrading Red Hat Enterprise Linux on Satellite or Capsule chapter.
root@sat.p1.lab ~ # dnf install --disableplugin=foreman-protector install leapp leapp-upgrade-el8toel9
The following answerfile can be created once you ensure VDO is not in use in your RHEL8 system:
root@sat.p1.lab ~ # cat /var/log/leapp/answerfile
[check_vdo]
# Title: None
# Reason: Confirmation
# ============================= check_vdo.confirm =============================
# Label: Are all VDO devices, if any, successfully converted to LVM management?
# Description: Enter True if no VDO devices are present on the system or all VDO devices on the system have been successfully converted to LVM management. Entering True will circumvent check of failures and undetermined devices. Recognized VDO devices that have not been converted to LVM management can still block the upgrade despite the answer.All VDO devices must be converted to LVM management before upgrading.
# Reason: To maximize safety all block devices on a system that meet the criteria as possible VDO devices are checked to verify that, if VDOs, they have been converted to LVM management. If the devices are not converted and the upgrade proceeds the data on unconverted VDO devices will be inaccessible. In order to perform checking the 'vdo' package must be installed. If the 'vdo' package is not installed and there are any doubts the 'vdo' package should be installed and the upgrade process re-run to check for unconverted VDO devices. If the check of any device fails for any reason an upgrade inhibiting report is generated. This may be problematic if devices are dynamically removed from the system subsequent to having been identified during device discovery. If it is certain that all VDO devices have been successfully converted to LVM management this dialog may be answered in the affirmative which will circumvent block device checking.
# Type: bool
# Default: None
# Available choices: True/False
confirm = True
root@sat.p1.lab ~ # leapp preupgrade
root@sat.p1.lab ~ # time leapp upgrade && reboot
[...]
Again, at this point leapp will perform the OS upgrade, perform a Selinux relabel of all files and re-run the Satellite installer once in RHEL9.
Note that at this time the following issue missing PES event to remove shim-ia32 when upgrading to RHEL9 is still pending to be released. It should be fixed in the next few days.
# cat /etc/redhat-release
Red Hat Enterprise Linux release 9.4 (Plow)
# rpm -qi satellite
Name : satellite
Version : 6.16.0
Release : 2.el9sat
Architecture: noarch
Install Date: Tue 05 Nov 2024 05:31:57 PM EST
All in all, a very painless upgrade that removes all requirements to have RHEL8 in your environment. Go RHEL9!
Happy hacking!