Re-configuring workers in Satellite 6.13 for performance tunning

While there is a very complete Satellite 6.13 Performance tuning guide, I always struggle to find these parameters whenever I happen to change the CPU and RAM resources when Satellite is configured as a Virtual Machine.

Usually I care about tunning two things:

  • a) The number of Puma workers (helps with Satellite WebUI responsiveness and with the number of dynflow tasks it can handle)

  • b) The number of pulp workers so I can synchronize more repositories in parallel.

To tune this, in a 16 vCPU machine, I would use something like:

satellite-installer \
 --foreman-foreman-service-puma-workers=8 \
 --foreman-foreman-service-puma-threads-min=16 \
 --foreman-foreman-service-puma-threads-max=16 \
 --foreman-proxy-content-pulpcore-worker-count=8

Happy hacking!

Checking SSL and TLS configuration with sslyze

Just a quick tip on checking a server's SSL/TLS configuration without too much issues.

sslyze is a python package that can be used to check the accepted ciphers of a certain service, eg:

 pip install sslyze

 python -m sslyze server.example.com

CHECKING CONNECTIVITY TO SERVER(S)
----------------------------------

server.example.com:443            => 192.168.1.208   WARNING: Server requested optional client authentication


SCAN RESULTS FOR server.example.com:443 - 192.168.1.208
-------------------------------------------------

* Certificates Information:
    Hostname sent for SNI:             server.example.com
    Number of certificates detected:   1


    Certificate #0 ( _RSAPublicKey )
    SHA1 Fingerprint:                  2e57a27485b980d25ea0d8d642ab31d5b6a64b6e
    Common Name:                       server.example.com
    Issuer:                            server.example.com
    Serial Number:                     217661717633682589085577779257221678089539599645
    Not Before:                        2023-02-15
    Not After:                         2038-01-18
    Public Key Algorithm:              _RSAPublicKey
    Signature Algorithm:               sha256
    Key Size:                          4096
    Exponent:                          65537
    SubjAltName - DNS Names:           ['server.example.com']

    Certificate #0 - Trust
    Hostname Validation:               OK - Certificate matches server hostname
    Android CA Store (13.0.0_r9):      FAILED - Certificate is NOT Trusted: self-signed certificate in certificate chain
    Apple CA Store (iOS 16, iPadOS 16, macOS 13, tvOS 16, and watchOS 9):FAILED - Certificate is NOT Trusted: self-signed certificate in certificate chain
    Java CA Store (jdk-13.0.2):        FAILED - Certificate is NOT Trusted: self-signed certificate in certificate chain
    Mozilla CA Store (2022-12-11):     FAILED - Certificate is NOT Trusted: self-signed certificate in certificate chain
    Windows CA Store (2023-02-19):     FAILED - Certificate is NOT Trusted: self-signed certificate in certificate chain
    Symantec 2018 Deprecation:         ERROR - Could not build verified chain (certificate untrusted?)
    Received Chain:                    server.example.com --> server.example.com
    Verified Chain:                    ERROR - Could not build verified chain (certificate untrusted?)
    Received Chain Contains Anchor:    ERROR - Could not build verified chain (certificate untrusted?)
    Received Chain Order:              OK - Order is valid
    Verified Chain contains SHA1:      ERROR - Could not build verified chain (certificate untrusted?)

    Certificate #0 - Extensions
    OCSP Must-Staple:                  NOT SUPPORTED - Extension not found
    Certificate Transparency:          NOT SUPPORTED - Extension not found

    Certificate #0 - OCSP Stapling
                                        NOT SUPPORTED - Server did not send back an OCSP response

* SSL 2.0 Cipher Suites:
    Attempted to connect using 7 cipher suites; the server rejected all cipher suites.

* SSL 3.0 Cipher Suites:
    Attempted to connect using 80 cipher suites; the server rejected all cipher suites.

* TLS 1.0 Cipher Suites:
    Attempted to connect using 80 cipher suites; the server rejected all cipher suites.

* TLS 1.1 Cipher Suites:
    Attempted to connect using 80 cipher suites; the server rejected all cipher suites.

* TLS 1.2 Cipher Suites:
    Attempted to connect using 156 cipher suites.

    The server accepted the following 23 cipher suites:
        TLS_RSA_WITH_AES_256_GCM_SHA384                   256                      
        TLS_RSA_WITH_AES_256_CCM                          256                      
        TLS_RSA_WITH_AES_256_CBC_SHA256                   256                      
        TLS_RSA_WITH_AES_256_CBC_SHA                      256                      
        TLS_RSA_WITH_AES_128_GCM_SHA256                   128                      
        TLS_RSA_WITH_AES_128_CCM                          128                      
        TLS_RSA_WITH_AES_128_CBC_SHA256                   128                      
        TLS_RSA_WITH_AES_128_CBC_SHA                      128                      
        TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256       256       ECDH: X25519 (253 bits)
        TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384             256       ECDH: prime256v1 (256 bits)
        TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA                256       ECDH: prime256v1 (256 bits)
        TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256             128       ECDH: prime256v1 (256 bits)
        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256             128       ECDH: prime256v1 (256 bits)
        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA                128       ECDH: prime256v1 (256 bits)
        TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256         256       DH (4096 bits) 
        TLS_DHE_RSA_WITH_AES_256_GCM_SHA384               256       DH (4096 bits) 
        TLS_DHE_RSA_WITH_AES_256_CCM                      256       DH (4096 bits) 
        TLS_DHE_RSA_WITH_AES_256_CBC_SHA256               256       DH (4096 bits) 
        TLS_DHE_RSA_WITH_AES_256_CBC_SHA                  256       DH (4096 bits) 
        TLS_DHE_RSA_WITH_AES_128_GCM_SHA256               128       DH (4096 bits) 
        TLS_DHE_RSA_WITH_AES_128_CCM                      128       DH (4096 bits) 
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA256               128       DH (4096 bits) 
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA                  128       DH (4096 bits)

    The group of cipher suites supported by the server has the following properties:
    Forward Secrecy                    OK - Supported
    Legacy RC4 Algorithm               OK - Not Supported


* TLS 1.3 Cipher Suites:
    Attempted to connect using 5 cipher suites.

    The server accepted the following 4 cipher suites:
        TLS_CHACHA20_POLY1305_SHA256                      256       ECDH: X25519 (253 bits)
        TLS_AES_256_GCM_SHA384                            256       ECDH: X25519 (253 bits)
        TLS_AES_128_GCM_SHA256                            128       ECDH: X25519 (253 bits)
        TLS_AES_128_CCM_SHA256                            128       ECDH: X25519 (253 bits)


* Deflate Compression:
                                        OK - Compression disabled

* OpenSSL CCS Injection:
                                        OK - Not vulnerable to OpenSSL CCS injection

* OpenSSL Heartbleed:
                                        OK - Not vulnerable to Heartbleed

* Client certificated required for --robot: use --cert and --key to provide one.

* Session Renegotiation:
    Client Renegotiation DoS Attack:   OK - Not vulnerable
    Secure Renegotiation:              OK - Supported

* Elliptic Curve Key Exchange:
    Supported curves:                  X25519, X448, prime256v1, secp384r1, secp521r1
    Rejected curves:                   prime192v1, secp160k1, secp160r1, secp160r2, secp192k1, secp224k1, secp224r1, secp256k1, sect163k1, sect163r1, sect163r2, sect193r1, sect193r2, sect233k1, sect233r1, sect239k1, sect283k1, sect283r1, sect409k1, sect409r1, sect571k1, sect571r1

SCANS COMPLETED IN 6.953993 S
-----------------------------

COMPLIANCE AGAINST MOZILLA TLS CONFIGURATION
--------------------------------------------

    Checking results against Mozilla's "MozillaTlsConfigurationEnum.INTERMEDIATE" configuration. See https://ssl-config.mozilla.org/ for more details.

    server.example.com:443: ERROR - Scan did not run successfully; review the scan logs above.

Happy hacking!

Introducing satellite-conf-report

tl;dr: I wrote a quick shell script that gathers your Satellite/Foreman configuration and generates a Markdown file with it. It uses hammer commands underneath, and can help you in the following scenarios:

  • Reporting the basic configuration of a Satellite system.
  • Help with configuration drift - you can compare reports from different dates.
  • Quickly share the configuration with others.
  • Help perform configuration analysis / health checks.

The satellite-conf-report script is available int he following Github repo:

https://github.com/frangdlt/satellite-conf-report

Happy hacking!

Installing and running shrewsoft VPN client in Fedora 38

Shrewsoft created a VPN client and last build was done in 2013. We can still run such an ancient software in Fedora 38 with this couple of tricks.

Installation

First you need to enable the COPR repository for legacy OpenSSL 1.10

dnf copr enable dioni21/compat-openssl10

After that, you can manually install the shrewsoft package with:

yum localinstall https://download.copr.fedorainfracloud.org/results/pessoft/ike/fedora-rawhide-x86_64/00780930-ike/ike-2.2.1-13.fc29.x86_64.rpm

Running Shrewsoft VPN client

First you need to ensure that the IKE daemon is runing. You can launch it under your regular user:

/usr/sbin/iked &

Then launch the VPN client GUI with:

/usr/bin/ikec

Finally import your VPN profile and enjoy :-)

Happy hacking!

Overriding the system hostname when registering into Red Hat Satellite

Sometimes you need to ensure that a system registered into Satellite has a specific hostname, because the current one is unacceptable for a number of reasons (it might collide with another one, etc).

The Satellite documentations points to the fact that you can override the name by configuring a setting in the subscription-manager facts, however this does not seem to work in newer (6.12+) versions of Red Hat Satellite.

In the latest testing I did, this way ensures the system is registered with the requested hostname in Satellite.

  • Ensure the old host and content hosts are deleted from Satellite (if required).
  • Add the requested hostname in /etc/hosts of the system, bound to one of its IPs. It does not matter if it's not the first entry associated to the IP.
  • Ensure that /etc/rhsm/facts/katello.facts has this settings:

    {
    "network.fqdn":"new-name.example.com",
    "network.hostname":"new-name.example.com",
    "network.hostname-override":"new-name.example.com"
    }
    
  • Register the system again with subscription-manager, eg:

    subscription-manager register --org="ORGANIZATION" --activationkey="ACTIVATIONKEY" --force --name=new-name.example.com
    

In addition to this, it seems that simply setting the network.hostname-override parameter does not longer work by default. Satellite can be configured to obey this parameter, but the rest of the configuration is still required.

Some relevant documents:

  • https://access.redhat.com/solutions/3166211 - Registered content host appears with a different hostname in Red Hat Satellite WebUI.
  • https://access.redhat.com/solutions/3732221 - network.hostname-override is not overriding the hostname of server for registration

Happy hacking!

Setting VSCode as the default Gnome editor

Quick post. I had the need to change the default Gnome editor from gedit to VSCode. There isn't a straight-forward way to do this, so I found this quick way and wanted to make a note in the blog so I don't forget ;-) .

➜ xdg-mime  query default text/plain 
gedit.desktop

➜ xdg-mime default code.desktop text/plain

➜ xdg-mime  query default text/plain  
code.desktop

You can also check the list of existing applications by searching for *.desktop files installed in your system, eg:

➜ find /usr/share/applications  -name "*.desktop"

Happy hacking!

Registering old RHEL systems into new Satellite versions

Sometimes you find out you need to register really ancient Red Hat Enterprise Linux systems into a new Satellite, but this poses a number of challenges, namely around support, and having the right tools in the system (subscription-manager, curl and others) available so the system can be successfully managed by Red Hat Satellite.

Supported Satellite client operating systems

Red Hat maintains a list of supported operating systems per version on Satellite here:

https://access.redhat.com/solutions/5607011 - What are the supported operating systems for hosts of Red Hat Satellite 6 ?

The reality is a bit more complex, as Red Hat only tests new Satellite versions with actively supported RHEL operating systems. For RHEL6, this means that the only tested combinations are those using RHEL 6.10 using the ELS (Extended Lifecycle Support) add-on.

For ancient RHEL 7.0 and RHEL 7.1 versions, it's even more complicated because curl lacks some cryptographic cyphers, which make connecting to Satellite impossible.

The workaround is to upgrade the relevant system packages to the latest version PRIOR to attempting to register the system into Satellite 6.x .

So here are some notes on how to accomplish that:

Registering RHEL 6 older than 6.10 are not supported with Satellite 6.10+. Registration does not work.

  • It is required to update the subscription-manager, rpm, and other components to those provided by RHEL 6.10 prior to onboarding the system into Satellite 6.12 .
  • This process might render the "subscription-manager" inoperable. It is recommended to perform a full backup of the system, and implement access to the RHEL 6.10 DVD prior to attempting this upgrade.
  • The RHEL 6.10 content can be made available via the RHEL ISO, or hosted in a web server or NFS server.
  • Red Hat Engineering does not certify a full list of packages that need to be upgraded. However, when testing the following list of packages seems to be sufficient to allow a successful registration:
  • yum upgrade -y "yum*" "subscription-manager*" "rpm" python-requests.noarch python-rhsm.x86_64 python-six.noarch python-urlgrabber.noarch python-urllib3.noarch openssl
  • More packages might be required depending on the installed packages in the managed system that needs to be migrated.

RHEL 7.0 and RHEL 7.1 cannot be registered with Satellite.

  • In a similar way to RHEL 6, RHEL 7.0 and RHEL 7.1 cannot connect to a new satellite because of openssl ciphers and because of subscription-manager components. Registration errors might include curl: (35) Peer reports incompatible or unsupported protocol version.
  • The problem can be solved by upgrading the following packages prior to registration. Packages should come from RHEL 7.9.
  • yum upgrade -y openssl curl "yum*" "subscription-manager*" "rpm" python-six python-rhsm
  • Again, this process might render the "subscription-manager" inoperable. It is recommended to perform a full backup of the system, and implement access to the RHEL 7.9 DVD prior to attempting this upgrade.
  • The RHEL 7.9 content can be made available via the RHEL ISO, or hosted in a web server or NFS server.

Happy Satellite-ing!

Installing a Brother DCP-L2550DN printer and scanner in Fedora 37

Bother DCP-L2550DN printer

My trusty, 15 year old Brother HL-2030 printer died with a Laser sensor error, so it was due time for a replacement. Brother has been known to make good printers and I wanted to get a replacement ASAP.

My requirements for the printer were:

  • Laser B&W / monochrome. I only print a few pages per year, but I really need to work OK (as I'm usually very remote to the printer).
  • Reasonable low cost (under 200€).
  • Network attachable (Ethernet), wifi optional.
  • Fedora compatible.

So I found this Brother DCP-L2550DN on Amazon, and after a couple of days for shipping, I was ready to get it installed.

Installing the printer

In my case, I just had to remove the old printer, and replace in place with the new one. I connected the same USB-A (computer) to USB-B (printer) cable, and it was immediately recognized by lsusb.

#> lsusb | grep -i Brother
Bus 001 Device 005: ID 04f9:0423 Brother Industries, Ltd DCP-L2550DN series

The Brother support page instructs you to download the installer script, which in my case was:

  • linux-brprinter-installer-2.2.3-1
  • dcpl2550dnpdrv-4.0.0-1.i386.rpm
  • brscan-skey-0.3.1-2.x86_64.rpm
  • brscan4-0.4.11-1.x86_64.rpm

After running the script and asking for the specific model (DCP-L2550DN in my case, hyphen included), CUPS reported it as configured.

#> ./linux-brprinter-installer-2.2.3-1 
Input model name ->DCP-L2550DN

You are going to install following packages.
   dcpl2550dnpdrv-4.0.0-1.i386.rpm
   brscan4-0.4.11-1.x86_64.rpm
   brscan-skey-0.3.1-2.x86_64.rpm
OK? [y/N] ->y

rpm -ihv --nodeps --replacefiles --replacepkgs dcpl2550dnpdrv-4.0.0-1.i386.rpm
Verifying...                          ########################################
Preparing...                          ########################################
Updating / installing...
dcpl2550dnpdrv-4.0.0-1                ########################################
ln: failed to create symbolic link '/opt/brother/Printers/DCPL2550DN/lpd/rawtobr3': File exists
ln: failed to create symbolic link '/opt/brother/Printers/DCPL2550DN/lpd/brprintconflsr3': File exists
ln: failed to create symbolic link '/etc/opt/brother/Printers/DCPL2550DN/inf/brDCPL2550DNrc': File exists
ln: failed to create symbolic link '/usr/lib/cups/filter/brother_lpdwrapper_DCPL2550DN': File exists
ln: failed to create symbolic link '/usr/share/cups/model/brother-DCPL2550DN-cups-en.ppd': File exists
ln: failed to create symbolic link '/usr/share/ppd/brother/brother-DCPL2550DN-cups-en.ppd': File exists
lpadmin -p DCPL2550DN -E -v usb://Brother/DCP-L2550DN%20series?serial=E78284M2N111836 -P /usr/share/ppd/brother/brother-DCPL2550DN-cups-en.ppd
lpadmin: Printer drivers are deprecated and will stop working in a future version of CUPS.
ValueError: File context for /etc/opt/brother/Printers/DCPL2550DN/inf(/.*)? already defined
ValueError: File context for /opt/brother/Printers/DCPL2550DN/inf(/.*)? already defined
ValueError: File context for /opt/brother/Printers/DCPL2550DN/lpd(/.*)? already defined
ValueError: File context for /opt/brother/Printers/DCPL2550DN/cupswrapper(/.*)? already defined
#
semanage fcontext -a -t bin_t /opt/brother
ValueError: File context for /opt/brother already defined
restorecon -R /opt/brother
semanage fcontext -a -t cupsd_rw_etc_t /etc/opt/brother
ValueError: File context for /etc/opt/brother already defined
restorecon -R /etc/opt/brother
semanage fcontext -a -t cupsd_rw_etc_t /opt/brother/Printers/(.*/)?inf(/.*)?
ValueError: File context for /opt/brother/Printers/(.*/)?inf(/.*)? already defined
restorecon -R /opt/brother/Printers
semanage fcontext -a -t cupsd_rw_etc_t /etc/opt/brother/Printers/(.*/)?inf(/.*)?
ValueError: File context for /etc/opt/brother/Printers/(.*/)?inf(/.*)? already defined
restorecon -R /etc/opt/brother/Printers
semanage fcontext -a -t bin_t /opt/brother/Printers/(.*/)?lpd(/.*)?
ValueError: File context for /opt/brother/Printers/(.*/)?lpd(/.*)? already defined
restorecon -R /opt/brother/Printers
semanage fcontext -a -t bin_t /opt/brother/Printers/(.*/)?cupswrapper(/.*)?
ValueError: File context for /opt/brother/Printers/(.*/)?cupswrapper(/.*)? already defined
restorecon -R /opt/brother/Printers
restorecon -RFv /usr/lib/cups/filter
setsebool -P cups_execmem 1
Will you specify the Device URI? [Y/n] ->


0: beh
1: ipp
2: cups-brf:/
3: https
4: socket
5: serial:/dev/ttyS0?baud=115200
6: lpd
7: http
8: ipps
9: smb
10 (I): Specify IP address.
11 (A): Auto. (usb://dev/usblp0)

select the number of destination Device URI. ->11

lpadmin -p DCPL2550DN -v usb:///etc/usblp0 -E
Test Print? [y/N] ->

You are going to install following packages.
   brscan4-0.4.11-1.x86_64.rpm
rpm -ihv --nodeps --replacefiles --replacepkgs brscan4-0.4.11-1.x86_64.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:brscan4-0.4.11-1                 ################################# [100%]
This software is based in part on the work of the Independent JPEG Group.
You are going to install following packages.
   brscan-skey-0.3.1-2.x86_64.rpm
rpm -ihv --nodeps --replacefiles --replacepkgs brscan-skey-0.3.1-2.x86_64.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:brscan-skey-0.3.1-2              ################################# [100%]
ln: failed to create symbolic link '/etc/opt/brother/scanner/brscan-skey/brscan_mail.config': File exists
ln: failed to create symbolic link '/etc/opt/brother/scanner/brscan-skey/brscan-skey.config': File exists
ln: failed to create symbolic link '/etc/opt/brother/scanner/brscan-skey/brscan-snmp.cfg': File exists
ln: failed to create symbolic link '/etc/opt/brother/scanner/brscan-skey/scantofile.config': File exists
ln: failed to create symbolic link '/etc/opt/brother/scanner/brscan-skey/scantoimage.config': File exists
ln: failed to create symbolic link '/etc/opt/brother/scanner/brscan-skey/scantoocr.config': File exists
ln: failed to create symbolic link '/etc/opt/brother/scanner/brscan-skey/scantoemail.config': File exists
ln: failed to create symbolic link '/usr/bin/brscan-skey': File exists

I tried printing a test page but nothing would work because it was configured to use a device called /dev/usb/lp0 , or /dev/usblp0 . Neither of those existed on my system so I was troubleshooting for a while.

Some useful resources:

https://fedoraproject.org/wiki/How_to_debug_printing_problems

Configuring the right CUPS queue

In the end, I used good old system-config-printer graphical assistant, which automatically detected the printer, and configured it in cups in the following manner:

#> lpstat -p
printer Brother-DCP-L2550DN-series is idle.  enabled since Thu Mar 30 14:46:53 2023
Description:    Brother DCP-L2550DN series
Location:   mycomputer
Driver: Brother DCPL2550DN for CUPS (grayscale, duplex)
Connection: usb://Brother/DCP-L2550DN%20series?serial=XXXXXXXXXXXXX
Default Options:    Banners=none, none paper=iso_a4_210x297mm sides=one-sided

Configuring SANE

This was my first try with SANE in 20+ years using Linux, and I only had to install the driver as performed with the automatic installer.

Fedora ships simple-scan, which takes care of scanning pages both from the external feeder or the internal scanner. It works nicely and can output images to both JPGs or PDFs (for documents).

Happy hacking!

Notes about pens

And now for something completely different, let's talk about pens. I'm a big fan of stationery, however I strongly refrain from buying lots of fancy pens and writing pads because they usually remain unused and taking lots of space.

I wanted to use this post to document some of the pens that work well for me so whenever I need to buy replacements, I don't spend lots of time figuring which one was good for what -- or even if I should buy a specific model at all.

Being left handed means I don't use many of the fountain pens at all - I haven't found a comfortable way of writing with them, plus the additional mess they make because ink is never dry enough when I put my hand on it to continue writing.

Anyway, my list of currently useful pens:

  • Uni-Ball Vision RT Black UBN-178. Writes well on my Moleskine; ink doesn't bleed and get dry fast enough so I can write without delays. With a 0.8 size, the stroke creates a ticker line.

  • Uni-Ball UB-150. With a thinner stroke (0.5) usually writes well on any paper, including Moleskine.

  • Pilot Hi-Tecpoint V5 Grip. Again a 0.5 ball creates a thin stroke. The ink in this case usually faints a bit after drying out, and produces a lighter blue - which is to be taken into account if you really want your text to strike out.

  • Pilot G2 07. Now a gel pen with a thicker stroke. I really like this pen however is probably the messiest one in terms of wet ink, so I should really not use this pen anymore

  • Pilot Hi-Tecpoint V5. An oldie, but goodie. I haven't used one of these in a while but they provided sharp lines with no traces of running ink.

I'll keep updating this page if I find new pens that I like -- I'm well stoked for the time being though!

Upgrading Satellite 6.11 from RHEL7 to RHEL8

Satellite 6.11 is the only version of Satellite capable of running both in RHEL7 and RHEL8. While Satellite 6.11 was published a few months ago, I didn't immediately upgrade to RHEL8, and here's a quick recap on what is needed to update a Satellite system in place to the next version of RHEL.

Preparations

You can review the official upgrade documentation in the Upgrading Satellite or Capsule to Red Hat Enterprise Linux 8 In-Place Using Leapp chapter of the documentation.

Prior to performing this upgrade, you should be on the latest Satellite 6.11 version, including the RHEL OS packages. You can upgrade to the latest version with a regular foreman-maintain upgrade run -y --target-version=6.11.z.

In addition to that, you need to enable the extras repository and install the leapp package:

# subscription-manager repos --enable rhel-7-server-extras-rpms
# satellite-maintain packages install leapp leapp-repository -y
Running install packages in unlocked session                                          
================================================================================
Confirm installer run is allowed:                                                                                                                                            

WARNING: This script runs satellite-installer after the yum execution          
to ensure the Satellite is in a consistent state.                                                                                                                            
As a result some of your services may be restarted.                                                                                                                          

Do you want to proceed?, [y(yes), q(quit)] y                                                                                                                                 
                                                                      [OK]                                                                                                  
--------------------------------------------------------------------------------                                                                                             
Unlock packages:                                                      [OK]                                                                                                   
--------------------------------------------------------------------------------                                                                                             
Install packages: Loaded plugins: product-id, search-disabled-repos, subscription-manager          
[...]
================================================================================
Install  2 Packages (+16 Dependent packages)

Total download size: 3.8 M
Installed size: 14 M
Is this ok [y/d/N] y

[...]
--------------------------------------------------------------------------------
Check status of version locking of packages: 
  Automatic locking of package versions is enabled in installer.
  Packages are locked.                                                [OK]
--------------------------------------------------------------------------------
# 

You will also need to perform this workaround to pass through a known caveat in the upgrade process documented in https://access.redhat.com/solutions/6966647 .

# subscription-manager repo-override --repo=satellite-6.11-for-rhel-8-x86_64-rpms --add=module_hotfixes:1 
Repository 'satellite-6.11-for-rhel-8-x86_64-rpms' does not currently exist, but the override has been added.

Running leapp preupgrade

LEAPP can be run to assess the system and prepare the upgrade process.

# time leapp preupgrade
==> Processing phase `configuration_phase`
====> * ipu_workflow_config
        IPU workflow config actor
==> Processing phase `FactsCollection`
====> * tcp_wrappers_config_read
        Parse tcp_wrappers configuration files /etc/hosts.{allow,deny}.
====> * grubdevname
        Get name of block device where GRUB is located
====> * scanmemory
        Scan Memory of the machine.
====> * scan_subscription_manager_info
        Scans the current system for subscription manager information
====> * scan_files_for_target_userspace
        Scan the source system and identify files that will be copied into the target userspace when it is created.
====> * sssd_facts
        Check SSSD configuration for changes in RHEL8 and report them in model.
====> * network_manager_read_config
        Provides data about NetworkManager configuration.
====> * scan_kernel_cmdline
        No documentation has been provided for the scan_kernel_cmdline actor.
====> * storage_scanner
        Provides data about storage settings.
====> * load_device_driver_deprecation_data
        Loads deprecation data for drivers and devices (PCI & CPU)
====> * register_yum_adjustment
        Registers a workaround which will adjust the yum directories during the upgrade.
====> * udevadm_info
        Produces data exported by the "udevadm info" command.
====> * scan_sap_hana
        Gathers information related to SAP HANA instances on the system.
====> * pci_devices_scanner
        Provides data about existing PCI Devices.
====> * authselect_scanner
        Detect what authselect configuration should be suggested to administrator.
====> * persistentnetnames
        Get network interface information for physical ethernet interfaces of the original system.
====> * common_leapp_dracut_modules
        Influences the generation of the initram disk
====> * persistentnetnamesdisable
        Disable systemd-udevd persistent network naming on machine with single eth0 NIC
====> * system_facts
        Provides data about many facts from system.
====> * read_openssh_config
        Collect information about the OpenSSH configuration.
====> * repository_mapping
        Produces message containing repository mapping based on provided file.
====> * xfs_info_scanner
        This actor scans all mounted mountpoints for XFS information
====> * sctp_read_status
        Determines whether or not the SCTP kernel module might be wanted.
====> * source_boot_loader_scanner
        Scans the boot loader configuration on the source system.
====> * scan_custom_repofile
        Scan the custom /etc/leapp/files/leapp_upgrade_repositories.repo repo file.
====> * biosdevname
        Enable biosdevname on the target RHEL system if all interfaces on the source RHEL
====> * rpm_scanner
        Provides data about installed RPM Packages.
Loaded plugins: foreman-protector, product-id, subscription-manager

WARNING: Excluding 13038 packages due to foreman-protector. 
Use foreman-maintain packages install/update <package> 
to safely install packages without restrictions.
Use foreman-maintain upgrade run for full upgrade.

====> * transaction_workarounds
        Provides additional RPM transaction tasks based on bundled RPM packages.
====> * scan_pkg_manager
        Provides data about package manager (yum/dnf)
====> * check_kde_apps
        Actor checks which KDE apps are installed.
====> * root_scanner
        Scan the system root directory and produce a message containing
====> * firewalld_facts_actor
        Provide data about firewalld
====> * scanclienablerepo
        Produce CustomTargetRepository based on the LEAPP_ENABLE_REPOS in config.
====> * pam_modules_scanner
        Scan the pam directory for services and modules used in them
====> * selinuxcontentscanner
        Scan the system for any SELinux customizations
====> * scandasd
        In case of s390x architecture, check whether DASD is used.
====> * scancpu
        Scan CPUs of the machine.
====> * removed_pam_modules_scanner
        Scan PAM configuration for modules that are not available in RHEL-8.
====> * satellite_upgrade_facts
        Report which Satellite packages require updates and how to handle PostgreSQL data
====> * get_enabled_modules
        Provides data about which module streams are enabled on the source system.
====> * repositories_blacklist
        Exclude target repositories provided by Red Hat without support.
====> * detect_kernel_drivers
        Matches all currently loaded kernel drivers against known deprecated and removed drivers.
====> * get_installed_desktops
        Actor checks if kde or gnome desktop environments
====> * checkrhui
        Check if system is using RHUI infrastructure (on public cloud) and send messages to
====> * red_hat_signed_rpm_scanner
        Provide data about installed RPM Packages signed by Red Hat.
====> * quagga_daemons
        Active quagga daemons check.
====> * ipa_scanner
        Scan system for ipa-client and ipa-server status
====> * rpm_transaction_config_tasks_collector
        Provides additional RPM transaction tasks from /etc/leapp/transaction.
====> * used_repository_scanner
        Scan used enabled repositories
====> * cups_scanner
        Gather facts about CUPS features which needs to be migrated
====> * spamassassin_config_read
        Reads spamc configuration (/etc/mail/spamassassin/spamc.conf), the
====> * pes_events_scanner
        Provides data about package events from Package Evolution Service.
====> * vsftpd_config_read
        Reads vsftpd configuration files (/etc/vsftpd/*.conf) and extracts necessary information.
====> * multipath_conf_read
        Read multipath configuration files and extract the necessary informaton
====> * setuptargetrepos
        Produces list of repositories that should be available to be used by Upgrade process.
==> Processing phase `Checks`
====> * check_luks_and_inhibit
        Check if any encrypted partitions is in use. If yes, inhibit the upgrade process.
====> * check_memcached
        Check for incompatible changes in memcached configuration.
====> * check_os_release
        Check if the current RHEL minor version is supported. If not, inhibit the upgrade process.
====> * authselect_check
        Confirm suggested authselect call from AuthselectScanner.
====> * checkacpid
        Check if acpid is installed. If yes, write information about non-compatible changes.
====> * tcp_wrappers_check
        Check the list of packages previously compiled with TCP wrappers support
====> * postgresql_check
        Actor checking for presence of PostgreSQL installation.
====> * check_root_symlinks
        Check if the symlinks /bin and /lib are relative, not absolute.
====> * check_kde_gnome
        Checks whether KDE is installed
====> * check_non_mount_boot_s390
        Inhibits on s390 when /boot is NOT on a separate partition.
====> * check_btrfs
        Check if Btrfs filesystem is in use. If yes, inhibit the upgrade process.
====> * check_se_linux
        Check SELinux status and produce decision messages for further action.
====> * check_rhsmsku
        Ensure the system is subscribed to the subscription manager
====> * check_sendmail
        Check if sendmail is installed, check whether configuration update is needed, inhibit upgrade if TCP wrappers
====> * open_ssh_deprecated_directives_check
        Check for any deprecated directives in the OpenSSH configuration.
====> * check_ipa_server
        Check for ipa-server and inhibit upgrade
====> * check_skipped_repositories
        Produces a report if any repositories enabled on the system are going to be skipped.
====> * check_ntp
        Check if ntp and/or ntpdate configuration needs to be migrated.
====> * check_chrony
        Check for incompatible changes in chrony configuration.
====> * check_firewalld
        Check for certain firewalld configuration that may prevent an upgrade.
====> * check_docker
        Checks if Docker is installed and warns about its deprecation in RHEL8.
====> * open_ssh_algorithms
        OpenSSH configuration does not contain any unsupported cryptographic algorithms.
====> * checkdosfstools
        Check if dosfstools is installed. If yes, write information about non-compatible changes.
====> * check_brltty
        Check if brltty is installed, check whether configuration update is needed.
====> * cups_check
        Reports changes in configuration between CUPS 1.6.3 and 2.2.6
====> * checktargetrepos
        Check whether target yum repositories are specified.
====> * check_sap_hana
        If SAP HANA has been detected, several checks are performed to ensure a successful upgrade.
====> * check_removed_envvars
        Check for usage of removed environment variables and inhibit the upgrade
====> * zipl_check_boot_entries
        Inhibits the upgrade if a problematic Zipl configuration is detected on the system.
====> * checkhybridimage
        Check if the system is using Azure hybrid image.
====> * quagga_report
        Checking for babeld on RHEL-7.
====> * unsupported_upgrade_check
        Checks enviroment variables and produces a warning report if the upgrade is unsupported.
====> * checkfstabxfsoptions
        Check the FSTAB file for the deprecated / removed XFS mount options.
====> * check_boot_avail_space
        Check if at least 100Mib of available space on /boot. If not, inhibit the upgrade process.
====> * python_inform_user
        This actor informs the user of differences in Python version and support in RHEL 8.
====> * check_system_arch
        Check if system is running at a supported architecture. If no, inhibit the upgrade process.
====> * check_etc_releasever
        Check releasever info and provide a guidance based on the facts
====> * removed_pam_modules
        Check for modules that are not available in RHEL 8 anymore
====> * check_cifs
        Check if CIFS filesystem is in use. If yes, inhibit the upgrade process.
====> * open_ssh_protocol
        Protocol configuration option was removed.
====> * check_nfs
        Check if NFS filesystem is in use. If yes, inhibit the upgrade process.
====> * check_postfix
        Check if postfix is installed, check whether configuration update is needed.
====> * multipath_conf_check
        Checks whether the multipath configuration can be updated to RHEL-8 and
====> * check_fips
        Inhibit upgrade if FIPS is detected as enabled.
====> * powertop
        Check if PowerTOP is installed. If yes, write information about non-compatible changes.
====> * check_installed_debug_kernels
        Inhibit IPU (in-place upgrade) when multiple debug kernels are installed.
====> * sctp_checks
        Parses collected SCTP information and take necessary actions.
====> * check_wireshark
        Report a couple of changes in tshark usage
====> * sssd_check
        Check SSSD configuration for changes in RHEL8 and report them.
====> * checkgrep
        Check if Grep is installed. If yes, write information about non-compatible changes.
====> * efi_check_boot
        Adjust EFI boot entry for first reboot
====> * check_bind
        Actor parsing BIND configuration and checking for known issues in it.
====> * vsftpd_config_check
        Checks whether the vsftpd configuration is supported in RHEL-8. Namely checks that
====> * checkmemory
        The actor check the size of RAM against RHEL8 minimal hardware requirements
====> * check_installed_devel_kernels
        Inhibit IPU (in-place upgrade) when multiple devel kernels are installed.
====> * check_detected_devices_and_drivers
        Checks whether or not detected devices and drivers are usable on the target system.
====> * red_hat_signed_rpm_check
        Check if there are packages not signed by Red Hat in use. If yes, warn user about it.
====> * check_ha_cluster
        Check if HA Cluster is in use. If yes, inhibit the upgrade process.
====> * spamassassin_config_check
        Reports changes in spamassassin between RHEL-7 and RHEL-8
====> * multiple_package_versions
        Check for problematic 32bit packages installed together with 64bit ones.
====> * satellite_upgrade_check
        Check state of Satellite system before upgrade
====> * check_rpm_transaction_events
        Filter RPM transaction events based on installed RPM packages
====> * removed_pam_modules_check
        Check if it is all right to disable PAM modules that are not in RHEL-8.
====> * detect_grub_config_error
        Check grub configuration for syntax error in GRUB_CMDLINE_LINUX value.
====> * open_ssh_use_privilege_separation
        UsePrivilegeSeparation configuration option was removed.
====> * checkirssi
        Check if irssi is installed. If yes, write information about non-compatible changes.
====> * openssh_permit_root_login
        OpenSSH no longer allows root logins with password.
====> * yum_config_scanner
        Scans the configuration of the YUM package manager.
====> * check_installed_kernels
        Inhibit IPU (in-place upgrade) when installed kernels conflict with a safe upgrade.
====> * check_grub_core
        Check whether we are on legacy (BIOS) system and instruct Leapp to upgrade GRUB core
====> * check_yum_plugins_enabled
        Checks that the required yum plugins are enabled.
====> * check_skip_phase
        Skip all the subsequent phases until the report phase.
==> Processing phase `Reports`
====> * verify_check_results
        Check all dialogs and notify that user needs to make some choices.
====> * verify_check_results
        Check all generated results messages and notify user about them.

============================================================
                     UPGRADE INHIBITED                      
============================================================

Upgrade has been inhibited due to the following problems:
    1. Inhibitor: Use of NFS detected. Upgrade can't proceed
    2. Inhibitor: Leapp detected loaded kernel drivers which have been removed in RHEL 8. Upgrade cannot proceed.
    3. Inhibitor: Newest installed kernel not in use
    4. Inhibitor: Missing required answers in the answer file
Consult the pre-upgrade report for details and possible remediation.

============================================================
                     UPGRADE INHIBITED                      
============================================================


Debug output written to /var/log/leapp/leapp-preupgrade.log

============================================================
                           REPORT                           
============================================================

A report has been generated at /var/log/leapp/leapp-report.json
A report has been generated at /var/log/leapp/leapp-report.txt

============================================================
                       END OF REPORT                        
============================================================

Answerfile has been generated at /var/log/leapp/answerfile

real    3m26.738s
user    2m48.344s
sys 0m11.675s

The output of the report can be reviewed at :

📋 : /var/log/leapp/leapp-report.txt

Answering update questions and amending configurations

LEAPP will probably point a number of blocker issues (inhibitors) that prevent RHEL from being directly upgraded into the next version. The most typical ones are:

  • Deprecated drivers (eg: floppy)
  • Multiple NICs following the naming standard (eg: eth0 and eth1).
  • Not running the latest installed kernel
  • NFS mountpoints
  • Changes in configuration.

Deprecated drivers can be removed online with a simple modprobe -r command, eg:

# modprobe -r floppy
# modprobe -r pata_acpi

If you need to rename your NICs prior to upgrade, review your satellite configuration to ensure no service depends on such nic names. This can be achieved looking at the current configuration:

# satellite-installer --scenario satellite -h | grep eth

This command will provide the output of any service using 'eth' .

Finally, you'll need to answer any pending questions at /var/log/leapp/answerfile. They can be answered by editing the file, or programatically with:

# leapp answer --section remove_pam_pkcs11_module_check.confirm=True

Performing the upgrade

You will need about:

  • 15-30 min to download the new RHEL8 RPMs.
  • 15-30 min to power down the system, start in single user and perform the RHEL upgrade (done automatically by leapp).
  • 15-30 min post-reboot, once in RHEL8, for the leapp-upgrade process to run satellite-installer once again.

Launching the upgrade

Once the prerequisites have been sorted out, you can launch the actual install phase with:

# time leapp upgrade --reboot 
==> Processing phase `configuration_phase`
====> * ipu_workflow_config
        IPU workflow config actor
==> Processing phase `FactsCollection`
====> * source_boot_loader_scanner
        Scans the boot loader configuration on the source system.
[...]
====> * target_userspace_creator
        Initializes a directory to be populated as a minimal environment to run binaries from the target system.
Red Hat Enterprise Linux 8 for x86_64 - AppStre  33 MB/s |  47 MB     00:01    
Red Hat Enterprise Linux 8 for x86_64 - BaseOS   34 MB/s |  53 MB     00:01    
[...]
 rpm-plugin-systemd-inhibit    x86_644.14.3-24.el8_6                rhel-8-for-x86_64-baseos-rpms         79 k
 kpartx                        x86_640.8.4-22.el8_6.2               rhel-8-for-x86_64-baseos-rpms         115 k

Transaction Summary
================================================================================
Install  199 Packages

Total download size: 111 M
Installed size: 707 M
Downloading Packages:
(1/199): pinentry-1.1.0-2.el8.x86_64.rpm        376 kB/s | 100 kB     00:00    
(2/199): libxkbcommon-0.9.1-1.el8.x86_64.rpm    295 kB/s | 116 kB     00:00    
[...]
Complete!
==> Processing phase `TargetTransactionCheck`
====> * tmp_actor_to_satisfy_sanity_checks
        The actor does NOTHING but satisfy static sanity checks
====> * local_repos_inhibit
        Inhibits the upgrade if local repositories were found.
====> * report_set_target_release
        Reports information related to the release set in the subscription-manager after the upgrade.
====> * dnf_transaction_check
        This actor tries to solve the RPM transaction to verify the all package dependencies can be successfully resolved.
Applying transaction workaround - yum config fix

Applying transaction workaround - PostgreSQL symlink fix

Last metadata expiration check: 0:01:05 ago on Sun Oct 30 05:49:50 2022.
Package foreman-installer-katello-1:3.1.2.8-1.el7sat.noarch is already installed.
Package rubygem-foreman_maintain-1:1.0.18-1.el7sat.noarch is already installed.
Package tfm-rubygem-smart_proxy_ansible-3.3.1-4.el7sat.noarch is already installed.
Package satellite-installer-6.11.0.7-1.el7sat.noarch is already installed.
Package katello-4.3.0-3.el7sat.noarch is already installed.
Package foreman-installer-1:3.1.2.8-1.el7sat.noarch is already installed.
[...]
Transaction Summary
====================================================================================================================================================================
Install    796 Packages
Upgrade    520 Packages
Remove     501 Packages
Downgrade   11 Packages

Total size: 1.1 G
Total download size: 1.0 G
DNF will only download packages, install gpg keys, and check the transaction.
Downloading Packages:
[SKIPPED] libcroco-0.6.12-4.el8_2.1.x86_64.rpm: Already downloaded          
[...]
(1323/1324): glib2-devel-2.56.4-158.el8_6.1.x86 1.9 MB/s | 425 kB     00:00    
(1324/1324): linux-firmware-20220210-108.git634  43 MB/s | 196 MB     00:04    
--------------------------------------------------------------------------------
Total                                           7.7 MB/s | 1.0 GB     02:12     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Complete!
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
==> Processing phase `InterimPreparation`
====> * upgrade_initramfs_generator
        Creates the upgrade initramfs
[...]
Transaction test succeeded.
Complete!
====> * add_upgrade_boot_entry
        Add new boot entry for Leapp provided initramfs.
====> * efi_interim_fix
        Adjust EFI boot entry for first reboot
Connection to sat.example.org closed by remote host.

When the system reboots, it will automatically enter the upgrade phase. Progress can be seen on the server console (if it has one), or similarly in the serial console:

[    0.000000] Linux version 4.18.0-372.32.1.el8_6.x86_64 (mockbuild@x86-vm-08.build.eng.bos.redhat.com) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)) #1 SMP Fri Oct 7 12:35:10 EDT 2022
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-upgrade.x86_64 root=UUID=989ac477-64f2-449f-8415-25b1a5f7d47f ro console=tty0 console=ttyS0,115200n8 no_timer_check net.ifnames=0 crashkernel=auto LANG=en_US.UTF-8 enforcing=0 rd.plymouth=0 plymouth.enable=0
[...]
[  OK  ] Reached target System Upgrade.
         Starting System Upgrade...
[    3.091330] upgrade[564]: starting upgrade hook
[    3.100141] upgrade[564]: /bin/upgrade: line 19: /sysroot/var/tmp/system-upgrade.state: Read-only file system
[    3.108025] upgrade[567]:   WARNING: locking_type (4) is deprecated, using --sysinit --readonly.
[    3.139068] upgrade[577]: Spawning container sysroot on /sysroot.
[    3.140429] upgrade[577]: Press ^] three times within 1s to kill container.
[    3.158634] upgrade[578]: Host and machine ids are equal (e6a3f27a614a4bafbce01f024fffa4fa): refusing to link journals
[   19.039129] upgrade[581]: ==> Processing phase `InitRamStart`
[   19.040185] upgrade[581]: ====> * remove_upgrade_boot_entry
[   19.041240] upgrade[581]:         Remove boot entry for Leapp provided initramfs.
[   20.201161] upgrade[581]: ==> Processing phase `LateTests`
[   20.202092] upgrade[581]: ====> * persistentnetnamesinitramfs
[   20.203015] upgrade[581]:         Get network interface information for physical ethernet interfaces with the new kernel in initramfs.
[   20.368520] upgrade[581]: ==> Processing phase `Preparation`
[   20.369458] upgrade[581]: ====> * applytransactionworkarounds
[   20.370454] upgrade[581]:         Executes registered workaround scripts on the system before the upgrade transaction
[   20.687348] upgrade[1127]: Applying transaction workaround - yum config fix
[   20.688443] upgrade[1127]: Applying transaction workaround - PostgreSQL symlink fix
[   20.713928] upgrade[581]: ====> * zipl_convert_to_blscfg
[   20.714805] upgrade[581]:         Convert the zipl boot loader configuration to the the boot loader specification on s390x systems.
[   20.810979] upgrade[581]: ====> * update_etc_sysconfig_kernel
[   20.811979] upgrade[581]:         Update /etc/sysconfig/kernel file.
[   20.928661] upgrade[581]: ====> * removed_pam_modules_apply
[   20.929962] upgrade[581]:         Remove old PAM modules that are no longer available in RHEL-8 from
[   21.008238] upgrade[581]: ====> * remove_boot_files
[   21.009385] upgrade[581]:         Remove Leapp provided initramfs from boot partition.
[   21.079783] upgrade[581]: ====> * bind_update
[   21.080723] upgrade[581]:         Actor parsing facts found in configuration and modifing configuration.
[   21.702416] upgrade[581]: ====> * selinuxprepare
[   21.703252] upgrade[581]:         Remove selinux policy customizations before updating selinux-policy* packages
[   37.974629] upgrade[581]: ==> Processing phase `RPMUpgrade`
[   37.975625] upgrade[581]: ====> * dnf_upgrade_transaction
[   37.976681] upgrade[581]:         Setup and call DNF upgrade command
[   56.672687] upgrade[1508]: Last metadata expiration check: 0:09:28 ago on Sun Oct 30 05:49:50 2022.
[   56.674187] upgrade[1508]: Package foreman-installer-katello-1:3.1.2.8-1.el7sat.noarch is already installed.
[   56.675740] upgrade[1508]: Package rubygem-foreman_maintain-1:1.0.18-1.el7sat.noarch is already installed.
[   56.677163] upgrade[1508]: Package tfm-rubygem-smart_proxy_ansible-3.3.1-4.el7sat.noarch is already installed.
[   56.678745] upgrade[1508]: Package satellite-installer-6.11.0.7-1.el7sat.noarch is already installed.
[   56.680170] upgrade[1508]: Package katello-4.3.0-3.el7sat.noarch is already installed.
[   56.681422] upgrade[1508]: Package foreman-installer-1:3.1.2.8-1.el7sat.noarch is already installed.
[   56.682846] upgrade[1508]: Dependencies resolved.
...
[  629.502384] upgrade[1508]:   Cleanup          : libffi-3.0.13-19.el7.x86_64                      2358/2377
[  629.504355] upgrade[1508]:   Running scriptlet: libffi-3.0.13-19.el7.x86_64                      2358/2377
[  629.506308] upgrade[1508]:   Cleanup          : libattr-2.4.46-13.el7.x86_64                     2359/2377
[  629.508318] upgrade[1508]:   Running scriptlet: libattr-2.4.46-13.el7.x86_64                     2359/2377
[  629.510145] upgrade[1508]:   Cleanup          : glibc-common-2.17-326.el7_9.x86_64               2360/2377
[  629.512042] upgrade[1508]:   Cleanup          : libselinux-2.5-15.el7.x86_64                     2361/2377
...
[  767.787262] upgrade[1508]:   yum-rhn-plugin-2.0.1-10.el7.noarch
[  767.789242] upgrade[1508]: Complete!
[  767.826797] upgrade[581]: ====> * scan_installed_target_kernel_version
[  767.828424] upgrade[581]:         Scan for the version of the newly installed kernel
[  768.206418] upgrade[581]: ====> * update_grub_core
[  768.208251] upgrade[581]:         On legacy (BIOS) systems, GRUB core (located in the gap between the MBR and the
[  769.967896] upgrade[581]: ====> * prepare_python_workround
[  769.969822] upgrade[581]:         Prepare environment to be able to run leapp with Python3 in initrd.
[  770.059375] upgrade[581]: ====> * check_leftover_packages
[  770.061088] upgrade[581]:         Check if there are any RHEL 7 packages present after upgrade.
[  785.130946] upgrade[581]: ====> * report_leftover_packages
[  785.132304] upgrade[581]:         Collect messages about leftover el7 packages and generate report for users.
[  785.479332] upgrade[581]: Debug output written to /var/log/leapp/leapp-upgrade.log
[  785.482295] upgrade[581]: ============================================================
[  785.485510] upgrade[581]:                            REPORT
[  785.488046] upgrade[581]: ============================================================
[  785.491356] upgrade[581]: A report has been generated at /var/log/leapp/leapp-report.json
[  785.494664] upgrade[581]: A report has been generated at /var/log/leapp/leapp-report.txt
[  785.497093] upgrade[581]: ============================================================
[  785.499437] upgrade[581]:                        END OF REPORT
[  785.501382] upgrade[581]: ============================================================
[  785.503777] upgrade[581]: Answerfile has been generated at /var/log/leapp/answerfile
[  785.541018] upgrade[577]: Container sysroot exited successfully.
[  785.569372] upgrade[23665]: Spawning container sysroot on /sysroot.
[  785.571293] upgrade[23665]: Press ^] three times within 1s to kill container.
[  785.589754] upgrade[23666]: Host and machine ids are equal (e6a3f27a614a4bafbce01f024fffa4fa): refusing to link journals
[  800.976473] upgrade[23669]: ==> Processing phase `Applications`
[  800.978106] upgrade[23669]: ====> * persistentnetnamesconfig
[  800.979944] upgrade[23669]:         Generate udev persistent network naming configuration
[  801.100341] upgrade[23669]: ====> * satellite_upgrade_data_migration
[  801.101788] upgrade[23669]:         Reconfigure Satellite services and migrate PostgreSQL data
[  801.200412] upgrade[23669]: ====> * sctp_config_update
[  801.201875] upgrade[23669]:         This actor updates SCTP configuration for RHEL8.
[  801.453848] upgrade[23669]: ====> * migrate_ntp
[  801.455196] upgrade[23669]:         Migrate ntp and/or ntpdate configuration to chrony.
[  801.570183] upgrade[23669]: ====> * cups_migrate
[  801.571466] upgrade[23669]:         cups_migrate actor
[  801.677384] upgrade[23669]: ====> * spamassassin_config_update
[  801.679294] upgrade[23669]:         This actor performs several modifications to spamassassin configuration
[  801.813341] upgrade[23669]: ====> * network_manager_update_config
[  801.814771] upgrade[23669]:         Updates NetworkManager configuration for Red Hat Enterprise Linux 8.
[  801.929467] upgrade[23669]: ====> * authselect_apply
[  801.930932] upgrade[23669]:         Apply changes suggested by AuthselectScanner.
[  802.030986] upgrade[23669]: ====> * firewalld_update_lockdown_whitelist
[  802.032780] upgrade[23669]:         Update the firewalld Lockdown Whitelist.
[  802.168557] upgrade[23669]: ====> * sanebackends_migrate
[  802.170137] upgrade[23669]:         Actor for migrating sane-backends configuration files.
[  802.369631] upgrade[23669]: ====> * migrate_sendmail
[  802.371174] upgrade[23669]:         Migrate sendmail configuration files.
[  802.506474] upgrade[23669]: ====> * quagga_to_frr
[  802.507824] upgrade[23669]:         Edit frr configuration on the new system.
[  802.589137] upgrade[23669]: ====> * set_etc_releasever
[  802.590430] upgrade[23669]:         Release version in /etc/dnf/vars/releasever will be set to the current target release
[  802.686639] upgrade[23669]: ====> * vim_migrate
[  802.688214] upgrade[23669]:         Modify configuration files of Vim 8.0 and later to keep the same behavior
[  803.087132] upgrade[23669]: ====> * vsftpd_config_update
[  803.088369] upgrade[23669]:         Modifies vsftpd configuration files on the target RHEL-8 system so that the effective
[  803.224567] upgrade[23669]: ====> * migrate_brltty
[  803.226273] upgrade[23669]:         Migrate brltty configuration files.
[  803.306197] upgrade[23669]: ====> * selinuxapplycustom
[  803.307482] upgrade[23669]:         Re-apply SELinux customizations from the original RHEL installation
[  813.958361] upgrade[23669]: ====> * network_manager_update_service
[  813.960107] upgrade[23669]:         Updates NetworkManager services status.
[  814.226576] upgrade[23669]: ====> * multipath_conf_update
[  814.227993] upgrade[23669]:         Modifies multipath configuration files on the target RHEL-8 system so that
[  814.343563] upgrade[23669]: ====> * cupsfilters_migrate
[  814.345277] upgrade[23669]:         Actor for migrating package cups-filters.
[  814.739751] upgrade[23669]: ==> Processing phase `ThirdPartyApplications`
[  814.741239] upgrade[23669]: ==> Processing phase `Finalization`
[  814.742808] upgrade[23669]: ====> * schedule_se_linux_relabelling
[  814.744659] upgrade[23669]:         Schedule SELinux relabelling.
[  814.884644] upgrade[23669]: ====> * grubenvtofile
[  814.886180] upgrade[23669]:         Convert "grubenv" symlink to a regular file on Azure hybrid images using BIOS.
[  814.959727] upgrade[23669]: ====> * kernelcmdlineconfig
[  814.961317] upgrade[23669]:         Append extra arguments to the target RHEL kernel command line
[  815.278592] upgrade[23669]: ====> * efi_finalization_fix
[  815.280153] upgrade[23669]:         Adjust EFI boot entry for final reboot
[  815.376693] upgrade[23669]: ====> * force_default_boot_to_target_kernel_version
[  815.378281] upgrade[23669]:         Ensure the default boot entry is set to the new target kernel
[  816.031702] upgrade[23669]: ====> * create_systemd_service
[  816.033670] upgrade[23669]:         Add a systemd service to launch Leapp.
[  816.169112] upgrade[23669]: ====> * target_initramfs_generator
[  816.170874] upgrade[23669]:         Regenerate the target RHEL major version initrd and include files produced by other actors
[  816.263190] upgrade[23669]: ====> * set_permissive_se_linux
[  816.264610] upgrade[23669]:         Set SELinux mode.
[  816.424797] upgrade[25854]: Running in chroot, ignoring request.
[  816.586634] upgrade[23669]: Debug output written to /var/log/leapp/leapp-upgrade.log
[  816.588318] upgrade[23669]: ============================================================
[  816.590201] upgrade[23669]:                            REPORT
[  816.591874] upgrade[23669]: ============================================================
[  816.594323] upgrade[23669]: A report has been generated at /var/log/leapp/leapp-report.json
[  816.596922] upgrade[23669]: A report has been generated at /var/log/leapp/leapp-report.txt
[  816.599435] upgrade[23669]: ============================================================
[  816.601852] upgrade[23669]:                        END OF REPORT
[  816.603864] upgrade[23669]: ============================================================
[  816.605804] upgrade[23669]: Answerfile has been generated at /var/log/leapp/answerfile
[  816.645563] upgrade[23665]: Container sysroot exited successfully.
[  816.650618] upgrade[564]: writing logs to disk and rebooting
[  816.784097] upgrade[25871]: Spawning container sysroot on /sysroot.
[  816.785843] upgrade[25871]: Press ^] three times within 1s to kill container.
[  816.805312] upgrade[25872]: Host and machine ids are equal (e6a3f27a614a4bafbce01f024fffa4fa): refusing to link journals
[  816.825354] upgrade[25871]: Container sysroot exited successfully.
[  817.100972] upgrade[564]: /bin/upgrade: line 19: /sysroot/var/tmp/system-upgrade.state: Read-only file system
[  OK  ] Stopped target Timers.
[  OK  ] Stopped target Remote File Systems (Pre).
...
[  817.883505] reboot: Restarting system
[  817.884817] reboot: machine restart

Now the system will restart and start a SELinux relabeling process:

[   29.079373] selinux-autorelabel[817]: Warning: Skipping the following R/O filesystems:
[   29.081344] selinux-autorelabel[817]: /sys/fs/cgroup
[   29.082834] selinux-autorelabel[817]: Relabeling / /dev /dev/hugepages /dev/mqueue /dev/pts /dev/shm /run /sys /sys/fs/cgroup/blkio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/cpuset /sys/fs/cgroup/devices /sys/fs/cgroup/freezer /sys/fs/cgroup/hugetlb /sys/fs/cgroup/memory /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/perf_event /sys/fs/cgroup/pids /sys/fs/cgroup/rdma /sys/fs/cgroup/systemd /sys/fs/pstore /sys/kernel/debug /sys/kernel/tracing
...
[  181.063734] reboot: Restarting system
[  181.064470] reboot: machine restart

At this moment, the system restarts with RHEL8 and initiates the last part of the configuration and upgrade process. The progress can be followed logging in via SSH and following the leapp-upgrade systemd unit, and the /var/log/foreman-installer/satellite.log :

# journalctl -u leapp_resume.service  -f
-- Logs begin at Sun 2022-10-30 06:15:40 EDT. --
Oct 30 06:15:46 sat.example.org systemd[1]: Starting Temporary Leapp service which resumes execution after reboot...
Oct 30 06:16:08 sat.example.org leapp3[1348]: ==> Processing phase `FirstBoot`
Oct 30 06:16:08 sat.example.org leapp3[1348]: ====> * network_manager_update_connections
Oct 30 06:16:08 sat.example.org leapp3[1348]:         Update NetworkManager connections.
Oct 30 06:16:08 sat.example.org leapp3[1348]: ====> * enable_rhsm_target_repos
Oct 30 06:16:08 sat.example.org leapp3[1348]:         On the upgraded target system, set release and enable repositories that were used during the upgrade
Oct 30 06:17:23 sat.example.org leapp3[1348]: ====> * satellite_upgrader
Oct 30 06:17:23 sat.example.org leapp3[1348]:         Execute installer in the freshly booted system, to finalize Satellite configuration
Oct 30 06:36:00 sat.example.org leapp3[6544]: Running the installer. This can take a while.
Oct 30 06:36:00 sat.example.org leapp3[1348]: ====> * remove_systemd_resume_service
Oct 30 06:36:00 sat.example.org leapp3[1348]:         Remove systemd service to launch Leapp.
Oct 30 06:36:01 sat.example.org leapp3[1348]: Debug output written to /var/log/leapp/leapp-upgrade.log
Oct 30 06:36:01 sat.example.org leapp3[1348]: ============================================================
Oct 30 06:36:01 sat.example.org leapp3[1348]:                            REPORT
Oct 30 06:36:01 sat.example.org leapp3[1348]: ============================================================
Oct 30 06:36:01 sat.example.org leapp3[1348]: A report has been generated at /var/log/leapp/leapp-report.json
Oct 30 06:36:01 sat.example.org leapp3[1348]: A report has been generated at /var/log/leapp/leapp-report.txt
Oct 30 06:36:01 sat.example.org leapp3[1348]: ============================================================
Oct 30 06:36:01 sat.example.org leapp3[1348]:                        END OF REPORT
Oct 30 06:36:01 sat.example.org leapp3[1348]: ============================================================
Oct 30 06:36:01 sat.example.org leapp3[1348]: Answerfile has been generated at /var/log/leapp/answerfile
Oct 30 06:36:01 sat.example.org systemd[1]: leapp_resume.service: Succeeded.
Oct 30 06:36:01 sat.example.org systemd[1]: Started Temporary Leapp service which resumes execution after reboot.

The satellite should be now up and running in the latest version!

We can verify with foreman-maintain, as usual:

# foreman-maintain service status
...
\ All services are running                                            [OK]      
--------------------------------------------------------------------------------

Post upgrade tasks

Set SELinux in enforcing mode

As you folks are running ALL your systems with SELinux in enforcing mode 😉 , you'll need to reenable it with:

# vim /etc/selinux/config   # (and set it to enforcing)
# dnf reinstall foreman-selinux katello-selinux --disableplugin=foreman-protector -y && reboot

Remove the package locks in /etc/yum.conf

Edit /etc/yum.conf so no packages are listed in the exclude section. The leapp process leaves the following configuration, which must be removed:

# grep exclude /etc/yum.conf
exclude=python2-leapp,snactor,leapp-upgrade-el7toel8,leapp

Remove the leapp package

As a part of the upgrade, the leapp package is not automatically removed and this can create issues.

You can remove the leapp package with:

# dnf remove leapp leapp-deps-el8 leapp-repository-deps-el8 leapp-upgrade-el7toel8  python2-leapp  --disableplugin=foreman-protector  -y

Optionally perform an update to the latest Satellite version

As a verification of the previous steps, we can perform an optional update of Satellite to ensure we didn't forget any relevant step. The update should do nothing (packages are already in the latest version), and we can quickly confirm no problems will occur on future updates.

# foreman-maintain upgrade run --target-version=6.11.z -y

Conclusion

All in all, great work of the LEAPP team creating a tool that will provide the framework to perform in-place upgrades of RHEL operating systems for the years to come!