Upgrading Ansible Tower to Ansible Automation Platform

It's been quite a while since I last touched Ansible Tower, and I'm glad to report that the latest Ansible Automation Platform introduces several enhancements that makes it a really attractive product.

An strategy to perform upgrades

The Ansible team at Red Hat has published a number of documents on how to perform the upgrade, as this upgrade changes some of the concepts traditionally used in Tower. Namely, virtual environments are replaced by a container-based technology named Execution Environments.

The guide is available here:

Performing the upgrade

In this case, I'll what I did to upgrade an existing clustered Ansible Tower installation from 3.8.x to Ansible Automation Platform 2.2.x, and enable the new features provided by the product (Automation Hub), and the SaaS service provided by Red Hat at console.redhat.com .

Review source environment

In this steps, you'll be noting how the source environment was configured infrastructure-wise, with things like:

• Check how servers are currently configured, including:
  • Filesystems and sizes
  • Networks
  • Operating system tuning
  • Operating system hardening
• Check your Ansible Tower installation:
  • Exact version
  • Database Schema status
  • Inventory file used for installation
• Firewall rules to required resources, such as:
  • Internet proxies
  • SCMs (Git, etc)
  • Authentication (AD/LDAP)
  • CMDB / dynamic inventory sources
  • Red Hat Satellite
  • Other shared resources

Perform a dry-run migration

It is possible to perform a mock upgrade in a separate system, starting from an Ansible Tower backup of the "old" system, even if the old system is a clustered one.

This can be accomplished by performing a backup on the source Tower system, and a fresh Tower install + restore process in the test system.

root@tower-old ~/ansible-tower-setup-bundle-3.8.6-2 # ./setup -b
(transfer backup to test system) 

Then you can create an inventory in the test system, and run the installer as if it was a new system, with a blank config. Then restore the database dump on it.

root@tower-test ~/ansible-automation-platform-setup-bundle-1.2.7-2 # ./setup.sh
root@tower-test ~/ansible-automation-platform-setup-bundle-1.2.7-2 # ./setup.sh -r -e 'restore_backup_file=/tmp/tower-backup.tar.gz'

In this case, you'll be looking to ensure your database schema is migrated successfully prior to engaging into the next upgrade step (eg, Tower 3.8.x to AAP 1.2.latest, to AAP 2.1.latest, finally to AAP 2.2.latest).

In my case, migrating from Tower 3.8.3 to AAP 1.2 (or Tower 3.8.latest) failed silently. The Ansible Tower update process (setup.sh) finished successfully, but the web page itself was showing a maintenance page.

This was solved by checking the database schema:

root@tower-test ~/ansible-automation-platform-setup-bundle-1.2.7-2 # awx-manage  showmigrations | grep -v [X]
auth
 [ ] 0012_alter_user_first_name_max_length
conf
contenttypes
main
oauth2_provider
 [ ] 0002_auto_20190406_1805
 [ ] 0003_auto_20201211_1314
sessions
sites
social_django
 [ ] 0009_auto_20191118_0520
 [ ] 0010_uid_db_index
sso
taggit
 [ ] 0004_alter_taggeditem_content_type_alter_taggeditem_tag

Re-running setup.sh fixed the issue, and further updates could be done successfully.

After this snag was fixed, the upgrade to 2.1 and 2.2 went smoothly.

Post upgrade tasks

Once your environment is upgraded to Ansible Automation Platform 2.2.x, you can also review the following settings:

Default Execution environment

Virtual Envs are deprecated in AAP 2.x, so you should move to Execution Environments (EEs) and probably create your own EEs based on the supported EEs shipped with AAP.

root@tower ~ # awx-manage list_custom_venvs 
· Discovered Virtual Environments:
/var/lib/awx/venv/myvenv

• To export the contents of a (deprecated) virtual environment, run the following command while supplying the path as an argument: awx-manage export_custom_venv /path/to/venv

• To view the connections a (deprecated) virtual environment had in the database, run the following command while supplying the path as an argument: awx-manage custom_venv_associations /path/to/venv

root@tower ~ # awx-manage custom_venv_associations  /var/lib/awx/venv/myvenv -q
inventory_sources: []
job_templates: []
organizations: []
projects: []

Integration with Automation Analytics

Red Hat provides Automation Analytics included in the Ansible Automation Platform, and can be enabled by:

In case a proxy is required, you can configure it in the AAP Job settings menu, then immediately trigger a sync:

# automation-controller-service restart

# awx-manage gather_analytics --ship                                        
/tmp/48627e92-4cfd-4f8d-86f2-c180adcaef42-2022-06-11-000448+0000-0.tar.gz   
/tmp/48627e92-4cfd-4f8d-86f2-c180adcaef42-2022-06-11-000448+0000-1.tar.gz  

Cleaning up instances

You might end up in a state of having leftover instances in your environment.

They can be purged in this way:

# awx-manage list_instances                                                
[controlplane capacity=178 policy=100%]
        localhost capacity=0 node_type=hybrid version=4.2.0
        aap.example.org capacity=178 node_type=hybrid version=4.2.0 heartbeat="2022-06-09 08:12:18"

[default capacity=178 policy=100%]
        localhost capacity=0 node_type=hybrid version=4.2.0
        aap.example.org capacity=178 node_type=hybrid version=4.2.0 heartbeat="2022-06-09 08:12:18"


# awx-manage remove_from_queue --hostname=localhost --queuename=controlplane

# awx-manage remove_from_queue --hostname=localhost --queuename=default

# awx-manage deprovision_instance --hostname localhost
Instance Removed
Successfully deprovisioned localhost
(changed: True)

Enabling the Private Automation Hub

Once your AAP control plane is up and running, you can add your Private Automation Hub by adding the new system into the inventory and re-running setup.sh .

Interesting links

Red Hat has put together a number of resources on this new Ansible Automation Platform, available here:

• What's new in Ansible Automation Platform 2.2.
• Self paced labs
• Ansible Automation workshops material
• The Ansible Automation Platform documentation
• The automation adoption journey

... and support notes

• https://access.redhat.com/articles/6239891 - Ansible Automation Platform 2 Migration Strategy Considerations
• https://access.redhat.com/articles/6185641 - AAP 2 Migration Considerations Checklist https://access.redhat.com/articles/4098921 - What are the Recommended Upgrade Paths for Ansible Tower/Ansible Automation Platform?
• https://access.redhat.com/solutions/6740441 - How Do I Perform Security Patching / OS Package Upgrades On Ansible Automation Platform Nodes Without Breaking Any Ansible Automation Platform Functionality?
• https://access.redhat.com/solutions/6834291 - May I only update one of the components I want on Ansible Tower or Ansible Automation Controller? https://access.redhat.com/solutions/4308791 - How Can I Bypass "noexec" Permission Issue On "/tmp" and "/var/tmp" During Ansible Tower and Ansible Automation Platform installation?
• https://access.redhat.com/articles/6177982 - What’s new with Ansible Automation Platform 2.0: Developing with ansible-builder and Automation execution environments.
• https://access.redhat.com/solutions/5115431 - How to configure Ansible Tower to use a proxy for Automation Analytics
• https://access.redhat.com/solutions/5519041 - Why Is The Manual Data Uploading To Red Hat Automation Analytics Failing With Status 401 In Ansible Tower?
• https://access.redhat.com/solutions/6446711 - How do I Replace All Execution Environments in Ansible Automation Platform using Private Images from Private Automation Hub?
• https://access.redhat.com/solutions/6539431 - How Do I Install Ansible Automation Platform 2.0 in a Disconnected Environment from the Internet?
• https://access.redhat.com/solutions/6635021 - How Do I Install Ansible Automation Platform 2.1 in a Disconnected Environment from the Internet in a Single Node?
• https://access.redhat.com/solutions/6219021 - In Ansible Automation Controller, How Do I Set a Proxy Just for Ansible Galaxy And Not Globally?
• https://access.redhat.com/solutions/3127941 - How do I Specify HTTP/HTTPS_PROXY using Ansible Tower?
• https://access.redhat.com/solutions/4798321 - How to Activate Ansible Tower License with Red Hat Customer Credentials under a Proxy Environment? (edit /etc/supervisord.conf file)

Other interesting resources

Porting guides

• https://docs.ansible.com/ansible/devel/porting_guides/porting_guides.html
• https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_2.10.html
• https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_3.html

Ansible lint

https://ansible-lint.readthedocs.io/en/latest/

AWX cli

https://github.com/ansible/awx/blob/devel/INSTALL.md#installing-the-awx-cli

Lifecycle

• https://access.redhat.com/support/policy/update_policies/
• https://access.redhat.com/support/policy/updates/ansible-automation-platform

... happy hacking!