ARA Documentation: Release 1.0.0 Guide
ARA Documentation: Release 1.0.0 Guide
Release 1.0.0
Red Hat
1 Table of Contents 3
1.1 FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Installing ARA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Configuring Ansible to use ARA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Configuring the ARA Ansible plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 ARA API server configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.6 ARA API Server authentication and security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.7 API Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.8 Using ARA API clients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.9 Setting playbook names and labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.10 Recording arbitrary data in playbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.11 How to contribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.12 ansible-role-ara-api . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.13 ansible-role-ara-web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
1.14 ansible-role-ara-tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
i
ii
ara Documentation, Release 1.0.0
Contents 1
ara Documentation, Release 1.0.0
2 Contents
CHAPTER 1
Table of Contents
1.1 FAQ
ARA Records Ansible playbooks and makes them easier to understand and troubleshoot.
ARA is currently composed of three different free and open source projects:
• [Link] for the REST API server and Ansible plugins
• [Link] for the standalone web interface
• [Link] for project-specific infrastructure needs (such as the
[Link] website)
3
ara Documentation, Release 1.0.0
Ansible Callbacks are essentially hooks provided by Ansible. Ansible will send an event and you can react to it with a
callback. You could use a callback to do things like print additional details or, in the case of ARA, record the playbook
run data in a database.
Yes, you can find persistent and up-to-date live demos at [Link] for the API and
[Link] for the ara-web standalone interface.
The upstream Ansible community and maintainers provide support for the latest three major stable releases and ARA
follows the same support cycle.
For example, if the latest version of Ansible is 2.8, then the latest release of ARA will support 2.8 as well as 2.7 and
2.6.
For more information on Ansible’s release and maintenance cycle, you can refer to the Ansible documentation.
If you are using a release of Ansible that is no longer supported, we strongly encourage you to upgrade as soon as
possible in order to benefit from the latest features and security fixes.
Older unsupported versions of Ansible can contain unfixed security vulnerabilities (CVE).
Before version 1.0 of ARA, both python2 and python3 were supported. Versions of ARA after 1.0 are not designed to
support python2 in consideration that python2 will reach end of life in January 2020.
Yes.
The defaults are set to have the callback use the offline API client which expects the server dependencies installed and
the data is saved to a local sqlite database.
However, the callback can also be configured to send data to a specified API server address and the API server can be
configured to use a remote database server such as PostgreSQL or MySQL.
The web client interface provided by ara-web is stateless and requires an API server address to connect to. It can be
installed anywhere that has access to the API server.
If you are standing up an API server or if your use case is about recording data locally or offline, the required depen-
dencies can be installed automatically by suffixing [server] to your pip install commands.
First, it is recommended to use a python virtual environment in order to avoid conflicts with your Linux distribution
python packages.
Two roles are built-in to help users configure their API and web deployments:
• ansible-role-ara-api to install the python parts (Ansible plugins, API server)
• ansible-role-ara-web to install the web client (nodejs, react+patternfly)
To begin using ARA, you’ll first need to tell Ansible where it is located.
Since this location will be different depending on your operating system and how you are installing ARA, there are
convenient python modules to help you figure out the right paths.
Once you’ve set up the callback_plugins configuration or the ANSIBLE_CALLBACK_PLUGINS environment
variable, Ansible will automatically use the ARA callback plugin to start recording data.
If you’d like to use the ara_record action plugin to record arbitrary data during your playbook, you’ll need to set
action_plugins and ANSIBLE_ACTION_PLUGINS as well.
$ python3 -m [Link]
/usr/lib/python3.7/site-packages/ara
$ python3 -m [Link]
/usr/lib/python3.7/site-packages/ara/plugins
$ python3 -m [Link].action_plugins
/usr/lib/python3.7/site-packages/ara/plugins/action
$ python3 -m [Link].callback_plugins
/usr/lib/python3.7/site-packages/ara/plugins/callback
$ python3 -m [Link]
[defaults]
callback_plugins=/usr/lib/python3.7/site-packages/ara/plugins/callback
action_plugins=/usr/lib/python3.7/site-packages/ara/plugins/action
ARA plugins uses the same mechanism and configuration files as Ansible to retrieve it’s configuration. It comes with
sane defaults that can be customized if need be.
The order of priority is the following:
1. Environment variables
2. ./[Link] (in the current working directory)
3. ~/.[Link] (in the home directory)
4. /etc/ansible/[Link]
When using the [Link] file, the configuration options must be set under the ara namespace, like so:
[ara]
variable = value
The ARA callback plugin is the component that recovers data throughout the execution of your playbook and sends it
to the API.
By default, the callback plugin is set up to use the local API server with the offline API client but you can also send
data to a remote API server, specify credentials or customize other parameters:
callback: ara
callback_type: notification
requirements:
- ara
short_description: Sends playbook execution data to the ARA API internally or over
˓→HTTP
description:
- Sends playbook execution data to the ARA API internally or over HTTP
options:
api_client:
description: The client to use for communicating with the API
default: offline
env:
- name: ARA_API_CLIENT
ini:
- section: ara
key: api_client
choices: ['offline', 'http']
api_server:
description: When using the HTTP client, the base URL to the ARA API server
default: [Link]
env:
- name: ARA_API_SERVER
ini:
- section: ara
key: api_server
api_username:
description: If authentication is required, the username to authenticate with
default: null
env:
- name: ARA_API_USERNAME
ini:
- section: ara
key: api_username
api_password:
description: If authentication is required, the password to authenticate with
default: null
env:
- name: ARA_API_PASSWORD
ini:
- section: ara
key: api_password
api_timeout:
description: Timeout, in seconds, before giving up on HTTP requests
default: 30
env:
- name: ARA_API_TIMEOUT
ini:
- section: ara
key: api_timeout
(continues on next page)
For example, a customized callback plugin configuration might look like this in an [Link] file:
[ara]
api_client = http
api_server = [Link]
api_username = user
api_password = password
api_timeout = 15
ignored_facts = '["ansible_env", "ansible_all_ipv4_addresses"]'
ignored_arguments = '["extra_vars", "vault_password_files"]'
or as environment variables:
export ARA_API_CLIENT=http
export ARA_API_SERVER="[Link]
export ARA_API_USERNAME=user
export ARA_API_PASSWORD=password
export ARA_API_TIMEOUT=15
export ARA_IGNORED_FACTS='["ansible_env", "ansible_all_ipv4_addresses"]'
export ARA_IGNORED_ARGUMENTS='["extra_vars", "vault_password_files"]'
The ara_record action plugin recovers it’s configuration from the callback plugin.
It is therefore not necessary to configure it explicitely other than enabling Ansible to find it by setting
action_plugins in [Link] or the ANSIBLE_ACTION_PLUGINS environment variable.
The API server ships with sane defaults, supports the notion of different environments (such as dev, staging, prod) and
allows you to customize the configuration with files, environment variables or a combination of both.
The API is a Django application that leverages django-rest-framework. Both Django and django-rest-framework have
extensive configuration options which are not necessarily exposed or made customizable by ARA for the sake of
simplicity.
1.5.1 Overview
This is a brief overview of the different configuration options for the API server. For more details, click on the
configuration parameters.
ARA_ALLOWED_HOSTS
ARA_BASE_DIR
ARA_CORS_ORIGIN_WHITELIST
dev:
CORS_ORIGIN_WHITELIST:
- [Link]
- [Link]
production:
CORS_ORIGIN_WHITELIST:
- [Link]
- [Link]
ARA_DATABASE_ENGINE
– [Link]
– [Link]
The Django database driver to use.
When using anything other than sqlite3 default driver, make sure to set the other database settings to allow the API
server to connect to the database.
ARA_DATABASE_NAME
ARA_DATABASE_USER
ARA_DATABASE_PASSWORD
ARA_DATABASE_HOST
ARA_DATABASE_PORT
ARA_DEBUG
ARA_ENV
# Default settings are used only when not provided in the environments
default:
READ_LOGIN_REQUIRED: false
WRITE_LOGIN_REQUIRED: false
LOG_LEVEL: INFO
DEBUG: false
# Increase verbosity and debugging for the default development environment
development:
LOG_LEVEL: DEBUG
DEBUG: true
SECRET_KEY: dev
# Enable write authentication when using the production environment
production:
WRITE_LOGIN_REQUIRED: true
SECRET_KEY: prod
With the example above, loading the development environment would yield the following settings:
• READ_LOGIN_REQUIRED: false
• WRITE_LOGIN_REQUIRED: false
• LOG_LEVEL: DEBUG
• DEBUG: true
• SECRET_KEY: dev
Another approach to environment-specific configuration is to use ARA_SETTINGS and keep your settings in different
files such as [Link] or [Link] instead.
Tip: If it does not exist, the API server will generate a default configuration file at ~/.ara/server/settings.
yaml. This generated file sets up all the configuration keys in the default environment. This lets users override only
the parameters they are interested in for specific environments.
ARA_LOGGING
LOGGING:
disable_existing_loggers: false
formatters:
normal:
format: '%(asctime)s %(levelname)s %(name)s: %(message)s'
handlers:
console:
class: [Link]
formatter: normal
level: INFO
stream: ext://[Link]
loggers:
(continues on next page)
ARA_LOG_LEVEL
ARA_SETTINGS
ARA_READ_LOGIN_REQUIRED
Determines if authentication is required before being authorized to query all API endpoints exposed by the server.
There is no concept of granularity: users either have access to query everything or they don’t.
Enabling this feature first requires setting up users.
ARA_SECRET_KEY
ARA_WRITE_LOGIN_REQUIRED
The API server ships with a default configuration that emphasizes simplicity to let users get started quickly.
By default:
• A random SECRET_KEY will be generated once if none are supplied
• No users are created
• API authentication and permissions are not enabled
• ALLOWED_HOSTS and CORS_ORIGIN_WHITELIST are configured for use on localhost
These default settings can be configured according to the requirements of your deployments.
By default, the API server randomly generates a token for the ARA_SECRET_KEY setting if none have been supplied
by the user.
This value is persisted in the server configuration file in order to prevent the key from changing on every instanciation
of the server.
The default location for the server configuration file is ~/.ara/server/[Link].
You can provide a custom secret key by supplying the ARA_SECRET_KEY environment variable or by specifying the
SECRET_KEY setting in your server configuration file.
The API server leverages Django’s user management but doesn’t create any user by default.
Note: Creating users does not enable authentication on the API. In order to make authentication required for using
the API, see Enabling authentication for read or write access.
In order to create users, you’ll need to create a superuser account before running the API server:
Tip: If you ever need to reset the password of a superuser account, this can be done with the “changepassword”
command:
Once the superuser has been created, make sure the API server is started and then login to the Django web adminis-
trative interface using the credentials you just set up.
By default, you can start the API server with ara-manage runserver and access the admin interface at http:/
/[Link]:8000/admin/.
Log in to the admin interface:
And from here, you can manage existing users or create new ones:
Once you have created your users, you can enable authentication against the API for read (ex: GET) and write (ex:
DELETE, POST, PATCH) requests.
This is done with the two following configuration options:
• ARA_READ_LOGIN_REQUIRED for read access
• ARA_WRITE_LOGIN_REQUIRED for write access
These settings are global and are effective for all API endpoints.
The callback plugin used to record playbooks as well as the ara_record action plugin will need to authenticate
against the API if authentication is enabled and required.
You can specify the necessary credentials through the ARA_API_USERNAME and ARA_API_PASSWORD environ-
ment variables or through your [Link] file:
[defaults]
# ...
[ara]
api_client = http
api_server = [Link]
api_username = ara
api_password = password
To instanciate an authenticated client with the built-in basic HTTP authentication provided by Django:
If you have a custom authentication that is supported by the python requests library, you can also pass the relevant
auth object directly to the client:
By default, ARA_ALLOWED_HOSTS authorizes localhost, ::1 and [Link] to serve requests for the API
server.
In order to host an instance of the API server on another domain, the domain must be part of this list or the application
server will deny any requests sent to it.
The ARA_CORS_ORIGIN_WHITELIST default is designed to allow a local development instance of an ara-web dash-
board to communicate with a local development instance of the API server.
The whitelist must contain the domain names where you plan on hosting instances of ara-web.
ARA ships with a helpful interface to navigate the API directly from your favorite web browser.
For example, if you run ara-manage runserver, this interface would be available at [Link]
1:8000/api/v1/:
You can navigate the interface and drill down to list views, for example:
You can also see what a detailed view looks like by querying a specific object id:
Alternatively, you may also find an up-to-date live demonstration of the API at [Link]
[Link].
The relationship between objects in the API should be straightforward with an understanding of how Ansible play-
books are executed.
Generally speaking, the data is organized in the following fashion:
labels
|
Playbook -> Play -> Task -> Result <- Host
| | | |
file file content facts
• Every object is associated to a playbook (except labels which are applied to playbooks)
• In a playbook you have plays
When installing ARA, you are provided with a REST API server and two API clients out of the box:
• AraOfflineClient can query the API without needing an API server to be running
• AraHttpClient is meant to query a specified API server over http
If your use case doesn’t require a remote or persistent API server, the offline client lets you query the API without
needing to start an API server.
In order to use this client, you would instanciate it like this:
#!/usr/bin/env python3
# Import the client
from [Link] import AraOfflineClient
AraHttpClient works with the same interface, methods and behavior as AraOfflineClient.
You can set your client to communicate with a remote ara-server API by specifying an endpoint parameter:
#!/usr/bin/env python3
# Import the client
from [Link] import AraHttpClient
# Instanciate the HTTP client with an endpoint where an API server is listening
client = AraHttpClient(endpoint="[Link]
#!/usr/bin/env python3
# Import the client
from [Link] import AraHttpClient
# Instanciate the HTTP client with an endpoint where an API server is listening
client = AraHttpClient(endpoint="[Link]
# If there are any results from our query, get more information about the
# failure and print something helpful
template = "{timestamp}: {host} failed '{task}' ({task_file}:{lineno})"
for playbook in playbooks["results"]:
# Get a detailed version of the playbook that provides additional context
detailed_playbook = [Link]("/api/v1/playbooks/%s" % playbook["id"])
Running this script would then provide an output that looks like the following:
ARA provides the ability for users to specify playbook names and labels in order to better distinguish playbooks run
in different environments or purposes.
Names and labels are also searchable by the ARA API, allowing you to find playbooks matching your query.
Names and labels are set as regular Ansible variables:
• ara_playbook_name
• ara_playbook_labels
These variables can be provided by your Ansible inventory, directly in your playbook file, as extra-vars or any other
way supported by Ansible.
For example, in an inventory:
[dev]
host1
host2
[dev:vars]
ara_playbook_name=deploy-dev
ara_playbook_labels='["deploy", "dev"]'
In a playbook:
Or as extra-vars:
It also supports different types of data which will have an impact on how a value might later be parsed or displayed:
It is possible to run an ara_record task on a specific playbook that might already be completed by specifying a
playbook. This is particularly useful for recording data that might only be available or computed after your playbook
run has been completed:
---
# Write data to a specific (previously run) playbook
- ara_record:
playbook: 14
key: logs
value: "{{ lookup('file', '/var/log/[Link]') }}"
type: text
Or as an ad-hoc command:
This data will be recorded inside ARA’s database and associated with the particular playbook run that was executed.
These records can later be retrieved through the API or through a web interface.
ARA Records Ansible is an open source community project and welcomes contributions, whether they are in the form
of feedback, comments, suggestions, bugs, documentation, code, or code reviews.
The ARA code review and CI infrastructure is hosted by [Link] which provides Gerrit for code review, Zuul for
CI/CD as well as many other systems.
All new patches are automatically tested with lint, unit and a variety of integration test scenarios. The end result is
higher standards, better code, more testing, less regressions and more stability.
OpenDev’s Gerrit instance currently uses Launchpad for authentication. If you do not already have a Launchpad
account, you will need to create one here.
If you’ll be contributing code or code reviews, you’ll need to set up your Gerrit code review account.
Once you have your Launchpad account, you will be able to sign in to [Link].
To be able to submit code, Gerrit needs to have your public SSH key in the same way Github does. To do that, click
on your name at the top right and go to the settings where you will see the tab to set up your SSH key.
Note that if the username from your local machine differs from the one in Gerrit, you might need to set it up in your
local ~/.ssh/config file like this:
Git Review is a python module that adds a “git review” command that wraps around the process of sending a commit
for review in Gerrit. You need to install it to be able to send patches for code reviews.
There are different ways to install git-review, choose your favorite.
When you send a commit for review, it’ll create a code review request in Gerrit for you. When that review is created,
it will automatically be tested by a variety of jobs that the ARA maintainers have set up to test every patch that is sent.
We’ll check for things like code quality (pep8/flake8), run unit tests to catch regressions and we’ll also run both
integration tests on different operating systems to make sure everything really works.
The result of the tests are added as a comment in the review when all of them are completed. If you’re interested in
digging into the logs for a particular test, clicking on the results of the test will take you to console, debug logs and a
built version of ARA’s web interface.
If you get a failed test result and you believe you have fixed the issue, add the files, amend your commit (git commit
--amend) and send it for review once again. This will create a new patchset that will be up for review and testing.
To be able to merge a patch, the tests have to come back successful and the core reviewers must provide their agreement
with the patch.
1.12 ansible-role-ara-api
This Ansible role provides a framework for installing one or many instances of ARA Records Ansible in a variety of
opinionated deployment topologies.
It is currently tested and supported against Ubuntu 18.04 and Fedora 29.
See defaults/[Link].
# Root directory where every file for the ARA installation are located
ara_api_root_dir: "{{ ansible_user_dir }}/.ara"
# When installing from source, the URL or filesystem path where the git source
# repository can be cloned from.
ara_api_source: "[Link]
# When installing from source, location where the source repository will be checked
˓→out to.
# - gunicorn: gunicorn will be installed and set up to run the API as a systemd
˓→service.
# - mod_wsgi [planned]
ara_api_wsgi_server: null
####################################
# ara API configuration settings
# For more information, see documentation: [Link]
####################################
1.12. ansible-role-ara-api 31
ara Documentation, Release 1.0.0
ara_api_cors_origin_allow_all: false
1.12.2 TL;DR
- name: Install ARA with default settings and no persistent API server
hosts: all
gather_facts: yes
roles:
- ara_api
This Ansible role is designed to support different opinionated topologies that can be selected with role variables.
For example, the following role variables are used to provide the topology from the TL;DR above:
• ara_api_install_method: source
• ara_api_wsgi_server: null
• ara_api_database_engine: [Link].sqlite3
• ara_api_web_server: null
The intent is that as the role gains support for other install methods, wsgi servers, database engines or web servers, it
will be possible to mix and match according to preference or requirements.
Perhaps ARA could be installed from pypi and run with gunicorn, nginx and mysql. Or maybe it could be installed
from distribution packages and set up to run with apache, mod_wsgi and postgresql. Or any combination of any of
those.
Install ARA and set up the API to be served by a persistent gunicorn service:
1.12. ansible-role-ara-api 33
ara Documentation, Release 1.0.0
Install ARA and set up the API to be served by nginx in front of gunicorn:
1.13 ansible-role-ara-web
This Ansible role provides a framework for installing one or many instances of ara-web in a variety of opinionated
deployment topologies.
It is currently tested and supported against Ubuntu 18.04 and Fedora 29.
See defaults/[Link].
# When using static builds without the dev server, path to ara-web static assets
ara_web_static_dir: "{{ ara_web_root_dir }}/www/ara-web"
# When installing from source, the location of the remote or local git repository
ara_web_source: "[Link]
# When using a frontend server, you can override the default vhost configuration
# template by specifying the path to your own template file.
ara_web_frontend_vhost: null
1.13.2 TL;DR
1.13. ansible-role-ara-web 35
ara Documentation, Release 1.0.0
This Ansible role is designed to support different opinionated topologies that can be selected with role variables.
For example, the following role variables are defaults used to provide the topology from the TL;DR above:
• ara_web_install_method: source
• ara_web_dev_server: true
• ara_web_frontend_server: null
The intent is that as the role gains support for other install methods or frontend servers, it will be possible to mix and
match according to preference or requirements.
Deploy the ARA API and web client on the same machine with defaults:
Deploy only ara-web behind nginx and point it to a remote API endpoint:
# Note: Don't forget to add the web fqdn in the remote cors_origin_whitelist.
# Otherwise, the web client might not be authorized to query the API.
- name: Deploy ara-web for remote API endpoint
hosts: all
gather_facts: yes
vars:
ara_web_fqdn: [Link]
ara_web_api_endpoint: "[Link]
ara_web_frontend_server: nginx
ara_web_frontend_vhost: [Link].j2
roles:
- ara_web
1.14 ansible-role-ara-tests
An Ansible role that installs ARA and Ansible to run quick and inexpensive tests that do not require superuser privi-
leges.
See defaults/[Link].
# Root directory where integration tests will prepare and store data
ara_tests_root: "/tmp/ara-integration-tests"
1.14.2 TL;DR
1.14. ansible-role-ara-tests 37