Rally v0.11.0
Overview
--------
+------------------+-----------------------+
| Release date | **02/16/2018** |
+------------------+-----------------------+
* Stabilize Rally releases (see requirements section)
* Publishing docker images is returned!
* Environment introduction (see Deployment section)
Details
-------
Requirements
~~~~~~~~~~~~
As for long time we tried to make our releases stable and workable even after
a year from release date. For this purpose, upper limits for all our
requirements were used. Such solution helped to make releases more
stable, but it did not work well and created more pain than a profit.
The main issue was related to new releases of not-direct rally
dependencies which can be incompatible which each other.
From Rally 0.11.0 we are stopping adding upper version limits for our
`requirements
<https://github.com/openstack/rally/blob/0.11.0/requirements.txt>`_ (this does
not apply to cases when we sure that some new releases of dependency broke us
for sure).
As alternative solution, the `upper-constraints file
<https://github.com/openstack/rally/blob/0.11.0/upper-constraints.txt>`_ is
selected. It is a file with a list of packages and their versions which we used
in our CI while testing. Versions from this list are suggested to use in
production.
Please note, that it also contains not direct Rally dependencies (dependencies
of rally dependencies and dependencies of dependencies of rally dependencies
as well) and packages which possibly doesn't relate to Rally at all.
The example ou usage:
.. console:: bash
$ git clone https://github.com/openstack/rally && cd rally
$ pip install . --constraint upper-constraints.txt
Logging
~~~~~~~
The default value of ``use_stderr`` option of Rally config is changed to
**True**. It is done to ensure that json output of rally commands will not be
messed with logging and integration with third-party tools and scripts should
become simpler.
Docker
~~~~~~
Unfortunately, we lost access to `rallyforge organization at DockerHub
<https://hub.docker.com/r/rallyforge/>`_, so our docker images were not
published anywhere officially for some time. Docker Support did not help us a
lot. At least, the original repo is removed now and we can start new
organization at DockerHub from the scratch.
The new docker images for Rally+OpenStack plugins with an updated HowTo you
can find at `xrally/xrally-openstack repo
<https://hub.docker.com/r/xrally/xrally-openstack>`_. It contains all Rally
releases + latest tag which maps to master branch.
Command Line Interface
~~~~~~~~~~~~~~~~~~~~~~
* Introduce `rally db ensure
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#rally-db-ensure>`_
command. It is going to create Rally DB if it doesn't exist, otherwise it
does nothing.
* Various improvements for `rally db
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#category-db>`_.
Such as printing results of operations as well as connection string to DB,
hiding credentials by default, etc.
* Various changes in returning error codes. Error codes become different for
different exceptions. Also, `rally task start
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#rally-task-start>`_
began to return 1 in case of any runtime issues and 2 if any workload doesn't
pass it's SLA.
* The new CLI for new component (see ``Environments & Deployments`` section
for more details) - `rally env
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#category-env>`_
Environments & Deployments
~~~~~~~~~~~~~~~~~~~~~~~~~~
Deployment Component is fundamental part of Rally which is used by Task and
Verification components. Whereas Task and Verification components experienced
redesigns (some parts were redesigned even several times), Deployment component
was only extended over the time.
Currently, we are at the point when further extending requires much work and
user experience become worth. It is a hard decision, as like we had done with
Verification component in Rally 0.8, the Deployment component is re-written
from the scratch. To be clear, the new version of the component has not only
the new code, but the new name as well - it is the Environment.
We are at the stage when Rally is suitable not only for OpenStack but for
various platforms and systems. The Environment is some entity against which
Rally should perform load. The specific Environment can include credentials for
OpenStack or for Kubernetes or for both. The Environment is a way to describe
the complex system/cloud which all of us have. As well it can be used for
simple systems as easy as for complex.
If you are regular Rally user which tests OpenStack clusters, nothing is
changing for you at this moment. `rally deployment
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#category-deployment>`_
CLI still works and it is not even deprecated. It will be kept for a while.
But you can start looking at our new CLI `rally env
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#category-env>`_
.
As for writing plugins for external platforms - we are working on updating our
documentary.
Task component
~~~~~~~~~~~~~~
* The json results are extended with context execution stats (i.e when and
which context was executed, how much time setup and cleanup methods took,
etc). Also, `volumes@openstack
<https://rally.readthedocs.io/en/0.11.0/plugins/plugin_reference.html#volumes-context>`_
and
`volume_types@openstack
<https://rally.readthedocs.io/en/0.11.0/plugins/plugin_reference.html#volume-types-context>`_
are ported to include detailed stats of executed actions. In further
releases, all existing contexts will be adopted to the similar behavior.
* Better OSProfiler integration
Rally environment&deployment config began accept
``profiler_conn_str`` property which is used to generate OSProfiler native
html-report and to embed it to Rally's html-report.
* HTML report is extended to include a timestamp of failures.
Plugins
~~~~~~~
As it mentioned above, the Deployment Component will be replaced soon. Such
decision led to abandoning all deployment-related plugins (engines, providers,
etc).
**Scenarios**:
* *NEW!!*
`NovaServers.boot_server_attach_volume_and_list_attachments
<https://rally.readthedocs.io/en/0.10.0/plugins/plugin_reference.html#novaservers-boot-server-attach-volume-and-list-attachments-scenario>`_
* *UPDATED!!*
- Extend several Nova&Neutron related scenarios with
``create_floating_ip_args`` parameter
`NovaServers.boot_and_associate_floating_ip
<https://rally.readthedocs.io/en/0.11.0/plugins/plugin_reference.html#novaservers-boot-and-associate-floating-ip-scenario>`_
`NovaServers.boot_server_associate_and_dissociate_floating_ip
<https://rally.readthedocs.io/en/0.11.0/plugins/plugin_reference.html#novaservers-boot-server-associate-and-dissociate-floating-ip-scenario>`_
- Modify Bgpvpn scenarios to test true bgpvpn
All Bgpvpn scenarios began to boot a server to add active port in the
network associated to the bgpvpn which allows to test not only the record in
the database, but true bgpvpn
**Contexts**:
*UPDATED!!*
`network@openstack
<https://rally.readthedocs.io/en/latest/plugins/plugin_reference.html#network-context>`_
context is extended with ability to specify external router information.
Fixed bugs
~~~~~~~~~~
* [backported into 0.10.1][deployment] Suppress deprecation warning about an
old format in case of using `--fromenv option of rally deployment create
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#deployment-create-fromenv>`_
* [backported into 0.10.1][deployment] Failure `rally deployment show
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#deployment-show>`_
while displaying the information about deployment with a config in an old
format.
* [backported into 0.10.1][task] New json report processed the hook results in
a wrong way
`Launchpad bug-report #1734336
<https://bugs.launchpad.net/rally/+bug/1734336>`_
* [backported into 0.10.1][task] Failure while generating trends reports in
case of failures in setup method of any context
`Launchpad bug-report #1732193
<https://bugs.launchpad.net/rally/+bug/1732193>`_
* [backported into 0.10.1][task] Failure to export results in ElasticSearch 5.x
cluster in case of extra ``/`` in the end of destination url.
* [deployment] OpenStack deployment creation with `--fromenv
<https://rally.readthedocs.io/en/0.11.0/cli_reference.html#deployment-create-fromenv>`_
option used old deprecated format.
* [verify] Rally did not support creating verifiers from Gerrit/SSH source.
`Launchpad bug-report #1737529
<https://bugs.launchpad.net/rally/+bug/1737529>`_
* [task][openstack] Removing default security group in users@openstack context
did not take into account that neutron can return multiple resources in some
configuration instead of one security group which relates to requested
tenant.
* [task][openstack] Existing openstack users get their roles un-assigned if
they are equal to what roles@openstack context is configured to assign.
`Launchpad bug-report #1720270
<https://bugs.launchpad.net/rally/+bug/1720270>`_
* [task][openstack] Validation step ignores roles@openstack context and
marks as "invalid" valid cases
Some actions in openstack can be performed only if the user has specific
role. Since these roles can be different in different OpenStack installations
Rally has `roles@openstack context
<https://rally.readthedocs.io/en/0.11.0/plugins/plugin_reference.html#roles-context>`_
context which can assign roles to the users.
Validation step did not check for specified roles in workload config and made
wrong assumption about accessibility of resources
`Launchpad bug-report #1539878
<https://bugs.launchpad.net/rally/+bug/1539878>`_
* [task][openstack] Wrong identifiers were used for filtering Mistral resources
while cleanup step.
* [task][openstack] `NovaServers.boot_and_live_migrate_server
<https://rally.readthedocs.io/en/0.11.0/plugins/plugin_reference.html#novaservers-boot-and-live-migrate-server-scenario>`_
does wrong target host selection
`Launchpad bug-report #1734914
<https://bugs.launchpad.net/rally/+bug/1734914>`_
Thanks
~~~~~~
2 Everybody!