tests are based on self-hosted runner which is physically close
to our staging deployment in aws, currently tests consist of
various configurations of pgbenchi runs.
Also these changes rework benchmark fixture by removing globals and
allowing to collect reports with desired metrics and dump them to json
for further analysis. This is also applicable to usual performance tests
which use local zenith binaries.
* Use logging in python tests
* Use f-strings for logs
* Don't log test output while running
* Use only pytest logging handler
* Add more info about pytest logging
It's quite hard to get python2 to exit gracefully when the code was
intended for python3, because the interpreter will SyntaxError before
running a single line of code. Thankfully, the pytest developers put a
version check in their .ini config, so that should gracefully handle
both wrong-pytest-version and wrong-python-version.
Also document the woes of trying to run the pytest version shipped by
e.g. Debian or Ubuntu.