mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 00:09:35 +00:00
Valeriy V. Vorotyntsev
26448a9820
``` ❌ Functions coverage is below threshold: 62.95% < 63.0% ✅ Lines coverage: 55.87% ✅ Regions coverage: 43.13% [Container] 2023/05/09 09:07:40 Command did not exit successfully ./coverage.sh exit status 1 [Container] 2023/05/09 09:07:40 Phase complete: BUILD State: FAILED [Container] 2023/05/09 09:07:40 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: ./coverage.sh. Reason: exit status 1 ``` As I wrote on Slack: > ❓ How useful is coverage.sh, particularly ‘Functions coverage’? > > I don't know how `cargo llvm-cov` calculates this ratio. Probably it divides the number of tested functions by the total number of functions. > > Splitting big functions into smaller pieces improves code readability. It also increases the total number of functions. If my assumption about `cargo llvm-cov` algorithm is correct, splitting code into smaller functions decreases the ‘Functions coverage’ ratio. > > If you look at [the patch] that fails the CI due to insufficient ‘Functions coverage’, you'll see that the new code there is accompanied with just-about-right amount of unit tests. I mean, apart from `MetricType`-to-string conversion, what else is there to be tested? How useful would those tests be? > > IMO, ‘Functions coverage’ target of `coverage.sh` adds more friction than benefit. > > I propose that we don't let `coverage.sh` fail the CI. [the patch]: https://github.com/zinclabs/zincobserve/pull/695/files
113 lines
2.5 KiB
Bash
Executable File
113 lines
2.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
set -eu -o pipefail
|
|
# set -x
|
|
export PS4='+ [${BASH_SOURCE[0]##*/}:${LINENO}${FUNCNAME[0]:+:${FUNCNAME[0]}}] '
|
|
|
|
export COVERAGE_FUNCTIONS=${COVERAGE_FUNCTIONS:-63}
|
|
export COVERAGE_LINES=${COVERAGE_LINES:-55}
|
|
export COVERAGE_REGIONS=${COVERAGE_REGIONS:-43}
|
|
|
|
usage() {
|
|
cat <<EOF
|
|
Usage: $0 [-h | --help] [<command>]
|
|
|
|
Commands:
|
|
check [TESTNAME] Run 'cargo llvm-cov test' and verify that code coverage statistics
|
|
are greater than or equal to the thresholds. Test name filtering
|
|
is supported.
|
|
help Show this help and exit.
|
|
html Generate coverage report in HTML format and open it in the default
|
|
browser.
|
|
show-env Show the coverage thresholds.
|
|
|
|
Omitted <command> defaults to 'check'.
|
|
EOF
|
|
}
|
|
|
|
_cov_test() {
|
|
cargo llvm-cov --version >/dev/null || cargo install cargo-llvm-cov
|
|
|
|
RUSTFLAGS='-C target-cpu=native' \
|
|
cargo llvm-cov test \
|
|
--verbose \
|
|
--ignore-filename-regex job \
|
|
--ignore-run-fail \
|
|
"$@"
|
|
}
|
|
|
|
cmd_html() {
|
|
_cov_test --html "$@"
|
|
open target/llvm-cov/html/index.html # HACK: `open` is not portable
|
|
}
|
|
|
|
cmd_show_env() {
|
|
cat <<EOF
|
|
COVERAGE_FUNCTIONS=$COVERAGE_FUNCTIONS
|
|
COVERAGE_LINES=$COVERAGE_LINES
|
|
COVERAGE_REGIONS=$COVERAGE_REGIONS
|
|
EOF
|
|
}
|
|
|
|
cmd_check() {
|
|
_cov_test --json --summary-only --output-path report.json "$@"
|
|
|
|
python3 <(
|
|
cat <<'EOF'
|
|
import json
|
|
import os
|
|
import sys
|
|
|
|
thresholds = {
|
|
'functions': float(os.environ['COVERAGE_FUNCTIONS']),
|
|
'lines': float(os.environ['COVERAGE_LINES']),
|
|
'regions': float(os.environ['COVERAGE_REGIONS']),
|
|
}
|
|
|
|
with open('report.json') as f:
|
|
report = json.load(f)
|
|
|
|
totals = report['data'][0]['totals']
|
|
|
|
for k, threshold in thresholds.items():
|
|
actual = totals[k]['percent']
|
|
k = k.capitalize()
|
|
if actual >= threshold:
|
|
print(f'✅ {k} coverage: {actual:.2f}%', file=sys.stderr)
|
|
else:
|
|
print(
|
|
f'❌ {k} coverage is below threshold: {actual:.2f}% < {threshold}%',
|
|
file=sys.stderr,
|
|
)
|
|
EOF
|
|
)
|
|
}
|
|
|
|
main() {
|
|
case "${1:-}" in
|
|
'')
|
|
cmd_check
|
|
;;
|
|
check)
|
|
shift
|
|
cmd_check "$@"
|
|
;;
|
|
html)
|
|
shift
|
|
cmd_html "$@"
|
|
;;
|
|
show-env)
|
|
cmd_show_env
|
|
;;
|
|
-h|--help|help)
|
|
usage
|
|
;;
|
|
*)
|
|
echo >&2 "Invalid argument: $1"
|
|
echo >&2 "Type '$0 --help' for usage"
|
|
exit 1
|
|
;;
|
|
esac
|
|
}
|
|
|
|
main "$@"
|