kettle-soup-cover

🥘 Kettle::Soup::Cover

Four lines of code to get a configured, curated, opinionated, set of dependencies for Test Coverage, and that’s including the two lines for require "simplecov", and SimpleCov.start.

Configured for what? To work out of the box on every CI*. Batteries included. For apps and libraries. Any test framework. Many code coverage related GitHub Actions (example configs 1, 2).

Test Framework	Helper	Config
MiniTest	test helper	.simplecov
RSpec	spec helper	.simplecov

Federated DVCS Repository	Status	Issues	PRs	Wiki	CI	Discussions
🧪 kettle-rb/kettle-soup-cover on GitLab	The Truth	💚	💚	💚	🏀 Tiny Matrix	➖
🧊 kettle-rb/kettle-soup-cover on CodeBerg	An Ethical Mirror (Donate)	➖	💚	➖	⭕️ No Matrix	➖
🐙 kettle-rb/kettle-soup-cover on GitHub	A Dirty Mirror	💚	💚	➖	💯 Full Matrix	💚
🎮️ Discord Server		Let’s	talk	about	this	library!

One of the major benefits of using this library is not having to figure out how to get multiple coverage output formats working. I did that for you, and I got all of them working, at the same time together, or al la carte. Kum-ba-ya.

A quick shot of 12-factor coverage power, straight to your brain:

export K_SOUP_COV_DO=true # Means you want code coverage
export K_SOUP_COV_FORMATTERS="html,tty" # Set to some slice of "html,xml,rcov,lcov,json,tty"
export K_SOUP_COV_MIN_BRANCH=53 # Means you want to enforce X% branch coverage
export K_SOUP_COV_MIN_HARD=true # Means you want the build to fail if the coverage thresholds are not met
export K_SOUP_COV_MIN_LINE=69 # Means you want to enforce X% line coverage
export MAX_ROWS=5 # Setting for simplecov-console gem for tty output, limits to the worst N rows of bad coverage

I hope I’ve piqued your interest enough to give it a ⭐️ if the forge you are on supports it.

What does the name mean?

A Covered Kettle of SOUP (Software of Unknown Provenance) The name is derived in part from the medical devices field, where this library is considered a package of [SOUP](https://en.wikipedia.org/wiki/Software_of_unknown_pedigree).

Format / Library x CI Matrix

This tool leverages other tools to make hard things easier, but sometimes those other tools break… I’ll try to track that here.

Format	Library	Status	Web	Circle CI	Git Lab	Travis CI	Jenkins X	Jenkins	Hudson	Semaphore	Bit Bucket	Team City	🤓 Nerds
`html`	`simplecov-html`	✅	✅										✅
`xml`	`simplecov-cobertura`	⚠️ works (with warnings); upvote #30!			✅			✅					✅
`rcov`	`simplecov-rcov`	✅							✅				✅
`lcov`	`simplecov-lcov`	✅		✅		✅	✅			✅		✅	✅
`json`	`simplecov_json_formatter`	✅		✅		✅	✅				✅		✅
`tty`	`simplecov-console`	✅											✅

If you find this working/not working different than above please open an issue / PR!

☝️ Not actually every CI

This gem does not add coverage parsing to CI’s that don’t have it, since that’s impossible. Vendor-specific formats which are not shared by other vendors are also not supported (e.g. BuildKite).

You’ll have to configure them manually if you use them:

BuildKite’s custom simplecov extension
GitHub Actions doesn’t parse test output, but…
- I configure my coverage workflow (see example) to upload coverage reports to SaaS services like:
  - codecov.io (needs token for upload)
  - QLTY.sh (needs token for upload)
  - coveralls.io
- This gem helps me configure my coverage workflow to use Github Actions designed to report coverage like:
  - Repo: irongut/CodeCoverageSummary
  - Repo: marocchino/sticky-pull-request-comment

This library is based on ideas I originally introduced in the gem rspec-stubbed_env.

💡 Info you can shake a stick at

Tokens to Remember
Works with MRI Ruby 3
Works with MRI Ruby 2
Source
Documentation
Compliance
Style
Support
Enterprise Support	💡Subscribe for support guarantees covering all FLOSS dependencies! 💡Tidelift is part of Sonar! 💡Tidelift pays maintainers to maintain the software you depend on! 📊`@`Pointy Haired Boss: An enterprise support subscription is “never gonna let you down”, and supports open source maintainers!
Comrade BDFL 🎖️
`...` 💖	🧊 🐙 🛖 🧪

✨ Installation

Install the gem and add to the application’s Gemfile by executing:

$ bundle add kettle-soup-cover

If bundler is not being used to manage dependencies, install the gem by executing:

$ gem install kettle-soup-cover

🔒 Secure Installation

kettle-soup-cover is cryptographically signed, and has verifiable SHA-256 and SHA-512 checksums by stone_checksums. Be sure the gem you install hasn’t been tampered with by following the instructions below.

Add my public key (if you haven’t already, expires 2045-04-29) as a trusted certificate:

gem cert --add <(curl -Ls https://raw.github.com/kettle-rb/kettle-soup-cover/main/certs/pboling.pem)

You only need to do that once. Then proceed to install with:

gem install kettle-soup-cover -P MediumSecurity

The MediumSecurity trust profile will verify signed gems, but allow the installation of unsigned dependencies.

This is necessary because not all of kettle-soup-cover’s dependencies are signed, so we cannot use HighSecurity.

If you want to up your security game full-time:

bundle config set --global trust-policy MediumSecurity

NOTE: Be prepared to track down certs for signed gems and add them the same way you added mine.

🔧 Basic Usage

RSpec or MiniTest

In your spec/spec_helper.rb or tests/test_helper.rb, just before loading the library under test, add two lines of code:

With Ruby 2.7+

require "kettle-soup-cover"
require "simplecov" if Kettle::Soup::Cover::DO_COV # `.simplecov` is run here!
# IMPORTANT: If you are using MiniTest instead of RSpec, also do this (and not in .simplecov):
# SimpleCov.external_at_exit = true

Projects that run tests against older Ruby versions, e.g. with Appraisals

# NOTE: Gemfiles for older rubies won't have kettle-soup-cover.
#       The rescue LoadError handles that scenario.
begin
  require "kettle-soup-cover"

  if Kettle::Soup::Cover::DO_COV
    require "simplecov" # `.simplecov` is run here!

    # IMPORTANT: If you are using MiniTest instead of RSpec, also do this (and not in .simplecov):
    # SimpleCov.external_at_exit = true
  end
rescue LoadError => error
  # check the error message, if you are so inclined, and re-raise if not what is expected
  raise error unless error.message.include?("kettle")
end

All projects

In your .simplecov file, add 2 lines of code:

require "kettle/soup/cover/config" # 12-factor, ENV-based configuration, with good defaults!
# you could do this somewhere else, up to you, but you do have to do it somewhere
SimpleCov.start

See Advanced Usage below for more info, but the simplest thing is to run all the coverage things, which is configured by default on CI. To replicate that locally you could:

CI=true bundle exec rake test # or whatever command you run for tests.

That’s it!

Rakefile

You’ll need to have your test task defined. If you use spec instead, you can make it a pre-requisite of the test task with:

desc "run spec task with test task"
task test: :spec

This gem provides a coverage task. It runs the test task (see just above about that), and opens the coverage results in a browser.

require "kettle-soup-cover"
Kettle::Soup::Cover.install_tasks

Filters

There are two built-in SimpleCov filters which can be loaded via Kettle::Soup::Cover.load_filters.

You could use them like this:

SimpleCov.add_group("Too Long", Kettle::Soup::Cover::Filters::GtLineFilter.new(1000))

Advanced Usage

There are a number of ENV variables that control things within this gem. All of them can be found, along with their default values, in lib/kettle/soup/cover.rb.

Handy List of ENV Variables

Most are self explanatory. I tried to follow POLS, the principle of least surprise, so they mostly DWTFYT. Want to help improve this documentation? PRs are easy!

K_SOUP_COV_COMMAND_NAME
K_SOUP_COV_DEBUG
K_SOUP_COV_DIR
K_SOUP_COV_DO
K_SOUP_COV_FILTER_DIRS
K_SOUP_COV_FORMATTERS
K_SOUP_COV_MERGE_TIMEOUT
K_SOUP_COV_MIN_BRANCH
K_SOUP_COV_MIN_HARD
K_SOUP_COV_MIN_LINE
K_SOUP_COV_MULTI_FORMATTERS
K_SOUP_COV_PREFIX
K_SOUP_COV_OPEN_BIN
K_SOUP_COV_USE_MERGING
K_SOUP_COV_VERBOSE

Additionally, some of the included gems, like simplecov-console, have their own complete suite of ENV variables you can configure.

Compatible with GitHub Actions for Code Coverage feedback in pull requests

If you don’t want to configure a SaaS service to update your pull requests with code coverage there are alternatives.

After the step that runs your test suite use one or more of the following.

irongut/CodeCoverageSummary

Repo: irongut/CodeCoverageSummary

      - name: Code Coverage Summary Report
        uses: irongut/CodeCoverageSummary@v1.3.0
        if: $
        with:
          filename: ./coverage/coverage.xml
          badge: true
          fail_below_min: true
          format: markdown
          hide_branch_rate: false
          hide_complexity: true
          indicators: true
          output: both
          thresholds: '100 100' # '<MIN LINE COVERAGE> <MIN BRANCH COVERAGE>'
        continue-on-error: $

marocchino/sticky-pull-request-comment

Repo: marocchino/sticky-pull-request-comment

      - name: Add Coverage PR Comment
        uses: marocchino/sticky-pull-request-comment@v2
        if: $
        with:
          recreate: true
          path: code-coverage-results.md
        continue-on-error: $

🚀 Release Instructions

See CONTRIBUTING.md.

🔐 Security

See SECURITY.md.

🤝 Contributing

If you need some ideas of where to help, you could work on adding more code coverage, or if it is already 💯 (see below) check issues, or PRs, or use the gem and think about how it could be better.

We so if you make changes, remember to update it.

See CONTRIBUTING.md for more detailed instructions.

Code Coverage

🪇 Code of Conduct

Everyone interacting in this project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the .

🌈 Contributors

Made with contributors-img.

Also see GitLab Contributors: https://gitlab.com/kettle-rb/kettle-soup-cover/-/graphs/main

⭐️ Star History

</a>

📌 Versioning

This Library adheres to . Violations of this scheme should be reported as bugs. Specifically, if a minor or patch version is released that breaks backward compatibility, a new version should be immediately released that restores compatibility. Breaking changes to the public API will only be introduced with new major versions.

📌 Is “Platform Support” part of the public API?

Yes. But I’m obligated to include notes…

SemVer should, but doesn’t explicitly, say that dropping support for specific Platforms is a breaking change to an API. It is obvious to many, but not all, and since the spec is silent, the bike shedding is endless.

dropping support for a platform is both obviously and objectively a breaking change

Jordan Harband (@ljharb, maintainer of SemVer) in SemVer issue 716

To get a better understanding of how SemVer is intended to work over a project’s lifetime, read this article from the creator of SemVer:

“Major Version Numbers are Not Sacred”

As a result of this policy, and the interpretive lens used by the maintainer, you can (and should) specify a dependency on these libraries using the Pessimistic Version Constraint with two digits of precision.

For example:

spec.add_dependency("kettle-soup-cover", "~> 1.0")

See CHANGELOG.md for list of releases.

📄 License

The gem is available as open source under the terms of the MIT License . See LICENSE.txt for the official Copyright Notice.

© Copyright

🤑 One more thing

You made it to the bottom of the page, so perhaps you’ll indulge me for another 20 seconds. I maintain many dozens of gems, including this one, because I want Ruby to be a great place for people to solve problems, big and small. Please consider supporting my efforts via the giant yellow link below, or one of the others at the head of this README.

This site is open source. Improve this page.