Attempt to make codecov less important

[kingbuzzman]

Depends on #1202
Related to #1201 -- had to close it due to CI not running

This all started because yesterday codecov was having issues, and p!$#ed me off...

This PR aims to reduce the emphasis on Codecov by adjusting coverage exclusions in pyproject.toml and modifying the CI workflow to handle coverage reporting differently.

• Update pyproject.toml by excluding "pytest_django/_version.py" in both run and report configurations.
• Revise the CI workflow to use ubuntu-latest, update Tox versions, and introduce dedicated coverage reporting and artifact handling steps.

Miscellaneous:

• CI now runs on all branches -- instead of those that were pointing to main only
• Updated tox version since python 3.8 is gone!
• Added html and the main coverage file to the archive
• Adds markdown details of coverage in the CI summary
• Moves the CI matrix up -- personal thing, i can move it back if its too many changes

[kingbuzzman]

@webknjaz

I, however, like how Codecov can produce multiple checks aggregating different aspects of coverage..

can you be specific about what it is you like about it?

as far as i’m concerned this pr hasn’t touched anything in codecov that you didn’t have before. i am combining all the coverages into a single one and uploading that, the benefit here is that i can keep all the context data (which codecov throws away) and generate a html artifact. all of the existing functionality of codecov should remain in place

——

@bluetech

If I understand correctly, the reason for doing this is that sometimes codecov have issues, and this provides a sort of "DIY" alternative to codecov?

However, if we are PRs still require passing codecov, doesn't it mean that codecov problems are still blocking?

lol yes, the name of the pr is “ Attempt to make codecov less important” not “removes codecov” 😇

[webknjaz]

I, however, like how Codecov can produce multiple checks aggregating different aspects of coverage..

can you be specific about what it is you like about it?

Having it apply flags for different runtimes et al. I was mostly pointing out that it's useful to have this combined thing as long as Codecov integration isn't compromised.
In general, I have a set of practices I deploy to make Codecov more predictable/responsive. This is largely about configuration. Additionally, pinning things is helpful. The official Codecov action downloads an uploader defaulting to latest instead of a specific version, for example. If pinned, it can contribute to overall stability. Two other things I typically do is having a plain-text token in the config (it's not really that important to have it secret) and replacing Codecov's polling for whether the CI completed and attempting to report statuses too early with a number of reports to wait for, which would match the number of uploading jobs.

[webknjaz]

FWIW, it's good to still upload separate reports from jobs because they get different flags/contexts assigned that are usable in the web UI, similar to the contexts that coveragepy can include.

[kingbuzzman]

@webknjaz

This is is the codecov report for this PR
This is main

I see extra context on the right in main, but seems a bit useless. what am i missing?

[kingbuzzman]

@webknjaz what am i missing from the codecov report, the summed up report vs individual ^ (see comment above)