Skip to content

chore(performance-issues): Create experimental N+1 API Call detector #90070

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 24, 2025

Conversation

leeandher
Copy link
Member

@leeandher leeandher commented Apr 22, 2025

For context, I'm auditing the current performance issue detectors and will likely be making some changes to them. Risky changes like touching fingerprinting should probably run silently beforehand to ensure we catch false positives or fingerprint explosion (see guidance).

In this PR, I'm creating a structure to make these experiments easier to review and colocated since I'll be working on different detectors in parallel.

Going forward, my workflow will probably be the following:

  1. Copy the detector and test files into /experiments and set up a dummy group type. This copy will be identical to the current detector. (1st PR, This one)
  2. Apply experimental changes in a follow up PR (2nd PR, making it easier to review)
  3. Add the options via sentry createissueflag (docs) to run in prod (3rd PR, in options-automator)
  4. If needed, follow up with a fix/change to the experimental detector. Repeat as needed
  5. When happy with the output, apply changes to the real detector and delete experiment files.

The reason I'm being so formal about this all is because I'll be parallelizing these experiments so I gotta keep this straight for myself as well 👍

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 22, 2025
Copy link
Member Author

@leeandher leeandher Apr 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy of the current test file with small edits for the experimental group type

Copy link
Member Author

@leeandher leeandher Apr 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy of the current detector file with small edits for the experimental group type

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi: Format on save did this

Copy link

codecov bot commented Apr 22, 2025

Codecov Report

Attention: Patch coverage is 96.78571% with 9 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...ctors/experiments/n_plus_one_api_calls_detector.py 93.87% 9 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           master   #90070    +/-   ##
========================================
  Coverage   87.72%   87.72%            
========================================
  Files       10247    10248     +1     
  Lines      578341   578601   +260     
  Branches    22693    22691     -2     
========================================
+ Hits       507347   507606   +259     
- Misses      70551    70552     +1     
  Partials      443      443            

"""

__slots__ = ["stored_problems"]
type = DetectorType.N_PLUS_ONE_API_CALLS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to create new DetectorType for each of the experiments? or is that just adding bloat that we will need to delete later once we replace the original ?

Copy link
Member Author

@leeandher leeandher Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! we don't need to for this case because the DetectorType is only used for settings (the ones in project > performance > [Detector Name]). It'd make sense to make a new one if we wanted to trial any new values or defaults, but we don't have to in this case.

@leeandher leeandher requested a review from roggenkemper April 23, 2025 19:32
Copy link
Member

@roggenkemper roggenkemper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for putting the effort into detailing how we are going to trial these experiments

@leeandher leeandher merged commit 1349128 into master Apr 24, 2025
64 checks passed
@leeandher leeandher deleted the leander/setup-npo-api-calls-experiment branch April 24, 2025 14:42
@asottile-sentry asottile-sentry added the Trigger: Revert Add to a merged PR to revert it (skips CI) label Apr 24, 2025
@asottile-sentry
Copy link
Member

reverting as this is breaking CI

@getsentry-bot
Copy link
Contributor

PR reverted: 657d08b

getsentry-bot added a commit that referenced this pull request Apr 24, 2025
…etector (#90070)"

This reverts commit 1349128.

Co-authored-by: asottile-sentry <103459774+asottile-sentry@users.noreply.github.com>
leeandher added a commit that referenced this pull request Apr 24, 2025
leeandher added a commit that referenced this pull request Apr 24, 2025
…alls detector (#90295)

Original PR: #90070 - CI was out of date.

This reverts commit 657d08b.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Scope: Backend Automatically applied to PRs that change backend components Trigger: Revert Add to a merged PR to revert it (skips CI)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants