Skip to content

Seperate out readme experiments from chart generation #328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 31, 2024

Conversation

scotts
Copy link
Contributor

@scotts scotts commented Oct 31, 2024

Makes the following changes:

  1. Separates the running of the experiments from the generation of the pretty graph we show to users. We want to do this so we can tweak the presentation without having to re-run experiments. We achieve this by:
    a. Splitting the current script into two: generate_readme_data.py and generate_readme_chart.py.
    b. Using a JSON file as the intermediary between the two.
  2. Checks-in a new chart. It's very similar to the old one.
  3. Checks-in the raw data backing up that chart as a JSON file - I think it's important to have the raw data around, and that we want the raw data and the graphs to match.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 31, 2024
@scotts scotts marked this pull request as ready for review October 31, 2024 18:40
@scotts scotts requested a review from ahmadsharif1 October 31, 2024 18:40
Copy link
Contributor

@ahmadsharif1 ahmadsharif1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can submit as it, but if it's decoupled like that we may want to have another key for non-test data like machine info, # of cpus, gpus, python version, etc.

@scotts
Copy link
Contributor Author

scotts commented Oct 31, 2024

@ahmadsharif1, I added the machine metadata into the data file. We can improve it over time if we want to - I wish we had the processor model name and speed.

@scotts scotts merged commit aa9b872 into pytorch:main Oct 31, 2024
37 of 40 checks passed
@scotts scotts deleted the exp_sep branch October 31, 2024 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants