Skip to content

llama iOS demo app documentation page feedback #10012

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
byjlw opened this issue Apr 9, 2025 · 2 comments
Closed

llama iOS demo app documentation page feedback #10012

byjlw opened this issue Apr 9, 2025 · 2 comments
Assignees
Labels
module: ios Issues related to iOS code, build, and execution triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@byjlw
Copy link
Contributor

byjlw commented Apr 9, 2025

📚 The doc issue

Update at the top

Remove the Update at the top
Section to remove

[UPDATE - 10/24] We have added support for running quantized Llama 3.2 1B/3B models in demo apps on the [XNNPACK backend](https://github.com/pytorch/executorch/blob/main/examples/demo-apps/apple_ios/LLaMA/docs/delegates/xnnpack_README.md). We currently support inference with SpinQuant and QAT+LoRA quantization methods.

We’re excited to share that the newly revamped iOS demo app is live and includes many new updates to provide a more intuitive and smoother user experience with a chat use case! The primary goal of this app is to showcase how easily ExecuTorch can be integrated into an iOS demo app and how to exercise the many features ExecuTorch and Llama models have to offer.

This app serves as a valuable resource to inspire your creativity and provide foundational code that you can customize and adapt for your particular use case.

Please dive in and start exploring our demo app today! We look forward to any feedback and are excited to see your innovative ideas.

Supported Models

Should include links to the models that users can download.

Building the Project

  • Don't ask them to choose a delegate. - **This is because it's unclear which delegate to use and the having to immediately go to another tutorial or documentation is a mental hurtle for people. **
    Have all details in line for doing things using pip install and export the model that they can run.
  • Pick whichever delegate is easier and most common to limit friction and reduce errors
  • Have a link and small blurb that users can follow if they know what they're doing and want to use a specific delegate

How to use the App

  • Move the note for for how to solve package dependencies to a section at the bottom called common issues or something like that. It's helpful if they run into issues and can scan the page or google for the issue if they encounter the issue. Include the actual error people get so it shows up when they search google for it
  • Provide step by step instructions to clone the repo. The XCode has instructions refers to a project on disk and it's not clear that the user needs to clone the repo and that they can find it there.

Copy the model to simulator

  • User needs to be given instructions on where to get the tokenizer and model
  • This section should be expanded to include preparing model where they download and export/quantize the model for use
  • Add screenshots for this section for clarity

Suggest a potential alternative/fix

No response

cc @shoumikhin @cbilgin

@metascroy
Copy link
Contributor

@shoumikhin can you have a look?

@lucylq lucylq added triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module module: ios Issues related to iOS code, build, and execution labels Apr 11, 2025
@shoumikhin
Copy link
Contributor

#10255

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: ios Issues related to iOS code, build, and execution triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

No branches or pull requests

4 participants