parent 5021c18

author Nikhil Viswanath Sivakumar <68182521+nil-is-all@users.noreply.github.com> 1741841357 -0500
committer Nikhil Viswanath Sivakumar <nikhilviswanath@meta.com> 1745433893 -0500
Migrate iOS Demo App

parent 5021c18
author Nikhil Viswanath Sivakumar <68182521+nil-is-all@users.noreply.github.com> 1741841357 -0500
committer Nikhil Viswanath Sivakumar <nikhilviswanath@meta.com> 1745433648 -0500
Migrate iOS Demo App

Migrate iOS app demo as README.md

Reproduced https://github.com/pytorch/executorch/tree/main/examples/demo-apps/apple_ios/ExecuTorchDemo and added last line in docs to point to this GitHub repo

Changed file path to executorch-examples/mv3/apple/MobileNetV3Demo/README.md

Changed directory and made changes after local testing

Final directory: executorch-examples/mv3/apple/ExecuTorchDemo

Changes from [current demo](https://pytorch.org/executorch/main/demo-apps-ios.html#):
1) Check Swift Package Manager version under Prerequisites
2) Export Model - export.py for backends
3) Directory changes in Final Steps

Files for iOS MV3 Demo App

export script for MV3.pte and XNNPack, CoreML, MPS backends

Update README.md

Rename mv3/apple/ExecuTorchDemo/export.py to mv3/apple/ExecuTorchDemo/python/export.py

Removed export.py code block from README.md

Added code formatting for Clone the Demo App subsection

added snapshot url

Delete mv3/apple/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet directory

atomically migrated apple/ExecuTorch to mv3/apple/ExecuTorchDemo

Updated directory structure to have python in mv3 folder

fixed typo in .gitignore

Update project.pbxproj to pin to 0.6.0 (#11)

MV2 cpp and python example (#13)

Update README.md (#12)

* Update README.md

* Update README.md

fix directory and resolve conflicts

MV2 cpp and python example (#13)

Update .gitignore (#17)

Author: nil-is-all

apple/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.pbxproj renamed to mv3/apple/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.pbxproj

@@ -0,0 +1,5 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict/>
+</plist>

mv3/apple/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.xcworkspace/contents.xcworkspacedata

@@ -0,0 +1,15 @@
+{
+  "originHash" : "18bbbaa47c24d88e1d32540f35aedf6e1bd14398eb435a331199a9d3995e4e24",
+  "pins" : [
+    {
+      "identity" : "executorch",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/pytorch/executorch",
+      "state" : {
+        "branch" : "swiftpm-0.6.0",
+        "revision" : "65f4d1656eba10ece31e340b814731b7f98d91b4"
+      }
+    }
+  ],
+  "version" : 3
+}

mv3/apple/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.xcworkspace/xcshareddata/WorkspaceSettings.xcsettings

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>BuildLocationStyle</key>
+	<string>UseAppPreferences</string>
+	<key>CustomBuildLocationType</key>
+	<string>RelativeToDerivedData</string>
+	<key>DerivedDataLocationStyle</key>
+	<string>Default</string>
+	<key>ShowSharedSchemesAutomaticallyEnabled</key>
+	<true/>
+</dict>
+</plist>

mv3/apple/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved

@@ -0,0 +1,29 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>SchemeUserState</key>
+	<dict>
+		<key>App.xcscheme_^#shared#^_</key>
+		<dict>
+			<key>orderHint</key>
+			<integer>0</integer>
+		</dict>
+		<key>ImageClassification.xcscheme_^#shared#^_</key>
+		<dict>
+			<key>orderHint</key>
+			<integer>2</integer>
+		</dict>
+		<key>MobileNetClassifier.xcscheme_^#shared#^_</key>
+		<dict>
+			<key>orderHint</key>
+			<integer>3</integer>
+		</dict>
+		<key>MobileNetClassifierTest.xcscheme_^#shared#^_</key>
+		<dict>
+			<key>orderHint</key>
+			<integer>1</integer>
+		</dict>
+	</dict>
+</dict>
+</plist>

mv3/apple/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.xcworkspace/xcuserdata/nikhilviswanath.xcuserdatad/UserInterfaceState.xcuserstate

@@ -0,0 +1,148 @@
+# Building an ExecuTorch iOS Demo App
+
+Welcome to the tutorial on setting up the ExecuTorch iOS Demo App!
+
+This app uses the
+[MobileNet v3](https://pytorch.org/vision/main/models/mobilenetv3.html) model to
+process live camera images leveraging three different backends:
+[XNNPACK](https://github.com/google/XNNPACK),
+[Core ML](https://developer.apple.com/documentation/coreml) and
+[Metal Performance Shaders (MPS)](https://developer.apple.com/documentation/metalperformanceshaders)
+(Xcode 15+ and iOS 17+ only).
+
+<p align="center">
+  <img src="https://github.com/pytorch/executorch/blob/main/docs/source/_static/img/demo_ios_app.png" width="50%">
+</p>
+
+## Prerequisites
+
+Before we start, make sure you have the following tools installed:
+
+### 1. Xcode 15+ and Command Line Tools
+
+Install Xcode 15+ from the
+[Mac App Store](https://apps.apple.com/app/xcode/id497799835) and then install
+the Command Line Tools using the terminal:
+
+```bash
+xcode-select --install
+```
+
+### 2. Python 3.10+
+
+Python 3.10 or above, along with `pip3`, should be pre-installed on MacOS 13.5+.
+If needed, [download Python](https://www.python.org/downloads/macos/) and
+install it. Verify the Python and pip versions using these commands:
+
+```bash
+which python3 pip3
+python3 --version
+pip3 --version
+```
+
+### 3. Check Swift Package Manager version
+The prebuilt ExecuTorch runtime, backend, and kernels are available as a Swift PM
+package. Ensure the latest SwiftPM version is linked to your Xcode installation.
+The steps to add the necessary package dependency are available in the
+[Using ExecuTorch for iOS](https://pytorch.org/executorch/main/using-executorch-ios.html#swift-package-manager) documentation.
+
+### 4. Set Up ExecuTorch
+
+Clone ExecuTorch and set up the environment as explained in the [Building from Source tutorial](https://pytorch.org/executorch/main/using-executorch-building-from-source.html):
+
+```bash
+git clone -b viable/strict https://github.com/pytorch/executorch.git && cd executorch
+
+python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip
+
+./install_executorch.sh
+```
+
+### 5. Backend Dependencies
+
+Install additional dependencies for [Core ML](https://pytorch.org/executorch/main/backends-coreml.html) and [MPS](https://pytorch.org/executorch/main/backends-mps.html) backends:
+
+```bash
+./backends/apple/coreml/scripts/install_requirements.sh
+
+./backends/apple/mps/install_requirements.sh
+```
+
+### 6. Clone the Demo App
+
+```bash
+git clone --depth 1 https://github.com/pytorch-labs/executorch-examples.git
+```
+
+## Models and Labels
+
+Now, let's move on to exporting and bundling the MobileNet v3 model.
+
+### 1. Export Model
+
+Export the MobileNet v3 model using the command line and move the exported model to
+a specific location where the Demo App will pick them up:
+
+```bash
+python3 -m examples.portable.scripts.export --model_name="mv3"
+
+APP_PATH="executorch-examples/mv3/apple/ExecuTorchDemo/ExecuTorchDemo"
+mkdir -p "$APP_PATH/Resources/Models/MobileNet/"
+mv mv3.pte "$APP_PATH/Resources/Models/MobileNet/"
+```
+
+Next, export the MobileNet v3 model with Core ML, MPS and XNNPACK backends by runnning this [`export.py`](https://github.com/pytorch-labs/executorch-examples/mv3/apple/ExecuTorchDemo/export.py) script.
+
+```bash
+python3 export.py
+```
+
+
+### 2. Download Labels
+
+Download the MobileNet model labels required for image classification:
+
+```bash
+curl https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt \
+  -o "$APP_PATH/Resources/Models/MobileNet/imagenet_classes.txt"
+```
+
+## Final Steps
+
+We're almost done! Now, we just need to open the project in Xcode, run the
+tests, and finally run the app.
+
+### 1. Open Project in Xcode
+
+Double-click on the project file under
+`executorch-examples/mv3/apple/ExecuTorchDemo/ExecuTorchDemo` to openit with Xcode, or run the command:
+
+```bash
+open $APP_PATH.xcodeproj
+```
+
+### 2. Run Tests
+
+You can run tests on Simulaltor directly in Xcode with `Cmd + U` or use the
+command line:
+
+```bash
+xcrun simctl create executorch "iPhone 15"
+xcodebuild clean test \
+     -project executorch-examples/mv3/apple/ExecuTorchDemo/ExecuTorchDemo.xcodeproj \
+     -scheme App \
+     -destination name=executorch
+xcrun simctl delete executorch
+```
+
+### 3. Run App
+
+Finally, connect the device, set up Code Signing in Xcode, and then run the app
+using `Cmd + R`. Try installing a Release build for better performance.
+
+Congratulations! You've successfully set up the ExecuTorch iOS Demo App. Now,
+you can explore and enjoy the power of ExecuTorch on your iOS device!
+
+Learn more about integrating and running [ExecuTorch on Apple](https://pytorch.org/executorch/main/using-executorch-ios.html) platforms.
+
+For specific examples, take a look at this [GitHub repository](https://github.com/pytorch-labs/executorch-examples/mv3/apple/ExecuTorchDemo/).

mv3/apple/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.xcworkspace/xcuserdata/nikhilviswanath.xcuserdatad/WorkspaceSettings.xcsettings

@@ -0,0 +1,48 @@
+#!/usr/bin/env python
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+import torch
+import torchvision.models as models
+from executorch.backends.apple.coreml.partition import CoreMLPartitioner
+from executorch.backends.apple.mps.partition import MPSPartitioner
+from executorch.exir.backend.backend_details import CompileSpec 
+from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
+from executorch.exir import to_edge_transform_and_lower
+import ssl
+import certifi
+
+
+def main() -> None:
+    model = models.mobilenet_v3_small(weights="DEFAULT").eval()
+    sample_inputs = (torch.randn(1, 3, 224, 224),)
+
+    et_program_coreml = to_edge_transform_and_lower(
+        torch.export.export(model, sample_inputs),
+        partitioner=[CoreMLPartitioner()],
+    ).to_executorch()
+
+    et_program_mps = to_edge_transform_and_lower(
+        torch.export.export(model, sample_inputs),
+        partitioner=[MPSPartitioner([CompileSpec("use_fp16", bytes([True]))])], 
+    ).to_executorch()
+
+    et_program_xnnpack = to_edge_transform_and_lower(
+        torch.export.export(model, sample_inputs),
+        partitioner=[XnnpackPartitioner()],
+    ).to_executorch()
+
+    save_path = "executorch-examples/mv3/apple/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet/"
+    with open(save_path+"mv3_coreml_all.pte", "wb") as file:
+        et_program_coreml.write_to_file(file)
+    with open(save_path+"mv3_mps_float16.pte", "wb") as file:
+        et_program_mps.write_to_file(file)
+    with open(save_path+"mv3_xnnpack_fp32.pte", "wb") as file:
+        et_program_xnnpack.write_to_file(file)
+
+
+if __name__ == "__main__":
+    main()