CoreML for Swift: Complete Guide with Examples

\n\n

1. What It Is and Why You Need It

\n\n

CoreML is a machine learning framework developed by Apple for integrating machine learning models directly into iOS, macOS, watchOS, and tvOS applications. Unlike cloud-based solutions, CoreML performs all computations on the user's device, ensuring instant data processing without network latency and complete user data privacy.

\n\n

Why do you need CoreML? First, it allows developers to implement complex AI features (image recognition, natural language processing, sound analysis) without needing to understand the intricacies of neural networks. Second, on-device operation means the application can function in offline mode, which is critical for many use cases. Third, Apple has optimized CoreML to work with Neural Engine, GPU, and CPU, ensuring maximum performance on all Apple devices.

\n\n

CoreML supports models created in popular frameworks (TensorFlow, PyTorch, scikit-learn), which can be converted to the .mlmodel format using the coremltools tool. The framework also integrates tightly with Vision (for image analysis), Natural Language (for text processing), and SoundAnalysis (for audio).

\n\n

2. Installation

\n\n

CoreML is a built-in Apple framework and does not require installation via package managers. To use it, simply import it into your project. However, to convert models from other formats, you will need the Python package coremltools.

\n\n

Installing coremltools (Python):

\n

pip install coremltools

\n\n

Integrating CoreML in an Xcode project:

\n

import CoreML

\n\n

For working with images and video, additionally import Vision:

\n

import Vision

\n\n

3. Quick Start — Minimal Working Example

\n\n

Let's create a simple application that classifies an image using the pre-trained MobileNetV2 model. First, download the MobileNetV2.mlmodel model from the Apple repository or convert it using coremltools.

\n\n

Minimal image classification example:

\n

import CoreML\nimport Vision\nimport UIKit\n\n// 1. Load the model\nguard let model = try? VNCoreMLModel(for: MobileNetV2().model) else {\n    fatalError("Failed to load model")\n}\n\n// 2. Create a request\nlet request = VNCoreMLRequest(model: model) { request, error in\n    guard let results = request.results as? [VNClassificationObservation],\n          let topResult = results.first else {\n        print("Failed to classify image")\n        return\n    }\n    print("Recognized: \\(topResult.identifier) with confidence \\(topResult.confidence)")\n}\n\n// 3. Load the image\nguard let image = UIImage(named: "cat.jpg"),\n      let ciImage = CIImage(image: image) else {\n    fatalError("Failed to load image")\n}\n\n// 4. Perform the request\nlet handler = VNImageRequestHandler(ciImage: ciImage)\ntry? handler.perform([request])

\n\n

4. Main Methods/Functions/Classes

\n\n

4.1. MLModel

\n

Signature: class MLModel

\n

What it does: The main class representing a loaded machine learning model. Allows making predictions and retrieving model information.

\n

Example:

\n

// Loading a model from a file\nguard let modelURL = Bundle.main.url(forResource: "MyModel", withExtension: "mlmodelc"),\n      let model = try? MLModel(contentsOf: modelURL) else {\n    fatalError("Failed to load mo