Access the SDK

The SDK is available provided as the CapturMicromobilityEvents.xcframework file for iOS devices, available in the Reference Implementation Repository

ℹ️ Note: this is restricted access. If you see a 404, please confirm that your github user has been granted access.

Requirements

1. iOS version 15.0 +
2. Target - iOS devices
3. You will need a target Workspace in the www.captur.ai platform, and:
   1. the associated API Key ; You can generate an API key by logging into the Captur Dashboard
   2. the associated assetType; e.g. .eBike, eScooter , seatedEScooter
   3. the associated locationName; e.g. London , Paris, KL

Installation

1. Open your Xcode project. In the Xcode Project Navigator, right-click on your root app folder and select Add Files to "[Your App Target Name]".
2. Navigate to the downloaded .xcframework file and select it. Ensure that the "Copy items if needed" checkbox is selected. Click Add.
3. Now it's time to link the .xcframework to your app target. Select your project in the Project Navigator to open the project settings and choose your app target from the list of targets.
4. Navigate to the General tab. In the Frameworks, Libraries, and Embedded Content section, click the + button. Find and add the .xcframework file. You can skip this step if the framework shows up by default.
5. In the Frameworks, Libraries, and Embedded Content section, ensure that the .xcframework is set to Embed & Sign.
6. Import and Use the SDK. Open any .swift file where you want to use the SDK. Import the SDK using the import statement.

Usage

🚧

Importing

Make sure to import the SDK wherever you require access to Captur's methods, classes and properties

import CapturMicromobilityEvents

🚧

Camera Usage Description

Add camera usage description to your app's info.plist file.

This is a documented requirement from Apple docs; failing to do so will not trigger the camera preview.

Read more: Apple's documentation

1 - setAPI() to Setup the SDK

1. Call the Captur.shared.setApi(key:) method before accessing any other Captur related methods or properties.
2. Configure the SDK early in the app lifecycle, typically during app launch or at the start of the ride. For example, in AppDelegate.swift

func initialiseSDK(apiKey: String) throws -> Bool {
  do {
    try Captur.shared.setApi(key: apiKey)
    print("SDK initialised")
    return true
  } catch let error {
    if let error = error as? CapturException {
      print(error.localizedDescription)
    }

    return false
  }
}

2 - prepareModel() at start of ride to initialise the latest model

1. The function will ensure the correct model is available, based on the assetType and locationName provided.
   1. If the on-device model is already the latest model, the function will initialise it
   2. If there is a model update available, the function will download the new model and over-write the existing model.

      🚧

      callprepareModel() early to allow sufficient time for model downloads

2. This can be a fire and forget call. You will be able to check if a model has been successfully initialised before you proceed to get the configuration and present the camera

Captur.shared.prepareModel(locationName: <LOCATION>, assetType: <ASSET_TYPE>) {
  if let error = error {
    //handle error
  }
  
  if status {
    print("Model initialisation success")
    
    //Proceed to get your configs
  }
}

3 - getConfig() at end-ride to retrieve the configuration

Each session requires a configuration from the Captur backend. Aconfiguration implements the policies that you set up in the control centre.

🚧

Check you are using the correct`locationName and assetType; matching the control centre.

The function will capture an error type of .modelInitialisationFailure if for any reason the previousprepareModel() function failed to initialise, download, or store the model.

Captur.shared.getConfig(locationName: <LOCATION>, assetType: .eScooter, latitude: 0, longitude: 0) { status, error in

  if let error = error {
     //handle error 
  } 

  if status {
     print("Configs success")

     //proceed to present your camera
   }
}

❗️

The camera view depends on prepareModel() and getConfig()

Only present the camera view after you have called the prepareModel() function and obtained the configuration via the getConfig() function for each verification session.

4 - Define CapturCameraManager to handle scanning

Most of the interface with the Captur SDK happens via the CapturCameraManager. Whether you are using SwiftUI or UIViewControllers, make sure to initialise CapturCameraManager at a place where you have an active reference to it.

We recommend configuring the CapturCameraManager within View Models, but this does depend on how you have architected your app.

4.1. Prepare your view model or host class

class YourViewModel: ObservableObject {
  init() {
  }
}

4.2 Initialise CapturCameraManager and subscribe to events

class YourViewModel: ObservableObject {
  
  var manager: CapturCameraManager!
  
  init(reference: String) {
    //Initialise the system and make sure to pass in a unique reference for each verification.
    //For ex: This could be your rideId (Id per ride)
    manager = CapturCameraManager(referenceId: reference)
  
    //Subscribe to events
    manager.subscribeToEvents(events: self)
  }
}

• Pass in a unique reference for each verification task. For example, your rideId. Records in the Captur dashboard are searchable by reference, and this is also used for invoicing, analytics, and debugging.

📘

Attempts vs Sessions

You can have more than one attempt per session. For example - you might ask a rider to retry parking compliance after a bad parking decision event.

To have multiple attempts, simply initialise CapturCameraManagerwith the same reference or alternatively, you can call manager.retake(), depending on what suits your architecture better.

In hybrid apps like Flutter, the camera is managed natively but your feedback views might be managed in the hybrid code. During cases like this, it is better to re-initialise the managerwith the same reference in order to increment attempt.

❗️

Unique Identifiers for Verification

Avoid passing in userId or bike/scooter IDs as they are not unique to each verification task. Usually, clients prefer passing in their rideId or equivalent as the reference.

5 - Handle Events

In the previous step, you can see that we subscribed to events by calling the capturCameraManager.subscribeToEvents(events: self).

This will of course throw a compiler error that YourViewModel does not conform to CapturEvents. Let us make YourViewModel conform to CapturEvents.

extension YourViewModel: CapturEvents {
  
}

Now the compiler will force you to implement the Captur callback methods. Let's add the protocol methods.

extension YourViewModel: CapturEvents {
  func capturDidGenerateEvent(state: CapturCameraState, metadata: CapturOutput?) {
    //Handle decision
  }
  
  func capturDidGenerateGuidance(metadata: CapturOutput?) {
    //Handle guidance
  }
  
  func capturDidRevokeGuidance() {
    //Handle cases where guidance is no longer relevant
  }
  
  func capturDidGenerateError(error: CapturError) {
    //Handle any error cases
  }
}

Let us go through the events one by one:

5.1 capturDidGenerateEvent(state: , metadata: )

The capturDidGenerateEvent event handles all the UI states that occur when the SDK presents the camera view. You can iterate over the various UI states to handle different use cases.

func capturDidGenerateEvent(state: CapturCameraState, metadata: CapturOutput?) {
  switch state {
  
    case .cameraRunning:
      //In this state, the SDK has presented the camera view to the user
                  
    case .cameraDecided:
      //In this state the SDK has finished its predictions and has come to a decision
      
      //You can unwrap this decision
      guard let metadata = metadata else {return}
            
      //..add your own logic to handle decision
      handleDecision(metadata: metadata)
  }
}

You can define a handleDecisions() function to handle the attempt decision returned by the SDK

func handleDecision(metadata: CapturOutput) {
  switch metadata.decision {
      
    case .goodParking:
      //Handle good parking flow
      
    case .badParking:
      //Handle bad parking flow
      
    case .improvableParking:
      //Handle improvable parking flow
      
    case .insufficientInformation:
      //Handle the flow when the SDK predicts insufficient information.
      //Example - vehicle too close, no vehicle in image, image quality too poor etc.,
  }
}

5.2 capturDidGenerateGuidance(metadata: )

Aguidance event is emitted while the scan is still active. These events should tell the user to move their phone, so that they capture enough of the required environment for a decision.

For example, an event can be emitted when the user is attempting to end the ride when pointing the camera to the handlebars, instead of the whole vehicle. While the scanning is ongoing, you can show a card view that says "The vehicle is too close. Please take a step back".

The event metadata includes guidanceTitle and guidanceDetail properties, which you can use to display feedback.

func capturDidGenerateGuidance(metadata: CapturOutput) {
  
  guard let guidanceTitle = metadata?.guidanceTitle else {return}
  guard let guidanceDetail = metadata?.guidanceDetail else {return}
  
  //Use these strings to handle your own guidance UI
  self.showGuidance(title: guidanceTitle, detail: guidanceDetail)
}

🚧

decision event and guidance event metadata is visible in the control centre, but not editable or localised.

The recommended implementation is to define copy based on thereason_code

5.3 capturDidRevokeGuidance()

If the SDK detects that the guidance is no longer required, the capturDidRevokeGuidance() event will be triggered. You can use this to stop showing any guidance UI.

func capturDidRevokeGuidance() {
  self.removeGuidance()
}

5.4 capturDidGenerateError(error: )

The SDK might encounter errors depending on hardware or software issues. Use this event to handle any sort of errors to mitigate blockers and give your user a seamless experience. Note: This event is fired with the below errors when the camera is presented and when you have subscribed to events. This is a camera runtime error

func capturDidGenerateError(error: CapturError) {
  //Handle errors here
  switch error.errorType {
    case modelVerificationFailed:
    case unknownError:
    case videoDeviceNotFound:
    case videoDeviceInputError:
    case videoDeviceOutputError:
    case videoDeviceLockError:
  }
  
  //To get the error message
  print(error.errorMessage)
}

🚧

Handling modelVerificationFailed Errors

You typically don’t need to handle the modelVerificationFailed error. This error might occur if the system fails to process a single frame, possibly due to temporary CPU overload, but it doesn’t mean the entire flow should stop.

The SDK processes multiple frames per second, and occasionally, some frames may not be verified. For example, if a high number of frames within a short period fail, the modelVerificationFailed error will be triggered multiple times.

A consistently high failure rate may indicate an issue with the model or frame capture. You can set an acceptable failure rate based on your requirements.

6 - Present the camera

The SDK comes with a fully managed CapturCamera. You can add your own UI layer on top of it. First create a new SwiftUI view. In this case, we'll call it YourCameraView.

struct YourCameraView {
  var body: some View {
    ZStack {
    }
  }
}

In this particular example, we will use SwiftUI's @StateObject to initialise the view model YourViewModel, which we defined in the previous steps.

struct YourCameraView {
  @StateObject var yourViewModel = YourViewModel(reference: <YOUR_UNIQUE_REFERENCE>)
  
  var body: some View {
    ZStack {
    }
  }
}

The CapturCamera requires an instance of the CapturCameraManager to be passed in. We already initialised it in YourViewModel in the previous steps. Add CapturCamera to the ZStack and stack other UI elements (as per your design) on top of it. This gives you complete freedom to customise.

struct YourCameraView {
  @StateObject var yourViewModel = YourViewModel(reference: <YOUR_UNIQUE_REFERENCE>)
   @State var isFlashOn: Bool = false
   @State var isZoomedIn: Bool = false
  
  var body: some View {
    ZStack {
      //Define a function in your view model to get the instance of the running CapturCameraManager
      //It is also important to pass in a flash / zoom enabled or disabled parameter
      CapturCamera(capturCameraHandler: yourViewModel.getCapturCameraManager, isFlashOn: $isFlashOn, isZoomedIn: $isZoomedIn)
      
      VerticalFadeView()
      
      //....Other views you might want to layer on top
    }
  }
}

For example - You might want to structure YourViewModel to refresh:

• when there are new guidance event generated by the SDK
• when there is a new decision event generated by the SDK

In this case, you might want to create a view that does the following:

struct YourCameraView {
  @StateObject var yourViewModel = YourViewModel(reference: <YOUR_UNIQUE_REFERENCE>)
  @State var isFlashOn: Bool = false
  
  var body: some View {
    ZStack {
      CapturCamera(capturCameraHandler: yourViewModel.getCapturCameraManager, isFlashOn: $isFlashOn)
      
      VerticalFadeView()
      
      VStack {
        if yourViewModel.isGuidanceGenerated {
          GuidanceView()
        }
        
        Spacer()
        
        //some tool bar views - maybe a close button to dismiss camera or a flash button
      }
      
      if yourViewModel.isDecisionGenerated {
        DecisionView()
      }
    }
  }
}

👍

Customising Feedback

The reference implementation shows a feedback screen for all outcomes. You can customize your view model to react to successful parking decisions by displaying an animated "success" screen to congratulate users, or skip this entirely for an experience that feels super-fast.

Understanding the flow

1. Prepare your on device model (either initialise the existing model, download a new model or update model) based on your locationName and assetType
2. Get your configuration from Captur right before the scanning session starts
3. Present the camera. The camera will run it's verification and send back events
4. Handle a guidance event to display realtime feedback to the user while the camera is running
5. Handle decision event once the model has made an attempt decision, or the scan has timed out

Some changes to the flow are supported - see: updates to the scanning flow