HomeSDK IntegrationDiscussions
Log In
SDK Integration
Start typing to search…

Android Walkthrough

Integrating Captur SDK into Your Android Application

This guide walks you through integrating the Captur SDK into your Android app, from initialization to handling events. Follow these steps in sequence:

  1. Set Up Prerequisites
  2. Initialize the SDK
  3. Configure the Camera Manager
  4. Handle SDK Events
  5. Present the Camera View

1. Check Prerequisites

  1. Android version 10.0 +
  2. Target - Android devices
  3. You will need a target Workspace in the www.captur.ai platform, and:
    1. the associated API Key; You can generate an API key by logging into the Captur Dashboard
    2. the associated assetType; e.g. Package

Ensure Camera and Flash permissions are in the manifest file:

<uses-feature android:name="android.hardware.camera.flash" />
<uses-feature android:name="android.hardware.camera" android:required="false" />
<uses-permission android:name="android.permission.CAMERA" />

Update your app dependencies to include:

    implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.5.1")
    implementation ("com.google.code.gson:gson:2.10.1")
    implementation ("androidx.camera:camera-core:1.3.3")
    implementation ("androidx.camera:camera-camera2:1.3.3")
    implementation ("androidx.camera:camera-lifecycle:1.3.3")
    implementation ("androidx.camera:camera-view:1.3.3")
    api("org.tensorflow:tensorflow-lite-task-vision:0.4.0")
    implementation("com.squareup.retrofit2:adapter-rxjava2:2.9.0")

    implementation("com.squareup.moshi:moshi-kotlin:1.14.0")
    implementation("com.squareup.retrofit2:retrofit:2.9.0")
    implementation("com.squareup.retrofit2:converter-moshi:2.9.0")
    implementation("com.squareup.okhttp3:okhttp:4.11.0")
    implementation("com.squareup.okhttp3:logging-interceptor:4.11.0")
    implementation ("androidx.lifecycle:lifecycle-viewmodel-ktx:2.8.0")

    implementation("com.google.accompanist:accompanist-permissions:0.31.0-alpha")

Sync project with Gradle files.

2. Initialize the SDK

Initialise the SDK early in the app lifecycle, typically during app launch or at the start of the relevant user session. For example, in ApplicationClass.kt

class ApplicationClass : Application() {
    override fun onCreate() {
        super.onCreate()
        try {
            Captur.init(
                this,
                "<YOUR_API_KEY>"
            )
        } catch (e:Exception){
            Log.d("error", e.message ?: "error with tf lite model")
        }
    }
}

3. Configure the Camera Manager

Most of the interface with the Captur SDK happens via the CapturCameraManager. Whether you are using Jetpack Compose or traditional Activity/Fragment, ensure you initialize CapturCameraManager where you have an active reference to it.

We recommend configuring the CapturCameraManager within ViewModels, but this depends on how you have architected your app.

3.1. Prepare your view model or host class

class CameraViewModel(val manager: CapturCameraManager) : ViewModel() {
  //Manage the camera via a view model class
}

3.2. Initialise CapturCameraManager and subscribe to events

class CameraViewModel(val manager: CapturCameraManager) : ViewModel() {  
   init {
        manager.subscribeToEvents(this)
    }
}

Calling the view model:

val captureManager = CapturCameraManager(reference)
viewModel = CameraViewModel(captureManager)
  • Pass in a unique reference for each verification task. For example, your rideId. Records in the Captur dashboard are searchable by reference, and this is also used for invoicing, analytics, and debugging.

📘

Attempts vs Sessions

You can have more than one attempt per session. For example - you might ask a rider to retry parking compliance after a bad parking prediction decision event.

To have multiple attempts, simply initialise CapturCameraManagerwith the same reference or alternatively, you can call manager.retake(), depending on what suits your architecture better.

In hybrid apps like Flutter, the camera is managed natively but your feedback views might be managed in the hybrid code. During cases like this, it is better to re-initialise the managerwith the same reference in order to increment attempt

❗️

Unique Identifiers for Verification

Avoid passing in userId or bike/scooter IDs as they are not unique to each verification task. Usually, clients prefer passing in their rideId or equivalent as the reference.

4. Handle SDK Events

In the previous step, you can see that we subscribed to events by calling the manager.subscribeToEvents(this).

We need to conform to the events interface:

class CameraViewModel(val manager: CapturCameraManager) : ViewModel(), CapturEvents {  
   init {
        manager.subscribeToEvents(this)
    }
}

Implement the interface methods

override fun capturDidGenerateEvent(state: CapturCameraState, metadata: CapturOutput?) {}

override fun capturDidGenerateError(error: CapturError) {}

override fun capturDidGenerateGuidance(metadata: CapturOutput) {}

Let us go through the events one by one:

4.1 capturDidGenerateEvent(state: , metadata: )

The capturDidGenerateEvent event handles two UI states that occur when the SDK presents the camera view. You can iterate over the various UI states to handle different use cases.

One is the CAMERA_RUNNING state which indicates that the camera has started to make predictions on live feed. The CAMERA_DECIDED state indicates that the camera has finished making predictions and has arrived at a decision: goodParking, badParking, improvableParkingand insufficientInformation.

You will also receive a final image which you can present in your UI, and upload to your backend. This data is available via the metaData property.

override fun capturDidGenerateEvent(state: CapturCameraState, metadata: CapturOutput?) {
  when (state) {
    CapturCameraState.CAMERA_RUNNING -> {
      //Camera running
    }

    CapturCameraState.CAMERA_DECIDED -> {
      metadata?.let {
        val decision = it.decision
        //Handle your flow based on what the decision is
      }
    }
  }
}

You can define a handleDecisions() function to handle the end decision predicted by the SDK

fun handleDecision(metadata: CapturOutput) {
    when (metadata.decision) {
        GOOD_PARKING -> {
            // Handle good parking flow
        }
        BAD_PARKING -> {
            // Handle bad parking flow
        }
        IMPROVABLE_PARKING -> {
            // Handle improvable parking flow
        }
        INSUFFICIENT_INFORMATION -> {
            // Handle the flow when the SDK predicts insufficient information.
            // Example - vehicle too close, no vehicle in image, image quality too poor etc.,
        }
    }
}

4.2 capturDidGenerateGuidance(metadata: )

Aguidance event is emitted while the scan is still active. These events should tell the user to move their phone, so that they capture enough of the required environment for a decision.

For example, an event can be emitted when the user is attempting to end the ride when pointing the camera to the handlebars, instead of the whole vehicle. While the scanning is ongoing, you can show a card view that says "The vehicle is too close. Please take a step back".

The event metadata includes guidanceTitle and guidanceDetail properties, which you can use to display feedback.

override fun capturDidGenerateGuidance(metadata: CapturOutput) {
  
  guard let guidanceTitle = metadata.guidanceTitle
  guard let guidanceDetail = metadata.guidanceDetail
  
  //Use these strings to handle your own guidance UI. Maybe a nice cardview on top of the camera
  showGuidance(title: guidanceTitle, detail: guidanceDetail)
}

🚧

decision event and guidance event metadata is visible in the control centre, but not editable or localised.

The recommended implementation is to define copy based on the decision or reason_code

4.3 capturDidRevokeGuidance()

If the SDK detects that the guidance is no longer required, the capturDidRevokeGuidance() event will be triggered. You can use this to stop showing any guidance UI.

4.4 capturDidGenerateError(error: )

The SDK might encounter errors depending on hardware or software issues. Use this event to handle any sort of errors to mitigate blockers and give your user a seamless experience. Note: This event is fired with the below errors when the camera is presented and when you have subscribed to events. This is a camera runtime error.

override fun capturDidGenerateError(error: CapturError) {
  //Handle camera runtime errors here
}

🚧

Handling modelVerificationFailed Errors

You typically don’t need to handle the modelVerificationFailed error. This error might occur if the system fails to process a single frame, possibly due to temporary CPU overload, but it doesn’t mean the entire flow should stop.

The SDK processes multiple frames per second, and occasionally, some frames may not be verified. For example, if a high number of frames within a short period fail, the modelVerificationFailed error will be triggered multiple times.

A consistently high failure rate may indicate an issue with the model or frame capture. You can set an acceptable failure rate based on your requirements

5 - Present the camera

The SDK comes with a fully managed CapturCameraPreview, which is a fully managed camera system. You can add your own UI layer on top of it. First create a new Compose view. In this case, we'll call it CameraScreen.

To tie all this together, first define interface and handler classes. There are many ways to go about this; what is shown here is one opinionated approach.

class CameraUiState {
    var shouldShowFeedBackScreen by mutableStateOf(false)
    var output by mutableStateOf<CapturOutput?>(null)
    var touchLightOn by mutableStateOf(false)
    var zoomOn by mutableStateOf(false)
    var backPress by mutableStateOf(false)
}

interface CameraScreenEventHandler {
    fun onFlashLightClicked()
    fun onZoomClicked()
    fun onRetryClicked()
    fun onFinishClicked()
}

Define the CameraScreen composable:

@Composable
fun CameraScreen(
    uiState: CameraUiState,
    eventHandler: CameraScreenEventHandler,
    manager: CapturCameraManager,
    refr: String,
    onFinish: () -> Unit,
) {
    val context = LocalContext.current

    if (uiState.backPress) {
        onFinish()
    } else if (uiState.shouldShowFeedBackScreen) {
        ContextCompat.startActivity(
            context,
            Intent(context, FeedBackActivity::class.java).putExtra(
                "state",
                Gson().toJson(uiState.output)
            ).putExtra("ref", refr),
            null
        )
        onFinish()
    } else {
        Box(modifier = Modifier.fillMaxSize()) {
            CapturCameraPreview(uiState.touchLightOn, uiState.zoomOn, manager)
            CameraOverlay(uiState, eventHandler)

        }
    }
}

The CapturCameraPreview requires an instance of the CapturCameraManager to be passed in. It also expects a state management toggle for flash. This is pretty important during lowlight conditions.

Now define the camera overlay to show other UI elements like the guidance (if captured via events), other important buttons like an exit button and a flash button.

@Composable
fun CameraOverlay(uiState: CameraUiState, eventHandler: CameraScreenEventHandler) {
    // Client overlay here...
    Box(
        modifier = Modifier.fillMaxSize()
    ) {

        uiState.output?.let {
            InformationText(R.drawable.infofilled, it.guidanceTitle, it.guidanceDetail)
        }

        Row(
            modifier = Modifier
                .align(Alignment.BottomCenter)
                .padding(AppSizes.current.Space3)
                .clip(shape = RoundedCornerShape(AppSizes.current.CornerSize2))
                .background(AppColors.current.White)
                .padding(AppSizes.current.Padding2)
        ) {
            CircularProgressIndicator(
                modifier = Modifier
                    .size(20.dp)
                    .align(Alignment.CenterVertically),
                color = AppColors.current.Purple
            )

            Text(
                text = stringResource(id = R.string.analysing_parking),
                modifier = Modifier.padding(start = AppSizes.current.Padding),
                style = AppTextStyles.current.Regular16,
                color = AppColors.current.Black
            )
        }

        IconButton(
            onClick = {
                eventHandler.onFinishClicked()
            },
            modifier = Modifier
                .padding(30.dp)
                .size(IconSize)
                .clip(shape = CircleShape)
                .align(Alignment.BottomStart)
                .background(AppColors.current.White)
        ) {
            Icon(Icons.Default.Clear, contentDescription = "", tint = AppColors.current.Black)
        }

        IconButton(
            onClick = {
                eventHandler.onFlashLightClicked()
            },
            modifier = Modifier
                .padding(30.dp)
                .size(IconSize)
                .clip(shape = CircleShape)
                .align(Alignment.BottomEnd)
                .background(AppColors.current.White)
        ) {

            Icon(
                painterResource(R.drawable.ic_flash_light),
                contentDescription = "",
                tint = Color.Black
            )
        }
    }
}

Define your CameraActivity

class CameraActivity : ComponentActivity() {
    lateinit var viewModel: CameraViewModel
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        enableEdgeToEdge()
        setContent {
            val c: String = intent.extras?.getString("ref").toString()
            val reference = rememberSaveable {
                if (c == "null") (UUID.randomUUID().toString()) else c
            }
            val manager = CapturCameraManager(reference)
            viewModel = CameraViewModel(manager)
            YourAppTheme {
                Scaffold(modifier = Modifier.fillMaxSize()) { innerPadding ->
                    Column(modifier = Modifier.padding(innerPadding)) {
                        Surface(
                            modifier = Modifier.fillMaxSize(),
                            color = MaterialTheme.colorScheme.background
                        ) {
                            CameraScreen(
                                uiState = viewModel.uiState,
                                viewModel,
                                viewModel.manager,
                                reference
                            ) {
                                finish()
                            }
                        }
                    }
                }
            }
        }
    }
}

Your final CameraViewModel will now look like

class CameraViewModel(val manager: CapturCameraManager) : ViewModel(), CapturEvents,
    CameraScreenEventHandler {

    val uiState = CameraUiState()

    init {
        manager.subscribeToEvents(this)
    }

    override fun onFlashLightClicked() {
        uiState.touchLightOn = !uiState.touchLightOn

    }

    override fun onRetryClicked() {
        uiState.shouldShowFeedBackScreen = false
        uiState.output = null
    }

    override fun onFinishClicked() {
        uiState.backPress = true
        dismissCamera()
    }

    private fun dismissCamera() {
        uiState.backPress = true
    }

    override fun capturDidGenerateEvent(state: CapturCameraState, metadata: CapturOutput?) {
        when (state) {
            CapturCameraState.CAMERA_RUNNING -> {
                uiState.shouldShowFeedBackScreen = false
            }

            CapturCameraState.CAMERA_DECIDED -> {
                metadata?.let {
                    uiState.output = it
                }
                uiState.shouldShowFeedBackScreen = true
            }
        }
    }

    override fun capturDidGenerateError(error: CapturError) {
        println("error $error.errorMessage")
    }

    override fun capturDidGenerateGuidance(metadata: CapturOutput) {
        metadata.let {
            uiState.output = it
        }
    }
}

👍

Customising Feedback

The reference implementation shows a feedback screen for all outcomes. You can customize your view model to react to successful parking decisions by displaying an animated "success" screen to congratulate users, or skip this entirely for an experience that feels super-fast.

Some changes to the flow are supported - see: updates to the scanning flow

Updated 5 months ago