Initial public release.
This commit is contained in:
Joe Kearney
parent
ed31acd60f
commit
58d87b11b7
249 changed files with 15831 additions and 4 deletions
86
subapp-sentry/README.md
Normal file
86
subapp-sentry/README.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
# Sentry
|
||||
|
||||
An autonomous sentry gun subapp that uses the device camera and on-device machine learning to detect people and automatically send a tag from a KTag blaster connected via USB when a person is detected in the crosshair.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. The back camera streams frames continuously through CameraX.
|
||||
2. Each frame is analyzed by **MediaPipe Tasks Vision** running the **EfficientDet-Lite0** object detection model entirely on-device — no network connection required.
|
||||
3. When a person is detected with ≥ 50 % confidence, a red bounding box is drawn over them and a continuous 880 Hz tone plays (if enabled).
|
||||
4. If the bounding box overlaps the crosshair at the centre of the screen, the sentry fires: it sends a trigger-press command followed by a trigger-release command over USB serial to the connected KTag device.
|
||||
5. A cooldown period prevents the sentry from firing again until the timer expires.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Download the detection model
|
||||
|
||||
The EfficientDet-Lite0 model is not bundled with the APK and must be placed manually:
|
||||
|
||||
```
|
||||
subapp-sentry/src/main/assets/efficientdet_lite0.tflite
|
||||
```
|
||||
|
||||
Download the **TFLite (efficientdet/lite0/detection/metadata/1)** file from:
|
||||
|
||||
> https://tfhub.dev/tensorflow/lite-model/efficientdet/lite0/detection/metadata/1
|
||||
|
||||
### 2. Connect a KTag device via USB
|
||||
|
||||
Plug a KTag device into the Android device using a USB OTG cable. The app will detect it automatically and show it in the device selector.
|
||||
|
||||
### 3. Grant camera permission
|
||||
|
||||
The app requests `CAMERA` permission on first launch. Deny it and the camera preview will not start.
|
||||
|
||||
## UI
|
||||
|
||||
| Element | Description |
|
||||
|----------------------|-----------------------------------------------------------------------------------------------------|
|
||||
| Camera preview | Full-screen live view from the back camera |
|
||||
| Red bounding box | Drawn around any detected person |
|
||||
| White crosshair | Fixed plus symbol at the centre of the screen; firing only occurs when it overlaps the bounding box |
|
||||
| **DETECTED** badge | Appears in the top-right corner whenever a person is visible |
|
||||
| Device selector | Drop-down listing connected USB serial devices |
|
||||
| Connect / Disconnect | Connects to the selected USB device or drops the connection |
|
||||
| Status bar | Shows connection state, configured team, and time since last trigger |
|
||||
|
||||
## Settings
|
||||
|
||||
Open the settings screen via the gear icon in the top-right corner.
|
||||
|
||||
| Setting | Options | Default | Description |
|
||||
|---------------------|----------------------|---------|----------------------------------------------------------|
|
||||
| Tag Team | Blue, Red, Purple | Blue | Team argument sent in the trigger command |
|
||||
| Trigger Cooldown | 3 s, 5 s, 10 s, 30 s | 5 s | Minimum time between consecutive trigger firings |
|
||||
| Play Detection Tone | On / Off | On | Plays a continuous 880 Hz tone while a person is visible |
|
||||
|
||||
## USB Serial Protocol
|
||||
|
||||
When the sentry fires, it simulates a trigger press by sending two commands in sequence:
|
||||
|
||||
```
|
||||
KEvent 7 0\n ← trigger press
|
||||
KEvent 8 0\n ← trigger release
|
||||
```
|
||||
|
||||
The device selector lists all recognised USB serial devices. The app requests USB permission automatically on first connect.
|
||||
|
||||
## Audio
|
||||
|
||||
| Sound | Condition | Details |
|
||||
|------------------------|-------------------------|----------------------------------------------------------------------------------------------|
|
||||
| Continuous 880 Hz tone | Person visible in frame | `AudioTrack` sine wave; plays until person leaves frame or "Play Detection Tone" is disabled |
|
||||
| Short beep | Sentry fires | 200 ms `ToneGenerator` beep |
|
||||
|
||||
Both sounds play on the media audio stream and respect the device's media volume.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Library | Version | Purpose |
|
||||
|------------------------|---------|----------------------------------------------------------|
|
||||
| CameraX | 1.4.0 | Camera preview and frame capture |
|
||||
| MediaPipe Tasks Vision | 0.10.32 | On-device object detection |
|
||||
| EfficientDet-Lite0 | 1 | Person detection model (Apache-2.0, download separately) |
|
||||
| usb-serial-for-android | 3.10.0 | USB serial communication with KTag hardware |
|
||||
|
||||
> **Note:** MediaPipe transitively includes Firebase. The `AndroidManifest.xml` explicitly removes `FirebaseInitProvider` via `tools:node="remove"` to prevent any Firebase initialisation.
|
||||
55
subapp-sentry/build.gradle.kts
Normal file
55
subapp-sentry/build.gradle.kts
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
|
||||
|
||||
plugins {
|
||||
alias(libs.plugins.android.library)
|
||||
alias(libs.plugins.kotlin.compose)
|
||||
}
|
||||
|
||||
android {
|
||||
namespace = "club.clubk.ktag.apps.sentry"
|
||||
compileSdk = 36
|
||||
|
||||
defaultConfig {
|
||||
minSdk = 24
|
||||
}
|
||||
|
||||
compileOptions {
|
||||
sourceCompatibility = JavaVersion.VERSION_17
|
||||
targetCompatibility = JavaVersion.VERSION_17
|
||||
}
|
||||
|
||||
buildFeatures {
|
||||
compose = true
|
||||
}
|
||||
|
||||
androidResources {
|
||||
noCompress += "tflite"
|
||||
}
|
||||
}
|
||||
|
||||
dependencies {
|
||||
implementation(project(":core"))
|
||||
implementation(project(":shared-services"))
|
||||
|
||||
implementation("com.github.mik3y:usb-serial-for-android:3.10.0")
|
||||
implementation("androidx.camera:camera-core:1.4.0")
|
||||
implementation("androidx.camera:camera-camera2:1.4.0")
|
||||
implementation("androidx.camera:camera-lifecycle:1.4.0")
|
||||
implementation("androidx.camera:camera-view:1.4.0")
|
||||
implementation("com.google.mediapipe:tasks-vision:0.10.32")
|
||||
|
||||
implementation(libs.androidx.core.ktx)
|
||||
implementation(libs.androidx.preference)
|
||||
implementation(libs.androidx.lifecycle.viewmodel.compose)
|
||||
implementation(libs.androidx.activity.compose)
|
||||
implementation(platform(libs.androidx.compose.bom))
|
||||
implementation(libs.androidx.compose.ui)
|
||||
implementation(libs.androidx.compose.material3)
|
||||
implementation(libs.androidx.startup)
|
||||
}
|
||||
|
||||
tasks.withType<KotlinCompile>().configureEach {
|
||||
compilerOptions {
|
||||
jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17)
|
||||
}
|
||||
}
|
||||
51
subapp-sentry/src/main/AndroidManifest.xml
Normal file
51
subapp-sentry/src/main/AndroidManifest.xml
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
|
||||
xmlns:tools="http://schemas.android.com/tools">
|
||||
|
||||
<uses-permission android:name="android.permission.CAMERA" />
|
||||
|
||||
<uses-feature
|
||||
android:name="android.hardware.camera"
|
||||
android:required="true" />
|
||||
<uses-feature
|
||||
android:name="android.hardware.usb.host"
|
||||
android:required="true" />
|
||||
|
||||
<application>
|
||||
<activity
|
||||
android:name=".SentryActivity"
|
||||
android:exported="true"
|
||||
android:label="Sentry"
|
||||
android:launchMode="singleTop">
|
||||
<intent-filter>
|
||||
<action android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED" />
|
||||
</intent-filter>
|
||||
<meta-data
|
||||
android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED"
|
||||
android:resource="@xml/device_filter" />
|
||||
</activity>
|
||||
|
||||
<activity
|
||||
android:name=".SentrySettingsActivity"
|
||||
android:exported="false"
|
||||
android:label="Sentry Settings"
|
||||
android:parentActivityName=".SentryActivity"
|
||||
android:theme="@style/Theme.AppCompat.Light.DarkActionBar" />
|
||||
|
||||
<provider
|
||||
android:name="androidx.startup.InitializationProvider"
|
||||
android:authorities="${applicationId}.androidx-startup"
|
||||
android:exported="false">
|
||||
<meta-data
|
||||
android:name="club.clubk.ktag.apps.sentry.SentryInitializer"
|
||||
android:value="androidx.startup" />
|
||||
</provider>
|
||||
<!-- Prevent Firebase (pulled in transitively by MediaPipe) from auto-initializing -->
|
||||
<provider
|
||||
android:name="com.google.firebase.provider.FirebaseInitProvider"
|
||||
android:authorities="${applicationId}.firebaseinitprovider"
|
||||
android:exported="false"
|
||||
tools:node="remove" />
|
||||
</application>
|
||||
|
||||
</manifest>
|
||||
8
subapp-sentry/src/main/assets/MODEL.txt
Normal file
8
subapp-sentry/src/main/assets/MODEL.txt
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
Place the EfficientDet-Lite0 person detection model here:
|
||||
|
||||
File: efficientdet_lite0.tflite
|
||||
Source: https://tfhub.dev/tensorflow/lite-model/efficientdet/lite0/detection/metadata/1
|
||||
License: Apache-2.0
|
||||
|
||||
Download the "TFLite (efficientdet/lite0/detection/metadata/1)" file from the URL above
|
||||
and save it to this directory as efficientdet_lite0.tflite.
|
||||
|
|
@ -0,0 +1,413 @@
|
|||
package club.clubk.ktag.apps.sentry
|
||||
|
||||
import android.Manifest
|
||||
import android.content.pm.PackageManager
|
||||
import android.os.Bundle
|
||||
import android.util.Log
|
||||
import androidx.activity.ComponentActivity
|
||||
import androidx.activity.compose.setContent
|
||||
import androidx.activity.result.contract.ActivityResultContracts
|
||||
import androidx.camera.core.CameraSelector
|
||||
import androidx.camera.core.ImageAnalysis
|
||||
import androidx.camera.core.Preview
|
||||
import androidx.camera.lifecycle.ProcessCameraProvider
|
||||
import androidx.camera.view.PreviewView
|
||||
import androidx.compose.foundation.Canvas
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.Row
|
||||
import androidx.compose.foundation.layout.Spacer
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.fillMaxWidth
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.foundation.layout.width
|
||||
import androidx.compose.material.icons.Icons
|
||||
import androidx.compose.material.icons.filled.Settings
|
||||
import androidx.compose.material3.Button
|
||||
import androidx.compose.material3.ButtonDefaults
|
||||
import androidx.compose.material3.DropdownMenu
|
||||
import androidx.compose.material3.DropdownMenuItem
|
||||
import androidx.compose.material3.ExperimentalMaterial3Api
|
||||
import androidx.compose.material3.Icon
|
||||
import androidx.compose.material3.IconButton
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.OutlinedButton
|
||||
import androidx.compose.material3.Scaffold
|
||||
import androidx.compose.material3.Surface
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.material3.TopAppBar
|
||||
import androidx.compose.material3.TopAppBarDefaults
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.DisposableEffect
|
||||
import androidx.compose.runtime.LaunchedEffect
|
||||
import androidx.compose.runtime.collectAsState
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableLongStateOf
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.remember
|
||||
import androidx.compose.runtime.setValue
|
||||
import kotlinx.coroutines.delay
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.geometry.Offset
|
||||
import androidx.compose.ui.geometry.Size
|
||||
import androidx.compose.ui.graphics.Color
|
||||
import androidx.compose.ui.graphics.drawscope.Stroke
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.platform.LocalLifecycleOwner
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.unit.sp
|
||||
import androidx.compose.ui.viewinterop.AndroidView
|
||||
import androidx.core.content.ContextCompat
|
||||
import androidx.lifecycle.Lifecycle
|
||||
import androidx.lifecycle.LifecycleEventObserver
|
||||
import androidx.lifecycle.viewmodel.compose.viewModel
|
||||
import club.clubk.ktag.apps.core.UsbSerialManager
|
||||
import com.hoho.android.usbserial.driver.UsbSerialDriver
|
||||
import java.util.concurrent.Executors
|
||||
|
||||
private const val TAG = "SentryActivity"
|
||||
|
||||
class SentryActivity : ComponentActivity() {
|
||||
|
||||
private val requestCameraPermission = registerForActivityResult(
|
||||
ActivityResultContracts.RequestPermission()
|
||||
) { granted ->
|
||||
if (!granted) {
|
||||
Log.w(TAG, "Camera permission denied")
|
||||
}
|
||||
}
|
||||
|
||||
override fun onCreate(savedInstanceState: Bundle?) {
|
||||
super.onCreate(savedInstanceState)
|
||||
if (ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA)
|
||||
!= PackageManager.PERMISSION_GRANTED
|
||||
) {
|
||||
requestCameraPermission.launch(Manifest.permission.CAMERA)
|
||||
}
|
||||
setContent {
|
||||
MaterialTheme {
|
||||
Surface(modifier = Modifier.fillMaxSize()) {
|
||||
SentryScreen()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@OptIn(ExperimentalMaterial3Api::class)
|
||||
@Composable
|
||||
fun SentryScreen(vm: SentryViewModel = viewModel()) {
|
||||
val context = LocalContext.current
|
||||
val lifecycleOwner = LocalLifecycleOwner.current
|
||||
|
||||
val connectionState by vm.connectionState.collectAsState()
|
||||
val availableDevices by vm.availableDevices.collectAsState()
|
||||
val detectionResult by vm.detectionResult.collectAsState()
|
||||
val lastTriggerMs by vm.lastTriggerMs.collectAsState()
|
||||
val config by vm.config.collectAsState()
|
||||
|
||||
var selectedDevice by remember { mutableStateOf<UsbSerialDriver?>(null) }
|
||||
var showDeviceMenu by remember { mutableStateOf(false) }
|
||||
val cameraExecutor = remember { Executors.newSingleThreadExecutor() }
|
||||
|
||||
// Register/unregister USB
|
||||
DisposableEffect(vm) {
|
||||
vm.registerUsb()
|
||||
onDispose { vm.unregisterUsb() }
|
||||
}
|
||||
|
||||
// Cleanup camera executor on disposal
|
||||
DisposableEffect(cameraExecutor) {
|
||||
onDispose { cameraExecutor.shutdown() }
|
||||
}
|
||||
|
||||
// Reload config on every resume so settings changes take effect immediately
|
||||
DisposableEffect(lifecycleOwner) {
|
||||
val observer = LifecycleEventObserver { _, event ->
|
||||
if (event == Lifecycle.Event.ON_RESUME) vm.reloadConfig()
|
||||
}
|
||||
lifecycleOwner.lifecycle.addObserver(observer)
|
||||
onDispose { lifecycleOwner.lifecycle.removeObserver(observer) }
|
||||
}
|
||||
|
||||
// Auto-select first available device
|
||||
LaunchedEffect(availableDevices) {
|
||||
if (selectedDevice == null && availableDevices.isNotEmpty()) {
|
||||
selectedDevice = availableDevices.first()
|
||||
}
|
||||
if (selectedDevice != null && availableDevices.none {
|
||||
it.device.deviceId == selectedDevice?.device?.deviceId
|
||||
}
|
||||
) {
|
||||
selectedDevice = null
|
||||
}
|
||||
}
|
||||
|
||||
val teamLabel = when (config.team) {
|
||||
"asblue" -> "Blue"
|
||||
"asred" -> "Red"
|
||||
"aspurple" -> "Purple"
|
||||
else -> config.team
|
||||
}
|
||||
|
||||
var currentTimeMs by remember { mutableLongStateOf(System.currentTimeMillis()) }
|
||||
LaunchedEffect(Unit) {
|
||||
while (true) {
|
||||
delay(500)
|
||||
currentTimeMs = System.currentTimeMillis()
|
||||
}
|
||||
}
|
||||
|
||||
val secondsSinceTrigger = if (lastTriggerMs > 0L) {
|
||||
(currentTimeMs - lastTriggerMs) / 1000
|
||||
} else null
|
||||
|
||||
Scaffold(
|
||||
topBar = {
|
||||
TopAppBar(
|
||||
title = { Text("Sentry") },
|
||||
colors = TopAppBarDefaults.topAppBarColors(
|
||||
containerColor = Color(0xFF323031),
|
||||
titleContentColor = Color(0xFFFFC857),
|
||||
actionIconContentColor = Color(0xFFFFC857)
|
||||
),
|
||||
actions = {
|
||||
IconButton(onClick = {
|
||||
context.startActivity(SentrySettingsActivity.createIntent(context))
|
||||
}) {
|
||||
Icon(Icons.Default.Settings, contentDescription = "Settings")
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
) { paddingValues ->
|
||||
Column(
|
||||
modifier = Modifier
|
||||
.fillMaxSize()
|
||||
.padding(paddingValues)
|
||||
) {
|
||||
// Camera preview + bounding box overlay
|
||||
Box(
|
||||
modifier = Modifier
|
||||
.weight(1f)
|
||||
.fillMaxWidth()
|
||||
) {
|
||||
// CameraX PreviewView - binds both Preview and ImageAnalysis
|
||||
val hasCameraPermission = ContextCompat.checkSelfPermission(
|
||||
context, Manifest.permission.CAMERA
|
||||
) == PackageManager.PERMISSION_GRANTED
|
||||
|
||||
if (hasCameraPermission) {
|
||||
AndroidView(
|
||||
factory = { ctx ->
|
||||
val previewView = PreviewView(ctx).apply {
|
||||
scaleType = PreviewView.ScaleType.FILL_CENTER
|
||||
}
|
||||
|
||||
val cameraProviderFuture = ProcessCameraProvider.getInstance(ctx)
|
||||
cameraProviderFuture.addListener({
|
||||
val cameraProvider = cameraProviderFuture.get()
|
||||
|
||||
val preview = Preview.Builder().build().also {
|
||||
it.surfaceProvider = previewView.surfaceProvider
|
||||
}
|
||||
|
||||
val imageAnalysis = ImageAnalysis.Builder()
|
||||
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
|
||||
.build()
|
||||
.also { analysis ->
|
||||
analysis.setAnalyzer(cameraExecutor, vm.buildAnalyzer())
|
||||
}
|
||||
|
||||
try {
|
||||
cameraProvider.unbindAll()
|
||||
cameraProvider.bindToLifecycle(
|
||||
lifecycleOwner,
|
||||
CameraSelector.DEFAULT_BACK_CAMERA,
|
||||
preview,
|
||||
imageAnalysis
|
||||
)
|
||||
} catch (e: Exception) {
|
||||
Log.e(TAG, "Camera binding failed", e)
|
||||
}
|
||||
}, ContextCompat.getMainExecutor(ctx))
|
||||
|
||||
previewView
|
||||
},
|
||||
modifier = Modifier.fillMaxSize()
|
||||
)
|
||||
} else {
|
||||
Box(
|
||||
modifier = Modifier
|
||||
.fillMaxSize()
|
||||
.background(Color.Black),
|
||||
contentAlignment = Alignment.Center
|
||||
) {
|
||||
Text("Camera permission required", color = Color.White)
|
||||
}
|
||||
}
|
||||
|
||||
// Bounding box + crosshair overlay
|
||||
Canvas(modifier = Modifier.fillMaxSize()) {
|
||||
detectionResult?.let { result ->
|
||||
val left = result.left * size.width
|
||||
val top = result.top * size.height
|
||||
val right = result.right * size.width
|
||||
val bottom = result.bottom * size.height
|
||||
drawRect(
|
||||
color = Color.Red,
|
||||
topLeft = Offset(left, top),
|
||||
size = Size(right - left, bottom - top),
|
||||
style = Stroke(width = 4f)
|
||||
)
|
||||
}
|
||||
|
||||
// Crosshair
|
||||
val cx = size.width / 2f
|
||||
val cy = size.height / 2f
|
||||
val arm = minOf(size.width, size.height) * 0.08f
|
||||
val gap = minOf(size.width, size.height) * 0.02f
|
||||
val crosshairColor = Color.White
|
||||
val strokeWidth = 3f
|
||||
drawLine(crosshairColor, Offset(cx - arm - gap, cy), Offset(cx - gap, cy), strokeWidth)
|
||||
drawLine(crosshairColor, Offset(cx + gap, cy), Offset(cx + arm + gap, cy), strokeWidth)
|
||||
drawLine(crosshairColor, Offset(cx, cy - arm - gap), Offset(cx, cy - gap), strokeWidth)
|
||||
drawLine(crosshairColor, Offset(cx, cy + gap), Offset(cx, cy + arm + gap), strokeWidth)
|
||||
}
|
||||
|
||||
// Detected badge
|
||||
if (detectionResult != null) {
|
||||
Box(
|
||||
modifier = Modifier
|
||||
.align(Alignment.TopEnd)
|
||||
.padding(8.dp)
|
||||
.background(Color.Red, shape = MaterialTheme.shapes.small)
|
||||
.padding(horizontal = 8.dp, vertical = 4.dp)
|
||||
) {
|
||||
Text("DETECTED", color = Color.White, fontSize = 12.sp)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Bottom control panel
|
||||
Column(
|
||||
modifier = Modifier
|
||||
.fillMaxWidth()
|
||||
.background(Color(0xFF323031))
|
||||
.padding(8.dp)
|
||||
) {
|
||||
// Device selector + connect button
|
||||
Row(
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
verticalAlignment = Alignment.CenterVertically
|
||||
) {
|
||||
Box(modifier = Modifier.weight(1f)) {
|
||||
OutlinedButton(
|
||||
onClick = { showDeviceMenu = true },
|
||||
modifier = Modifier.fillMaxWidth()
|
||||
) {
|
||||
Text(
|
||||
text = selectedDevice?.device?.deviceName
|
||||
?: if (availableDevices.isEmpty()) "No devices" else "Select device",
|
||||
maxLines = 1,
|
||||
color = Color.White
|
||||
)
|
||||
}
|
||||
DropdownMenu(
|
||||
expanded = showDeviceMenu,
|
||||
onDismissRequest = { showDeviceMenu = false }
|
||||
) {
|
||||
if (availableDevices.isEmpty()) {
|
||||
DropdownMenuItem(
|
||||
text = { Text("No USB serial devices found") },
|
||||
onClick = { showDeviceMenu = false }
|
||||
)
|
||||
} else {
|
||||
availableDevices.forEach { driver ->
|
||||
DropdownMenuItem(
|
||||
text = {
|
||||
Text(
|
||||
"${driver.device.deviceName} " +
|
||||
"(${driver.javaClass.simpleName.removeSuffix("Driver")})"
|
||||
)
|
||||
},
|
||||
onClick = {
|
||||
selectedDevice = driver
|
||||
showDeviceMenu = false
|
||||
}
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Spacer(modifier = Modifier.width(8.dp))
|
||||
|
||||
Button(
|
||||
onClick = {
|
||||
when (connectionState) {
|
||||
UsbSerialManager.ConnectionState.CONNECTED -> vm.disconnect()
|
||||
UsbSerialManager.ConnectionState.DISCONNECTED,
|
||||
UsbSerialManager.ConnectionState.ERROR -> {
|
||||
selectedDevice?.let { vm.connect(it) }
|
||||
}
|
||||
else -> {}
|
||||
}
|
||||
},
|
||||
enabled = selectedDevice != null &&
|
||||
connectionState != UsbSerialManager.ConnectionState.CONNECTING &&
|
||||
connectionState != UsbSerialManager.ConnectionState.AWAITING_PERMISSION,
|
||||
colors = ButtonDefaults.buttonColors(
|
||||
containerColor = if (connectionState == UsbSerialManager.ConnectionState.CONNECTED)
|
||||
MaterialTheme.colorScheme.error
|
||||
else
|
||||
MaterialTheme.colorScheme.primary
|
||||
)
|
||||
) {
|
||||
Text(
|
||||
when (connectionState) {
|
||||
UsbSerialManager.ConnectionState.CONNECTED -> "Disconnect"
|
||||
UsbSerialManager.ConnectionState.CONNECTING -> "Connecting..."
|
||||
UsbSerialManager.ConnectionState.AWAITING_PERMISSION -> "Requesting..."
|
||||
else -> "Connect"
|
||||
}
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// Status row
|
||||
Row(
|
||||
modifier = Modifier
|
||||
.fillMaxWidth()
|
||||
.padding(top = 4.dp),
|
||||
verticalAlignment = Alignment.CenterVertically
|
||||
) {
|
||||
val stateColor = when (connectionState) {
|
||||
UsbSerialManager.ConnectionState.CONNECTED -> Color(0xFF4BA838)
|
||||
UsbSerialManager.ConnectionState.ERROR -> Color.Red
|
||||
else -> Color.Gray
|
||||
}
|
||||
Text(
|
||||
text = connectionState.name,
|
||||
color = stateColor,
|
||||
fontSize = 12.sp
|
||||
)
|
||||
Text(
|
||||
text = " | Team: $teamLabel",
|
||||
color = Color.White,
|
||||
fontSize = 12.sp
|
||||
)
|
||||
if (secondsSinceTrigger != null) {
|
||||
Text(
|
||||
text = " | Last trigger: ${secondsSinceTrigger}s ago",
|
||||
color = Color(0xFFFFC857),
|
||||
fontSize = 12.sp
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
package club.clubk.ktag.apps.sentry
|
||||
|
||||
import android.content.Context
|
||||
import androidx.preference.PreferenceManager
|
||||
|
||||
data class SentryConfig(
|
||||
val team: String,
|
||||
val cooldownMs: Long,
|
||||
val playDetectionTone: Boolean
|
||||
) {
|
||||
companion object {
|
||||
fun fromPreferences(context: Context): SentryConfig {
|
||||
val prefs = PreferenceManager.getDefaultSharedPreferences(context)
|
||||
val team = prefs.getString(SentryPreferenceKeys.TEAM, "asblue") ?: "asblue"
|
||||
val cooldownS = prefs.getString(SentryPreferenceKeys.COOLDOWN_S, "5")?.toLongOrNull() ?: 5L
|
||||
val playDetectionTone = prefs.getBoolean(SentryPreferenceKeys.PLAY_DETECTION_TONE, true)
|
||||
return SentryConfig(team = team, cooldownMs = cooldownS * 1000L, playDetectionTone = playDetectionTone)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
package club.clubk.ktag.apps.sentry
|
||||
|
||||
import android.content.Context
|
||||
import androidx.startup.Initializer
|
||||
import club.clubk.ktag.apps.core.SubAppRegistry
|
||||
|
||||
class SentryInitializer : Initializer<Unit> {
|
||||
override fun create(context: Context) {
|
||||
SubAppRegistry.register(SentrySubApp())
|
||||
}
|
||||
|
||||
override fun dependencies(): List<Class<out Initializer<*>>> = emptyList()
|
||||
}
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
package club.clubk.ktag.apps.sentry
|
||||
|
||||
object SentryPreferenceKeys {
|
||||
const val TEAM = "sentry_team"
|
||||
const val COOLDOWN_S = "sentry_cooldown_s"
|
||||
const val PLAY_DETECTION_TONE = "sentry_play_detection_tone"
|
||||
}
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
package club.clubk.ktag.apps.sentry
|
||||
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import club.clubk.ktag.apps.sharedservices.BaseSettingsActivity
|
||||
|
||||
class SentrySettingsActivity : BaseSettingsActivity() {
|
||||
companion object {
|
||||
@JvmStatic
|
||||
fun createIntent(context: Context): Intent {
|
||||
return BaseSettingsActivity.createIntent(
|
||||
context,
|
||||
R.xml.preferences_sentry,
|
||||
SentrySettingsActivity::class.java
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
package club.clubk.ktag.apps.sentry
|
||||
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import club.clubk.ktag.apps.sharedservices.SettingsSubApp
|
||||
|
||||
class SentrySubApp : SettingsSubApp {
|
||||
override val id = "sentry"
|
||||
override val name = "Sentry"
|
||||
override val icon = R.drawable.ic_sentry
|
||||
override val settingsPreferencesResId = R.xml.preferences_sentry
|
||||
override val usesMqtt = false
|
||||
|
||||
override fun createIntent(context: Context): Intent {
|
||||
return Intent(context, SentryActivity::class.java)
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,273 @@
|
|||
package club.clubk.ktag.apps.sentry
|
||||
|
||||
import android.app.Application
|
||||
import android.media.AudioAttributes
|
||||
import android.media.AudioFormat
|
||||
import android.media.AudioManager
|
||||
import android.media.AudioTrack
|
||||
import android.media.ToneGenerator
|
||||
import kotlin.math.sin
|
||||
import android.os.Handler
|
||||
import android.os.Looper
|
||||
import android.util.Log
|
||||
import android.graphics.Bitmap
|
||||
import android.graphics.Matrix
|
||||
import androidx.camera.core.ImageAnalysis
|
||||
import androidx.camera.core.ImageProxy
|
||||
import androidx.lifecycle.AndroidViewModel
|
||||
import club.clubk.ktag.apps.core.UsbSerialManager
|
||||
import com.hoho.android.usbserial.driver.UsbSerialDriver
|
||||
import com.google.mediapipe.framework.image.BitmapImageBuilder
|
||||
import com.google.mediapipe.tasks.core.BaseOptions
|
||||
import com.google.mediapipe.tasks.vision.core.RunningMode
|
||||
import com.google.mediapipe.tasks.vision.objectdetector.ObjectDetector
|
||||
import kotlinx.coroutines.flow.MutableStateFlow
|
||||
import kotlinx.coroutines.flow.StateFlow
|
||||
import kotlinx.coroutines.flow.asStateFlow
|
||||
|
||||
data class DetectionResult(
|
||||
val left: Float,
|
||||
val top: Float,
|
||||
val right: Float,
|
||||
val bottom: Float,
|
||||
val confidence: Float
|
||||
)
|
||||
|
||||
class SentryViewModel(application: Application) : AndroidViewModel(application) {
|
||||
|
||||
private val mainHandler = Handler(Looper.getMainLooper())
|
||||
|
||||
private val _connectionState = MutableStateFlow(UsbSerialManager.ConnectionState.DISCONNECTED)
|
||||
val connectionState: StateFlow<UsbSerialManager.ConnectionState> = _connectionState.asStateFlow()
|
||||
|
||||
private val _availableDevices = MutableStateFlow<List<UsbSerialDriver>>(emptyList())
|
||||
val availableDevices: StateFlow<List<UsbSerialDriver>> = _availableDevices.asStateFlow()
|
||||
|
||||
private val _detectionResult = MutableStateFlow<DetectionResult?>(null)
|
||||
val detectionResult: StateFlow<DetectionResult?> = _detectionResult.asStateFlow()
|
||||
|
||||
private val _lastTriggerMs = MutableStateFlow(0L)
|
||||
val lastTriggerMs: StateFlow<Long> = _lastTriggerMs.asStateFlow()
|
||||
|
||||
private val _config = MutableStateFlow(SentryConfig.fromPreferences(application))
|
||||
val config: StateFlow<SentryConfig> = _config.asStateFlow()
|
||||
|
||||
private val _statusMessage = MutableStateFlow<String?>(null)
|
||||
val statusMessage: StateFlow<String?> = _statusMessage.asStateFlow()
|
||||
|
||||
private val usbSerialManager = UsbSerialManager(
|
||||
application,
|
||||
object : UsbSerialManager.Listener {
|
||||
override fun onDataReceived(data: ByteArray) {
|
||||
// Sentry doesn't process incoming serial data
|
||||
}
|
||||
|
||||
override fun onConnectionStateChanged(state: UsbSerialManager.ConnectionState) {
|
||||
mainHandler.post { _connectionState.value = state }
|
||||
}
|
||||
|
||||
override fun onError(message: String) {
|
||||
mainHandler.post { _statusMessage.value = message }
|
||||
}
|
||||
|
||||
override fun onDevicesChanged(devices: List<UsbSerialDriver>) {
|
||||
mainHandler.post { _availableDevices.value = devices }
|
||||
}
|
||||
|
||||
override fun onAutoConnecting(driver: UsbSerialDriver) {
|
||||
// Auto-connect is handled implicitly
|
||||
}
|
||||
},
|
||||
actionUsbPermission = "club.clubk.ktag.apps.sentry.USB_PERMISSION"
|
||||
)
|
||||
|
||||
private val toneGenerator = ToneGenerator(AudioManager.STREAM_MUSIC, ToneGenerator.MAX_VOLUME)
|
||||
@Volatile private var isDetectionTonePlaying = false
|
||||
|
||||
// Model file must be placed in src/main/assets/efficientdet_lite0.tflite.
|
||||
// Download from: https://tfhub.dev/tensorflow/lite-model/efficientdet/lite0/detection/metadata/1
|
||||
private val objectDetector: ObjectDetector by lazy {
|
||||
val baseOptions = BaseOptions.builder()
|
||||
.setModelAssetPath("efficientdet_lite0.tflite")
|
||||
.build()
|
||||
val options = ObjectDetector.ObjectDetectorOptions.builder()
|
||||
.setBaseOptions(baseOptions)
|
||||
.setRunningMode(RunningMode.IMAGE)
|
||||
.setMaxResults(5)
|
||||
.setScoreThreshold(0.5f)
|
||||
.build()
|
||||
ObjectDetector.createFromOptions(getApplication(), options)
|
||||
}
|
||||
|
||||
fun registerUsb() {
|
||||
usbSerialManager.register()
|
||||
}
|
||||
|
||||
fun unregisterUsb() {
|
||||
usbSerialManager.unregister()
|
||||
}
|
||||
|
||||
fun connect(driver: UsbSerialDriver) {
|
||||
usbSerialManager.connect(driver)
|
||||
}
|
||||
|
||||
fun disconnect() {
|
||||
usbSerialManager.disconnect()
|
||||
}
|
||||
|
||||
fun reloadConfig() {
|
||||
_config.value = SentryConfig.fromPreferences(getApplication())
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns an ImageAnalysis.Analyzer to be attached to the CameraX ImageAnalysis use case.
|
||||
* The composable is responsible for creating and binding the use case.
|
||||
*/
|
||||
fun buildAnalyzer(): ImageAnalysis.Analyzer = ImageAnalysis.Analyzer { imageProxy ->
|
||||
processImage(imageProxy)
|
||||
}
|
||||
|
||||
private fun processImage(imageProxy: ImageProxy) {
|
||||
try {
|
||||
val bitmap = imageProxy.toBitmap()
|
||||
val rotationDegrees = imageProxy.imageInfo.rotationDegrees
|
||||
val rotatedBitmap = if (rotationDegrees != 0) {
|
||||
val matrix = Matrix().apply { postRotate(rotationDegrees.toFloat()) }
|
||||
Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true)
|
||||
} else {
|
||||
bitmap
|
||||
}
|
||||
|
||||
val mpImage = BitmapImageBuilder(rotatedBitmap).build()
|
||||
val results = objectDetector.detect(mpImage)
|
||||
|
||||
val personDetection = results.detections()
|
||||
.filter { it.categories().any { cat -> cat.categoryName() == "person" } }
|
||||
.maxByOrNull { it.categories().first { cat -> cat.categoryName() == "person" }.score() }
|
||||
|
||||
if (personDetection == null) {
|
||||
mainHandler.post {
|
||||
_detectionResult.value = null
|
||||
stopDetectionTone()
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
val box = personDetection.boundingBox()
|
||||
val w = rotatedBitmap.width.toFloat()
|
||||
val h = rotatedBitmap.height.toFloat()
|
||||
val result = DetectionResult(
|
||||
left = (box.left / w).coerceIn(0f, 1f),
|
||||
top = (box.top / h).coerceIn(0f, 1f),
|
||||
right = (box.right / w).coerceIn(0f, 1f),
|
||||
bottom = (box.bottom / h).coerceIn(0f, 1f),
|
||||
confidence = personDetection.categories().first { it.categoryName() == "person" }.score()
|
||||
)
|
||||
val intersectsCrosshair = result.left <= 0.5f && 0.5f <= result.right &&
|
||||
result.top <= 0.5f && 0.5f <= result.bottom
|
||||
mainHandler.post {
|
||||
_detectionResult.value = result
|
||||
if (_config.value.playDetectionTone) startDetectionTone() else stopDetectionTone()
|
||||
if (intersectsCrosshair) {
|
||||
maybeTrigger()
|
||||
}
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "Detection failed", e)
|
||||
mainHandler.post {
|
||||
_detectionResult.value = null
|
||||
stopDetectionTone()
|
||||
}
|
||||
} finally {
|
||||
imageProxy.close()
|
||||
}
|
||||
}
|
||||
|
||||
private fun maybeTrigger() {
|
||||
val cfg = _config.value
|
||||
val now = System.currentTimeMillis()
|
||||
if (now - _lastTriggerMs.value >= cfg.cooldownMs) {
|
||||
_lastTriggerMs.value = now
|
||||
val trigger_press_command = "KEvent 7 0\n"
|
||||
usbSerialManager.write(trigger_press_command.toByteArray(Charsets.UTF_8))
|
||||
Log.i(TAG, "Triggered: $trigger_press_command")
|
||||
playTriggerSound()
|
||||
val trigger_release_command = "KEvent 8 0\n"
|
||||
usbSerialManager.write(trigger_release_command.toByteArray(Charsets.UTF_8))
|
||||
}
|
||||
}
|
||||
|
||||
private fun startDetectionTone() {
|
||||
if (isDetectionTonePlaying) return
|
||||
isDetectionTonePlaying = true
|
||||
|
||||
val sampleRate = 44100
|
||||
val frequency = 880.0 // A5 — one octave above the standard 440 Hz dial tone
|
||||
val bufferSize = AudioTrack.getMinBufferSize(
|
||||
sampleRate, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT
|
||||
)
|
||||
val track = AudioTrack.Builder()
|
||||
.setAudioAttributes(
|
||||
AudioAttributes.Builder()
|
||||
.setUsage(AudioAttributes.USAGE_MEDIA)
|
||||
.setContentType(AudioAttributes.CONTENT_TYPE_SONIFICATION)
|
||||
.build()
|
||||
)
|
||||
.setAudioFormat(
|
||||
AudioFormat.Builder()
|
||||
.setSampleRate(sampleRate)
|
||||
.setEncoding(AudioFormat.ENCODING_PCM_16BIT)
|
||||
.setChannelMask(AudioFormat.CHANNEL_OUT_MONO)
|
||||
.build()
|
||||
)
|
||||
.setBufferSizeInBytes(bufferSize)
|
||||
.setTransferMode(AudioTrack.MODE_STREAM)
|
||||
.build()
|
||||
track.play()
|
||||
|
||||
Thread {
|
||||
val buffer = ShortArray(bufferSize / 2)
|
||||
var phase = 0.0
|
||||
val phaseIncrement = 2.0 * Math.PI * frequency / sampleRate
|
||||
try {
|
||||
while (isDetectionTonePlaying) {
|
||||
for (i in buffer.indices) {
|
||||
buffer[i] = (Short.MAX_VALUE * sin(phase)).toInt().toShort()
|
||||
phase += phaseIncrement
|
||||
if (phase >= 2.0 * Math.PI) phase -= 2.0 * Math.PI
|
||||
}
|
||||
if (track.write(buffer, 0, buffer.size) < 0) break
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "Detection tone thread error", e)
|
||||
} finally {
|
||||
track.stop()
|
||||
track.release()
|
||||
}
|
||||
}.start()
|
||||
}
|
||||
|
||||
private fun stopDetectionTone() {
|
||||
isDetectionTonePlaying = false
|
||||
// The tone thread sees the flag, exits its loop, and releases the AudioTrack in its finally block.
|
||||
}
|
||||
|
||||
private fun playTriggerSound() {
|
||||
try {
|
||||
toneGenerator.startTone(ToneGenerator.TONE_PROP_BEEP, 200)
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "Could not play trigger sound", e)
|
||||
}
|
||||
}
|
||||
|
||||
override fun onCleared() {
|
||||
super.onCleared()
|
||||
stopDetectionTone()
|
||||
objectDetector.close()
|
||||
toneGenerator.release()
|
||||
}
|
||||
|
||||
companion object {
|
||||
private const val TAG = "SentryViewModel"
|
||||
}
|
||||
}
|
||||
BIN
subapp-sentry/src/main/res/drawable/ic_sentry.png
Normal file
BIN
subapp-sentry/src/main/res/drawable/ic_sentry.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.5 MiB |
26
subapp-sentry/src/main/res/values/arrays.xml
Normal file
26
subapp-sentry/src/main/res/values/arrays.xml
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<resources>
|
||||
<string-array name="sentry_team_entries">
|
||||
<item>Blue</item>
|
||||
<item>Red</item>
|
||||
<item>Purple</item>
|
||||
</string-array>
|
||||
<string-array name="sentry_team_entry_values">
|
||||
<item>asblue</item>
|
||||
<item>asred</item>
|
||||
<item>aspurple</item>
|
||||
</string-array>
|
||||
|
||||
<string-array name="sentry_cooldown_entries">
|
||||
<item>3 seconds</item>
|
||||
<item>5 seconds</item>
|
||||
<item>10 seconds</item>
|
||||
<item>30 seconds</item>
|
||||
</string-array>
|
||||
<string-array name="sentry_cooldown_entry_values">
|
||||
<item>3</item>
|
||||
<item>5</item>
|
||||
<item>10</item>
|
||||
<item>30</item>
|
||||
</string-array>
|
||||
</resources>
|
||||
15
subapp-sentry/src/main/res/xml/device_filter.xml
Normal file
15
subapp-sentry/src/main/res/xml/device_filter.xml
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<resources>
|
||||
<!-- FTDI -->
|
||||
<usb-device vendor-id="1027" />
|
||||
<!-- Prolific PL2303 -->
|
||||
<usb-device vendor-id="1659" />
|
||||
<!-- Silabs CP210x -->
|
||||
<usb-device vendor-id="4292" />
|
||||
<!-- Qinheng CH340/CH341 -->
|
||||
<usb-device vendor-id="6790" />
|
||||
<!-- Arduino -->
|
||||
<usb-device vendor-id="9025" />
|
||||
<!-- Qinheng CH9102 -->
|
||||
<usb-device vendor-id="6790" product-id="21972" />
|
||||
</resources>
|
||||
19
subapp-sentry/src/main/res/xml/preferences_sentry.xml
Normal file
19
subapp-sentry/src/main/res/xml/preferences_sentry.xml
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android">
|
||||
<ListPreference
|
||||
android:key="sentry_team"
|
||||
android:title="Tag Team"
|
||||
android:entries="@array/sentry_team_entries"
|
||||
android:entryValues="@array/sentry_team_entry_values"
|
||||
android:defaultValue="asblue" />
|
||||
<ListPreference
|
||||
android:key="sentry_cooldown_s"
|
||||
android:title="Trigger Cooldown"
|
||||
android:entries="@array/sentry_cooldown_entries"
|
||||
android:entryValues="@array/sentry_cooldown_entry_values"
|
||||
android:defaultValue="5" />
|
||||
<SwitchPreferenceCompat
|
||||
android:key="sentry_play_detection_tone"
|
||||
android:title="Play Detection Tone"
|
||||
android:defaultValue="true" />
|
||||
</PreferenceScreen>
|
||||
Loading…
Add table
Add a link
Reference in a new issue