News

By Charles Drutman and Mark Hutcheson - The world is in the midst of a major paradigm shift in computing and connectivity resulting from the availability of low cost, high-power mobile electronic devices. These devices are — somewhat dismissively — called smart phones and tablets but are actually eminently portable, always-connected computers. Smart phones and tablets are expected to give the warfighter new and highly significant capabilities on the battlefield. Optical Alchemy Inc. (OAI), based in Nashua, N.H., is coupling the capabilities of these highly mobile devices to unmanned vehicle payloads.

A live image stream from OAI’s KJ-640 gimbaled sensor is highlighted by having the live transport stream simultaneously displayed on a ruggedized laptop, a Galaxy tablet and a Droid X phone. Photo courtesy OAI.

In-Air Processing

Historically speaking, it has been very difficult to get data from remote payloads to warfighters. Data is often streamed to only one location where it is then captured and distributed to other nodes in the net-centric system. Because of this architecture, delays in data distribution and command and control response compromise functionality. OAI has addressed this problem by putting all processing in the sensor payload, making each sensor a central distribution point for real-time video and data. The result has been measured video and data latencies under 200 milliseconds, which appear as real time for a human operator.

For more than 10 years, OAI has been developing cutting-edge ultralight geo-referenced gimbaled sensor payloads for UAS applications. OAI has implemented an Internet protocol (IP)-based system incorporating both command and control functions and video and data streaming. These streams are multicast to all users in the field, as well as to command centers anywhere in the world via the ubiquitous protocol. IP streaming also allows mobile devices at the receiving end to replace larger, more traditional computer systems.

With these two capabilities (in-air processing and the IP-based architecture), OAI’s sensor payloads are able to easily integrate and interact with mobile devices in a real-time and fully functional way.

Smart Phone Capabilities

OAI is developing a suite of programs based on the Android operating system and iOS, Apple’s operating system, that interacts with its lightweight, georeferenced sensors to give the warfighter these new smart phone capabilities.

OAI is adding several new smart phone-based capabilities to its KJ- 640 gimbaled sensor system, including new user interface modalities and the integration of field-generated inputs into a common operating picture.

A major focus of OAI’s work on command and control has been to support multiple interaction modalities. For example, Geocam4D supports user interaction through a touch screen, mouse and handcontroller; which one a user selects may depend as much on conditions as on preference. For example, while a user may have a personal preference for the touch screen interface, conditions on the ground (i.e., in a moving truck on a bumpy dirt road) may force them to use a hand-controller as the only means to effectively interact with the KJ-640.

The company has implemented touch screen operations that mimic the common operations that mobile users are familiar with, such as pinch to control zoom. It is also experimenting with additional interface modalities enabled by many mobile devices’ internal sensor packages. For example, the three-axis orientation sensors have allowed the company to implement sensor control based upon user motion and orientation. Geoid4D has a follow mode where the sensor tracks the orientation of the user. That is, if the user faces 20 degrees west of north, so does the sensor.

OAI has also demonstrated voice control using built-in speech-to-text libraries. Geoid4D currently supports a limited vocabulary that supports azimuth and zoom control and the KJ-generated metadata sent through text messaging. The company plans to integrate a separate custom GIS (geographic information system) program running on the Galaxy Tab into Geoid4D to allow sensor control based upon touch screen GIS interaction. This will mimic the Geocam4D capability to slew the sensor from a map-based context.

One of the key issues the company has addressed in supporting field operations is the detection of and response to intermittent communication links. From a user perspective, intermittent communications create delays between user-entered command and control functions and visual confirmation. These delays can cause intense user frustration. To alleviate this problem, Optical Alchemy has developed proprietary algorithms to monitor the health of the link and automatically adjust to changing conditions. For example, when the system detects that the link has been compromised, it forces the user into a view-only mode. Aside from preventing user frustration, this also prevents any possibility that a large number of commands will be stacked and sent in burst mode when the link returns. When the link is weak but still available, the system automatically modifies communication parameters to support as much command and control as possible without overloading the link.

The company has already ported most of this code over to the Droid, so the current implementation of its handheld based system contains a similar level of functionality and robustness to changing link conditions.

All of the capabilities discussed so far have focused on sending data generated from the KJ-640 sensor to users in the field, but mobile devices can also support the dissemination of data generated by users in the field.

The company can extend this capability to allow in-field users to send geo-referenced imagery and, through image processing, determine, for example, the locations of shadow-producing objects. Combining this with an algorithm for enhancing shadowed objects, the system can automatically trigger this algorithm in real time, based upon the sun’s position, to highlight objects that might otherwise be missed.

Possibilities

The introduction of what are essentially low-cost mini-computers with multiple sensing modalities has opened a wide range of possibilities for supporting net-centric operations and providing a common operating picture to all echelons of warfighters. While there are differences in capability and usability among different manufacturers’ devices, there is enough commonality to allow any given device to be mastered relatively quickly. This means minimal training and quick acceptance by users in the field.

The keys to acceptance are robustness of operation and ease of use. For field operations with remote assets, like UAV-based gimbaled sensor payloads, it is therefore critical to respond to changing link conditions so that the user knows what they can and cannot do at any given time. OAI has found that the code it has implemented on a PC translates in large part to mobile devices to create a robust command and control system based on mobile devices.

OAI believes mobile devices will likely find their way into warfighters’ hands in increasing numbers as a means to provide a common operating picture in a net-centric environment and will greatly expanded capabilities for ground based troops.

Charles Drutman is vice president of software and algorithm development at Optical Alchemy. His primary responsibilities include the development of ground station software and the exploitation of imagery and data from the KJ class of gimbals in support of our warfighters.

Mark Hutcheson is the director of business development at Optical Alchemy. His responsibilities include understanding warfighter requirements, driving product developments that meet those requirements and developing customer relationships to deliver those solutions to the warfighter.

Download entire article (.pdf 254KB)