Deployment of the Next Generation Machine Vision Technology and Integration with SCOOT in the City of Minneapolis
ABSTRACT
Whereas virtually all the deployed and operational machine vision sensors are multi-camera units, there are potential applications where multiple cameras are not needed. Examples in which multi-camera units may not be appropriate are downtown intersections of one-way streets, work zone monitoring, data collection, and arterial status monitoring. A new class of machine vision sensors is emerging to fill this need. This new sensor integrates the camera optics with an image processor to offer a compact single-camera package. This new technology is currently being deployed in downtown Minneapolis. The deployment is a part of the Adaptive Urban Signal Control and Integration operational test of adaptive control sponsored by the Federal Highway Administration.
INTRODUCTION
Successful deployment of state-of-the-art technology in the field is essential for Intelligent Transportation Systems (ITS) to benefit the traveler. One of these new technologies, which is gaining acceptance among practicing traffic engineers, is wide area video detection (machine vision). Video detection has now been available commercially for several years (1) and is gaining acceptance as a more effective tool than conventional inductive loop detectors. The additional benefits of using video are many. Key is the ability to cover many lanes with a single camera and to extract wide area measurements such as spatial density, queue size, speed profile, and others. Additionally, lane closures are typically not needed during installation. This results in increased driver and construction crew safety and minimal traffic disruption. In fact, once installed, they are typically used during subsequent road construction or resurfacing as needed as the road geometry varies. Finally, if desired, the video can be used to supplement existing video surveillance. For these and other practical reasons video detection systems have generated much interest in advanced traffic management.
The machine vision systems available for traffic detection purposes have typically been multi-camera units. That is, video from multiple (typically up to eight) cameras is carried to a separate location where a Machine Vision Processor (MVP) is used for processing the video (2). The MVP is typically housed in a cabinet located several hundred feet away from the cameras.
There are many applications where no more than one or two cameras are needed to detect traffic. For example, downtown areas with one-way streets require only two cameras at each intersection. A multi-camera MVP is neither required nor desirable at an intersection with only two cameras. This compatibility gap between the needs and available solutions will very definitely limit the acceptance of video vehicle detection technology in ITS.
To fill this gap, an exciting new integrated machine vision technology has emerged offering well-matched solutions. This technology integrates the camera optics and the image processing electronics into one compact, single-camera unit, eliminating the need for a multi-camera MVP for single-camera applications. In addition, co-located electronics and optics allow the image processor to control the camera gain, brightness, illumination, and electronic zoom. The single-camera units are uniquely well suited for several applications. Such applications include Central Business Districts (CBD), one-way streets, rural monitoring and safety, arterial and block and incident detection, smart work zones, data collection stations, freeway surveillance, management and control, and others.
The new integrated machine vision system offers many benefits. For instance, it:
- Eliminates the need for high bandwidth video transmission between the camera and the MVP. This lowers installation cost, eliminates transmission-induced loss of image quality, and makes the deployment more rapid.
- Makes the system more readily portable by eliminating a major physical component and long, bulky, video cables.
- Enables closed loop control of the camera optics, such as illumination, gain, brightness, and electronic zoom, by the vision processor itself.
Thus, integrating the camera optics and the vision processor into a single-camera unit offers the users two alternative families of machine vision systems for different applications. The integrated system is currently being deployed in Minneapolis downtown as a part of the Adaptive Urban Signal Control and Integration (AUSCI) Program, sponsored by the Federal Highway Administration (FHWA) (3). The new sensor and the deployment establish a new ITS standard to be followed in further developing wide area traffic sensors.
MACHINE VISION SYSTEM
In 1995, the Minnesota Department of Transportation (Mn/DOT) specified the requirements for a machine vision sensor for adaptive control in the CBD of Minneapolis, Minnesota as part of the AUSCI program (4). The adaptive control scheme to be used was the Split Cycle Offset Optimization Technique (SCOOT). SCOOT requires demanding, precise detection of vehicles.
The Minneapolis CBD, like many other high traffic volume metro CBDs, has a grid of one-way streets. This implied that most intersections would need two machine vision cameras, making a four-camera MVP an overkill. Furthermore, SCOOT requires the detection zones on the pavements to be significantly away from the stop line. This is different from SCATS, where the detection zones are at the stop line (5). So, in SCATS the machine vision cameras are installed at the intersection itself; in SCOOT the cameras need to be installed away from the intersection, making the video transmission distances from the cameras to the cabinets at the intersection containing the MVPs much longer than in SCATS. Thus, the AUSCI deployment required an integrated sensor with the MVP integrated with the camera. To meet this need a new sensor that combines camera optics and the MVP into one integrated unit was developed.
One of the major technical benefits of co-locating the CPU and the optics is closed loop feedback control of the optics by the CPU. Traditionally, any adverse sudden change in the camera illumination is compensated for by the detection software in the CPU. In the integration sensor in AUSCI, the CPU constantly monitors selected metrics that relate the detector operation to the camera illumination. Based on the metrics, the CPU controls the optics so the detector performance is always optimized.
As can be seen in Figure 1, the external form of the sensor is not much different than that of a traffic video camera, yet the need for a separate MVP has been eliminated. Additionally, the sensor eliminates the need for the video transmission of a traditional multi-camera system deployment. There are four sets of input/output to/from the sensor through the sensor cable: power, image, data, and control. The power options are 24 volt d.c. or a.c. The 24V d.c. power supply option allows for the remote, isolated operation of the sensor with battery power, backed up by solar cells if desired. In the AUSCI program, 24V a.c. power supply was provided through a step-down transformer from a standard 110V supply.
The image output from the sensor is provided over a twisted pair in the sensor drop cable. The image at the sensor output is already processed by the MVP and includes the detector layout superimposed on it. This image is intended for display or video archival purposes. The superimposed detectors flash when a detection occurs, indicating sensor operation to the viewer. AUSCI deployment uses a set of Mil-Lectron video signal repeaters to transmit full motion surveillance video from the field. The machine vision video is multiplexed with the surveillance video and is transmitted uncompressed.
The data and control signals are available in RS232 or RS485 format. AUSCI deployment uses RS485 format. With commercially available 24 GHz spread spectrum wireless transmitters and receivers, both data and video can be transmitted wirelessly, making the integrated sensor an extremely convenient candidate for portable or rural applications. The sensor provides both real-time detection output as well as station data accumulated in the flash memory of the sensor over desired data collection intervals. The sensor drop cable, carrying the image and data, can then be connected to a central computer. In AUSCI deployment twisted pair wires were used to connect to the sensor drop cable.
The new sensor would result in a lower installation cost for single-camera system deployments. The deployment cost would also be lower because a multi-camera MVP and an MVP housing would be replaced with an equivalent of a single-camera MVP without any separate MVP housing. Elimination of the MVP housing and the video cables would improve portability of a single-camera application. Any degradation in the video quality due to transmission would be eliminated. The integrated sensor technology offers the users “smart cameras” that have the potential to individually perform a full range of vehicle detections starting from loop emulation and ranging to vehicle tracking, queue detection, and automated incident detection (6). Most importantly, such integration leads to the natural evolution of combining detection and surveillance in a single camera housing rather than having two separate systems, i.e. one for surveillance and one for detection.
COMMUNICATION SYSTEM
The infrastructure used for communication is the twisted pair communication system existing in Minneapolis CBD, integrated with additional twisted pair where desired. Figure 2 shows the communication infrastructure. For communicating over this mostly pre-existing twisted pair infrastructure, Image Sensing Systems provided a flexible Hub, shown in Figure 3. These Hubs facilitate all networking of data and video between the sensors and the TCC.
The Hub is a flexible, modular device that can be custom-configured for each location depending on its networking need. First, and foremost for the AUSCI project, the Hub has an EIM-P module that provides NEMA TS1 interface between the integrated sensor and the communications modification unit (CMU). Each EIM-P module provides 16 outputs and 8 inputs. A Hub can have multiple EIM-P modules if more outputs are required. The output of the CMU is then transmitted over a twisted pair to the TCC for analysis and processing by the SCOOT adaptive control system. There is also an EIM-S module option available for NEMA TS2 SDLC interface.
The Hub also has a video mux module for multiplexing live full motion video from the sensors to the TCC. The video mux module provides local manual selection or remote computerized selection of full motion video output from up to 8 sensors. In addition to multiplexing, the video mux module also performs relaying of the transmitted video.
The City of Minneapolis has nine (9) pan-tilt-zoom (PTZ) surveillance video cameras in the AUSCI test area. The video from the integrated traffic sensor is multiplexed with the PTZ camera video using the Hubs and transmitted to the TCC over twisted pair lines. At each intersection, the video multiplexer in the Hub can be manually switched to view the full motion video from any integrated machine vision sensor or any PTZ surveillance camera connected to the Hub. This video switching can also be done remotely from the TCC by software control. The full motion video can then be archived at the TCC using video tape recorders.
A third module available in the Hubs is the RS485 module. This module provides two functions: 1) Communications between the sensors and the supervisory computer at TCC and 2) relay communication for longer haul transmission over twisted pair and for transmission branch out to get the desired network topology. There is also an RS232 module option available for the use of devices such as modems.
SENSOR NETWORK BROWSER
The network of integrated sensors and communication Hubs in the AUSCI project can be browsed and managed through a network browser software at the TCC. Through the browser, one of several client software can be launched to:
- manage the network
- create, edit and remotely download desired detector configurations (e.g. number and type of detectors) to the sensors
- display sensor output (e.g. video, detection)
- remotely install software on the sensors or the Hubs, individually, or en-masse
- display the operations log of the sensors
- remotely set the operational parameters for the sensors and the Hubs
- remotely read traffic data collected by the individual sensors
- remotely monitor the status of the sensors and the Hubs.
The browser software interfaces with the sensors and the Hubs through a communication server (Comserver) software. The Comserver software resides on an IBM-compatible desktop or notebook PC platform operating with Windows NT Workstation 4.0 operating system.
The Network Manager client automatically configures communication addresses for the sensors and the Hubs. The users may, instead, define their own addressing scheme, if desired. The Network Manager polls each channel and automatically locates any sensor or Hub on the polled channel. Thus, the Network Manager automatically learns the network configuration and graphically displays the network (see Figure 4).
The Detector Editor client allows the operator at the TCC to:
- read video snapshots from the video sensors
- layout and save detector (configuration files)
- edit an existing detector file
- write the detector files to the video sensors in the field
Each video sensor in the field maintains an automatic, comprehensive operations log (Oplog) that is used to trouble shoot problems, if desired.
The Oplog Viewer client allows an operator at the TCC to access any or all of the video sensors and Hubs, read any or all of the operations logs, and store them in a Central Oplog File. The Oplog Viewer allows the operator to read and filter the Central Oplog to display the information and print it.
CURRENT STATUS OF SENSOR DEPLOYMENT IN AUSCI
The AUSCI operational test is focusing on a section of the northwest side of the CBD of Minneapolis. The test area, shown in Figure 2, will involve 65 intersections, out of 780 signalized intersections in Minneapolis. The location was chosen because it is an events area with large parking facilities to serve stadium and arena sporting events and ramps connecting to two major freeways, which results in irregular surges of traffic in addition to normal daily surges of rush hour traffic.
The operational test will include 138 integrated video sensors and 65 Hubs provided by Image Sensing Systems, Inc. The city has completed the installation of the sensors and almost all of the Hubs. The sensor installation typically took place on weekdays after 9:00 A.M. and before 3:00 P.M. This time constraint was designed so there was minimum disruption of downtown traffic due to the installation. The installation of each sensor was extremely quick, taking approximately 1.5 hours.
Currently, the city has just started the process of calibration and validation of the SCOOT system in a small subset, approximately 15 intersections, of the operational test area. The evaluation of the completion system is scheduled to start in September 1998.
CONCLUSION
A new traffic video sensor technology is presented. The new technology combines the camera optics and the machine vision processor into a compact, integrated video sensor. The integrated video sensor is well suited for certain intersection and freeway applications. The prior technology involving a multi-camera unit is also desirable in some applications. It is expected that the two synergistic and complementary technologies will co-exist and offer solutions in their own niches. The integrated sensor has been installed in downtown Minneapolis for SCOOT adaptive control as a part of the AUSCI operational test. The full AUSCI system evaluation is scheduled to start in September 1998.
REFERENCES
(1)Fenichell, et al. Advanced Traffic Detection Emerging Technologies and Market Forecast. Transport Technology Publishing, 1995.
(2) Michalopoulos, P.G. Vehicle Detection Through Video Image Processing: The AUTOSCOPE System. IEEE Transactions on Vehicular Technology, Vol. 40, No. 1, 1991, pp. 21-29.
(3) Remer, M., A. S. Klugman, D.W. Hildebrand and W.M. Belrose Implementing Adaptive Signal Control in a Downtown Area: The Minneapolis AUSCI project, presented at the 8th Annual Meeting of Intelligent Transportation Society of America, May 1998, Detroit, Michigan.
(4) Minnesota Department of Transportation. Request for Partnership Proposals, Video Detection Systems, Adaptive Urban Signal Control Integration (AUSCI) Project, Sept. 1995.
(5) Vinger, S. Autoscope and SCATS Together in FAST-TRAC. Traffic Technology International, Sept. 1997, pp. 51-54.
(6) Michalopoulos, P.G., Jacobson, R.D., Anderson, C.A., and DeBruycker, T.B. Automatic Incident Detection Through Video Image Processing. Traffic Engineering & Control, Feb. 1993, pp. 66-75.
Authors
Durga Panda, Image Sensing Sytems, Inc. and Panos Michalopoulos of the University of MinnesotaAs presented at the 5th ITS World Congress 12-16 October 1998
©1998 Image Sensing Systems, Inc.
