A New Integrated Video Sensor Technology for Traffic Management Applications

ABSTRACT

An exciting new machine vision technology has emerged to complement the current vehicle detection technology in Intelligent Transportation Systems applications. Machine vision sensor technology offers several advantages over conventional in-pavement sensors in traffic management. However, current systems do not meet certain needs of advanced traffic management. Whereas virtually all the deployed and operational machine vision sensors are multi-camera units, there are potential applications where multiple cameras are not needed. Examples in which multi-camera units may not be appropriate are downtown intersections of one-way streets, work zone monitoring, data collection, and arterial status monitoring. A new class of machine vision sensor is emerging to fill this need. This new sensor integrates the camera optics with an image processor to offer the traffic engineer choices such as incident detection, queue size measurement, turning movement extraction, vehicle tracking, and traditional loop emulation in a compact single-camera package. Such an integrated unit does not require a relatively expensive multi-camera processor box or chassis. It also reduces infrastructure installation cost by eliminating the need for the transmission of high bandwidth video from the camera to the processor. It establishes a new standard and opens up exciting possibilities of a whole new breed of systems that could lead to much wider scale accelerated deployment of non-intrusive, wide area sensors. This new technology is currently being deployed in downtown Minneapolis in partnership with the Minnesota Department of Transportation. The deployment is a part of the Adaptive Urban Signal Control and Integration operational test of the Split Cycle Offset Optimization Technique adaptive control sponsored by the Federal Highway Administration.

INTRODUCTION

Successful deployment of state-of-the-art technology in the field is essential for Intelligent Transportation Systems (ITS) to benefit the traveler. One of these new technologies, which is gaining acceptance among practicing traffic engineers, is wide area video detection (machine vision). Video detection has now been available commercially for several years and is gaining acceptance as a more effective tool than conventional inductive loop detectors. A very good summary of available technology alternatives can be found in (1). The additional benefits of using video are many. Key is the ability to cover many lanes with one camera and extract wide area measurements such as spatial density, queue size, speed profile, and others. Additionally, lane closures are typically not needed during installation. This results in increased driver and construction crew safety and minimal traffic disruption. In fact, once installed, they are typically used during subsequent road construction or resurfacing as needed as the road geometry varies. Finally, if desired, the video can be used to supplement existing video surveillance. For these and other practical reasons video detection systems have generated much interest in advanced traffic management.

The machine vision systems available for traffic detection purposes have typically been multi-camera units (2,3,4). That is, video from multiple (typically from four up to eight) cameras is carried to a separate location where a Machine Vision Processor (MVP) is used for processing the video. The MVP is typically housed in a cabinet located several hundred feet away from the cameras (5).

There are many applications where no more than one or two cameras are needed to detect traffic. For example, downtown areas with one-way streets require only two cameras at each intersection. A multi-camera MVP is neither required nor desirable at an intersection with only two cameras. This compatibility gap between the needs and available solutions will very definitely limit the acceptance of video vehicle detection technology in ITS.

To fill this gap, an exciting new integrated machine vision technology has emerged offering well matched solutions. This technology integrates the camera optics and the image processing electronics into one compact single-camera unit, eliminating the need for a multi-camera MVP for single-camera applications. In addition, co-located electronics and optics allow the image processor to control the camera gain, brightness, and illumination. The single-camera units are uniquely well suited for several applications. Some example applications are:

  • Central Business District (CBD)—Traffic control on the one-way streets of CBDs can be done with nominally two, single-camera units per intersection without a four-camera MVP.
  • Arterial Status Monitoring—Mid-block monitoring of arterial status, e.g. volume and speed, can be done with a single-camera unit.
  • Rural Highway Incident Detection—Highways where the traffic alarm condition is to be monitored with a very sparse spacing of cameras are good candidates for single-camera units. For example, monitoring of rural interstate freeways for incidents and white-out conditions in northern states or automatic monitoring of traffic speeds for safety at rural road curvatures could be better done with a single-camera unit. However, incident detection on high volume metro area expressways, such as on the Gowanus Expressway in New York or freeways in Atlanta, Georgia (4,5,6), where dense camera spacing is desired and optical fiber link infrastructure is available for video transmission at no extra cost, four-camera or eight-camera units are appropriate.
  • Smart Work Zone—Automatic safety monitoring of work zone areas typically requires one or two cameras (7). Thus, single-camera units would be more desirable for these applications.
  • Data Collection—Stations where wide-area traffic data extraction is required are good candidates for single-camera units.

The system offers several other benefits as well. For example, it:

  • Eliminates the need for high bandwidth video transmission between the camera and the MVP. This lowers installation cost, eliminates transmission-induced loss of image quality, and makes the deployment more rapid.
  • Makes the system more readily portable by eliminating a major physical component and long, bulky, video cables.
  • Enables closed loop control of the camera optics, such as illumination, gain, and brightness by the vision processor itself.

Thus, integrating the camera optics and the vision processor into a single-camera unit offers the users two alternative families of systems for different applications. The system, called Autoscope Solo, is currently being deployed in Minneapolis downtown as a part of the Adaptive Urban Signal Control and Integration (AUSCI) Program, sponsored by the Federal Highway Administration (FHWA)(8). The new sensor and the deployment establish a new ITS standard to be followed in further developing wide area traffic sensors.

The next section gives a brief background leading to the development of the new sensor. This is followed by a description of the integrated sensor system design. The subsequent section describes the AUSCI deployment. A brief description of freeway application potential follows the description of the AUSCI intersection application. The paper concludes with a vision of the future of wide-area video sensor deployment.

BACKGROUND

Large scale deployment of the wide-area video vehicle sensor started in 1992 in Oakland County, Michigan with the FAST-TRAC program (9). It used a multi-camera video sensor initially developed at the University of Minnesota (10). FAST-TRAC deployment was aimed at the use of adaptive control to improve traffic management in Oakland County, in the suburbs of Detroit, Michigan.

The adaptive control scheme selected for the FAST-TRAC program was the Sydney Coordinated Adaptive Traffic System (SCATS), requiring demanding precision from the detection sensor. For this demanding application, the Road Commission for Oakland County (RCOC) selected the machine vision sensor because of its non-intrusive nature, the ability to perform repairs and maintenance activities in harsh winter climate, and because of the potential of expanded benefits from this technology in the future.

Today, RCOC has machine vision technology installed at approximately 300 intersections, the largest installation worldwide, and is still expanding (5). In Phase II-B and Phase III of the FAST-TRAC program, RCOC is developing surface street incident detection and automatic video archiving using machine vision sensors.

The intersections under machine vision detection in Oakland County have two-way streets and four, or sometimes more, approaches per intersection. Thus, the intersections use four-camera units and eight-camera units of machine vision processors. This is in contrast to the requirements of the AUSCI adaptive control deployment.

In 1995, the Minnesota Department of Transportation (Mn/DOT) specified the requirements for a machine vision sensor for adaptive control in the CBD of Minneapolis, Minnesota as part of the AUSCI program. The adaptive control scheme to be used was not SCATS but the Split Cycle Offset Optimization Technique (SCOOT). SCOOT requires equally demanding, precise detection of the vehicles, and so the machine vision technology used in Oakland County should be applicable in Minneapolis. However, there are some differences in the SCOOT application in Minneapolis’s CBD and the SCATS application in Oakland County that required new sensor technology development.

First of all, the Minneapolis CBD, like many other high traffic volume metro CBDs, has a grid of one-way streets. This implied that most intersections would need two machine vision cameras, making a four-camera MVP an overkill.

Secondly, SCOOT requires the detection zones on the pavements to be significantly away from the stop line. This is different from SCATS, where the detection zones are at the stop line. So, in SCATS the machine vision cameras are installed at the intersection itself; in SCOOT the cameras need to be installed away from the intersection, making the video transmission distances from the cameras to the cabinets at the intersection containing the MVPs much longer than in SCATS. Thus, the AUSCI deployment required an integrated sensor with the MVP integrated with the camera.

Technology for integrated image sensing and processing has been making steady progress over the last decade. A silicon-based Complementary Metal Oxide Semiconductor (CMOS) imaging sensor is being developed that will enable integrated processing chips on the same silicon substrate. However, this technology is not yet ready for commercial deployment on the roads. The Jet Propulsion Laboratory (JPL) has been developing an Active Pixel Sensor (APS) with the potential for integrated image sensing and processing which could see commercial use in the next three to four years. A practical solution for the AUSCI deployment was to develop a new integrated sensor whose detection software would be compatible with that used in Oakland County. This compatibility would allow the detection software, proven in the demanding SCATS adaptive control in Oakland County, to be easily ported to the system performing equally demanding detection for SCOOT adaptive control in the Minneapolis CBD.

INTEGRATED VIDEO SENSOR ARCHITECTURE

For the SCOOT detection requirements in the AUSCI program, a new integrated sensor was developed by Image Sensing Systems, Inc. The latest technology available, allowing the utmost miniaturization and integration, was used to develop the sensor. The technology allowed integration of Charge Coupled Devices (CCD) imagers with an Intel processor into Autoscope Solo, an integrated, wide-area traffic video sensor.

The architecture of the integrated traffic video sensor is shown in Figure 1. In the integrated sensor there is one MVP dedicated for each camera, and it is co-located with the camera in one compact housing. Each camera has a CCD transducer that converts the sensed optical photon energy into electrical video signals. Each MVP has a Central Processing Unit (CPU) based on Intel’s X86 family of chips. The choice of the processor chip makes the sensor software compatible with the multi-camera video detection sensor for the SCATS application in Oakland County. This allows all the proven detection software and artifact treatment software to be easily ported from the multi-camera MVP to the single-camera unit.

One of the major technical benefits of co-locating the CPU and the optics is closed loop feedback control of the optics by the CPU. Traditionally, any adverse, sudden change in the camera illumination is compensated for by the detection software in the CPU. In the integration sensor in AUSCI, the CPU constantly monitors selected metrics that relate the detector operation to the camera illumination. Based on the metrics, the CPU controls the optics so the detector performance is always optimized.

The integrated sensor design also includes an option for zoom and pan/tilt control. Electronic Pan/Tilt/Zoom (PTZ) control would enable a user to use the camera for scanning the scene in case of incidents, for example. However, this option was not intended, nor available, for use in the AUSCI deployment. Due to the precision required by SCOOT in the occupancy measurement, AUSCI machine vision sensors use fixed focal length cameras. Where the traffic detection is not required to be as precise, PTZ control could be used to alternately perform vehicle detection and manual surveillance. Examples of such applications are traffic status monitoring and data collection.

The sensor housing is designed for extremely easy installation. The camera housing is filled with nitrogen gas pressurized to 5 psi and is sealed to ensure long-term operation in outdoor environments. The sealed sensor housing is covered with a cylindrical-shaped sun shield that allows free rotation of the sensor for proper alignment of the image during installation. The sensor is equipped with a temperature-controlled heater to keep the faceplaste free of condensation. The backplate of the sensor is equipped with an environmentally-sealed, military standard connector that enables all the external connection in one quick snap. A single set of sensor drop cables emerges from the backplate connector to provide all the necessary external connections.

As can be seen in Figure 1, the external form of the sensor is not much different than that of a traffic video camera, yet the need for a separate MVP has been eliminated. Additionally, the sensor eliminates the need for the high quality video transmission of a traditional multi-camera system deployment (see Figure 2). There are four sets of wires to the sensor through the sensor cable: power, video, data, and control. The power options are 24 volt dc or ac. The 24V dc power supply option allows for the remote, isolated operation of the sensor with battery power (Figure 3), backed up by solar cells if desired. In the AUSCI program, 24V ac power supply was provided through a step-down transformer from a standard 110V supply.

The full frame-rate video output from the sensor is provided over a twisted pair in the sensor drop cable. The image at the sensor output is already processed by the MVP and may include the detector layout superimposed on it. This image is intended for display or detection performance verification purposes. The superimposed detectors flash when a detection occurs, indicating sensor operation to the viewer. The video compression option shown in Figure 3 is not used in AUSCI deployment. AUSCI deployment uses a set of Mil-Lectron video signal repeaters to transmit full motion surveillance video from the field. The machine vision video is multiplexed with the surveillance video and is transmitted uncompressed.

The data and control signals are available in RS485 format. With commercially available 24 GHz spread spectrum wireless transmitters and receivers, both data and video can be transmitted wirelessly, making the integrated sensor an extremely convenient candidate for portable or rural applications. The sensor provides both real-time detection output as well as station data accumulated in the flash memory of the sensor over desired data collection intervals. The sensor drop cable, carrying the image and data, can then be connected to available underground infrastructure, twisted pair, leased line, or optical fiber for transmission to a central computer. In AUSCI deployment twisted pair wires were used to connect to the sensor drop cable.

For SCOOT application, it was required that the sensor output be converted to contact closure equivalent to loop output in the intersection controller cabinet; and so the sensor data was carried to a communication Hub in the controller cabinet, where it was converted to NEMA standard TS-1 output.

The CPUs of the integrated sensor and the multi-camera MVP are compatible with each other. Consequently, any traffic application software developed for one can be easily ported to the other. This keeps the functional performance of each of the systems consistent in their individual niche applications. In addition, the integrated system offers some advantages in applications where a multi-camera unit is not required. Because there is less infrastructure installation required, deployment can be done more rapidly, especially for certain applications, such as data collection.

This can result in a lower installation cost for single-camera system deployments. The deployment cost can be lower because a multi-camera MVP and an MVP housing would be replaced with an equivalent of a single-camera MVP without any separate MVP housing. Elimination of the MVP housing and the video cables would improve portability of a single-camera application. Also, any degradation in the video quality due to transmission is eliminated. The integrated sensor technology offers the users “smart cameras” that can individually perform a full range of vehicle detections starting from loop emulation and ranging to vehicle tracking, queue detection, and automated incident detection (11). Most importantly, such integration leads to the natural evolution of combining detection and surveillance in a single camera housing rather than having two separate systems, i.e. one for surveillance and one for detection.

VIDEO SENSOR FEATURES AT THE TRAFFIC CONTROL CENTER

Each video sensor detection output in the AUSCI project is required to provide electrical contact closure signals through a Communications Modification Unit (CMU) located in the traffic signal controller cabinet. To achieve this, the output of each integrated video sensor is first transmitted to a communication Hub located in each cabinet. The Hub converts the sensor detection output to NEMA standard TS-1 signal and provides it to the CMU. The output of the CMU is then transmitted over a twisted pair communication network to the Traffic Control Center (TCC) for analysis and processing by the SCOOT adaptive control system.

Full motion video from each sensor is also transmitted to the TCC. This is done by multiplexing the sensor video with the surveillance camera video and using the surveillance camera communication path to transmit sensor video. The video multiplexing is done by the Hub in each cabinet. At each intersection, the video multiplexer in the Hub can be manually switched to view the full motion video from any video sensor or any surveillance camera connected to the Hub. This video switching can also be done remotely from the TCC by software control. The full motion video can be archived at the TCC using video tape recorders.

An operator at the TCC can also call up a digital video snapshot from any sensor to view or archive the snapshot in a computer. There are several other video sensor features available at the central system. The central system features are accessed by a set of Supervisor software. The Supervisor software provides communications and certain utility functions from the TCC to the video sensors in the field.

A block diagram of the Supervisor software is shown in Figure 4. Referring to Figure 4, the communication server software (Comserver) enables access to the field video sensors from various client routines.

The Comserver resides on an IBM-compatible desktop PC platform operating with Windows NT Workstation 4.0 operating system. Comserver also operates on a notebook computer that supports the Windows NT 4.0 operating system. Comserver supports multidrop communications (up to 57,600 buad) to the video sensors and Hubs in the field. Communication between the video sensors and the Comserver uses UDP/IP protocol. The multidrop communication allows a user to:

  • Read and write detector (configuration) files to the video sensors and Hubs (a detector file is a file documenting what detection functions are required by the operator and where on the filed of view the detections are to be done)
  • Collect video snapshots
  • Install software
  • Read the interval data flash file
  • Collect video sensor status information
  • Read operations logs
  • Read and write communications parameters

Communications between the client applications and the Comserver uses TCP/IP protocol. The client software may reside on the Comserver platform PC, i.e. the PC hosting the Comserver software, or on a separate PC. In the AUSCI program, the client applications reside on the Comserver platform PC. The client applications are shown in Figure 4.

The Detector Editor/Monitor client allows the operator at the TCC to:

  1. Read video snapshots from the video sensors
  2. Layout and save detector (configuration) files
  3. Edit an existing detector file
  4. Write the detector files to the video sensors in the field

The Install client allows the operator at the TCC to install new releases of video sensor software. The Install client allows the operator to install new releases of software on all video sensors in the field automatically.

The video sensor is capable of recording station detector interval data in its local flash memory. The Data Archiver client enables this recorded interval data to be retrieved and archived at the TCC. The Data Archiver can retrieve the flash stored data at user-specified intervals, e.g. one hour, six hours, and 24 hours. The archived data includes date and time stamp, type of data, and an indication of memory full (if and when it occurs). Data Archiver automatically starts and operates the data retrieval and archiving function on a time scheduled basis. It automatically retries the retrieval of all or portions of the data to obtain data missed during previous unsuccessful data collection passes. The data is archived in a format that automatically ports to EXCEL or other data analysis tools. Data Archiver maintains a log describing the data collection effort including the successful and the unsuccessful ones. The detection function by the field video sensor continues uninterrupted during the retrieval operation by the Data Archiver.

Each video sensor in the field maintains a comprehensive operations log that is required to record the following events:

  • Communication connections/disconnections
  • System reboots
  • Corrupted configuration files
  • Other serious or fatal errors

The Oplog Viewer client allows an operator at the TCC to access any or all of the video sensors and Hubs, read any or all of the operations logs, and store them in a Central Oplog File. The Oplog Viewer allows the operator to read and filter the Central Oplog to display the information and print it.

Each video sensor in the field will contain a set of diagnostic tests. The tests are intended to verify the operation of:

  • Processor
  • Memory
  • Camera Function
  • Communication

The Diagnostics client allows the operator at the TCC to access any of the video sensors and run one or more of the diagnostic tests. The Diagnostics client allows the operator to view the results of the diagnostic tests using its own Windows or the Oplog Viewer.

The Network Browser client provides a network “explorer” view of the video sensor network. It also provides a “layout” network view that can be overlaid on a map. It enables the operator at the TCC to browse the sensors and video snapshot images on the PC screen guided by a computerized map of the City of Minneapolis (see Figure 5).

STATUS OF DEPLOYMENT IN AUSCI

The AUSCI operational test is focussed on a section of the northwest side of the CBD of Minneapolis (12). The test area, shown in Figure 6, will involve 65 intersections, out of 780 signalized intersections in Minneapolis. The location was chosen because it is an events area with large parking facilities to serve stadium and arena sporting events and ramps connecting to two major freeways, which results in irregular surges of traffic in addition to normal daily surges of rush hour traffic.

First, an evaluation unit of the integrated sensor was installed for pre-deployment evaluation purposes at the intersection of Dunwoody Avenue and Colfax Avenue, one of the intersections in the AUSCI test area. This installation took place at the end of June 1997. In-pavement loops had been installed in two of the lanes for comparison with the integrated sensor output. The City of Minneapolis conducted evaluations covering a broad range of ambient conditions such as day, night, day/night transition, and others. In mid-September the City concluded the evaluation to its satisfaction and gave the go-ahead for the sensor deployment.

There are 138 integrated video sensors and 65 Hubs being provided by Image Sensing Systems, Inc. In mid-September of 1997, classroom training was given to the video sensor installation subcontractor along with the installation inspectors. The installation was restricted to weekdays after 9:00 A.M. and before 3:00 P.M., the time constraint designed to minimize disruption of downtown traffic. The installation of the sensors was quick and simple. After the initial learning curve, the installation took approximately one hour per sensor. Figure 7 shows the installation of one of the sensors with the Minneapolis skyline in the background. As can be seen in the figure, the installation requires very little lane closure as it can be done from a bucket truck on the roadside.

Currently the system is undergoing field integration. The City of Minneapolis plans to test the SCOOT adaptive control with the video sensors on a test area involving 23 intersections in the spring of 1998. The full AUSCI system turn-on is scheduled for September 1998.

CONCLUSION

A new traffic video sensor technology is presented. The new technology combines the camera optics and the machine vision processor into a compact, integrated video sensor. The integrated video sensor is well suited for certain intersection and incident detection applications. The prior technology involving a multi-camera unit is also desirable in some applications. It is expected that the two synergistic and complementary technologies will co-exist and offer solutions in their own niches. The integrated sensor is currently being installed in downtown Minneapolis for SCOOT adaptive control.

Authors

Durga Panda and Craig Anderson, Image Sensing Sytems, Inc. and Panos Michalopoulos of the University of Minnesota
As presented at the 8th Annual Meeting of ITS America 4-7 May 1998
©1998 Image Sensing Systems, Inc.

PDF

View all Publications