An Integrated Video Sensor Design for Traffic Management and Control
ABSTRACT
Conventionally, in-pavement inductive loops have been used as the sensors of choice for traffic management and control applications. Destruction of the roadbed and limited spatial sensing are two major limitations of the conventional sensors. For more than a decade the development of alternative sensors has been explored. Machine vision technology has emerged as an excellent candidate for traffic sensing.
The use of machine vision for video vehicle detection was first proposed by the Jet Propulsion Laboratory in Pasadena, California, U.S.A. Since then, several alternative systems have been developed and tested. The University of Minnesota developed the first practical system for wide-area video vehicle detection. A rich repertoire of technology, such as neural network and multi-resolution processing, has been applied to machine vision systems for traffic management applications. Acceptance of such systems by traffic engineers is constantly on the rise. However, affordability and reliability have been two significant barriers limiting explosive growth of machine vision sensor application in Intelligent Transportation Systems.
A new design has emerged that solves many of the problems associated with affordability and reliability of video sensors in transportation applications. This design takes advantage of the advancement in the miniaturization of digital electronics to integrate the opto-electrical transducer electronics and the computing electronics into an integrated vision sensor. The integrated vision sensor is supported by a new communications architecture for the sensor system and a new architecture for the sensor management software. The resulting video sensor system offers the users many advantages not previously available. The sensor uses improved tracking and speed measurement algorithms that contribute to enhanced accuracy in vehicle detection and classification. Enhanced speed measurement makes machine vision based automated speed enforcement feasible. Automated incident detection and queue measurements at surface street intersections are also feasible. The new communication architecture offers optimal routing of the machine vision detection results, full motion video as well as digital imagery, and supervisory control over a wide-area network suitable for central management of the network. Standard Internet Protocol formats are adapted for this sensor network offering intuitive user-friendly navigation.
Key-Words: Machine vision sensor, video vehicle detection, vehicle tracking, traffic data collection, queue measurement, automated incident detection, automated speed enforcement, software architecture, network communication architecture
1. Introduction
Traffic control at surface street intersections has traditionally used sensors that are buried in the pavement. Decades ago the workhorse of the detection system was the magnetic (brass torpedo) detector. It was installed in a PVC pipe under the roadbed. This magnetic detector gave good performance with little or no maintenance problems year round. However, the magnetic detector would only provide a pulse-type call and could not detect vehicle presence or occupancy. Inductive loop detectors have been the more predominant sensors of choice. The inductive loop sensors detect vehicle presence but are fixed in location and require intrusive installation in the roadbed. Because of the limitations of these sensors, researchers have been continually exploring alternative sensing technology. In 1978, the Federal Highway Administration (FHWA) conducted a study, prepared by the Jet Propulsion Laboratory of Pasadena, California, U.S.A., which concluded that machine vision technology offers a feasible alternative for traffic video detection [1].
Several years later researchers at the University of Minnesota [2] developed an enhanced system with then state-of-the-art industrial computers. Testing of the system in field test conditions indicated the feasibility of outdoor operation and the feasibility of real-time operation. These two indications signaled that it was feasible to use video sensors for practical real-time traffic control applications. Researchers in Japan and Europe were also developing machine vision systems for traffic control. Some examples include the works by Versavel et al [3], Takatoo et al [4], and Blosseville et al [5]. Advanced Traffic Detection is a recent publication that includes a very good brief summary of the various imaging sensor technologies being applied to traffic management problems [6].
Over the last decade the evolution of the traffic video detection system has been marked by several key evaluation reports. In 1991, researchers at the California Polytechnic Institute (Cal Poly) conducted an evaluation of the technology for highway applications [7]. This report was quite extensive in cataloging different, existing and emerging machine vision systems for traffic vehicle detection. Subsequently, in a recent paper Chatziioanou and Sullivan from Cal Poly indicated that some of the recently introduced machine vision systems, with their exotic techniques requiring complex computation, are less reliable for practical application [8].
In 1994, Hughes Aircraft Company reported a comparative evaluation study of several video, as well as other, detector technologies [9]. This study (although it did not explicitly conclude so) implied that the status of machine vision is ready and mature enough for traffic signal application. This was followed by a similar, but more rigorous and extensive, comparative study, done by the Minnesota Department of Transportation (Mn/DOT) for the FHWA [10]. This study, concluded in 1996, indicated that video detectors, in general, do offer reasonably high accuracy and reliability.
While these evaluations were proceeding, FHWA was further advancing the video technology to measure additional traffic parameters, such as approach queue length and approach stops at intersections [11]. In 1995, Mn/DOT conducted a large-scale test of various sensor technologies on operational road traffic [12]. The technologies included in the test were: active and passive infrared, magnetic, radar, passive acoustic, ultrasonic, and video. One of the conclusions of this test was that illumination conditions had significant impact on the video sensor performance.
2. Problem Formulation
The problems in the video sensor technology in 1995 could be broadly generalized into two categories: reliability and affordability. Affordability includes the initial capital cost of the system, as well as on-going operational and maintenance costs. Although video sensors offer many benefits not achievable with inductive loop sensors, it is desirable to significantly improve the affordability of the sensor.
Typically, machine vision sensors have a machine vision processor (MVP), processing the output of several video cameras. At a typical surface street intersection with four approaches, the MVP is designed to process four camera outputs in real time. Having a single processor for four cameras achieves the economy of scale. However, at a site that needs just one camera, such as on motorways, the four-camera MVP is economically sub-optimal. This is also true for central business district intersections with one-way traffic that needs only two cameras, not four, at an intersection. Also, there are many intersections, especially on European and Asian surface streets, which have five or six approaches. These applications would need two, four-camera MVPs at an intersection, again making it economically sub-optimal.
Additional cost considerations come from the locations of the MVPs. A four-camera MVP must be placed at a location central to all the four cameras. This has the built-in cost of coaxial or optical fibre or wireless video transmission from the cameras to the MVP. Also, operation and maintenance of the sensors incurs the cost of personnel traveling to each of the MVP locations in the field. Such travel could be for programming, installing new software, or trouble-shooting the MVPs. These costs contribute to the affordability problem of video sensors.
The current generation video sensors perform well in overcast daytime conditions. As the daytime conditions become partly sunny to bright full sunshine, the sensor performance begins to deteriorate from the optimum.
Shadows are a major cause of reliability problems during the daytime operation of video sensors. In urban areas with narrow streets and tall buildings, the shadow problem is compounded. There are three types of shadow artifacts: dynamic shadows from moving vehicles, static shadows from fixed objects, and what can be termed slow moving shadows from fixed objects. Video sensors use software techniques to treat the shadow artifacts. However, the ability to adapt to the illumination variation using software alone is somewhat limited. Often shadows or very bright sunlight cause a sharp reduction in the dynamic range of the image gray scale. This limits the information available to the vehicle detection software from the sensor. It is desirable to widen the dynamic range of the imagery, especially in the bright regions with over-saturated illumination and shadow regions with virtually no dynamic range. Figure 1 shows this phenomenon.
An added contributing element to the reliability problem is the degradation of video quality due to transmission. As mentioned earlier, often video is transmitted from the camera to the MVP over buried coaxial cable or wireless transmitter. Electro-magnetic interference (EMI) and transmission loss combine to degrade the image quality at the MVP. In addition, this degradation is intermittent, caused by external factors not easily modeled in the vehicle detection algorithms. Minimization or elimination of this cause of the reliability problem is desirable.
3. Integrated Sensor Hardware
A totally new sensor system was designed to address the problems identified above. The sensor system consists of two hardware subsystems: the MVP and the communication network. The sensor MVP consists of four elements: the processor, the transducer, packaging, and other physical entities. The processor is the major element in the sensor. The objective of the new sensor architecture was to enhance the cost and reliability over previous state-of-the-art. Two technical alternatives were considered for the basis of the processing unit: Motorola M6800 series and Intel X86 series. The Intel series processor offered somewhat lower cost, and it offered higher reliability due to the potential of backward compatibility of the embedded detection software with the previous sensors [13].
There were two major candidates for the transducer: charge coupled devices (CCD) and complementary metal oxide semiconductor (CMOS) devices. There is a third category of transducers, active pixel sensors (APS), that could be considered as a variant of CMOS sensors [14]. CMOS offered the potential for lower cost, especially due to higher integrability with the digital processor; but the reliability risks of the CMOS sensor, especially APS, were considered too high at the current state-of-the-art. This risk outweighed the benefit of integrability, and the CCD medium was selected as the transducer.
One of the design goals of the integrator sensor was improved reliability under varying illumination conditions. To achieve this, the CCD camera control was integrated with the CPU. This allowed the machine vision processor to perform closed-loop gain and bias control of the image being processed. This minimizes the relative variability of the image gray scales being processed by the vehicle detection algorithm in the MVP.
The sensor hardware has the following input/output:
- Vehicle detection decision
- Traffic data
- Supervisory control
- System status
- Analog imagery
- Digital imagery
The vehicle detection output is a very low bandwidth electrical contact closure, compatible with the National Electrical Manufacturers Association (NEMA) controllers for traffic control. Traffic data and supervisor control output is converted to serial EIA-RS232/RS485 format. This allows transmission over low cost twisted-pair copper wires. Power, 24V AC or DC, to the sensor is provided over 1-1/2 pair of twisted copper wires. Thus, all the input/output connectivity of the sensor with the external world is through twisted-pair copper wires. This design simplifies sensor interconnection and lowers cost.
Packaging form factor design of the sensor was also intended to minimize sensor and installation costs and maximize reliability. Of the various alternatives, cylindrical configuration with uniform circular cross-section is the least obtrusive when mounted on outdoor overhead fixtures. The camera optics is at one end of the longitudinal axis of the cylindrical form factor. This configuration allows much more rotational freedom, making the installation simpler and quicker. A single longitudinal board holds all the processing electronics. This allows the closed loop camera control to be located on one end of the axis, next to the sensor external connections. This configuration keeps the electronics assembly and packaging simple and low cost.
Other physical considerations include the heating element, the sensor shield, and the external connector. All the twisted-pair wire connections are brought out through a single quick-snap connector. The connector is a military standard (Mil-Std D) connector, needing a simple quarter-turn to connect all the input/output terminals. The sensor shield is semi-cylindrical, conformed to the sensor configuration. The shield protects the sensor from exposure to direct sunrays and precipitation. The heater prevents accumulation of snow, ice, and condensation. Figure 2 shows the sensor design.
A special communication hub was designed to facilitate low cost management of the sensors from a Traffic Management Center (TMC) or some other similar central facility.
The hub has three major communication functions:
- Data Multiplexing
- Video Multiplexing
- Hub-to-Hub Interconnection
Data from up to eight integrated sensors is multiplexed at a hub for the long haul transmission to the TMC. Similarly, video from up to eight integrated sensors is also multiplexed at the hub.
Figures 3 and 4 show the block diagrams for sensor network management from the TMC. At each node in the network, a hub multiplexes data and imagery from up to eight sensors. Twisted-pair wires link one communication hub to the next. The final hub is connected to a desktop PC through its communication ports. This desktop PC functions as a communication server (Comserver). The users may perform sensor management functions from this Comserver PC or any other PC connected to the Comserver over local area network (LAN).
4. Integrated Sensor Software
Referring to Figures 3 and 4, the Comserver is central to all the sensor management functions. The Comserver allows an operator at the central facility to establish communication with any of the sensors or the communication hubs in the field network. A user at the TMC can access any video sensor in the field network through the Comserver. The Sensor Network Browser software, similar to an Internet Browser, is available on the PC workstation of the operator for easy graphical interface with the sensors (see Figure 5). Any desktop or notebook PC with appropriate Windows operating environment connected to the local area network (LAN) linking the Comserver PC in the operation center can execute the Sensor Network Browser.
The Comserver is designed to communicate with the client applications and other sensor management software using industry standard TCP/IP communication protocol.
The sensor manager software consists of the following modules:
- Network Configuration Management: Includes defining, modifying, and verifying the communication servers and channels and the network topology.
- System Test: Performs diagnostics of the video sensors and the communication hub equipment.
- Software Installation: Includes automated software installation for all the sensors and hubs in the field network. New sensors in the field can have software installed from the traffic operation center, just as well as the existing sensors could receive software upgrades. The software allows installation in one unit (sensor or hub) or automatic batch installation for an entire set of units.
- Detector Editor: Includes setting up new detectors, modifying existing detector configurations, and adding or deleting detectors to the existing set.
- Operational Log Review: The video sensors and the communication hubs continuously perform self-diagnosis during their operations. Any anomaly or deviation from the normal operation is automatically stored in the unit’s Operations Log. The Operations Log of any sensor or hub can be remotely accessed, reviewed, and documented at the traffic operations center.
- Traffic Data Archival and Display: From the traffic operations center the sensors can be requested to collect and save any desirable traffic data accumulated over desired intervals. When ready, an operator can unload the traffic data saved in the sensors, review and format the data for display or analysis.
- Video Management: Digital snapshots, as well as full motion video with flashing detector operation from any of the sensors in the network, can be accessed and displayed that the traffic operations center.
The vehicle detection software embedded in each sensor has been evolved from the previous generation software baseline [13]. Typical traffic data measurements performed by the detection software are shown in Figure 6. Significant enhancements have been made to the detection software over the previous generation baseline.
Figure 7 shows the detection zones in the sensors filed-of-view. Vehicles and detected and tracked within the tracking zone in each lane. The spatial signature of each detected vehicle is integrated with the temporal signature of its motion obtained from the vehicle tracker to measure speed of the detected vehicle. This information is used in obtaining the measurements shown in Figure 6. Tracking and speed measurement help in the treatment of artifacts such as shadows and perspective interference of neighboring vehicles (perspective interference makes the vehicles sometimes appear to be moving in two adjacent lanes).
A queue measurement application software was developed using the baseline detection logic described above. Figures 8 and 9 show tracking of moving vehicles and the detection of the vehicles stopped in a queue at an intersection.
Direct measurement of queue size/length has several significant uses for a traffic engineer. The main benefits of queue measurement are described below.
- Better Adaptive Control: Queue size (or queue length) can be used as a state variable in intersection and ramp control. Therefore, new generation adaptive control schemes could now use the queue measurement as an input. Also, the new generation actuated controllers could use queue size for more effective control. In addition, queue measurement can be used for capacity and level of service (LOS) analysis, geometric improvements at intersections, and advanced freeway ramp control.
- MOE Derivation: Of equal importance is the employment of queue size in deriving Measures Of Effectiveness (MOE), such as delays. Delays and stops can also be used to estimate excess energy consumption and emission of atmospheric pollutants due to vehicles idling in the queue or stopping. Real-time MOE calculation can be used to generate an alarm if stops or delays exceed a certain user-selected threshold.
- Currently queue lengths, delays, and stops are measured manually. This is a very expensive and time consuming process that can only be conducted during limited time periods versus 24-hour, long-term basis.
- Performance Evaluation: Direct and automatic measurement of queue size can also be used in real-time, 24-hour evaluation of the performance of the signal timing plan. Relative performance improvement can be evaluated by measuring queue length before and after the signal timing plan.
The queue measurement software detects and tracks the position of vehicles within each tracking detector as shown in Figure 9. In the process of detecting and tracking vehicles, it also calculates the vehicles’ speeds and lengths; and this, in turn, enables the calculation of the size of the queue, the number of vehicle stops, the percentage of roadway covered by vehicles, and the number of vehicles entering and exiting the detector. Based on the way the detector detects and tracks vehicles, the following MOEs can be measured:
- Queue Size: The number of stopped vehicles (a stopped vehicle is defined as one traveling less than 5 mph).
- Queue Length: The distance from the first stopped vehicle to the end of the last stopped vehicle.
- Queue Exit Speed: The average speed of vehicles exiting the detector during an interval.
- Queue Exit Volume: The number of vehicles exiting the detector during an interval.
- Queue Flow Length: The cumulative length of vehicles exiting during an interval.
- Total Stops: The total number of stops made by vehicles that have exited the detector during an interval.
- Spatial Occupancy: The percentage of length of the roadway that is occupied by vehicles.
For freeway application an automatic incident detection algorithm (AIDA) has been developed using the baseline detection algorithms discussed above. AIDA uses temporal variations in the traffic parameters such as volume, occupancy, and speed to detect shockwave from an incident (see Figure 10). It can also detect stopped vehicles in its field-of-view. This allows automatic detection of vehicles pulled over to the pavement shoulder due to emergency.
5. System Test
Tests of the integrated video sensor system were conducted over an 18-month period, from May 1997 to September 1998. Two types of tests were conducted: system validation and performance evaluation. The two tests were conducted at two different sites because of instrumentation requirements. These tests and sample test results are briefly described below.
The objective of the system validation test was to validate in the field operational conditions the novel concepts introduced in the integrated sensor design. The validation was conducted in the Autoscope Proving Ground (APG) in St. Paul, Minnesota, U.S.A. The integrated sensor was installed at two different sites in the APG. The first site was at a surface street intersection, the intersection of Snelling Avenue and Spruce Tree Street. This was for validation under slow speed traffic with small headway. The sensor was viewing south for maximum variability in illumination conditions. The second site was at an interstate motorway location on I-94, for validation under high speed conditions. Twisted pair wires from the sensors were brought in through a communication hub to remote offices with a local area network of desk/laptop PCs. The sensors performed well under both the test conditions. These tests especially validated the proper operation of three elements in the system: the closed loop camera control, the embedded vehicle tracking and speed algorithm, and the network browser subsystem.
In September 1998, a performance evaluation was conducted at Dunwoody Boulevard and Colfax Avenue in Minneapolis, Minnesota. At this location approximately 2 meter by 2 meter inductive loops were installed in the pavement at the calibrated detection location where the integrated video sensor was to aim . The output of both the inductive loops and the integrated video sensor were sampled 30 times a second and integrated to simulate realistic operational conditions. Vehicle detection count was extracted from the loops and the video sensor and then compared.
Table 1 shows the comparison of the count data on two days, as an example, during the test. Detailed test results will be presented at the conference.
HOURS |
12 SEPT. |
13 SEPT. |
0000 - 0600 |
14 |
8 |
0600 - 0800 |
4 |
3 |
0800 - 1100 |
1 |
5 |
1100 - 1500 |
1 |
2 |
1500 - 1800 |
0 |
1 |
1800 - 2000 |
1 |
1 |
2000 - 2200 |
5 |
5 |
2200 - 2400 |
5 |
6 |
Table 1. Vehicle Count Error (%)
Figure 11 shows a sample plot of the test data over a 24-hour period. The results show excellent correlation between the loop measurement and the integrated video sensor measurement. Data in Table 1 has been grouped into several time periods to observe variations in the performance. The time periods are as follows:
0000 hrs to 0600 hrs night time
0600 hrs to 0800 hrs night-to-day transition
0800 hrs to 1100 hrs morning sun angle
1100 hrs to 1500 hrs overhead sun
1500 hrs to 1800 hrs afternoon sun angle
1800 hrs to 2000 hrs day-to-night transition
2000 hrs to 2400 hrs night time
Due to day light saving time observation, dawn illumination during night-to-day transition does not significantly illuminate the detection area until approximately 0600 hours. Also, during the period of overhead sun angle the sun is more to the south than truly overhead, casting reasonable shadows. This solar angle is due to the northerly latitude of Minneapolis. Passing clouds during the bright days made the daytime test a good evaluation of performance reliability.
Lastly, a large network of the integrated video sensor was interconnected using twisted pair wires and the network architecture as briefly described above [15]. The output from different sensors was brought into a Traffic Control Center (TCC) over five different communication channels. A diagram of the integrated sensor network is shown in Figure 12. Different engineers at TCC were able to use their individual laptop/desktop PCs and successfully perform all of the sensor network management functions.
6. Conclusion
An innovative video sensor design is presented. The sensor combines transducer electronics and MVP electronics into one compact integrated package.
This integrated video sensor has many features and resulting benefits for the users. Integration of the transducer and the MVP into one compact unit improves detection reliability and lowers susceptibility to EMI, lowers installation costs, reduces installation time, enables rapid deployment, and makes the system more readily portable. Direct real-time control of the transducer allows for precise adjustment of illumination based on detection zones and enables control of illumination.
Twisted-pair wire communication makes the sensor more cost-effective and simple to use and allows longer distance data transmission and deployment at more locations because twisted-pair is more readily available. Industry-standard TCP/IP protocol allows for multiple access capability from within the local area network at the TMC. An Internet browser-like user interface makes it easy for traffic engineers to configure detectors, collect traffic data, and manage the sensors.
Results of field tests conducted in operation conditions indicate excellent reliability in performance. System-wide cost, both initial as well as on-going operational, is expected to be lower than current generation sensors for many applications.
The compact integrated video sensor could be well suited for certain traffic management and control applications, such as motorway incident detection, tunnel incident detection, ramp metering, mid-block arterial traffic status monitoring, and control of intersections of one-way streets.
7. Acknowledgement
The author wishes to thank Mr. Marcus Culver of Image Sensing Systems, Inc. for his assistance in preparing this paper. The author also wishes to thank Mr. Roger Plum and Mr. Steve Mosing for their guidance in the testing of the integrated sensor network.
References:
[1] Wide Area Detection System, Conceptual Design Study, Report No. FHWA-RD-77-86, Federal Highway Administration, Washington, D.C., February 1978.
[2] Michalopoulos, P.G., Vehicle Detection Through Video Image Processing: The AUTOSCOPE System, IEEE Transactions on Vehicular Technology, Vol. 40, No. 1, 1991, pp. 21-29.
[3] J. Versavel et al, Camera and Computer Aided Traffic Sensor, IEE Second International Conference on Road Traffic Monitoring, London, 1989.
[4] M. Takatoo et al, Traffic Flow Measuring System Using Image Processing, SPIE, Vol. 1197, 1989.
[5] J.M. Blosseville et al, Titan: A Traffic Measurement System Using Image Processing Techniques, IEE Second International Conference on Road Traffic Monitoring, London, 1989.
[6] D. Fenichell et al, Advanced Traffic Detection Emerging Technologies and Market Forecast, Transport Technology Publishing, 1998.
[7] Final Report, Evaluation of Image Processing Technology for Applications on Highway Operations, Technical Report TR 91-2, Department of Transportation, State of California, Prepared by California Polytechnique State University, San Luis Obispo, California, June 1991.
[8] Chatziioanou, A., and E.C. Sullivan, Traffic Video Image Processing Systems Installations--An Examination of VIPS Standardization, Proceedings of IFAC Transportation Systems Conference, Chania, Greece, February 1997.
[9] Klein, L.A., and M.K. Mills, Evaluation of Modern Traffic Detector Technologies for IVHS Applications, Presented at NATDAC, Rocky Hill, Connecticut, September 1994.
[10] Task 2 Report, Life Cycle Cost Comparison of Autoscope Video Detection System to Inductive Loop Detection, Mn/DOT Agreement No. 70619, SP-8809-129, Prepared by Image Sensing Systems, Inc., St. Paul, Minnesota, May 1994.
[11] Sensor Development Final Report, Traffic Surveillance and Detection Technology Development, Federal Highway Administration, U.S. Department of Transportation, Prepared by the Jet Propulsion Laboratory, Pasadena, California, March 1997.
[12] Final Report, Field Test of Monitoring of Urban Vehicle Operations Using Non-Intrustive Technologies, Federal Highway Administration, U.S. Department of Transportation, Prepared by SRF Consulting Group, Inc., June 1997.
[13] Autoscope User’s Guide, Econolite Control Products, Inc., 1995.
[14] J. Larish, Photobit and the Next Generation Digital Camera Sensor, Advanced Imaging, Vol. 14, No. 1, January 1999.
[15] Panda, D.P., and P. Michalopoulos, Deployment of the Next Generation Machine Vision Technology and Integration with SCOOT in the City of Minneapolis, Presented at the 5th World Congress, Seoul, Korea, October 1998.
Authors
Durga Panda, Image Sensing Sytems, Inc.As presented at the IMACS IEEE CSCC '99 International Multi-conference held in Athens, Greece 4-8 July 1999
©1999 Image Sensing Systems, Inc.
