Advanced
Image-based Subway Secur ity System by Histogr am Projection Technology
Image-based Subway Secur ity System by Histogr am Projection Technology
Journal of Korea Multimedia Society. 2015. Mar, 18(3): 287-297
Copyright © 2015, Korea Multimedia Society
  • Received : November 27, 2014
  • Accepted : January 29, 2015
  • Published : March 30, 2015
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Zhiguo Bai
Dept. of Smart Distribution Research Center, Korea Electrotechnology Research Institute
Sung-Hwan Jung
Dept. of Computer Engineering, Changwon National University
sjung@ changwon. ac.kr

Abstract
A railway security detection system is very important. There are many safety factors that directly affect the safe operation of trains. Security detection technology can be divided into passive and active approaches. In this paper, we will first survey the railway security systems and compare them. We will also propose a subway security detection system with computer vision technology, which can detect three kinds of problems: the spark problem, the obstacle problem, and the lost screw problem. The spark and obstacle detection methods are unique in our system. In our experiment using about 900 input test images, we obtained about a 99.8% performance in F- measure for the spark detection problem, and about 94.7% for the obstacle detection problem.
Keywords
1. INTRODUCTION
Since the birth of trains, the security issue of trains has been associated with the development of trains. We can say that regardless of the development of trains, the security of trains will always be regarded as the most important issue. Especially with the birth of Japan's Shinkansen, train speeds have been elevated to unprecedented heights.
A high-speed train’s rapid velocity has caused the world to focus lots of resources and effort on studying high-speed train technology and building high-speed rail networks. But at the same time, with the rapid development of high-speed train technology, terrible accidents have occurred sometimes. For example, on July 23, 2011, there was a collision involving two of China’s high-speed trains. Thirty-nine people died when a train ran into the back of another.
In order to prevent similar incidents from happening again, how to maintain a high-speed train’s safety is a very important issue. However, because of the train’s rapid speed, ensuring safety is very difficult. Firstly, when a train is running, if there are obstacles on a track, brakes need to have a short braking distance. Secondly, while running, a train operator cannot give a quick response. Thirdly, once a train is out of control, a huge level of kinetic energy will become a powerful destructive force, causing serious accidents. Although the world has done lots of research to ensure train security, accidents still occur sometimes. Therefore, how to ensure the safety of high-speed railways is an urgent and difficult technical issue [1] .
Train safety is very complicated. It involves management, operational quality of personnel, equipment reliability, and natural disasters. For example, the failures of wired electrical systems might cause fires. As train speeds increased, operators who rely on conventional visual detection methods have difficulty ensuring safety.
Therefore, the research of new intelligent detection technology for railway safety can provide technical means to ensure safety and reduce the number of railway accidents, and it has significant socio-economic and academic value.
In this paper, first we analyze general railway security systems and compare their features, and then we also propose a specific solution using computer vision technology for subway train security problems: the spark, the obstacle, and the lost screw problem.
In chapter 2, we survey the railway security systems and compare them. In chapter 3, we propose a security detection system for the subway train. We show and discuss our experiment in chapter 4. Finally, in chapter 5, we make some conclusions.
2. RAILWAY SECURITY SYSTEM
- 2.1 Present Research
In the world, even before the construction of high-speed railways, traffic safety for a passenger’s life and property was a high priority. High-speed railway security technology has been regarded as the forerunner of core technology. It has been improving continuously in practical applications.
Japan's Shinkansen, renowned for high security, has run for more than 30 years. The early automation system of train operation management includes traffic scheduling, vehicle scheduling, power dispatching, communication signal scheduling, and equipment scheduling. It is not only responsible for the management of line scheduling equipment and maintenance, but also for collecting data for weather and earthquakes, and other kinds of information about railways.
The French high-speed train has obtained a maximum test speed of 515.3km/h, which is the highest railway speed in the current world. The automatic train control system which is signal-based has shown gradual development from TVM-300, to TVM-400, and TVM-430. In the TVM-430 system, it increases equipment monitoring and alarm subsystems. Also it further strengthens the safety of train operation functions. The main functions are catenaries’ voltage monitoring, hot-axis monitoring, rainfall monitoring, snow monitoring, wind monitoring, and overpass falling objects monitoring [1] . Germany also used a fire alarm system on its high-speed railway. It not only monitors the application state of line equipment, but also identifies and promptly reports the impact of the environment on railway safety.
Many countries' railway operations have adopted the Railway Intelligent Transportation System (RITS), especially in developed countries. It is Intelligent Transportation System (ITS) technology which is applied in the field of railway transport applications and development. In the United States and Europe, RITS research and development has been ongoing for more than 20 years. It has resulted in a number of representative systems, such as the American Railway's Advanced Train Control System(ATCS), the French railways' continuous real-time automated tracking system, and Rail Europe's All the European Train Control System(ETCS) [2 - 4] .
- 2.2 The Features of Security Detection System
Because high-speed railways have high speed and high traffic density, consequently there are high level requirements for the traffic safety system, which is characterized mainly in the following areas [2 , 5] :
1) Automatic exchange of information between a train and the railroad ground, with real-time transmission: When the train’s speed is over 160km/h, the operator cannot recognize signal displays and line states clearly. It requires more signal and line status information to be transferred to the train from the ground in real time.
2) Traffic scheduling has a unified command, and it can do real-time processing of security information and train operation under automatic control: The driving track interval of a normal speed railway is about 6~10 minutes, while the minimum driving track interval of a high-speed railway is up to about 3 minutes. At such high traffic density, it requires building a center which can do real-time monitoring, transmission, processing, and decision-making control of security information. It also has to unify command and control to ensure the security of train operations.
3) It can maintain the high smoothness of tracks and high stability: When high-speed trains are running, if there is small surface irregularity on the rail, it may cause strong train vibration and it ingenerates the reduction of a train's comfort, stability, security, or even worse. The high smoothness of the track depends on high stability. It also depends on the high stability and uniformity of a rail's base. So it should adopt the dynamic monitoring measures on the high stability of a rail's base and track.
4) Real-time self-diagnosis on the status of critical equipment: Because high-density traffic requires equipment to run continuously, the power, communication signals, and other critical equipment must have a self-diagnostic function. It can transmit information to the traffic control center. And it has a security information processing center in time to ensure train operation safety.
- 2.3 Technical Features
For a railway security detection system, obstacle detection is important. According to current technology, there are two ways of detection: active and passive approaches [5 - 9] .
First, active detection: It sends a signal to the location where obstacles are detected and detects the reflected back signal through sensors. It detects obstacles this way. Ultrasonic detection, laser detection, and radar detection are all active detection. The advantages of this method are as follows: It can accurately detect the location of obstacles. It uses simple algorithms. It needs little calculation. It is easy to implement, and is not effected by weather.
Second, passive detection: It is mainly about image detection which is based on computer vision technology, compared with active detection, it has many advantages: It’s non-invasive. It does not increase environmental noise. It has wide spatial coverage, and interferences will not occur among image sensors [10] .
Now in the world, there are many kinds of railway security detection systems, and these systems have their own character. They also have advantages and disadvantages. For example, at a technical level, they use different kinds of computer vision technologies for detecting railway security problems. Also, the equipment used in their systems are often different. The Shinkansen security detection system uses inspection cars and sets up detection cameras on it. Next we will give some comparisons among these systems from different viewpoints.
Some technical features in the typical security detection systems are shown in Table 1 .
Comparison of security detection systems
PPT Slide
Lager Image
Comparison of security detection systems
In Table 2 , there is the comparison of some features in the obstacle detection part of some railway detection systems
Comparison of some typical obstacle detection systems
PPT Slide
Lager Image
Comparison of some typical obstacle detection systems
3. OUR SECURITY DETECTION SYSTEM
We propose a security detection system with computer vision technology for the subway train running in B city. A train takes many image shots by 7 mounted cameras as shown in Fig. 1 while it runs in the subway: two cameras for the electric car line, two cameras for the tunnel structure detection, two cameras for the track inspection and one camera for the real time monitoring. To monitor the security of a train, staff members currently look over each image from the huge image database after getting image shots from 7 cameras manually. As an initial research stage for the automation, we propose our security detection system.
PPT Slide
Lager Image
Cameras of our security detection system.
The general block diagram of the whole security detection system is shown in Fig. 2 . Our security detection system consists of three units. The detection system unit analyzes images and finds out problems such as the spark, the obstacle, and the lost screw problem. Then it reports the problems to the data processing unit.
PPT Slide
Lager Image
Conceptual block diagram of our security detection system.
In the detection system unit, there are three detection parts: the spark detection, the obstacle detection, and the lost screw detection as shown in Fig. 2 . At the technical level, our security detection system uses computer vision technology to do the three main functions: the spark, the obstacle and the lost screw detection.
Fig. 3 shows the procedure of data processing in our system. Because there are three cases in our detection system, these cases need different technologies and operations. If the case type is the spark or the obstacle detection, it should follow the left side of the flowchart. If the case type is the lost screw detection, it should follow the right side.
PPT Slide
Lager Image
Processing flow of our security detection system.
In the flowchart, there are some preprocessing operations. For example, in the obstacle and the spark detection cases, we must use image cutting operation on the test images. Also, in the lost screw detection case, the test image must firstly be converted into a grayscale image. Each problem case will be explained in the following subsections.
- 3.1 Spark Detection
The spark problem always comes out at the top of train body when a train is running at high speed on a track, because of the friction of fixed metals as shown in Fig. 4 . Sparks are more serious, because when sparks come out, they easily cause the non-metallic materials such as wire to be burned at the top of train body. If wire burns, the train will run abnormally, resulting in a greatly increased risk of traffic accidents.
PPT Slide
Lager Image
Example of a spark image.
Therefore, when the train is running, the detection of sparks is very important and the problem areas must be inspected. The Fig. 5 shows the process of spark detection. An example of a spark after binarization process is shown in Fig. 6 .
PPT Slide
Lager Image
Flowchart of the spark detection.
PPT Slide
Lager Image
Example of the spark image after binarization.
The processing steps of the spark detection are as follows:
  • 1) Delete the top 2/3 part of a test image.
  • 2) Do binarization operation on a part of the test image.
  • 3) Use functions to get the x-histogram.
  • 4) Calculate the biggest area of x-histogram.
  • 5) Analyze the histogram information.
  • 6) Do the post-processing.
- 3.2 Obstacle Detection
Sometimes there are obstacles dropped on the railroad, like a stone, a block and other things as shown in Fig. 7 . For the railway security, they must be detected and clean them up. Our system helps a staff member to find obstacles easily by computer vision technology, so the railway company can confirm obstacles along the railroad and clean them up.
PPT Slide
Lager Image
Example of an abnormal railway image.
When we obtain a railway image like Fig. 8 from the monitoring unit, let us call the horizontal darkest lines among the crossties in the Figure as boundary lines. First, we find the boundary lines, and use them to split out the railway image into four part images like Fig. 9 . Then we can detect an obstacle in the image.
PPT Slide
Lager Image
Boundary lines for obstacle detection.
PPT Slide
Lager Image
Four divided parts.
In Fig. 10 , the obstacle detection process is described as the followings:
PPT Slide
Lager Image
Flowchart of the obstacle detection.
The processing steps of the obstacle detection are as follows:
  • 1) Input test images.
  • 2) Find boundary lines of each part, and then cut a test image into four parts.
  • 3) Get the x-histogram of each part.
  • 4) According to the pattern of histogram, find out obstacles.
To find obstacles, we can divide histogram into four areas. Two areas in left are called as LP1 and LP2, and two areas in right, as RP1 and RP2 as shown in Fig. 11 . We can detect obstacles by using the ratio of each histogram area like an example in Fig. 12 .
PPT Slide
Lager Image
Four areas in histogram.
PPT Slide
Lager Image
The part of an obstacle image (left) and the corresponding histogram (right).
- 3.3 Lost Screw Detection
When the screw used for fixing track and railroad crosstie is lost, it is very dangerous because the track is not fixed. For the railway security, this kind of problem must be detected and solved. There are some kinds of typical lost screw cases: the examples of the left side lost and the right side lost are shown in Fig. 13 and Fig. 14 , respectively.
PPT Slide
Lager Image
Example of a lost screw on the left side track.
PPT Slide
Lager Image
Example of a lost screw on the right side track.
And we use the image template matching technology to detect the lost screws on the railroad. In Fig. 15 , it shows the process of the lost screw detection.
PPT Slide
Lager Image
Flowchart of the lost screw detection.
The experimental steps of the lost screw detection are as follows:
  • 1) Input the test image and the template image.
  • 2) Convert the input images to grayscale.
  • 3) Get the mean value of the template image and subtracting it from the template image.
  • 4) Form a search image from the search region by using the size of the template image.
  • 5) Do similar operation like step 3 on a part of search image.
  • 6) Calculate the correlation coefficient, C.
  • 7) Find out the match point by using correlation coefficient matrix.
  • 8) Display result.
In step 3, here we take an average on the template image and subtract it from the template image to get the so-called mean value template image by using formula (1).
PPT Slide
Lager Image
Where, R is the template image of size l x m , called the reference image. After this, we get the input test image F and we do the same operation on the image as our doing on the reference image. Then, we use the formula (2) to get the linear cross correlation Ci,j between the input test image F and the reference image R . Where, mod is modular arithmetic operation.
PPT Slide
Lager Image
Finally we can find the location of the lost screw in the test image by using maximum value of Ci,j .
4. EXPERIMENTAL RESULTS AND DISCUSSION
In this experiment, we used a PC with Intel Pentium 4 CPU, 4GHz, and 1.5GB RAM. Our system was mainly implemented by using MATLAB 6.0 [11] . As an initial research of the subway security problem, we use only 900 test images for the following experiment because of the limitation of acquiring test images. We propose an automatic image-based subway detection system which can replace the current manual detection method. We involve some measures to evaluate the performance of our detection system as shown in Table 3 .
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
Definition of TP, FP, FN and TN
PPT Slide
Lager Image
Definition of TP, FP, FN and TN
Where, TP is the number of correctly recognized good ones, FP is the number of good ones which are falsely recognized as defective ones. FN is the number of defective ones which are falsely recognized as good ones. TN is the number of correctly recognized defective ones. The F- measure is the weighted mean of precision and recall. In our experiment, we chose α =1.
In the spark detection, we got the results shown in Table 4 after our experiment.
Results of our experiment on the spark problem
PPT Slide
Lager Image
Results of our experiment on the spark problem
In the experiment, our system received 900 input test images which consist of 898 normal (good) images and 2 spark (defect) images. The detection system correctly detected 895 images as normal ones (TP). 3 normal images were classified as spark images (FP). There are no spark images misclassified as normal ones (FN). 2 spark images were correctly classified as spark images (TN).
Precision, recall, and F-measure are 99.67%, 100%, and 99.84%, respectively. The average spark detection time per one image is about 0.1 second in our system.
In the obstacle detection problem, We obtained the detection results shown in Table 5 through the experiment.
Results of our experiment on the obstacle problem
PPT Slide
Lager Image
Results of our experiment on the obstacle problem
900 input test images consist of 482 normal (good) images and 418 obstacle (defect) images. When compared with the above spark problem exposed in a very short time, we could take many defect images with obstacles in this problem.
In the obstacle detection, precision, recall, and F-measure are 99.38%, 90.38%, and 94.67%, respectively. It presents about 5.17 % lower performance than that of the spark detection case in terms of F-measure. The average obstacle detection time per one image is about 0.19 second in our system.
In the lost screw detection, our detection system took tens of minutes per one image. Therefore, we first need to reduce the lost screw detection time for our system. We may apply morphological template matching method such as hit or miss transform or other methods to solve this problem.
5. CONCLUSIONS
We surveyed the general railway security systems and compared their features, and we also proposed a specific solution for a subway train security problems as an initial research stage for automation. To solve the specific problems - the spark, obstacle, and lost screw problems - we have developed a railway security detection method using computer vision technology.
In our detection system, we used information extracted from histograms to solve the spark and obstacle problems. In study of spark and obstacle detection, our system showed about 99.8% performance in F-measure for the spark detection and about 94.7% for the obstacle detection problem. Consequently, we could solve the spark and obstacle problem to some extent. However, in the lost screw detection problem, when using template-matching technology to detect the lost screw, it took tens of minutes. The reason is because it needed lots of time when we calculated the correlation coefficient C. In the future, we need to solve the time consumption problem and to demonstrate better performance for our detection system.
BIO
Bai zhiguo
He graduated from the XIDIAN University in China with his bachelor’s degree in computer engineering in 2007. He studied at Multimedia Information Processing System (MIPS) Lab. of Changwon National University from March 2010 to February 2012 and he acquired his master’s degree in computer engineering. He has been working at Smart Distribution Research Center of KERI since 2012. His research interest includes computer vision, image and video processing, action recognition, etc.
Sung-Hwan Jung
He received the B.S., M.S., and Ph. D. degrees from in Electronic Engineering (information and communication major) from Kyungpook National University, Korea in 1979, 1983, and 1988, respectively. He has worked for the Electronic and Telecommunication Research Institute in Korea as a research staff, where he had experienced some national research projects including developing a portable computer. In 1988, he joined the faculty of Department of Computer Engineering at Changwon National University in Korea, where he is currently working as a full professor. From 1992 to 1994, he was a postdoctoral research staff of the Dept. of Electrical and Computer Engineering at the Univ. of California (UCSB). From 1999 to 2000, he also worked for the Colorado School of Mine (CSM) as an exchange professor. From 2008 to 2009, he has experience on the medical information processing at the Dental School of the Univ. of Missouri at Kansas City (UMKC), as a visiting professor. He is an Information System Auditor and P.E. in the area of Information Processing and Electronic Computer. His research interests include image retrieval, steganography, watermarking, medical image processing, computer vision and pattern recognition, etc. He is a co-author of eight image processing related books including ‘Visual C++ Digital Image Processing Using Open Source CxImage’, ‘Digital Image Processing Using MATLAB’ and ‘Learning Computer Vision and Application in OpenCV’
References
Li D.M. 2007 “Intelligence Image Processing based Railway Obstacle Detection On-vehicle Technology,” Journal of Southwest Jiaotong University 2 (4) 12 - 19
Carlson A. , Frincke D. , Laude M. 2003 “Railway Security Issues: A Survey of Developing Railway Technology,” Proceeding of International Conference on Computer, Communication, and Control Technologies 1 (3) 1 - 6
Seyve C. 2005 “Metro Railway Security Algorithms with Real World Experience Adapted to the RATP Dataset,” Proceeding of IEEE International Conference on Advanced Video and Signal Based Surveillance 15 (16) 177 - 182
Tong L. , Zhu L. 2012 “Railway Obstacle Detection Using Onboard Forward-Viewing Camera,” Journal of Transportation Systems Engineering and Information Technology 12 (4) 80 - 83
Terakubo S. , Morii M. , Kashihara T. 1998 “Development of a Road Obstacle Sensor Combining Image Processing and a Laser Radar,” Proceeding of the 1998 IEEE International Conference on Intelligent Vehicles 2 521 - 526
Kato W. , Kawami T. 2005 “Application of Image Processing Technology to Shinkansen Inspection Car,” Institute for Mathematics and Computer Science World Congress 3 (21) 1003 - 1009
Passarella R. , Tutuk B. 2011 “Design Concept of Train Obstacle Detection System in Indonesia,” International Journal of Research and Reviews in Applied Sciences 9 (3) 20 - 27
Mockel S. , Scherer F. , Schuster P.F. 2003 “Multi-sensor Obstacle Detection on Railway Tracks,” Proceeding of IEEE Intelligent Vehicles Symposium 9 (11) 42 - 46
Möller H. , Möller H. , Hulin B. , Krötz W. , Sarnes B. 2001 “Video Based Obstacle Detection in Catenaries of Railways,” Proceeding of 6th International Conference on Pattern Recognition and Information Processing 1 (7) 275 - 287
Jung S.H. , Jo B.H. 2011 “Line Detection in the Image of a Wireless Mobile Robot Using an Efficient Preprocessing and Improved Hough Transform,” Journal of Multimedia Information System 14 (6) 719 - 729
Jung S.H. , Lee M.H. 2005 Practical Digital Image Processing Using MATLAB Hongrung Publishing Co. Seoul