Intelligent Control and Automation
 Vol. 3  No. 1 (2012) , Article ID: 17579 , 8 pages DOI:10.4236/ica.2012.31011

Building an Intelligent Home Space for Service Robot Based on Multi-Pattern Information Model and Wireless Sensor Networks

Fei Lu, Guohui Tian*, Fengyu Zhou, Yinghua Xue, Baoye Song

School of Control Science and Engineering, Shandong University, Jinan, China

Email: {lawyerlf, *g.h.tian, zhoufengyu}@sdu.edu.cn, yhua_xue@yahoo.comc.cn, b.y.song@mail.sdu.edu.cn

Received November 3, 2011; revised December 3, 2011; accepted December 10, 2011

Keywords: Intelligent Home Environment; Service Robot; Artificial Mark; Wireless Sensor Networks

ABSTRACT

This paper is concerned with constructing a prototype intelligent home environment for home service robot. In this environment, multi-pattern information can be represented by some intelligent artificial marks. Light-packs service robots can provide reliable and intelligent service by interacting with the environment through the wireless sensor networks. The intelligent space consists the following main components: smart devices with intelligent artificial mark; home server that connects the smart device and maintains the information through wireless sensor network; and the service robot that perform tasks in collaboration with the environment. In this paper, the multi-pattern information model is built, the construction of wireless sensor networks is presented, the smart and agilely home service is introduced. Finally, the future direction of intelligent space system is discussed.

1. Introduction

As the aging problem becomes increasingly serious in many countries and in order to improve the life quality of eldly persons and persons with physical disabilities, the development to home service robot and the study of practical technology of home service robot are very important (as discussed by Tian Guohui [1]). In recent decade, many attention has been put into this field.

It is necessary for robots to enter the home and provide service for human (as discussed by Seung-Ho Baeg [2]). For home service robot, how to percept the environmental change, understand human’s behavior and provide autonomous, accurate and intelligent home service work in complicated and various home environment becomes a problem which needs to be solved quickly. However, an very important basis for robot to complete these tasks is to obtain the environment information. Currently, in classical robotic platform, the main method for robot to get these information is by equip camera and many expensive sensors on the robot. Tasks such as location, navigation and decision are made by robot through complex algorithm with huge data. At same time, information (such as environmental information and human behavior) obtained by the robot with its onboard sensors is limited and the accuracy is depended on the performance of sensors. In this case, while facing to unstructured home environment, robot must process mass information and many dynamic disturbances. This unstructured home environment takes great challenge to the robot performance, increases robot’s manufacturing cost and becomes a bottleneck of expanding application for service robot (as discussed by Tian Guohui [3]).

Intelligent space technology is a new research field (as discussed as Joo-Ho Lee [4]). It is an important research field of ubiquitous computing. The concept of “Intelligent space” is proposed by Hashimoto Lab in University of Tokyo in 1996 (as discussed by Joo-Ho Lee [5]). In recent 15 years, more and more studies have been focused on it. The research to intelligent space has expanded its meaning from human-machine interaction to intelligent space system which interact with physical space. One of the important application fields of the intelligent space is combination with robot, it opened a new direction to the research and application for the robot, especially for home service robot.

The key idea of the intelligent space oriented to home service robot (Intelligent home environment) is the family environment is installed many types of sensors and actuators to establish an artificial home service environment where human and robots coexist. The intelligent space can percept the changing of the objects through wireless sensor network, transfer information with robot by network, record, track and manage the objects in intelligent space. Combining with the intelligent space technology, without adding more sensors on home service robot, a service robot can obtain more comprehendsive environmental information, perceive human and objects in space more completely.

With this method, robot can do more service tasks and do them better with “light packs” in a complex home environment, for example, robot can complete many tasks which can’t be solved depending on its onboard sensors. At same time, intelligent space can be used as a common platform for multi-service robot systems. Multiunit and multi-type service robots can work in the same platform with “light packs”, share the information resources (as discussed by Tian Guohui [3]). This technology can expand application fields of service robots and speed up its development. In home environment, how to build an intelligent space also can be called an intelligent home environment) and how to get more and accuracy information of the environment are an important basis for home service robot to finish service task.

In this paper, the structure of the intelligent space oriented to home service robot is introduced in Section 2, a new multi-pattern information model are proposed in Section 3, in Section 4, a construction of wireless sensor networks in intelligent space is introduced, the smart and agilely home service provided by home service robot based on the multi-pattern information and wireless sensor network is introduced detailed in Section 5. Finally a discussion and the conclusion are presented in the last section.

2. The Architecture of the Intelligent Space Oriented to Home Service Robot

An intelligent home is a residence in which computing and information technology apply to expect and respond to the occupants’ needs and can be used to enhance the everyday life at home. The architecture of an intelligent space is shown as Figure 1.

Figure 1. Architecture of our intelligent space.

Our intelligent home environment system mainly consists of three components: smart objects; home server and service robot. Smart objects are the basic elements of the intelligent space. They are objects with artificial land mark or objects with either sensor capabilities or actuator capabilities or both. They can communicate with the home server through wireless networks. Home server manages smart objects, collects data in the intelligent space, makes decisions based on the collected data and controls the smart objects. Service robot provides reliable service by symbiotic interactions with the environment through the wireless communication networks. Suppose robot needs to turn on the TV in the intelligent home for some reason, in the traditional robot platform, the robot should calculate the position of the TV switch through vision processing and object recognition system while moving to the TV, then move its arm to touch the switch. In our intelligent home environment, what the robot need to know is the network address of the TV. Some complex tasks can be finished easily and satisfied.

There exit two important research fields in the study to intelligent home environment. The first one is information model representation and multi-pattern acquisition, the second is the wireless sensor networks. Robot can obtain more and high level information with the help of these two technologies, can understand and perceive the environment more completely.

3. Multi-Pattern Information Model in Intelligent Space

3.1. Multi-Pattern Information Representation Model

In order to complete tasks, it is better for robot to obtain more and more environment information. In the traditional research to robot, the main method for robot to get the environment information is by the “eyes” of the robot: camera and sensors. After complex algorithm and computation, these information can be understood by robot. Usually, the information obtained by this way is limited and lower level, this will effect the quality of the task implementation for robot, especially in complicated and unstructured environment. In intelligent space, artificial marks are adopted to build information representation model, such as landmark and object mark (as discussed elsewhere [6-8]). These low-cost information mediums are an effective supplement to the information network in intelligent space system. Robot can get more and highlevel information through the information network in intelligent space without equipping more expensive sensors, these high-level information usually can’t be obtained by traditional pattern.

QR Code (Quick Response code) and RFID (Radio Frequency Identification) technologies are used to design the artificial landmarks and object marks in our intelligent home environment, which construct a complete information network and become the foundation for multipattern information acquisition.

1) QR Code Information Representation Model The artificial landmarks which is designed should have the following characteristics: can be detected, can be identified and can be positioned.

The artificial landmark designed in our system includes two parts: outside circle and inner information representation, shown in Figure 2. The function of the blue outside circle is to ensure this artificial landmark can be found by robot easily from a distance. The inner part uses QR Code to record the environment information and context-sensitive information, such as the attribute information and operating information of an object. QR Code is a kind of two dimensional barcode invented by Denso Company in Japan. Besides the advantages of all two-dimensional code, such as large capacity, high reliability, and high safety, QR Code still has other virtues, such as the high decoding speed, can be recognized from any direction, and can store the Chinese characters effectively (as discussed elsewhere [9-11]).

The new artificial landmark we designed is a real twodimensional landmark, because the outside circle and QR Code are all two-dimensional. Compared with one-dimensional landmark, the two-dimensional landmark can be read from any direction and can provide more information.

In Figure 2, on the left is the design of artificial landmark based on QR code and the right is the object which is stuck with QR Code.

2) RFID Information Representation Model RFID system mainly includes three parts: antenna, reader, and RFID tag. Because of the uniqueness of the RFID tag, the reader can locate and track the target once it is attached on an RFID tag. In our system, we use 900MHz RFID product made by Shenzhen Javs Technology Corporation, as shown in Figure 3. Passive tags are used to attach on the target object as shown in Figure

Figure 2. QR code model, on the left is artifical landmark based on QR code, on the right is object sticking with QR code.

Figure 3. RFID system, on the left is object attached with tag, in the middle is RFID reader and on the right is RFID antenna.

3 (on left) because they are much cheaper, long lived, lightweight and have a smaller foot print. The reader as shown in Figure 3 (in the middle) can communicated with host computer through RS232 and one reader can receive the data from 4 antennas at most. There are two kinds of antennas: linear polarized antenna (LPA) and circular polarized antenna (CPA), which looks similar in appearance, as shown in Figure 3 (on the right). The LPA is installed on the robot because the narrow recognition angle of LPA can guarantee the correct time sequence when the robot read the tags (as discussed elsewhere [12-16]).

RFID has the following advantages: noncontactable, identify multi-target at the same time, high speed and far distance, etc. The combination of RFID and QR Code can build the information network, provide more and high level information of the environment to the robot (as discussed by Yinghua Xue [17]).

Define abbreviations and acronyms the first time they are used in the text, even after they have been defined in the abstract. Abbreviations such as IEEE, SI, MKS, CGS, sc, dc, and rms do not have to be defined. Do not use abbreviations in the title or heads unless they are unavoidable.

3.2. Multi-Pattern Information Acquisition Based on Data Fusion

An intelligent home environment is a physical world where smart items, sensors and actuators are placed and they are invisibly and seamlessly interwoven through a continuous network. These equipment can percept the environment information or represent the environment information to the robot. In order to obtain more accuracy and more comprehensive environment information, robot must fuse all types of information provided by intelligent space, usually these information are incomplete and uncertain. The information in intelligent space can be divided into 3 levels: data level, feature level and decision level. With the help of computation ability in intelligent space, deeper information can be acquired through the data fusion to multi-pattern information (as discussed elsewhere [18,19]). Data fusion is the foundation for the intelligent space and robot to provide intelligent and autonomous service to people. The structure of data fusion in our intelligent space system is shown as Figure 4.

The information of data level mainly come from the sensors in the intelligent space and the sensors on robot, such as temperature, humility. Data of feature level include time, date, address, people, etc. In process of data fusion, different algorithm can be adopted according to the data feature. For example, in the first example in Section 5, robot can get different information from the QR code, RFID and the sensors equipped on its board. We can adopt some algorithm to fuse the information, so the robot can decide the path to access the object and the way to catch the object. The result of data fusion is to make a decision to control the devices, to understand the people behavior, to provide intelligent service.

4. The Construction of Wireless Sensor Networks

In intelligent space, intelligent devices, such as sensors, artificial landmark, camera can provide many environment information. In order to realize the interaction with intelligent space, these information must be seamlessly interconnected and be transferred. One of the core technologies for studying intelligent space is communications technologies which connects devices with the environment. With the help of communication technologies, all kinds of information can be transferred among the intelligent device. The realization of information communication depends on network. In order to share these information, ZigBee, Wi-Fi and TCP/IP protocols are adopted to built a heterogeneous network in intelligent space system. This network becomes a bridge of information exchange. Service robot, distributed intelligent nodes and smart devices communicated with server through wireless or wired network. The heterogeneous network in intelligent space system is shown in Figure 5.

In the model of heterogeneous network in intelligent space, different network is assigned to different tasks. Wi-Fi is used to transfer video and control command between server and robot. TCP/IP is used to transfer the video from distributed network camera. The purpose of GPRS is to transfer the information to mobile, PDA (Personal Digital Assistant).

In the inner network, the ZigBee protocol is adopted in to built wireless sensor and actor network in intelligent space. ZigBee is a new wireless communication technology based on wireless standard 802.15.4 (as discussed elsewhere [20-24]).Compared with other protocol standards, ZigBee technology has the following advantages: 1) short range 2) low-power 3) low transmission rate 4) large Network capacity 5) the network is very strong in self-grouping 6) short delay. In our intelligent space system, Zigbee is used to transfer the environmental sensor data, such as temperature, humidity, it also can transfer the control command to the intelligent device to realize the intelligence of the home.

Figure 4. The structure of data fusion of different data level.

Figure 5. Heterogeneous network in intelligent space system.

The transformation of sensor information and control information which cover more range can be achieved in intelligent space through the building of wireless sensor network. It extends the working scope of intelligent space and home service robot.

5. Examples and Result of Smart and Agilely Home Service Acknowledgements

Robot can provide smart and agilely home service with the interaction with intelligent space environment. In this paper, we select 2 examples to introduce.

5.1. Object Search and Recognition

One of the main tasks of service robot is to provide different kinds of services to persons in complex indoor environment, such as delivering water or medicine. So the robot must have the ability to search the target and recognize the target autonomously. Computer vision is widely used in the field of object recognition (as discussed elsewhere [25-28]). But in complex indoor environment, there are many difficulties to recognize the object through vision [17]. In intelligent space system, passive RFID technology and artificial object mark are combined together to provide a location for target. An RFID reader and antenna are mounted on the robot that moves autonomously through the space. As the robot moves, the RFID reader can read the reference tags pasted on the large furniture and target tags pasted on the target one by one, and the rough position of target can be calculated by the time sequence of reference tags and target tags. RFID system can only obtain the rough position of the target, so object mark is used to recognize the target at close distance. We stick artificial object mark (with QR Code) on the target object. The object mark proposed by us can not only assist the robot to distinguish the target in different cases, but also can provide the attribute information and operating information of the target to the robot. The attribute information and operating information of the target are stored in QR Code, and the outer circle is used to locate the mark from a more far distance rapidly. The flowchart of object search and recognition is shown as Figure 6.

5.2. Detection and Help System for Elderly Abnormal Behaviors

Another important task for service robot is to detect the elderly abnormal behavior and provide help. In our intelligent space system, when abnormal behavior, such as fall suddenly is detected, robot should move to the elderly. Abnormal detection and help system based on ZigBee wireless sensor networks is shown in Figure 7.

The ceiling projector is used for robot navigation in intelligent space. In intelligent space, the information is gathered and integrated to plan a reasonable path for the mobile robot. The path data are sent to the controller of the ceiling projector through ZigBee wireless sensor networks, which casts a spot along the path. Then, the mobile robot follows the moving spot by using its on-

Figure 6. Flowchart of object search and recognition.

Figure 7. ZigBee wireless sensor networks based abnormal detection and help system.

board sensors (e.g. CCD camera). This navigation method reduces the sensors for navigation carried on mobile robot. The path data are transmitted using our ZigBee wireless sensor networks.

The ceiling projector consists of a pan-tilt and a laser projector installed at the end effect, which is shown in Figure 8(a). is the base coordinate system, and are the joint coordinate system, respectively. is the laser coordinate system installed on the pan-tilt. Figure 8(b) illustrates the D-H kinematic parameters of the laser projector.

Further, we can get the Jacobian matrix representing the joint speed and the spot velocity as,

where, represents the Jacobian matrix, is the coordinate in. The input of the system is the velocity of spot in base coordinate as shown in Figure 9, and the control law is selected as,

6. Conclusion and Future work

In this paper, the intelligent home environment based on multi-pattern information and Zigbee wireless sensor networks is introduced. With the interaction with the intelligent space, service robot can obtain more comprehensive environmental information without adding more sensors on service robot. In the intelligent space, robot can work with “light packs” and the service quality can be improved. But there should require further research about the multi-pattern information model, such as the optimization set of the artificial land marks. In additional, the algorithm of data fusion is an important research topic to improve the accuracy of decision-making.

7. Acknowledgements

The authors gratefully acknowledge the funds provided by National Natural Science Foundation of China under Grant 61075092,National High-tech Program of China (863 Program) under Grant 2009AA04Z220, Natural

(a) (b)

Figure 8. (a) Structure of ceiling projector; (b) D-H model of the ceiling projector.

Figure 9. Model of pan-tilt control system.

Science Foundation of Shandong Province under Grant ZR2011FM011 and Self-determined and innovation foundation of Shandong University under Grant 2011JC017.

REFERENCES

  1. G. H. Tian, “Wide Future for Home Service Robot Research,” International Academic Developments, No. 1, 2007, pp. 28-29.
  2. S.-H. Baeg, J.-H. Park, J. Koh, et al., “Building a Smart Home Environment for Service Robots Based on RFID and Sensor Networks,” International Conference on Control, Automation and Systems, Seoul Korea, 17-20 October 2007, pp. 1078-1082.
  3. G. H. Tian, X. L. Li, S. P. Zhao, et al., “Research and Development of Intelligent Space Technology for Home Service Robot,” Journal of Shandong University: Engineering Science, Vol. 37, No. 5, 2007, pp. 53-59.
  4. J.-H. Lee and H. Hashimoto, “Intelligent Space,” Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Takamatsu, October 30-5 November 2000, pp. 1358-1363.
  5. J.-H. Lee, N. Ando and H. Hashimoto, “Design Policy of Intelligent Space,” Proceedings of the IEEE International Conference on Systems, Man, and Cyberneticsi, Tokyo, 12-15 October 1999, pp. 1077-1082.
  6. R. Katsuki, J. Ota, T. Mizuta, T. Kito, T. Arai, et al., “Design of an Artificial Mark to Determine 3D Pose by Monocular Vision,” IEEE International Conference on Robotics and Automation, Taipei, 14-19 September 2003, pp. 995-1000.
  7. R. Katsuki, J. Ota, Y. Tamura, T. Mizuta, T. Kito, et al., “Handling of Objects with Marks by a Robot,” IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, 27-31 October 2003, pp. 130-135.
  8. H.-T. Xue, G.-H. Tian, X.-L. Li and F. Lu, “Application of the QR Code for Various Object Identification and Manipulation,” Journal of Shandong University (Engineering Science), Vol. 37, No. 6, 2007, pp. 25-30.
  9. D. Scharstein and A. J. Briggs, “Real-time Recognition of Self-Similar Landmarks,” Image and Vision Computing, Vol. 19, No. 11, 2001, pp. 763-772. doi:10.1016/S0262-8856(00)00105-0
  10. B. Zitova and J. Flusser, “Landmark Recognition Using Invariant Features,” Pattern Recognition Letters, Vol. 20, No. 5, 1999, pp. 541-547. doi:10.1016/S0167-8655(99)00031-8
  11. D. Liu and X.-Q. Gao, “Research on Algorithm of Processing and Identification of QR Barcode Image,” Information Technology, Vol. 28, No. 1, 2004, pp. 61-63.
  12. N. Strobel, S. Spors and R. Rabenstein, “Joint AudioVideo Object Localization and Tracking,” IEEE Signal Processing Magazine, Vol. 18, No. 1, 2001, pp. 22-31. doi:10.1109/79.911196
  13. C. Cerrada, S. Salamanca, A. Adan, E. Perez, J.-A. Cerrada and I. Abad, “Improved Method for Object Recognition in Complex Scenes by Fusioning 3-D Information and RFID Technology,” IEEE Transactions on Instrumentation and Measurement, Vol. 58, No. 10, 2009, pp. 3473-3480. doi:10.1109/TIM.2009.2018000
  14. T. Kim, J. Shin and S. Tak, “Cell Planning for Indoor Object Tracking Based on RFID,” International Conference on Mobile Data Management: Systems, Services and Middleware, Taipei, 18-21 May 2009, pp. 709-713.
  15. S. Roh, H. and R. Choi, “3-D Tag-Based RFID System for Recognition of Object,” IEEE Transactions on Automation Science and Engineering, Vol. 6, No. 1, 2009, pp. 55-65. doi:10.1109/TASE.2008.2008119
  16. P. Kamol, S. Nikolandis, R. Ueda and T. Arai, “RFID Based Object Localization System Using Ceiling Cameras with Particle Filter,” International Conference on Future Generation Communication and Networking, Jeju Island, 6-8 December 2007, pp. 37-42.
  17. Y. H. Xue, G. H. Tian, R. K. Li and H. T. Jiang, “A New Object Search and Recognition Method Based on Artificial Object Mark in Complex Indoor Environment,” The 8th World Congress on Intelligent Control and Automation, Jinan, 7-9 July 2010, pp. 6648-6653.
  18. D. Smith and S. Singh, “Approaches to Multisensor Data Fusion in Target Tracking: A Survey,” IEEE Transactions on Knowledge and Data Engineering, Vol. 18, No. 12, 2006, pp. 1696-1710. doi:10.1109/TKDE.2006.183
  19. H. S. Carvalho, W. B. Heinzelman, A. L. Murphy, et al., “ A General Data Fusion Architecture,” Proceedings of the IEEE International Conference on Information Fusion, Queensland, 8-11 July 2003, pp. 1465-1472.
  20. I. F. Akyildiz, W. Su and Y. Sankarasubramaniam, “Wireless Sensor Networks: A Survey,” Computer Networks, Vol. 38, No. 4, 2001, pp. 393-422. doi:10.1016/S1389-1286(01)00302-4
  21. J.-S. Lee, Y.-W. Su and C.-C. Shen, “A Comparative Study of Wireless Protocols: Bluetooth, UWB, ZigBee, and Wi-Fi,” Proceedings of 33rd Annual Conference of the IEEE Industrial Electronics Society, Taipei, 5-8 November 2007, pp. 46-51.
  22. D. Y. He, “The ZigBee Wirelesss Sensor Network in Medical Care Application,” International Conference on Computer, Mechatronics, Control and Electronic Engineering, Changchun, 24-26 August 2010, pp. 497-500.
  23. E. Callaway, P. Gorday, L. Hester, et al., “Home Networking with IEEE 802.15.4 Developing Standard for Low-Rate Wireless Personal Area Networks,” IEEE Communications Magazine, Vol. 40, No. 8, 2002, pp. 70-77. doi:10.1109/MCOM.2002.1024418
  24. K. Gill, S.-H. Yang, F. Yao and X. Lu, “A ZigBee-Based Home Automation System,” IEEE Transactions on Consumer Electronics, Vol. 55, No. 2, 2009, pp. 422-430. doi:10.1109/TCE.2009.5174403
  25. M. Brejl and M. Sonka, “Object Localization and Border Detection Criteria Design in Edge-Based Image Segmentation: Automated Learning from Examples,” IEEE Transactions on Medical Imaging, Vol. 19, No. 10, 2000, pp. 973-985. doi:10.1109/42.887613
  26. S. Ekvall, D. Kragic and F. Hoffmann, “Object Recognition and Pose Estimation Using Color Cooccurrence Histograms and Geometric Modeling,” Image and Vision Computing,” Vol. 23, No. 11, 2005, pp. 943-955. doi:10.1016/j.imavis.2005.05.006
  27. M. Ulrich, C. Steger and A. Baumgartner, “Real-Time Object Recognition Using a Modified Generalized Hough Transform,” Pattern Recognition, Vol. 36, No. 11, 2003, pp. 2557-2570. doi:10.1016/S0031-3203(03)00169-9
  28. P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” Proceedings. of International Conference on Computer Vision and Pattern Recognition, Kauai, 8-14 December 2001, pp. 511- 518.

NOTES

*Corresponding author.

Journal Menu >>