While visual appearances play a main role in recognizing the concepts captured in images, additional information can provide complementary information for fine-grained image recognition, where concepts with similar visual appearances such as species of birds need to be distinguished. Especially for recognizing geospatial concepts, which are observed only at specific places, geographical locations of the images can improve the recognition accuracy. However, such geo-aware fine-grained image recognition requires prior information about the visual and geospatial features of each concept or the training data composed of high-quality images for each concept associated with correct geographical locations. By using a large number of images photographed in various places and described with textual tags which can be collected from image sharing services such as Flickr, this paper proposes a method for constructing a geospatial concept graph which contains the necessary prior information for realizing the geo-aware fine-grained image recognition, such as a set of visually recognizable fine-grained geospatial concepts, their visual and geospatial features, and the coarse-grained representative visual concepts whose visual features can be transferred to several fine-grained geospatial concepts. Leveraging the information from the images captured by many people can automatically extract diverse types of geospatial concepts with proper features for realizing efficient and effective geo-aware fine-grained image recognition.

One of the most serious problems in modern society is that internet users are deceived by various fake information. In this paper, we analyse a scenario based on an actual incident of fraud called business email compromise (BEC). We suppose that each email, in the incident, step-wisely changes user's thinking and makes him or her believe the emails. If fraud has such a step-by-step mechanism, it allows us to consider counter measures for dissuading a user from decision-making on the way. We discuss features and factors of the incident based on formulations of Channel theory. Our analysis revealed that each email message influenced user's decision-making by what kind of logical trap. It can be fundamental knowledge that is capable of warning the user with predicting logical traps. This paper contributes to providing a novel viewpoint to develop systems for detecting deception of BEC.

Online ISSN : 2186-7364

The problem that old people are sometimes deceived by means of remittance fraud has become a great concern in our society. In this paper, we consider deceived person's thinking using Barwise-Seligman's framework, which is a logic for representing distributed systems among people or artifacts. The framework has been used to consider the conversation with comical misconception. We think that it is similar to remittance fraud because in telephone conversation, a criminal of the fraud uses victim's misconception so that the criminal can impersonate somebody. In this paper, we consider a logical system to describe the communication of a typical remittance fraud, and discuss the representative ability of the formulas to express the situation where a person is deceived or not deceived. The formulas have indicated that a person may be deceived as a result of handling exceptions in terms of logic. This paper contributes to providing a novel viewpoint to consider why people are deceived.

In this paper, we propose a method for generating photorealistic video in which a person virtually performs a motion that is not performed in the real world. we refer to such video as motion video clones (MVCs). The proposed method requires only two kinds of source information as input data: a reference video, in which a person A performs some motion, and a target image, which includes the whole body of another person B. Using these data, our method generates a MVC in which the person A's motion is re-enacted by the person B's body. Since our method does not need 3D human body model nor any training phase, it is suitable to MVC-based entertainment systems. To handle the self-occlusion of the human body in the reference video, we employ a part-based approach. For each part such as the right arm and the left leg, we first extract its skeleton from the target image and move it so that the motion represented by the reference video is re-enacted. Next, we compute a 2D affine transform between the original and moved positions of the skeleton. This transform is used to map the texture of the target image onto each frame of a resultant MVC. Finally, we extend the part-wise affine transforms to pixel-wise ones by computing their linear combination for each pixel, whose combination weights are computed according to the geodesic distance between the pixel and the center of each part. This allows us to avoid unnatural appearance around the joints of the body parts. In our experiments, the proposed method generated much visually-natural MVCs than existing methods.

8 pages

This paper proposes a method for discovering the knowledge about geospatio-temporal concepts, which are strongly related to specific locations and times, from images tagged with latitude and longitude coordinates, time-stamps, and textual tags. The proposed method firstly extracts textural tags used either in specific locations or times as the geospatial or temporal concepts with their location or time information. Then, the semantic relations among the geospatio-temporal concepts are extracted based on the co-occurrence of the corresponding tags to be represented as a graph. Finally, visually discriminative geospatio-temporal concepts are linked to representative visual concepts. The usefulness of the discovered knowledge is evaluated based on the automatic tagging for the images tagged with latitude and longitude coordinates and time-stamps.

Since a large number of users of various social networking services post what they observe around themselves, what is going on around the world can be known in real time by extracting such real-world observations. Especially, observations covering miscellaneous areas of interest are posted to Twitter as short text messages. Our goal is to extract such observations to better understand the current situations of the real world. Since the observations at specific locations are often described with words representing the observed locations or events, the locations or events all over the world can be discovered by finding such local words which are uniquely used at specific locations or times. Then, parts of their observations can be obtained by using the local words to define the semantics of each local word. Finally, the observations which do not contain the local words but are related to the locations or events can be extracted based on their semantic relevancy to each local word.

Recently, geotagged posts to social media such as Twitter have become major sources for geospatial information referenced to a set of geographic coordinates. Especially, many existing work collects geospatial terms which identify geographic locations by examining the spatial locality of the term usage patterns observed in the geotagged posts accumulated for a certain period of time. Although the spatial locality of the collected geospatial terms can only be temporary, such time variability difference among the geospatial terms have not been considered in the existing work. Thus, in order to separately collect the stationary and temporary geospatial terms with proper timing, we propose an on-line method for constructing a geographical dictionary containing the up-to-date geospatial terms and their locations by continuously examining the spatial locality of terms in streaming geotagged posts. The geospatial terms can be distinguished between stationary and temporary terms according to their usage patterns in the recent set of geotagged posts and their locations recorded in the geographical dictionary. Additionally, images representing each geospatial term can also be collected from the posts containing the corresponding term. The usefulness of the collected geospatial information is demonstrated in comparison to existing geographical dictionaries constructed by experts, crowdsourcing, and batch method.

In this paper, we focus on the problem of group detection in crowd, which is a task of partitioning a set of pedestrians in a scene into small subsets called groups based on their trajectories. Most of previous methods use only a single model for representing a relationship between trajectories of pedestrians who belong to the same group. However, such relationship would vary depending on the activity state (e.g. walking together, approaching, splitting, and so on) of the group. In this paper, we propose a novel group detection method which can cope with a variation of groups' activity state. The proposed method constructs different models for each activity state in order to appropriately evaluate the relationship of pedestrians' trajectories. In addition, our method regards groups' activity state as hidden variables and estimates their probability distributions, which is used for integrating the constructed models. The proposed method outperforms existing methods in the experiment on the public dataset.

Nowadays mobile devices such as smartphones are widely used all over the world. Moreover, the performance of image recognition has dramatically increased by deep learning technologies. From these backgrounds, we think that the following scenario of information services could be realized in the near future: users take a photo and send it to a server, who recognizes the location in the photo and returns the users some information about the recognized location. However, this kind of client-server-based image recognition can cause a privacy issue because image recognition results are sometimes privacy sensitive. To tackle the privacy issue, in this paper, we propose a novel framework for privacy-preserving image recognition in which the server cannot uniquely identify the recognition result but users can do so. An overview of the proposed framework is as follows: First users extract a visual feature from their taken photo and transform it so that the server cannot uniquely identify the recognition result. Then users send the transformed feature to the server, who returns a candidate set of recognition results to the users. Finally, the users compare the candidates and the original visual feature for obtaining the final result. Our experimental results demonstrate the effectiveness of the proposed framework.

In this paper, we focus on the problem of inter-concept distance measurement (ICDM), which is a task of computing the distance between two concepts. ICDM is generally achieved by constructing a visual model of each concept and calculating the dissimilarity score between two visual models. The process of visual concept modeling often suffers from the problem of junk images, i.e., the images whose visual content is not related to the given text-tags. Similarly, it is naively expected that junk images also give a negative effect on the performance of ICDM. On the other hand, junk images might be related to its text-tags in a certain (non-visual) sense because the text-tags are given by not automated systems but humans. Hence, the following question arises: Is the effect of junk images on the performance of ICDM positive or negative? In this paper, we aim to answer this non-trivial question from experimental aspects using a unified framework for ICDM and junk image detection. Surprisingly, our experimental result indicates that junk images give a positive effect on the performance of ICDM.

Anthropometric biometrics are the most suitable features for long-term people reidentification. However, feature selection is still a major problem in the anthropometric biometrics literature. In this paper, we aim at improving people reidentification by enhancing feature selection. Based on a statistical analysis of body measurements on a large-scale dataset, a new anthropometric signature is introduced. The proposed signature describes both the size and shape of human body at specific anatomical landmarks. While size is measured by Euclidian distance between four skeleton joint pairs, shape is described by the surface distance along four circular body parts. A novel algorithm is proposed to automatically segment the circular body parts from the subject point cloud using body geometry, cylindrical fitting and soft clustering. The overall system is evaluated on two public datasets using CMC and nAUC metrics. Experimental results show the effectiveness of our method compared to state of the art.

The recent popularization of social network services (SNSs), such as YouTube, Dailymotion, and Facebook, enables people to easily publish their personal videos taken with mobile cameras. However, at the same time, such popularity has raised a new problem: video privacy. In such social videos, the privacy of people, i.e., their appearances, must be protected, but naively obscuring all people might spoil the video content. To address this problem, we focus on videographers' capture intentions. In a social video, some persons are usually essential for the video content. They are intentionally captured by the videographers, called intentionally captured persons (ICPs), and the others are accidentally framed-in (non-ICPs). Videos containing the appearances of the non-ICPs might violate their privacy. In this paper, we developed a system called BEPS, which adopts a novel conditional random field (CRF)-based method for ICP detection, as well as a novel approach to obscure non-ICPs and preserve ICPs using background estimation. BEPS reduces the burden of manually obscuring the appearances of the non-ICPs before uploading the video to SNSs. Compared with conventional systems, the following are the main advantages of BEPS: (i) it maintains the video content, and (ii) it is immune to the failure of person detection; false positives in person detection do not violate privacy. Our experimental results successfully validated these two advantages.

One way to prevent privacy intrusion is by blurring or blocking out facial images using image processing. However, this technique's effectiveness depends on viewers' familiarity with the subjects as well as on the subjects' conspicuousness.

A large number of real-world observations by social sensors all over the world can be obtained from various social networking services. Especially, observations covering miscellaneous areas of interest are posted to Twitter as short text messages. Our goal is to extract a wide range of observations related to the target of interest specified by the user from Twitter regardless of their popularity. Assuming that the related observations are likely to contain words people often associate with each other, the associative relations among words are learned from the past messages. When a user gives a keyword representing his/her current interest, recent related observations are extracted based on their word frequency distributions in terms of the associative relations to the keyword.

An enormous number of images are currently shared through social networking services such as Facebook. These images usually contain appearance of people and may violate the people's privacy if they are published without permission from each person. To remedy this privacy concern, visual privacy protection, such as blurring, is applied to facial regions of people without permission. However, in addition to image quality degradation, this may spoil the context of the image: If some people are filtered while the others are not, missing facial expression makes comprehension of the image difficult. This paper proposes an image melding-based method that modifies facial regions in a visually unintrusive way with preserving facial expression. Our experimental results demonstrated that the proposed method can retain facial expression while protecting privacy.

This paper proposes a template updating method for improving authentication accuracy in behavioral biometric authentication with hand/arm motion gestures for mobile devices. We introduce an extended version of the standard K-Medoids based clustering algorithm called supervised K-Medoids, which can handle with 2-class data such as positive samples and negative samples. Using the supervised K-Medoids, the template corresponding to each owner is selected as the one that is the most identifiable as the actual owner, and, at the same time, the most distinguishable from the others. Therefore, our method can decrease False-Rejection-Rate (FRR) and False-Acceptance-Rate (FAR) simultaneously, compared to the conventional work that is based on the template update with only the owner's data to decrease FRR. Our template update with multi-owner data attains Equal-Error-Rate (EER) of 5.2% whereas the conventional template update method with owner's own data results in 12.0% when 10 subjects authenticate with gestures for 10 days.

Our society has been getting more privacy-sensitive. Diverse information is given by users to information and communications technology (ICT) systems such as IC cards benefiting them. The information is stored as so-called big data, and there is concern over privacy violation. Visual information such as images and videos is also considered privacy-sensitive. The growing deployment of surveillance cameras and social network services has caused a privacy problem of information given from various sensors. To protect privacy of subjects presented in visual information, their face or figure is processed by means of pixelization or blurring. As image analysis technologies have made considerable progress, many attempts to automatically process flexible privacy protection have been made since 2000, and utilization of privacy information under some restrictions has been taken into account in recent years. This paper addresses the recent progress of privacy protection for visual information, showing our research projects: PriSurv, Digital Diorama (DD), and Mobile Privacy Protection (MPP). Furthermore, we discuss Harmonized Information Field (HIFI) for appropriate utilization of protected privacy information in a specific area.

This paper proposes a sensing-based real-world content called Digital Diorama, which is a three-dimensional miniature model of the dynamically changing real world created from the real-time data continuously published over the Internet by stationary cameras distributed in public spaces. Digital Diorama is designed to preserve both the privacy of the monitored persons and the intelligibility of the content by superimposing the latest background images and human icons each of which represents a monitored person in the three-dimensional model. By using the data continuously published from 10 stationary cameras installed on one floor of a shopping mall, our prototype Digital Diorama browser was able to construct and dynamically update Digital Diorama in approximately 1 fps. The results of subjective evaluations indicated that utilizing appropriate human icons can improve the intelligibility of the content while preserving the privacy of the monitored persons.

Most of the existing methods for measuring the inter-concept distance (ICD) between two concepts from their image instances use only a single kind of visual feature extracted from each instance. However, a single kind of feature is not enough for appropriately measuring ICDs due to a wide variety of perspectives for similarity evaluation (e.g., color, shape, size, hardness, heaviness, and functions); the relationships between different concept pairs are more appropriately modeled from different perspectives provided by multiple kinds of features. In this paper, we propose extracting two or more kinds of visual features from each image instance and measuring ICDs using these multiple features. Moreover, we present a method for adaptively weighting the visual features on the basis of their appropriateness for each concept pair. Experiments demonstrated that the proposed method outperformed a method using only a single kind of visual feature and one combining multiple kinds of features with a fixed weight.

Recently, geotagged posts to social media such as Twitter have been used to automatically construct a geographical dictionary containing diverse types of local words which indicate specific locations in the real world. The existing methods typically examine the spatial locality of the usage patterns observed in the geotagged posts accumulated for a certain period of time to select the local words; however, how long the geotagged posts need to be accumulated depends on the usage frequency of the word, and additionally, some local words can indicate different locations at different times. Thus, we propose a real-time method for constructing a local word database which consistently keeps the local words and their locations up to date by iteratively adding new local words, removing old temporary local words, and updating the locations indicated by the local words. These functions are realized by adaptively recording/resetting the usage history of each word to properly examine its spatial locality and by assigning the weight for each geotag which is used to represent the locations indicated by the local words according to their temporal variations. The local word database constructed by our proposed method was verified to contain more up-todate local words and locations compared to other types of geographical dictionary constructed by experts, crowdsourcing, and from the geotagged tweets accumulated for a fixed period of time based on the performance evaluations of tweet location estimation as an example of applications utilizing the geographical dictionaries.

Visual SLAM algorithms exploit natural scene features to infer the camera motion and build a map of the environment landmarks. SLAM algorithm has two interrelated processes localization and mapping. For accurate localization, we need the features location estimates to converge quickly. On the other hand, to build an accurate map, we need accurate localization. Recently, a biologically inspired approach exploits deliberate camera oscillation has been used to improve the convergence speed of depth estimate. In this paper, we explore the effect of camera oscillation pattern on the accuracy of VSLAM. Two main oscillation patterns are used for distance estimation: translational and rotational. Experiments, using static and moving robot, are made to explore the effect of these oscillation patterns on the VSLAM performance.

Visual SLAM algorithms exploit natural scene features to infer the camera motion and build a map of a static environment. In this paper, we relax the severe assumption of a static scene to allow for the detection and deletion of dynamic points. A new 'virtual correction' method is introduced which serves to detect the dynamic points by checking the re-projection error of the points before and after the virtual measurement update. It can also recover the erroneously excluded useful features, particularly the distant points which may be deleted because of the change in its position after new measurement observation. Deliberate camera oscillations are also used to improve the VSLAM accuracy and the camera observability. The simulation results showed the effectiveness of the virtual correction when combined with camera oscillation in recovering the misclassified features and detecting the dynamic features even in difficult scenarios.

This paper proposes a real-world event detection method by using the time and location information and text tags attached to the images in Flickr. Events can generally be detected by extracting images captured at the events which are annotated with text tags frequently used only in specific times and locations. However, such approach can not detect events where only a small number of images were captured. We focus on the fact that semantically related events often occur around the same time at different locations. Considering a group of these events as an event class, the proposed method firstly detects event classes from all images in Flickr based on their similarity of the captured time and text tags. Then, from the images consisting each event class, events are detected based on their similarity of the captured locations. Such two-step approach enables us to detect events where a small number of images were captured. © 2014 Springer International Publishing.

First International Workshop on Information Hiding and its Criteria for evaluation (IWIHC2014), (in conjunction with ASIACCS 2014), Kyoto, Japan

San Jose

The popularity of hand-held video camcorders has increased the amount of poor-quality home videos captured by amateur camcorder users. This paper introduces the content analysis techniques, namely, techniques for segmentation, indexing, and static and dynamic representation generation, which have been developed to help viewers watch such poor-quality videos by considering the characteristics of home videos.

This paper concerns depth-estimation-free conditions for projective factorization. We first show that, using an algebraic approach, the estimation of the projective depth is avoidable if and only if the origins of all camera coordinate systems are lying on a single plane, and optical axes of the coordinate systems point the same direction that is perpendicular to the plane. Next, we generalize the result to the case where the points are possibly restricted on a plane or on a line. The result clearly reveals the trade-off between the freedom of camera motion and that of point location. We also give a least-square-based method for Euclidean reconstruction from the result of the projective reconstruction. The proposed method is evaluated through simulation from the viewpoint of computational time. © 2013 Springer-Verlag.

In this paper, a new algorithm which adopts the techniques of double prediction and the Pareto probability model was applied to encode the DC term in the JPEG compression process. Conventionally, the DC term was encoded by differential coding, i.e., the difference of the DC values between the current block and the previous block. In this paper, we first use the DC terms of four adjacent blocks to predict the current DC value. We then further use the prediction error of the four adjacent blocks to estimate the variance of the prediction error of the current block. We call it the double prediction algorithm. Next, the Pareto distribution is applied to model the probability distribution of the prediction error. Simulation results show that, with the proposed algorithms, the data size required for DC terms is significantly reduced by 25% ∼ 60% and a much higher compression rate can be achieved. © 2013 IEEE.

Our goal is to develop a method for counting the number of people traveling over a wide area monitored by spatially disjoint multiple cameras with non-overlapping fields of view. The proposed method counts the number of people traversing across each pair of cameras' fields of view by estimating the flows between the foreground regions which have disappeared from and appeared in the camera views within a short time interval. The approach aims at resolving two problems: people in a foreground region can split and merge outside the cameras' fields of view, and the appearance variance of the same person and persons with similar appearance can lead to errors in person re-identification across cameras. The average errors of 0.184 persons in counting the number of people traveling over an area in a university campus monitored by four virtual cameras for 100 minutes have demonstrated the effectiveness of the proposed method. © 2013 IEEE.

Most social videos, which are uploaded and shared through social networking services (SNSs), e.g., YouTube and Facebook, contain not only intentionally-captured persons (ICPs) but also non-ICPs who are unexpectedly framed in, such as passers-by. Sharing such social videos may infringe on the non-ICPs' privacy but not on the ICPs' in many cases
however, existing systems for video privacy protection simply obscure persons without distinguishing ICPs from non-ICPs. This naive obscuration may spoil the videos. Since this is a critical problem especially for social videos, in this paper, we propose a novel system for automatically generating privacy-protected videos in real-time. Our system localizes ICPs and non-ICPs using ICP detection leveraging the spatial and temporal consistency of ICPs/non-ICPs and obscures the non-ICPs. We have experimentally evaluated the performance of ICP detection and demonstrated the applicability of our system. © 2013 IEEE.

With the recent popularization of mobile video cameras including camera phones, a new technology, mobile video surveillance, which uses mobile video cameras for video surveillance has been emerging. Such videos, however, may infringe upon the privacy of others by disclosing privacy sensitive information (PSI), i.e., their appearances. To prevent videos from infringing on the right to privacy, new techniques are required that automatically obscure PSI regions. The problem is how to determine the PSI regions to be obscured while maintaining enough video content to present the camera persons' capture-intentions, i.e., what they want to record in their videos to achieve their surveillance tasks. To this end, we introduce a new concept called intended human objects that are defined as human objects essential for capture-intentions, and develop a new method called intended human object detection that automatically detects the intended human objects in videos taken by different camera persons. Through the process of intended human object detection, we develop a system for automatically obscuring PSI regions. We experimentally show the performance of intended human object detection and the contributions of the features used. Our user study shows the potential applicability of our proposed system.

In this paper, a new framework of hybrid mobile-fixed video surveillance system (HMFVSS) is introduced. The purpose of this framework is to overcome common problems of existing mobile or fixed video surveillance systems: (1) moral harassment due to unfriendly or unnaturally installed mobile sensors, and (2) blind areas due to narrow-scope moving of fixed cameras. A case study of abandoned object's owner alert system (AOOAS) is also presented to emphasize the framework's advantages. IP cameras and "Spyglass" (i.e. a mobile camera embedded on glasses) are used as fixed and mobile sensors, respectively. There are three main tasks are inherited, developed, and integrated: (1) image registration for automatically locating abandoned object, (2) common histogram based abandoned object's owner detection, and (3) faces recognition. The experimental results with careful evaluation and comparison with others shows that the proposed framework moves a step ahead in video surveillance system.

This paper proposes a method for retrieving images containing a specific target person from a given image collection of group photos. This can be realized by query-by-example methods which compare the facial visual features of the target person in the given query image and of each person in the images in the image collection. However, since images are often taken under various conditions, facial appearance of the same person can vary. Since socially related people such as family and friends are often taken photos together, the people co-occurrence relations in the same images can also be a useful clue for image retrieval. Focusing on such people co-occurrence relations, we propose Bag of People (BoP) features which represent both the facial appearances of persons and their co-occurrence relations in the same images. By using the BoP features, a classifier for classifying images into two classes, images containing the target person and other images, can be trained from a small number of images labeled by user's relevance feedback. Furthermore, since the labeled images obtained by relevance feedback are much fewer than unlabeled images in the image collection, an active learning method is used to select useful images to train the classifier. When retrieving images of 24 persons in total from 550 images, after five feedback iterations, the mean average precision of 0.94 was obtained by considering the people co-occurrence relations, as against 0.69 when considering only the target person. Copyright © 2012 ACM.

Recommender systems automatically determine suitable items for users. Although preferences or context of users have been widely utilized in order to evaluate the suitability of the items for users, the surrounding context have little been considered. Focusing on that many ordinary human beings voluntarily report their observations of the current situation of the world to microblogs, this paper proposes a recommender system which not only recommends suitable restaurants to users based on their preferences and context but also provides the surrounding context information reported to microblogs which will further affect the users' restaurant selection behaviors. In particular, considering that such influential surrounding context information in microblogs includes keywords related to restaurant assessment, we propose a method for automatically determining the keywords to extract the context information by analyzing online reviews, which have been gathered also from ordinary human beings over a long period of time. The experiments by using Twitter as microblogs and Tabelog, a popular online restaurant review site in Japan, to obtain online reviews, indicated that the influential context information can be extracted from Twitter with the highest recall of 93.3% by using the area-related keywords. Additionally using the restaurant-related keywords was effective in removing irrelevant information obtaining the precision of 15.9%.

This paper presents a delivery method for viewer-specific privacy protected video, where "viewer-specific" implies that the way of privacy protection (e.g., box, mosaic, transparency) varies according to viewer's authority. In this method, in order to reduce the load of a delivery server, viewer-specific privacy protected videos are not produced at the delivery server, but they are produced at each viewer's terminal. The delivery server extracts and decomposes the information of human objects, and then embeds the information into background images using information hiding technique. Each viewer extracts a part of the decomposed information of human objects according to his/her authority, and then produces privacy protected video by integrating the extracted information. In this method, the discrete wavelet transform (DWT) plays a key role. The proposed method is evaluated through experiments.

Most videos taken by videographers contain intentionally-captured persons (ICPs), who are essential for what the videographers want to express in their video. This paper presents a method to detect ICPs in real-time. Whether a person in a video is an ICP or not is reflected in features such as the person's motion and camera motion, which are thus beneficial for detecting ICPs. However, estimating camera motion is computationally expensive. For real-time detection, we use samples of acceleration and angular velocity obtained from inertial sensors instead of estimating camera motion. Considering that pairwise constraints based on differences between persons' sizes also improve the detection performance, we model the ICPs using Markov random field. We experimentally evaluate the performance of our method and demonstrate that it works in real-time.

This paper proposes a method for tablet owner authentication based on behavioral characteristics of multiple fingers' actions called multi-touch actions. The method is based on dynamic time warping, which has been commonly used for authentication using pentablet or single finger's actions, but another problem arises due to the use of multi-touch actions (e. g., identifying fingers). We also provide methods for these problems. Using proposed method, we evaluate the authentication accuracies for several types of multi-touch actions through experiments.

Recently, a number of location-based services such as navigation and mobile advertising have been proposed. Such services require real-time user positions. Since a global positioning system (GPS), which is one of the most well-known techniques for real-time positioning, is unsuitable for indoor uses due to unavailability of GPS signals, many indoor positioning systems (IPSs) using WLAN, radio frequency identification tags, and so forth have been proposed. However, most of them suffer from high installation costs. In this paper, we propose a novel IPS for real-time positioning that utilizes a digital audio watermarking technique. The proposed IPS first embeds watermarks into an audio signal to generate watermarked signals, each of which is then emitted from a corresponding speaker installed in a target environment. A user of the proposed [PS receives the watermarked signals with a mobile. device equipped with a microphone, and the watermarks are detected in the received signal. For positioning, we model various effects upon watermarks due to propagation in the air, i.e., delays, attenuation, and diffraction. The model enables the proposed IPS to accurately locate the user based on the watermarks detected in the received signal. The proposed IPS can be easily deployed with a low installation cost because the IPS can work with off-the-shelf speakers that have been already installed in most of the indoor environments such as department stores, amusement arcades, and airports. We experimentally evaluate the accuracy of positioning and show that the proposed IPS locates the user in a 6 m by 7.5 m room with root mean squared error of 2.25 m on average. The results also demonstrate the potential capability of real-time positioning with the proposed IPS.

A video remix is generally created by arranging selected video clips and combining them with other media streams such as audio clips and video transition effects. This paper proposes a system for semi-automatically creating video remixes of good expressive quality. Given multiple original video clips, audio clips, and transition effects as the input, the proposed system generates a video remix by five processes: I) video clip sequence generation, II) audio clip selection, III) audio boundary extraction, IV) video segment extraction, and V) transition effect selection, based on the spatial and temporal structural patterns automatically learned from professionally created video remix examples. Experiments using movie trailers of action genre as video remix examples not only demonstrate that video remixing by professionals can be imitated based on examples but also reveal that the video clip sequence generation and audio clip selection are the most important processes to improve the perceived expressive quality of video remixes.

Recently, video sharing services such as YouTube and Daily-motion have become popular and many videos taken with mobile video cameras are uploaded to such a video sharing service. However, such videos can infringe on the privacy right of people in the videos because they may contain privacy sensitive information (PSI) of the people, i.e., their appearances. This strongly motivates us to develop a technique to generate privacy-protected videos. In this paper, we propose a novel system for automatic generation of privacy-protected videos based on background estimation. In most conventional techniques, objects that contain PSI are detected and obscured by, e.g., blurring. Conversely, in our system, background pixels are estimated and then substituted with intended human objects that are essential for the camera person's capture intention. We quantitatively evaluate our system to demonstrate its potential applicability. © 2011 IEEE.

There are several zoom-in video display methods including full-zoom and fisheye view that magnify the regions of interest (ROIs). However, those methods usually discard or deform the remaining regions without considering their content. In this paper, we propose a method for generating a content-preserving zoom-in view which magnifies ROIs and at the same time preserves the content of the remaining regions. Targeting on surveillance videos, our method firstly extracts moving objects from every input frame as ROIs. Then, the importance score is calculated for each pixel in the input frame based on its content to determine where the deformation, which may cause the destruction of the content, should be avoided. Finally, a mapping problem from the input frame to the zoom-in view with respect to the importance score is formulated to deform less important regions more than the important ones. Experiments are conducted to study the effectiveness of considering the content importance. We also compare the results of our method with those of other methods, fisheye view and a method of using uniform scaling and seam carving.

The recent popularity of digital cameras allows us to take a large number of images. There is an increasing need for efficiently and accurately retrieving images containing a specific person from such image collections. While only the visual features of the specific person are used in many query-by-example retrieval methods, we focus on the fact that some people such as family or friends are more likely to appear in the same images than others and use visual features of not only the queried person but also people who have strong co-occurrence relations with the queried person to improve the retrieval performance. The relevance feedback is used to learn who co-occur with the queried person in the same images, their faces, and the strength of their co-occurrence relations. For 116 images collected from 6 persons, after five feedback iterations, the recall rate of 53% was obtained by considering the co-occurrence relations among people, as against 34% when using only features of the queried person.

When camera persons take videos with mobile video cameras, they usually have capture intentions, i.e., what they want to express in their videos, and there are intentionally captured regions (ICRs) in the video frames that are essential for the capture intentions. Extracting ICRs is thus beneficial for wide range of applications such as video summarization and video adaptation for small displays. In this paper, we present a novel method for automatically extracting ICRs. A camera person usually moves his/her camera so that ICRs can be arranged in appropriate positions in video frames
therefore, ICRs can yield specific motion. This observation indicates that such specific motion is a vital cue for extracting ICRs. The proposed method represents motion by point trajectories, which are long-term trajectories of spatially dense points in video frames, and extracts ICRs using an ICR model based on the point trajectories. We experimentally evaluate the proposed method to demonstrate its potential applicability. Copyright 2011 ACM.

Video remixes are generally created by sequentially arranging selected video clips and mixing them with other media streams such as audio clips and transition effects. Especially, mixing music clips often effectively improves the expressive quality of video clips. This paper proposes an example-based system for supporting average users in the 3 steps in video remixing: I)video shot sequence creation, II)music clip selection, and III)audio volume adjustment. The proposed system creates a template for a video remix and gives suggestions to users on an interface such as which video clips should be selected to create a video shot sequence and which music clips should be mixed to the created video shot sequence based on professionally created video remix examples. Then, the audio volume of each video shot is automatically adjusted based on its audio content so that the sounds in the video shots and the music clips would not interfere with each other. Experiments have verified that our system was able to create a video shot sequence whose quality was improved equally as the professionally created one by mixing the selected music clips, doubling the subjective scores from 1.7 to 3.7 on a scale of 1-5. Automatic audio volume adjustment improved the subjective scores by approximately 0.5 points on average. Further, the suggestions provided on the interface was evaluated useful by 5 subjects when creating a video remix by selecting 32 video clips and 4 music clips from 265 video clips and 180 music clips. © 2011 ACM.

Video remixes are generally created by sequentially arranging selected video clips and combining them with other media streams such as audio clips. In this paper, an example-based approach is adopted for semi-automatically creating video remixes of good expressive quality from home videos. Given multiple home video clips and audio clips, the proposed system generates a video remix by four processes: I)video clip sequence generation, II)audio clip selection, III)audio boundary extraction, and IV)video segment extraction based on professionally created video remix examples. A user interface which presents video clips according to their suitability to the examples and their perceived quality is developed so that users can efficiently and effectively select and arrange suitable video clips in video clip sequence generation. By using movie trailers of action genre as video remix examples, a 43-second video remix was created from 45-minute home videos and was subjectively evaluated better than the one created considering only the perceived quality of video clips. © 2011 IEEE.

This paper proposes an image retrieval method which retrieves images of a specific person from group photos. Many query-by-example methods have focused only on the visual features of the queried person. However, since socially related people such as family and friends are often taken photos together, their co-occurrence relations can be useful information. Thus, we propose an image retrieval method which uses the visual features of not only the queried person but also those who co-occur with the queried person in the same images. Relevance feedback is used to learn who co-occur with the queried person, their faces, and how strong their co-occurrence relations are. When retrieving the images of 19 persons in total from 158 images, after five feedback iterations, the recall rate of 50% was obtained by considering the people co-occurrence relations, as against 33% when considering only the queried person. With human errors in giving relevance feedback, the recall rate still improved to 40%. © 2011 ACM.

Digital Diorama, the sensing-based real-world content, can be constructed by integrating real-time information obtained from sensors monitoring the real world. In order to increase the benefits of viewers without violating the privacy of monitored persons, this paper proposes three-level privacy control over the monitored persons based on their agreement to the usage of their information obtained from sensors: privacy control with i) no agreement, ii) partial agreement. and iii) mutual agreement. i) presents only their positions to show where persons are. ii) additionally presents the information which can be automatically obtained from sensors such as age and gender to show what kinds of persons are where without disclosing the visual appearances. iii) presents their visual appearances based on their mutual agreement with specific viewers. Our evaluation indicated that the representation simulating each privacy control presented information from sensors with acceptable privacy protection. © 2011 ACM.

We propose a method of using a random forest algorithm to quickly detect semantically high-level features such as specific objects. The random forest has a lower computation cost than that of the common algorithm such as a support vector machine (SVM). However, it cannot cope with training data that have a large bias in the number of negative and positive examples. We improve the conventional training algorithm to ensure sampling the data with equal probability from each class when creating bootstrap samples, which increases the classification accuracy. Experiments on the Caltech-101 dataset resulted in a recall of 64.3% and precision of 71.1%, which were comparable to those of conventional methods. The average time needed for training and for detection were reduced to one sixteenth and one twenty-seventh that of SVM, respectively.

DISI Seminar Series, University of Trento, Italy

In this paper, a new spatio-temporal method for adaptively detecting events based on Allen temporal algebra and external information support is presented. The temporal information is captured by presenting events as the temporal sequences using a lexicon of non-ambiguous temporal patterns. These sequences are then exploited to mine undiscovered sequences with external text information supports by using class associate rules mining technique. By modeling each pattern with linguistic part and perceptual part those work independently and connect together via transformer, it is easy to deploy this method to any new domain (e.g baseball, basketball, tennis, etc.) with a few changes in perceptual part and transformer. Thus the proposed method not only can work well in unwell structured environments but also can be able to adapt itself to new domains without the need (or with a few modification) for external re-programming, re-configuring and re-adjusting. Results of automatic event detection progress are tailored to personalized retrieval via click-and-see style using either conceptual or conceptual-visual query scheme. Experimental results carried on more than 30 hours of soccer video corpus captured at different broadcasters and conditions as well as compared with well-known related methods, demonstrated the efficiency, effectiveness, and robustness of the proposed method in both offline and online processes.

Digital video now plays an important role in supporting more profitable online patient training and counseling, and integration of patient training videos from multiple competitive organizations in the health care network will result in better offerings for patients. However, privacy concerns often prevent multiple competitive organizations from sharing and integrating their patient training videos. In addition, patients with infectious or chronic diseases may not want the online patient training organizations to identify who they are or even which video clips they are interested in. Thus, there is an urgent need to develop more effective techniques to protect both video content privacy and access privacy. In this paper, we have developed a new approach to construct a distributed Hippocratic video database system for supporting more profitable online patient training and counseling. First, a new database modeling approach is developed to support concept-oriented video database organization and assign a degree of privacy of the video content for each database level automatically. Second, a new algorithm is developed to protect the video content privacy at the level of individual video clip by filtering out the privacy-sensitive human objects automatically. In order to integrate the patient training videos from multiple competitive organizations for constructing a centralized video database indexing structure, a privacy-preserving video sharing scheme is developed to support privacy-preserving distributed classifier training and prevent the statistical inferences from the videos that are shared for cross-validation of video classifiers. Our experiments on large-scale video databases have also provided very convincing results.

This paper aims at analyzing the performance of an anonymous communication system 3-Mode Net with respect to the number of relay nodes required for communication and sender anonymity. As for the number of relay nodes, we give explicit formulas of the probability distribution, the expectation, and the variance. Considering sender anonymity, we quantify the degree of sender anonymity under a situation where some relay nodes collude with each other. The above analyses use random walk theory, a probability generating function, and their properties. From obtained formulas, we show several conditions for avoiding a situation where the number of relay nodes becomes large, and for providing high sender anonymity. Furthermore, we investigate the relationship between the number of relay nodes and sender anonymity, and give a condition for providing a better performance of 3MN.

This paper presents a method which attains recoverable privacy protection for video content distribution. The method is based on discrete wavelet transform (DWT), which generates scaling coefficients and wavelet coefficients. In our method, scaling coefficients, which can be regarded as a low-resolution image of an original image, are used for producing privacy-protected image. On the other hand, wavelet coefficients, which can be regarded as privacy information, are embedded into the privacy-protected image via information hiding technique. Therefore, privacy protected image can be recovered by authorized viewers if necessary. The proposed method is fully analyzed through experiments from the viewpoints of the amount of the embedded privacy information, the deterioration due to the embedding, and the computational time. © 2009 Guangzhen Li et al.

Video summarization is one of the most efficient methods for retrieval from large video archives. An automated method is described for generating TV program trailers (short video clips to advertise the program). Our method uses introductory text from an electronic program guide, which is a short description of the program highlights. We assume that the video segments corresponding to closed caption sentences with similar expressions to those of the introductory text contain appealing scenes, and our system generates a program trailer by linking these segments together. The AdaBoost algorithm is used to learn the textual characteristics of the introductory text. The proposed method was used to summarize actual TV programs to test its effectiveness.

The growing popularity of video sharing services such as YouTube enables us to upload and share consumer generated videos (CGVs) easily, resulting in disclosure of the privacy sensitive information (PSI) of persons, i.e., their appearances. Therefore, we need a technique for automatically protecting the privacy in CGVs
however, the main problem is how to determine PSI regions automatically. In this paper, we propose a novel system for automatically protecting the privacy in CGVs. The proposed system tackles the problem of determining PSI regions by using an intended human object detector that detects human objects which the camera person wanted to capture to achieve his/her capture intention. In addition, the proposed system adopts several PSI obscuring methods such as blocking out, blurring and seam carving. We present the results of subjective evaluations of a privacy protected video in terms of the visual quality and acceptability of PSI disclosure, as well as the performance of the intended human object detector. © 2010 ACM.

When people take videos, they always want to capture intended objects, which are essential for presenting what they want to express in their videos, and to share the intended objects with others. The concept of intended objects provide a novel perspective for video content analysis, and detecting intended objects may be beneficial for wide range of applications such as video understanding and semantics interpretation, video summarization, video adaptation, video privacy protection, and so on. In this paper, we focus on a particular type of intended objects, i.e., intended human objects, and an interesting method is developed for detecting intended human objects automatically from human-captured videos. We also investigate the correlation between intended human objects and visual attention. Our experimental results indicate that our method can successfully detect the intended human objects. © 2010 IEEE.

In this paper, we propose a novel anomality detection method based on external situational information and hierarchical analysis of behaviors. Past studies model normal behaviors to detect anomality as outliers. However, normal behaviors tend to differ by situations. Our method combines a set of simple classifiers with pedestrian trajectories as inputs. As mere path information is not sufficient for detecting anomality, trajectories are first decomposed into hierarchical features of different abstract levels and then applied to appropriate classifiers corresponding to the situation it belongs to. Effects of the methods are tested using real environment data. © 2010 IEEE.

Given a single face image of a specific person as a query, it is very difficult to retrieve all of his/her images from a personal image collection stored for a long term clue to age-related changes in facial appearances. This paper proposes to apply relevance feedback to enhance the performance of image retrieval from the image collections with age variation. Specifically, we propose two types of update schemes: i) query expansion and ii) weight updating and show the effects of each scheme by experiments with two actual image collections. For an image collection, the recall rate improved from 40.8% to 72.5% after five iterations of relevance feedback.

Recently, event detection in sports videos has been gaining some remarkable results. Unfortunately, there is a lack of useful tools for users to explore and exploit these event clips on their own demands. In this paper, a novel method using spatio-temporal patterns to analyze event tactics in sports videos is introduced. The proposed method aims to understand tactics of events such as distributions and speeds of players, or attacking/defensive formations throughout a time when such events happen without tracking objects. The major contribution of the proposed method is to model event tactics by using sequence of symbols. Each symbol represents a distribution of players in a certain period of time. Therefore, a sequence of symbols intrinsically is concerned as spatio-temporal patterns. By using these patterns, an event tactic is detected, explained, and integrated into an event video to create a visualizing abstract. This visualizing abstract is very useful to help users understand an event tactic without watching whole clip. Moreover, users could query by an example or by a text to find all events sharing the same tactic. Thorough testing with over 100 goal clips in soccer domain demonstrate the superiority of the proposed method in terms of precision recall ratios.

In a consumer video, there are not only intended objects, which are intentionally captured by the camcorder user, but also unintended objects, which are accidentally framed-in. Since the intended objects are essential to present what the camcorder user wants to express in the video, discriminating the intended objects from the unintended objects are beneficial for many applications, e.g., video summarization, privacy protection, and so forth. In this paper, focusing on human objects, we propose a method for discriminating the intended human objects from the unintended human objects. We evaluated the proposed method using 10 videos captured by 3 camcorder users. The results demonstrate that the proposed method successfully discriminates the intended human objects with 0.45 of recall and 0.80 of precision. © 2010 IEEE.

Many sensors around the world are consistently collecting the real-time real-world data. The data streams captured by these sensors can give us an idea of what is going on in a specific area
however, it is not easy for humans to understand their spatial and temporal relationships by just looking at them independently. This paper proposes to construct Digital Diorama, a three-dimensional view where viewers can see at a glance how people are moving around the monitored space without violating their privacy, by integrating multiple data streams captured by stationary cameras and RFID readers in real time. Digital Diorama realizes such real-world visualization with the following features: 1) view control, 2) real-time camera image superimposition, and 3) privacy control. We have demonstrated that Digital Diorama for a shopping center was able to present the current positions of persons and real-time camera images in approximately 1 frame per second. © Springer-Verlag Berlin Heidelberg 2010.

Although event detection in sports videos has been gaining some remarkable results, tools for exploring and exploiting these event clips on their own demands are far from user's expectation. In this paper, a novel method using spatio-temporal patterns to model visual information tailored to analyze event tactics in sports videos without tracking objects is introduced. The major contribution of the proposed method is to represent visual information by using a sequence of symbols. Each symbol represents a distribution of players in a certain period of time. By using these sequences, event tactics are detected, explained, and visualized to help users understand event tactics without watching whole clips. Moreover, users could query by an example or by a text to find all events sharing the same tactic. Thorough testing with over 100 goal clips in soccer domain demonstrates the superiority of the proposed method in terms of precision recall ratios. ©2010 IEEE.

In this paper, we propose a method for estimating the user position where a user is holding a microphone in an indoor environment using digital watermarking for audio signals. The proposed method utilizes detection strengths, which are calculated while detecting spread-spectrum-based watermarks. Taking into account delays and attenuation of the watermarked signals emitted from multiple loudspeakers and other factors, we construct a model of detection strengths. The user position is estimated in real-time using the model. The experimental results indicate that the user positions are estimated with 1.3 m of root mean squared error on average for the case where the user is static. We demonstrate that the proposed method successfully estimates the user position even when the user moves. © 2010 IEEE.

Video summarization is defined as creating a video summary which includes only important scenes in the original video streams. In order to realize automatic video summarization, the significance of each scene needs to be determined. When targeted especially on broadcast sports videos, a play scene, which corresponds to a play, can be considered as a scene unit. The significance of every play scene can generally be determined based on the importance of the play in the game. Furthermore, the following two issues should be considered: 1) what is important depends on each user's preferences, and 2) the summaries should be tailored for media devices that each user has. Considering the above issues, this paper proposes a unified framework for user and device adaptation in summarizing broadcast sports videos. The proposed framework summarizes sports videos by selecting play scenes based on not only the importance of each play itself but also the users' preferences by using the metadata, which describes the semantic content of videos with keywords, and user profiles, which describe users' preference degrees for the keywords. The selected scenes are then presented in a proper way using various types of media such as video, image, or text according to device profiles which describe the device type. We experimentally verified the effectiveness of user adaptation by examining how the generated summaries are changed by different preference degrees and by comparing our results with/without using user profiles. The validity of device adaptation is also evaluated by conducting questionnaires using PCs and mobile phones as the media devices.

This paper proposes a new linguistic-perceptual event model tailoring to spatio-temporal event detection and conceptual-visual personalized retrieval of sports video sequences. The major contributions of the proposed model are hierarchical structure, independence between linguistic and perceptual part, and ability of capturing temporal information of sports events. Thanks to these advanced contributions, it is very easy to upgrade model events from simple to complex levels either by self-studying from inner knowledge or by being taught from plug-in additional knowledge. Thus, the proposed model not only can work well in unwell structured environments but also is able to adapt itself to new domains without the need (or with a few modification) for external reprogramming, re-configuring and re-adjusting. Thorough experimental results demonstrate that events are modeled and detected with high accuracy and automation, and users' expectation of personalized retrieval is highly satisfied.

This paper presents a matrix-based algorithm for integrating inheritance relations of access rights for generating integrated access control policies which unify management of various access control systems. Inheritance relations of access rights are found in subject, resource, and action categories. Our algorithm first integrates inheritance relations in each category, and next, integrates inheritance relations of all categories. It is shown that these operations can be carried out by basic matrix operations. This enables us to implement the integration algorithm very easily.

Abstract As video surveillance systems are widely deployed, concerns continue to grow about invasion of privacy. We have built a privacy protected video surveillance system called PriSurv. Although PriSurv protects subject privacy using image processing, criteria of controlling the subject's visual information that is privacy-sensitive should be clarified. Visual information must be disclosed by considering the trade-off between privacy and security. The level of privacy-sensitive visual information that could be disclosed to a viewer is simply called disclosable privacy in this chapter. Disclosable privacy, which deeply involves the personal sense, is affected by many factors. A sense of privacy is individual, but in some cases it might have common factors. A sense of privacy is individual, but in some cases it might have common factors. In this chapter, we analyze what factors determine and affect disclosable privacy by applying statistical analysis to questionnaire-based experimental results. These results indicate that disclosable privacy is concerned with how much a subject has feeling of closeness to a viewer and expects the viewer's responsibility. They also show that disclosable privacy differs greatly by individuals. Reflecting the obtained findings in PriSurv's design, we adapt PriSurv to reflect a personal sense of privacy. © 2009 Springer London.

Trajectories of moving objects are known as one of the most important cues for understanding semantics in video data. Although there are a lot of significant researches dealing with trajectory analysis tailored to indexing and retrieval, several problems still remain. One of them is a trade-off between whole trajectory- and sub-trajectories- based methods. The former problem is that representing a trajectory as a whole is not appropriate for detecting similar patterns of the trajectory. In contrast, the latter is that even though some key portion of two trajectories share similar patterns, the whole trajectories may be totally different. Therefore, this paper proposes a novel method to optimize such trade-off. By representing a trajectory as a combination of sequence of "word" - each word's character represents one distinct feature extracted from sub-trajectories (i.e. segments), and a topological graph of trajectory's segments, the proposed method is shift and scale invariant, can handle occlusion and distortion, and can discover similar patterns among trajectories. Thorough comparisons with well-known methods demonstrate the superiority of the proposed method in terms of precision recall ratios. Copyright 2009 ACM.

This paper analyzes the performance of 3-Mode Net (3MN), which is a new anonymous communication system proposed in [1]. In particular, we give the probability distributions of the number of relay nodes as well as the number of encryption required for communications. The expectations and variances of these two numbers are also given. These results enable us to grasp the influence of the probabilities of mode selections in 3MN. Numerical examples are also presented to illustrate these results.

When a scene is recorded with a camcorder, there may be disruptions such as object interference or battery exhaustion resulting in missing segments in the recorded video. A temporal video completion method that inserts a video segment captured by another camera in the place of such a faulty segment is proposed. In the proposed method, a user initially uses an interface to select both faulty and preferred segments from videos captured by two different cameras. To minimize the visual difference at the transition points between the two videos, the most similar frames to those previous to and following the faulty segment are detected around both ends of the preferred segment as the start and the end frames of the inserted segment. Note that, in order to prevent artifacts as much as possible, the length of the inserted segment can be different from the length of the faulty segment. Visually plausible video completion is then achieved by smoothing the motion and color gaps between the transition frames and completing the missing regions by using spatio-temporal features of the video. Experiments were conducted to evaluate the visual plausibility of the completed videos by changing the distance between the camera and the recorded target or between two cameras.

In this paper, we propose a fast and effective LSB steganalysis for detecting the existence of hidden message and estimating hidden message length when the embedding is performed using both of two distinct embedding paradigms in one or more than one LSB. The method is based on the analysis of image statistics and quadratic equation. Compared with weighted stego-image based method, Ker's method and Luo et.al.'s method, the detection accuracy is high. Experimental results and theoretical verification show that the proposed method is an effective method of LSB steganalysis.

This paper presents a novel scheme of a recoverable image processing for privacy protection in real-time video surveillance system. The privacy information is embedded into the video using information hiding. Thus, the original privacy information can be recoverable with secrete key if necessary. In the proposed system, the privacy information is defined as information of objects that consist of detailed data to recover the original image of objects. The scheme is based on discrete wavelet transform (DWT) which is used for generating privacy-protected low resolution image, as well as the high resolution data including privacy information. An amplitude modulo modulation based information hiding scheme is used to hide the privacy information. Experimental results have shown that the proposed system can reduce the amount of the privacy information significantly, and allows the privacy information to be revealed after being embedded in real time.

The demand for information services considering personal preferences is increasing. In this paper, we propose a system for automatically acquiring personal preferences from TV viewer's behaviors. Our system firstly extracts intervals of interest and estimates the interest degree for each extracted interval based on the temporal patterns in facial changes by Hidden Markov Models (HMMs). Then, the viewer profile is created by associating the interest degrees with the content information described in the metadata of the watched program. Experimental results have shown that the proposed methods are able to correctly estimate interest degrees for extracted intervals with a precision rate of 73.1% and a recall rate of 68.8%, and that the created viewer profiles are comparable to the actual preferences of each viewer.

In this paper, we propose an improved steganalysis algorithm to detect spatial domain least significant bit (LSB) matching steganography, which is much harder than the detection of LSB replacement. Firstly we propose run length based features, which are sensitive to LSB matching in spatial domain of images. At the same time, we extend Ker's features, the statistical moments, to higher orders, which are sensitive to LSB matching in transform domain. Then the improved method is constructed based on these features. Experimental results on two datasets demonstrate that this method has superior results compared with other recently proposed algorithms, and shows that the proposed method is efficient to detect the LSB matching steganography on compressed or uncompressed images.

In this paper, we propose a steganalysis algorithm to detect spatial domain least significant bit (LSB) matching steganography, which is much harder than the detection of LSB replacement. We use the fusion of histogram of run length and histogram characteristic function to detect the LSB Matching. Experimental results on two datasets demonstrate that this method has superior results compared with other recently proposed algorithms, and shows that the proposed method is efficient to detect the LSB matching steganography on compressed or uncompressed images.

Event detection is one of the essential tasks by which the performance of sports video content analysis and access becomes more efficient and effective. Among internal information which are extracted from inside raw videos, the temporal information is critical to convey event meaning. In this paper, the new method for adaptively detecting event based on Allen temporal algebra and external information support is presented. The temporal information is captured by presenting events as the temporal sequences using a lexicon of non-ambiguous temporal patterns. These sequences are then exploited to mine undiscovered sequences with external text information supports by using class associate rules mining technique. By modeling each pattern with "linguistic part" and "perceptual part" those work independently and connect together via "transformer", it is easy to deploy this method to any new domain (e.g baseball, basketball, tennis, etc.) with a few changes in "perceptual part" and "transformer". Thus the proposed method not only can work well in unwell structured environments but also can be able to adapt itself to new domains without the need (or with a few modification) for external re-programming, re-configuring and re-adjusting. Experimental results that are carried on more than 30 hours of soccer video corpus captured at different broadcasters and conditions as well as compared with well-known related methods, demonstrated the efficiency, effectiveness, and robustness of the proposed method in both offline and online processes. Copyright 2008 ACM.

The paper proposes a method for LSB steganalysis of images, where the secret message is embedded in a given number L of the least significant bits. The proposed estimation method is based on weighted stego-Image and no assumption of cover images is required. The method can detect the presence of stego message using LSB steganography from the case L = 1 to arbitrary L>1. The estimation formula is clean and computation complexity is low. To evaluate the proposed steganalytic method, two experiments of detection and estimation are performed. It is shown that the accuracy of detecting the existence of secret messages in images and of estimating the embedding ratio of secret messages is relatively high.

Automatic prosody labeling is the task of automatically annotating prosodic labels such as syllable stresses or break indices into speech corpora. Prosody-labeled corpora are important for speech synthesis and automatic speech understanding. However, the subtleness of physical features makes accurate labeling difficult. Since errors in the prosodic labels can lead to incorrect prosody estimation and unnatural synthetic sound, the accuracy of the labels is a key factor for text-to-speech (TTS) systems. In particular, mora accent labels relevant to pitch are very important for Japanese, since Japanese is a pitch-accent language and Japanese people have a particularly keen sense of pitch accents. However, the determination of the mora accents of Japanese is a more difficult task than English stress detection in a way. This is because the context of words changes the mora accents within the word, which is different from English stress where the stress is normally put at the lexical primary stress of a word. In this paper, we propose a method that can accurately determine the prosodic labels of Japanese using both acoustic and linguistic models. A speaker-independent linguistic model provides mora-level knowledge about the possible correct accentuations in Japanese, and contributes to reduction of the required size of the speaker-dependent speech corpus for training the other stochastic models. Our experiments show the effectiveness of the combination of models.

This paper is concerned with an audio-based method for estimating speaker directions in meeting environment. It is well-known that cross-power spectrum phase (CSP) analysis is a very powerful tool for localizing sound sources. However, when we adopt the CSP-based method together with a circular microphone array system to estimate the speaker directions in 360-degree range (e.g. round-table discussions), the method fails to estimate the directions due to the existence of imaginary peaks of CSP coefficients. In order to circumvent the above problem, we propose a method to suppress the imaginary peaks, which uses a circular-array version of the method proposed by Nishiura and appropriate scaling around the imaginary peaks. Experimental results are also shown to demonstrate the effectiveness of the proposed method.

In this paper, we propose a novel application of digital water-marking, determination of recording locations. This application enables us to determine the seat location in an auditorium, where a recording was made. Precisely measured synchronization positions of the spread-spectrum, watermarks are used for the determination. To avoid use of mismeasured synchronization positions, the algorithm, discards synchronization positions with the corresponding normalized correlation values below a threshold. The experiments with our implementation resulted in accurate determinations
almost all. of the locations can be determined within the error of 0.5 m. These experimental results successfully show the potential applicability of our application. © 2007 IEEE.

In this paper, we propose a maximum-likelihood method for estimating a recording position inside several loudspeakers as a brand-new application of digital audio watermarking, which could be useful for protecting from illegal recordings of movies. An error model of times of arrival (TOAs) is utilized to estimate the recording position. The error model is constructed based on watermarking strengths, which are calculated by a watermark detection algorithm. This enables us to estimate the position if some of the TOAs are accurately measured. Experimental results indicate that the algorithm can estimate the recording position with the mean value of errors of 0.81 m.

This paper presents a new approach utilizing the sensor's pattern noise and linear prediction synthesis filter for steganography. The pattern noise is extracted from the images using denoising filter (for example wavelet-based filter). Then the approach introduces the linear prediction synthesis filter, whose parameters are derived from the extracted noise. After being filtered by such a filter, the secret message can be embedded by adapting the characteristics of the sensor's pattern noise. As a result, the embedding process violate little of the natural image statistics, and hence the detectability of steganalytic method is noticeably decreased. The experimental results prove the effectiveness of the new approach. © 2007 IEEE.

This paper proposes a detailed framework of privacy preserving techniques in real-time video surveillance systems. In the proposed system, the protected video data can be released in such a way that the identity of any individual contained in video cannot be recognized while the surveillance data remains practically useful, and if the original privacy information is demanded, it can be recoverable with a secrete key. The proposed system attempts to hide a face (real face, privacy information) in a face (new generated face for anonymity). To deal with the huge payload problem of privacy information hiding, an Active Appearance Model (AAM) based privacy information extraction and recovering is proposed in our system. A quantized index modulation based data hiding scheme is used to hide the privacy information. Experimental results have shown that the proposed system can embed the privacy information into video without affecting its visual quality and keep its practical usefulness, at the same time, allows the privacy information to be revealed in a secure and reliable way.

Several methods for summarizing sports videos by selecting only important scenes based upon their semantic content have been proposed so far. However, there remain two problems: 1)what is important depends on each user's preferences, and 2)the summaries should be tailored for media devices that each user has. To solve these problems, we discuss user and device adaptation for sports video summarization. The proposed framework dynamically adapts the video content to fit user's preferences using user profiles which describe the preference degrees for keywords. The video content is then presented through proper media such as image or text according to the confinement of media devices. For sports videos, the framework is tested using PCs and mobile phones as the media devices.

We propose a video completion method for automatically removing unnecessary objects from video. Using this method, a user selects an object from a frame on our interface, and the corresponding objects are extracted from all frames of the video. Based on estimated camera/object motions, visually plausible video completion with spatio-temporal features is achieved by repeatedly using the temporal correlation between adjacent frames and the spatial correlation between adjacent pixels in a frame. Experiments were conducted that targeted and removed video captions to evaluate processing time and visual plausibility.

With the increasing demand for greater security, video surveillance technologies have recently received a lot of attention. As video surveillance cameras become ubiquitous, there are growing concerns over the cost of monitoring these systems and the possible invasion of privacy. In this paper, we discuss a system architecture of privacy-preserving video surveillance for a community that achieves a good balance between security and privacy. We call the designed system 'PriSurv'. A subset of PriSurv is implemented, and privacy protected image processing modules are installed. The privacy preserving image processing is called visual abstraction. We conducted an experiment using five abstracted images and a questionnaire to evaluate PriSurv. Analyzing the results of the experiment using factor analysis, we obtained seven factors. With a hierarchical cluster analysis, we divided the subjects into three clusters. A significant difference in the choice of images among the clusters was found using the Pearson -squared test. From this analysis, we show that the relationships between subjects and monitors affect the subjects' sense of privacy
furthermore, the subjects' sense of privacy depends on the individual person. The results of analysis supports PriSurv's design principle, which can adapt the personal sense of privacy. Copyright 2006 ACM.

The importance of personalized information services has been increasing. Description of personal preferences needs to be prepared beforehand to realize such services. We propose a system for automatically acquiring personal preferences from TV viewer's behaviors. Considering "when" a viewer is watching TV is highly related to the viewer's preferences, we focus on estimating the time interval during which a pre-registered viewer is watching TV. In this paper, we firstly describe the outline of the personal preference acquisition system, and address a method for estimating the TV viewing intervals based on the appearance of frontal faces. Experiments resulted in a precision rate of 97.1% and a recall rate of 70.6% on average for TV viewing interval estimation. © 2006 IEEE.

14th International World Wide Web Conference (WWW2005), Proc. WWW2005 Special Interest Tracks and Posters

This paper presents a model to represent a broadcasted sports video in a semantic way and proposes a method of automatically generating semantic descriptions of significant scenes. Representation of a video should clarify the semantic content of the video as accurately as possible. Our model structurizes the video and specifies suitable semantic descriptions for video segments paying attention to the structure of both a sports game and a sports TV program. As the elements of these semantic descriptions, the proposed method tries to obtain the information about the plays and their related players from the closed-caption stream by searching key phrases. Finding the corresponding segments of the video by means of template matching for the image stream attaches these textual descriptions to the proper portion of the video. In this paper, we discuss some experimental results of our method and the potentiality for integrating these results into the standardized MPEG-7 description tools.

This paper proposes an automatic American football video parsing method based on transition rules of an American football game. Combining the results of live scene extraction and superimposed text detection based on image features enables us to segment the video into play units of a game. Temporally associating the segmented play units with the detected superimposed texts and the closed-caption text attaches possible semantic content information to the play units. Finallly, selecting only the play units which comform to transition rules of the sports game from the obtained play unit sequence, while discarding or complementing unnecessary or insufficient play units and attached semantic content information, realizes the semantic video parsing. © 2005 IEEE.

Structuring video data is necessary for its effective retrieval and summarization. In particular, collecting similar scenes from semantic aspects highly contributes to the structuring. In this paper, we propose a method of clustering the scenes with relevance feedback, which may be able to bridge the gap between the video data and its semantics. First, spatio-temporal video segments of a fixed length are clustered according to image features of each segment. Then, a user performs feedback to the results of clustering, whether each segment is relevant to the cluster it belongs to. The clustering accuracy can be improved through the interaction based on the feedback information. For diverse kinds of video streams, we investigated how the feedback should be given and demonstrated the effectiveness of the interactive clustering. © 2005 IEEE.

This paper proposes the chart-style video playback interface Play Watch
it displays a chart of semantic indices for locating video scenes. The main features of Play Watch are: 1) the user understands the distribution of the scenes because PlayWatch shows the indices in order. 2) The user can access a desired scene directly through the indices since they also act as link buttons. This paper also describes the evaluation of PlayWatch. Experiments on scene searching show that PlayWatch is effective in accessing precisely indexed scenes. ©2005 IEEE.

Video summarization is defined as creating a shorter video clip or a video poster which includes only the important scenes in the original video streams. In this paper, we propose two methods of generating a summary of arbitrary length for large sports video archives. One is to create a concise video clip by temporally compressing the amount of the video data. The other is to provide a video poster by spatially presenting the image keyframes which together represent the whole video content. Our methods deal with the metadata which has semantic descriptions of video content. Summaries are created according to the significance of each video segment which is normalized in order to handle large sports video archives. We experimentally verified the effectiveness of our methods by comparing the results with man-made video summaries. © 2005 IEEE.

Video abstraction is defined as creating shorter video clips or video posters from an original video stream. In this paper, we propose a method of generating a personalized abstract of broadcasted American football video. We first detect significant events in the video stream by matching textual overlays appearing in an image frame with the descriptions of gamestats in which highlights of the game are described. Then, we select highlight shots which should be included in the video abstract from those detected events reflecting on their significance degree and personal preferences, and generate a video clip by connecting the shots augmented with related audio and text. An hour-length video can be compressed into a minute-length personalized abstract. We experimentally verified the effectiveness of this method by comparing man-made video abstracts.

This paper presents a stability criterion for 2-D discrete systems described by the Fornasini-Marchesini second model. The method presented in this paper is based on linear matrix inequalities (LMI), and hence, it is computationally tractable. In deriving the method, finite-order Fourier series approximation of the solution for frequency-dependent LMI (FDLMI), and the properties of quadratic form representation of finite-order Fourier series play key roles. From the view point of the proposed method, the existing LMI-based condition can be regarded as the one which is obtained by Fourier series approximation of order zero, and thus, it is expected that the proposed method leads to less conservative results. This is illustrated by a numerical example.

This paper presents a method of retrieving similar scenes from video streams with sign sequence matching. The visual and auditory streams are partitioned into fixed-length video/audio packets. Based on feature vectors extracted from these packets, sign sequences are formed. The sign sequence can be viewed as abstraction of video and audio features. DP Matching between the target and query sign sequences allows us to find scenes similar to the query in the video stream. For the purpose of efficient processing, packets, histogram based features, and sign sequences, which are the key ideas in this method, are introduced. The preliminary experimental results show that this method is promising for quick retrieval for similar scenes.

A moving object that has many complex moving parts is very hard to detect and its motion is not easy to estimate. In this paper, we present a new technique for motion estimation and detection of moving complex objects by analyzing the resampled motions of the parts of objects. The Kalman filter is used to track all resampled movements and the tracked routes are classified into groups that share the same fundamental movements. Our simulation show that recall of motion estimation and detection is approximately 0.8, while the computation drops exponentially.

Personalization is one of the most important mechanisms to make multimedia systems easy to use. In video applications, its embodiment is to tailor video contents for a particular viewer. For this purpose, we are now developing a system of retrieving and browsing video segments, called video portal with personalization (VIPP). VTPP is characterized by 1) supporting the viewer's access to video contents and making a summarized video clip by taking his/her preference into account, and 2) acquiring the viewer's profile from his/her operations automatically. In this paper, we discuss the effect of personalization on retrieval and summarization of sports videos on VIPP.

Recently, the necessity for a system which supports an access to videos has been increasing. In addition, it is strongly desired to take a user's preference into account to realize a more flexible system. For this background, we propose a system which can automatically acquire the user's preference by monitoring the actions by the user, as well as which can provide the functions such as efficient retrieval and browsing of video segments, by taking advantage of their metadata. Furthermore we experimentally show its usefulness.

Recently, video abstraction has become a demanding application in multimedia computing. It is defined as creating shorter video clips or video posters from the original video stream. In this paper, we present a basic approach towards abstracting sports video by highlights, dealing with American football games, Using event based indexing, we create an abstracted video clip automatically. To select the appropriate highlights of the game, an impact factor reflecting on the importance of the event is newly introduced. It was possible to make an about 5-minute clip from the 3-hour original video.

In this paper, we propose an event network, which is a structured representation oriented for the contents of continuous media, as well as present two methods of detecting events as the first step to construct the network. We here deal with sports TV programs, considering American football as a case study. The first method is simple intermodal collaboration: linking between visual and linguistic (closed caption) streams. Using domain knowledge about state transitions of football games, the second method attempts to extract specific visual objects including the information about contents. The experimental results indicate that the both methods are effective for event detection.

We propose a non-interactive method for solving the contradictions caused by exceptions to ordinary rules. It is realized by 1) detecting the ordinary rules which have the instances in their bodies, and 2) converting them into the default; rules. To reduce the default rules which need much time in reasoning, we convert the minimal sets of ordinary rules to solve contradictions. Since the proposed method is executed without interaction with human, it contributes to automatic rule-base maintenance.

In this paper, we propose a method of generating a sketch map drawing and its instructions in order to support the route understanding of a human. Both are given from a structured data, called the road network, which is a graph augmented by the attributes of roads and crossings. In generating a sketch map drawing, the pictorial information about the shape and the direction of roads is modified. The sketch map drawing can be generated at any simplification level by controlling a parameter called simpleness. The instructions corresponding to the sketch map are produced by assigning appropriate terms into sentence templates. We verified that favorable results are obtained by this method. © 1996 IEEE.

This paper discusses the role of knowledge in document image understanding from the viewpoints of representation, utilization and acquisition. For the representation of knowledge, we propose two models, a layout model and a content model, which represent knowledge about the layout structure and content of a document, respectively. For the utilization of knowledge, we implement layout analysis and content analysis which utilize a layout model and a content model, respectively. The strategy of hypothesis generation and verification is introduced in order to integrate these two kinds of analysis. For the acquisition of knowledge, we propose a method of incremental acquisition of a layout model from a stream of example documents. From the experimental results of document image understanding and knowledge acquisition using 50 samples of visiting cards, we verified the effectiveness of the proposed method.

The shape analysis of a binary picture is of great importance for pictorial pattern recognition. In this paper, we propose a useful geometric feature parameter called curvedness. Curvedness represents which lines are dominant in a binary picture, straight or curved. Our algorithm for measuring curvedness is based on the relationship between the distance to a boundary point on each black point along each quantized direction and the mean width of a binary picture. We investigate the fundamental property of curvedness. The experimental results for Japanese Hiragana and Kanji characters show the validity of curvedness. © 1987.