Faculty Profiles - Noboru Babaguchi

Papers - Noboru Babaguchi

Division display >> ／ All the affair displays 1 - 171 of about 171

Paladin: Understanding Video Intentions in Political Advertisement Videos Reviewed

1. Hong Liu, Yuta Nakashima, Noboru Babaguchi

Proc. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2025.2

Enhancing Fake News Detection in Social Media via Label Propagation on Cross-modal Tweet Graph Reviewed International coauthorship International journal

Wanqing Zhao,Yuta Nakashima, Haiyuan Chen, Noboru Babaguchi

Proceedings of the 31st ACM International Conference on Multimedia 2400 - 2408 2023.10

Fake news detection in social media has become increasingly important due to the rapid proliferation of personal media channels and the consequential dissemination of misleading information. Our method constructs a cross-modal tweet graph using CLIP, which encodes images and text into a unified space, allowing us to extract potential connections based on similarities in text and images. We then design a Feature Contextualization Network with Label Propagation (FCN-LP) to model the interaction among tweets as well as positive or negative correlations between predicted labels of connected tweets. The propagated labels from the graph are weighted and aggregated for the final detection. To enhance the model’s generalization ability to unseen events, we
introduce a domain generalization loss that ensures consistent features between tweets on seen and unseen events. We use three publicly available fake news datasets, Twitter, PHEME, and Weibo, for evaluation. Our method consistently improves the performance
over the state-of-the-art methods on all benchmark datasets and effectively demonstrates its aptitude for generalizing fake news detection in social media.

An Experimental Consideration on Gait Spoofing Reviewed

Y. Hirose, K. Nakamura, N. Nitta, and N. Babaguchi

Proc International Conference on Computer Vision Theory and Applications (VISAPP2023) (8) 2023.2

Social IoT Approach to Cyber Defense of a Deep-Learning-Based Recognition System in Front of Media Clones Generated by Model Inversion Attack Reviewed International coauthorship

Mahdi Khosravy, Kazuaki Nakamura, Naoko Nitta, Nilanjan Dey, Rubén González Crespo, Enrique Herrera-Viedma, and Noboru Babaguchi

IEEE Trans Systems, Man, and Cybernetics: Systems (11) 2022.11

Anonymization of Human Gait in Video Based on Silhouette Deformation and Texture Transfer Reviewed International journal

Yuki Hirose, Kazuaki Nakamura, Naoko Nitta, and Noboru Babaguchi

IEEE Trans. Information Forensics and Security 17 3375 - 3390 2022.9

This paper proposes a method for anonymizing the appearance of walking people, namely human gait, in video. In the proposed method, we first crop human regions from all frames in an input video and binarize them to get their silhouettes. Next, we slightly deform the silhouettes from the aspects of static body shape and dynamic walking rhythm so that the person in the input video cannot be correctly identified by gait recognition techniques. After that, the textures of the original human regions are transferred onto the deformed silhouettes. Finally, the anonymized human regions with the transferred textures are filled back into the input video. In the results of our experiments, we successfully degraded the accuracy of CNN-based gait recognition systems from 100% to 1.57% in the lowest case without yielding serious distortion in the appearance of the human regions, which demonstrated the effectiveness of the proposed method.

Effective De-identification Generative Adversarial Network for Face Anonymization Reviewed

Z. Kuang, H. Liu, J. Yu, A. Tian, L. Wang, J. Fan, N. Babaguchi

Proc. of 29th ACM International Conference on Multimedia 3182 - 3192 2021.10

Unnoticeable synthetic face replacement for image privacy protection Reviewed

Z. Kuang, Z. Guo, J. Fang, J. Yu, N. Babaguchi, J. Fan

Neurocomputing 457 322 - 333 2021.10

Model Inversion Attack: Analysis under Gray-box Scenario on Deep Learning based Face Recognition System Reviewed

M. Khosravy, K. Nakamura, Y. Hirose, N. Nitta, N. Babaguchi

KSII Transactions on Internet and Information Systems 15 ( 3 ) 1100 - 1118 2021.3

Generation and Detection of Media Clones Invited Reviewed

I. Echizen, N. Babaguchi, J. Yamagishi, N. Nitta, Y. Nakashima, K. Nakamura, K. Kono, F. Fang, S. Myojin, Z. Kuang, H. H. Nguyen, N-D. T. Tieu

IEICE Transactions on Information and Systems E104-D ( 1 ) 12 - 23 2021.1

Preventing Fake Information Generation Against Media Clone Attacks Invited Reviewed

N. Babaguchi, I. Echizen, J. Yamagishi, N. Nitta, Y. Nakashima, K. Nakamura, K. Kono, F. Fang, S. Myojin, Z. Kuang, H. H. Nguyen, N-D. T. Tieu

IEICE Transactions on Information and Systems E104-D ( 1 ) 2 - 11 2021.1

Semi-Supervised Outdoor Image Generation Conditioned on Weather Signals Reviewed

S. Kawakami, K. Okada, N. Nitta, K. Nakamura, N. Babaguchi

Proc. of International Conference on Pattern Recognition (ICPR2021) 4268 - 4275 2021.1

Deep Face Recognizer Privacy Attack: Model Inversion Initialization by a Deep Generative Adversarial Data Space Discriminator Reviewed

M. Khosravy, K. Nakamura, N. Nitta, N. Babaguchi

Proc. of a-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2020) 1400 - 1405 2020.12

Detection of Cloned Recognizers: A Defending Method against Recognizer Cloning Attack Reviewed

Y. Mori, K. Nakamura, N. Nitta, N. Babaguchi

Proc. of a-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2020) 1375 - 1380 2020.12

Update of Graduate School of Engineering Invited

Noboru Babaguchi

72 ( 3 ) 1 - 2 2020.8

Constructing Geospatial Concept Graphs from Tagged Images for Geo-Aware Fine-Grained Image Recognition Reviewed

N. Nitta, K. Nakamura, N. Babaguchi

ISPRS International Journal of Geo-Information 9 ( 6 ) Article No. 354 - 23 pages 2020.5

While visual appearances play a main role in recognizing the concepts captured in images, additional information can provide complementary information for fine-grained image recognition, where concepts with similar visual appearances such as species of birds need to be distinguished. Especially for recognizing geospatial concepts, which are observed only at specific places, geographical locations of the images can improve the recognition accuracy. However, such geo-aware fine-grained image recognition requires prior information about the visual and geospatial features of each concept or the training data composed of high-quality images for each concept associated with correct geographical locations. By using a large number of images photographed in various places and described with textual tags which can be collected from image sharing services such as Flickr, this paper proposes a method for constructing a geospatial concept graph which contains the necessary prior information for realizing the geo-aware fine-grained image recognition, such as a set of visually recognizable fine-grained geospatial concepts, their visual and geospatial features, and the coarse-grained representative visual concepts whose visual features can be transferred to several fine-grained geospatial concepts. Leveraging the information from the images captured by many people can automatically extract diverse types of geospatial concepts with proper features for realizing efficient and effective geo-aware fine-grained image recognition.

Probabilistic Stone’s Blind Source Separation with application to channel estimation and multi-node identification in MIMO IoT green communication and multimedia systems Reviewed

M. Khosravy, N. Gupta, N. Patel, N. Dey, N. Nitta, N. Babaguchi

Computer Communications, Elsevier 157 423 - 433 2020.5

A logical consideration on fraudulent email communication Reviewed

S. Myojin, N. Babaguchi

ARTIFICIAL LIFE AND ROBOTICS 25 ( 3 ) 7pages 2020.3

One of the most serious problems in modern society is that internet users are deceived by various fake information. In this paper, we analyse a scenario based on an actual incident of fraud called business email compromise (BEC). We suppose that each email, in the incident, step-wisely changes user's thinking and makes him or her believe the emails. If fraud has such a step-by-step mechanism, it allows us to consider counter measures for dissuading a user from decision-making on the way. We discuss features and factors of the incident based on formulations of Channel theory. Our analysis revealed that each email message influenced user's decision-making by what kind of logical trap. It can be fundamental knowledge that is capable of warning the user with predicting logical traps. This paper contributes to providing a novel viewpoint to develop systems for detecting deception of BEC.

Speech-driven Face Reenactment for a Video Sequence Reviewed

Y. Nakashima, T. Yasui, L. Nguyen, N. Babaguchi

ITE Transactions on Media Technology and Applications 8 ( 1 ) 60 - 68 2020.1

Anonymization of Gait Silhouette Video by Perturbing Its Phase and Shape Components Reviewed

Y. Hirose, K. Nakamura, N. Nitta, N. Babaguchi

Proc.of Asia-Pacific Signal and Processing Association Annual Summit and Conference (APSIPA ASC 2019) 2019.11

Discrimination between Handwritten and Computer-Generated Texts using a Distribution of Patch-Wise Font Features Reviewed

N. Hamasaki, K. Nakamura, N. Nitta, N. Babaguchi

Proc. of Asia-Pacific Signal and Processing Association Annual Summit and Conference (APSIPA ASC 2019) 2019.11

Generating Spoofing Tweets considering Points of Interest of Target User Reviewed

J. Lim, N. Nitta, K. Nakamura, N. Babaguchi

Proc. of Asia-Pacific Signal and Processing Association Annual Summit and Conference (APSIPA ASC 2019) 2019.11

Discrimination between Genuine and Cloned Gait Silhouette Videos via Autoencoder-based Training Data Generation Reviewed

Y. Hirose, K. Nakamura, N. Nitta, N. Babaguchi

IEICE Transactions on Information and Systems E102.D ( 12 ) 2535 - 2546 2019.9

Constructing Geographic Dictionary from Streaming Geotagged Tweets Reviewed

J. Lim, N. Nitta, K. Nakamura, N. Babaguchi

ISPRS International Journal on Geo-Information 8 ( 5 ) Article No. 216 - 24 pages 2019.5

Encryption-Free Framework of Privacy-Preserving Image Recognition for Photo-Based Information Services Reviewed

K. Nakamura, N. Nitta, N. Babaguchi

IEEE Transactions on Information Forensics and Security 14 ( 5 ) 1264 - 1279 2019.5

A logical consideration on deceived person's thinking Reviewed

S. Myojin, N. Babaguchi

ARTIFICIAL LIFE AND ROBOTICS 24 ( 1 ) 114 - 118 2019.3

The problem that old people are sometimes deceived by means of remittance fraud has become a great concern in our society. In this paper, we consider deceived person's thinking using Barwise-Seligman's framework, which is a logic for representing distributed systems among people or artifacts. The framework has been used to consider the conversation with comical misconception. We think that it is similar to remittance fraud because in telephone conversation, a criminal of the fraud uses victim's misconception so that the criminal can impersonate somebody. In this paper, we consider a logical system to describe the communication of a typical remittance fraud, and discuss the representative ability of the formulas to express the situation where a person is deceived or not deceived. The formulas have indicated that a person may be deceived as a result of handling exceptions in terms of logic. This paper contributes to providing a novel viewpoint to consider why people are deceived.

Semiotic and Logical Approach for Analysis of Augmented World Reviewed

S. Myojin, N. Babaguchi

Proc. of 12th Asia Pacific Workshop on Mixed and Augmented Reality (APMAR2019) 2019.3

Modeling deceptive communication based on information flow Reviewed

S. Myojin, N. Babaguchi

Proc. of International Symposium on Artificial Life and Robotics, AROB 24th 2019 2019.1

A Comic-Style Chat System with Japanese Expression Techniques for More Expressive Communication. Reviewed

J. Itou, K. Matsumura, J. Munemori, N. Babaguchi

Collaboration Technologies and Social Computing - 25th International Conference(CRIWG/CollabTech) 172 - 187 2019

TRAINING-FREE METHOD FOR GENERATING MOTION VIDEO CLONES FROM A STILL IMAGE CONSIDERING SELF-OCCLUSION OF HUMAN BODY Reviewed

T. Tsutsumi, K. Nakamura, S. Myojin, N. Nitta, N. Babaguchi

2019 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP) 509 - 513 2019

In this paper, we propose a method for generating photorealistic video in which a person virtually performs a motion that is not performed in the real world. we refer to such video as motion video clones (MVCs). The proposed method requires only two kinds of source information as input data: a reference video, in which a person A performs some motion, and a target image, which includes the whole body of another person B. Using these data, our method generates a MVC in which the person A's motion is re-enacted by the person B's body. Since our method does not need 3D human body model nor any training phase, it is suitable to MVC-based entertainment systems. To handle the self-occlusion of the human body in the reference video, we employ a part-based approach. For each part such as the right arm and the left leg, we first extract its skeleton from the target image and move it so that the motion represented by the reference video is re-enacted. Next, we compute a 2D affine transform between the original and moved positions of the skeleton. This transform is used to map the texture of the target image onto each frame of a resultant MVC. Finally, we extend the part-wise affine transforms to pixel-wise ones by computing their linear combination for each pixel, whose combination weights are computed according to the geodesic distance between the pixel and the center of each part. This allows us to avoid unnatural appearance around the joints of the body parts. In our experiments, the proposed method generated much visually-natural MVCs than existing methods.

Passive Video Forgery Detection Considering Spatio-Temporal Consistency Reviewed

K. Kono, T. Yoshida, S. Ohshiro, N. Babaguchi

Proceedings of 14th International Conference on Information Assurance and Security 2018.12

Initial Consideration on Human Factor in Security Incident Reviewed

S. Myojin, N. Babaguchi

Proc. of The 13th International Workshop on Security 2018.9

Generating Handwritten Character Clones from an Incomplete Seed Character Set using Collaborative Filtering Reviewed

K. Nakamura, E. Miyazaki, N. Nitta, N. Babaguchi

68 - 73 2018.8

Extracting Related Real-World Observations from Microblog Reviewed

N. Nitta, M. Yoshitake, K. Nakamura, N. Babaguchi

vol.16-J Article No. 22 - 8 pages 2018.3

Formulating what a deceived person thinks Reviewed

S. Myojin, N. Babaguchi

23rd International Symposium on Artificial Life and Robotics, AROB 23rd 2018 2018.1

Development of a Stroll Support System Using Route Display on a Map and Photograph Sharing Service. Reviewed

J. Itou, T. Mori, J. Munemori, N. Babaguchi

Proc. of 10th International Conference on Collaboration Technologies (CollabTech2018) 48 - 55 2018

Constructing Geospatio-Temporal Concept Graphs from Tagged Images Reviewed

H. Honjo, N. Nitta, K. Nakamura, N. Babaguchi

Proceedings - 2017 IEEE 3rd International Conference on Multimedia Big Data, BigMM 2017 169 - 176 2017.6

This paper proposes a method for discovering the knowledge about geospatio-temporal concepts, which are strongly related to specific locations and times, from images tagged with latitude and longitude coordinates, time-stamps, and textual tags. The proposed method firstly extracts textural tags used either in specific locations or times as the geospatial or temporal concepts with their location or time information. Then, the semantic relations among the geospatio-temporal concepts are extracted based on the co-occurrence of the corresponding tags to be represented as a graph. Finally, visually discriminative geospatio-temporal concepts are linked to representative visual concepts. The usefulness of the discovered knowledge is evaluated based on the automatic tagging for the images tagged with latitude and longitude coordinates and time-stamps.

Extracting Real-World Observations from Microblog Reviewed

M. Yoshitake, Naoko Nitta, Kazuaki Nakamura, Noboru Babaguchi

Proceedings - 2017 IEEE 3rd International Conference on Multimedia Big Data, BigMM 2017 232 - 237 2017.6

Since a large number of users of various social networking services post what they observe around themselves, what is going on around the world can be known in real time by extracting such real-world observations. Especially, observations covering miscellaneous areas of interest are posted to Twitter as short text messages. Our goal is to extract such observations to better understand the current situations of the real world. Since the observations at specific locations are often described with words representing the observed locations or events, the locations or events all over the world can be discovered by finding such local words which are uniquely used at specific locations or times. Then, parts of their observations can be obtained by using the local words to define the semantics of each local word. Finally, the observations which do not contain the local words but are related to the locations or events can be extracted based on their semantic relevancy to each local word.

On-line Geospatial Term Extraction from Streaming Geotagged Tweets Reviewed

T. Kamimura, N. Nitta, K. Nakamura, N. Babaguchi

Proceedings - 2017 IEEE 3rd International Conference on Multimedia Big Data, BigMM 2017 322 - 329 2017.6

Recently, geotagged posts to social media such as Twitter have become major sources for geospatial information referenced to a set of geographic coordinates. Especially, many existing work collects geospatial terms which identify geographic locations by examining the spatial locality of the term usage patterns observed in the geotagged posts accumulated for a certain period of time. Although the spatial locality of the collected geospatial terms can only be temporary, such time variability difference among the geospatial terms have not been considered in the existing work. Thus, in order to separately collect the stationary and temporary geospatial terms with proper timing, we propose an on-line method for constructing a geographical dictionary containing the up-to-date geospatial terms and their locations by continuously examining the spatial locality of terms in streaming geotagged posts. The geospatial terms can be distinguished between stationary and temporary terms according to their usage patterns in the recent set of geotagged posts and their locations recorded in the geographical dictionary. Additionally, images representing each geospatial term can also be collected from the posts containing the corresponding term. The usefulness of the collected geospatial information is demonstrated in comparison to existing geographical dictionaries constructed by experts, crowdsourcing, and batch method.

Detection of groups in crowd considering their activity state Reviewed

K. Nakamura, T. Ono, N. Babaguchi

Proceedings - International Conference on Pattern Recognition 277 - 282 2017.4

In this paper, we focus on the problem of group detection in crowd, which is a task of partitioning a set of pedestrians in a scene into small subsets called groups based on their trajectories. Most of previous methods use only a single model for representing a relationship between trajectories of pedestrians who belong to the same group. However, such relationship would vary depending on the activity state (e.g. walking together, approaching, splitting, and so on) of the group. In this paper, we propose a novel group detection method which can cope with a variation of groups' activity state. The proposed method constructs different models for each activity state in order to appropriately evaluate the relationship of pedestrians' trajectories. In addition, our method regards groups' activity state as hidden variables and estimates their probability distributions, which is used for integrating the constructed models. The proposed method outperforms existing methods in the experiment on the public dataset.

A Framework of Privacy-Preserving Image Recognition for Image-Based Information Services Reviewed

K. Fujii, K. Nakamura, N. Nitta, N. Babaguchi

MULTIMEDIA MODELING (MMM 2017), PT I 10132 40 - 52 2017

Nowadays mobile devices such as smartphones are widely used all over the world. Moreover, the performance of image recognition has dramatically increased by deep learning technologies. From these backgrounds, we think that the following scenario of information services could be realized in the near future: users take a photo and send it to a server, who recognizes the location in the photo and returns the users some information about the recognized location. However, this kind of client-server-based image recognition can cause a privacy issue because image recognition results are sometimes privacy sensitive. To tackle the privacy issue, in this paper, we propose a novel framework for privacy-preserving image recognition in which the server cannot uniquely identify the recognition result but users can do so. An overview of the proposed framework is as follows: First users extract a visual feature from their taken photo and transform it so that the server cannot uniquely identify the recognition result. Then users send the transformed feature to the server, who returns a candidate set of recognition results to the users. Finally, the users compare the candidates and the original visual feature for obtaining the final result. Our experimental results demonstrate the effectiveness of the proposed framework.

Effect of Junk Images on Inter-concept Distance Measurement: Positive or Negative? Reviewed

Y. Nagasawa, K. Nakamura, N. Nitta, N. Babaguchi

MULTIMEDIA MODELING, MMM 2017, PT II 10133 173 - 184 2017

In this paper, we focus on the problem of inter-concept distance measurement (ICDM), which is a task of computing the distance between two concepts. ICDM is generally achieved by constructing a visual model of each concept and calculating the dissimilarity score between two visual models. The process of visual concept modeling often suffers from the problem of junk images, i.e., the images whose visual content is not related to the given text-tags. Similarly, it is naively expected that junk images also give a negative effect on the performance of ICDM. On the other hand, junk images might be related to its text-tags in a certain (non-visual) sense because the text-tags are given by not automated systems but humans. Hence, the following question arises: Is the effect of junk images on the performance of ICDM positive or negative? In this paper, we aim to answer this non-trivial question from experimental aspects using a unified framework for ICDM and junk image detection. Surprisingly, our experimental result indicates that junk images give a positive effect on the performance of ICDM.

Tag Chat: A Tag-Based Past Topics Recollection Support System. Reviewed

J. Itou, R. Tanaka, J. Munemori, N. Babaguchi

Proc. of 9th International Conference on Collaboration Technologies (CollabTech 2017) LNCS 10397 29 - 36 2017

Long-term people reidentification using anthropometric signature Reviewed

M. Hasan, N. Babaguchi

2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems, BTAS 2016 2016.12

Anthropometric biometrics are the most suitable features for long-term people reidentification. However, feature selection is still a major problem in the anthropometric biometrics literature. In this paper, we aim at improving people reidentification by enhancing feature selection. Based on a statistical analysis of body measurements on a large-scale dataset, a new anthropometric signature is introduced. The proposed signature describes both the size and shape of human body at specific anatomical landmarks. While size is measured by Euclidian distance between four skeleton joint pairs, shape is described by the surface distance along four circular body parts. A novel algorithm is proposed to automatically segment the circular body parts from the subject point cloud using body geometry, cylindrical fitting and soft clustering. The overall system is evaluated on two public datasets using CMC and nAUC metrics. Experimental results show the effectiveness of our method compared to state of the art.

Hierarchical detection of rectangles in images Reviewed

M. Hasan, M. Abdellatif, N. Babaguchi

First International Workshop on Pattern Recognition 7 pages 2016.7

Privacy Protection for Social Video via Background Estimation and CRF-Based Videographer's Intention Modeling Reviewed

Yuta Nakashima, Noboru Babaguchi, Jianping Fan

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E99D ( 4 ) 1221 - 1233 2016.4

The recent popularization of social network services (SNSs), such as YouTube, Dailymotion, and Facebook, enables people to easily publish their personal videos taken with mobile cameras. However, at the same time, such popularity has raised a new problem: video privacy. In such social videos, the privacy of people, i.e., their appearances, must be protected, but naively obscuring all people might spoil the video content. To address this problem, we focus on videographers' capture intentions. In a social video, some persons are usually essential for the video content. They are intentionally captured by the videographers, called intentionally captured persons (ICPs), and the others are accidentally framed-in (non-ICPs). Videos containing the appearances of the non-ICPs might violate their privacy. In this paper, we developed a system called BEPS, which adopts a novel conditional random field (CRF)-based method for ICP detection, as well as a novel approach to obscure non-ICPs and preserve ICPs using background estimation. BEPS reduces the burden of manually obscuring the appearances of the non-ICPs before uploading the video to SNSs. Compared with conventional systems, the following are the main advantages of BEPS: (i) it maintains the video content, and (ii) it is immune to the failure of person detection; false positives in person detection do not violate privacy. Our experimental results successfully validated these two advantages.

Evaluating Protection Capability for Visual Privacy Information Reviewed

Y. Nakashima, T. Ikeno, N. Babaguchi

IEEE Security and Privacy 14 ( 1 ) 55 - 61 2016.1

Real-World Observation Extraction from Microblog based on Word Associative Relations Reviewed

Masato Yoshitake, Naoko Nitta, Noboru Babaguchi

2016 IEEE SECOND INTERNATIONAL CONFERENCE ON MULTIMEDIA BIG DATA (BIGMM) 450 - 455 2016

A large number of real-world observations by social sensors all over the world can be obtained from various social networking services. Especially, observations covering miscellaneous areas of interest are posted to Twitter as short text messages. Our goal is to extract a wide range of observations related to the target of interest specified by the user from Twitter regardless of their popularity. Assuming that the related observations are likely to contain words people often associate with each other, the associative relations among words are learned from the past messages. When a user gives a keyword representing his/her current interest, recent related observations are extracted based on their word frequency distributions in terms of the associative relations to the keyword.

Facial expression preserving privacy protection using image melding Reviewed

Yuta Nakashima, Tatsuya Koyama, Naokazu Yokoya, Noboru Babaguchi

Proceedings - IEEE International Conference on Multimedia and Expo 2015- 1 - 6 2015.8

An enormous number of images are currently shared through social networking services such as Facebook. These images usually contain appearance of people and may violate the people's privacy if they are published without permission from each person. To remedy this privacy concern, visual privacy protection, such as blurring, is applied to facial regions of people without permission. However, in addition to image quality degradation, this may spoil the context of the image: If some people are filtered while the others are not, missing facial expression makes comprehension of the image difficult. This paper proposes an image melding-based method that modifies facial regions in a visually unintrusive way with preserving facial expression. Our experimental results demonstrated that the proposed method can retain facial expression while protecting privacy.

Owner authentication for mobile devices using motion gestures based on multi-owner template update Reviewed

Shigeki Karita, Kumi Nakamura, Kazuhiro Kono, Yoshimichi Ito, Noboru Babaguchi

2015 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2015 2015.7

This paper proposes a template updating method for improving authentication accuracy in behavioral biometric authentication with hand/arm motion gestures for mobile devices. We introduce an extended version of the standard K-Medoids based clustering algorithm called supervised K-Medoids, which can handle with 2-class data such as positive samples and negative samples. Using the supervised K-Medoids, the template corresponding to each owner is selected as the one that is the most identifiable as the actual owner, and, at the same time, the most distinguishable from the others. Therefore, our method can decrease False-Rejection-Rate (FRR) and False-Acceptance-Rate (FAR) simultaneously, compared to the conventional work that is based on the template update with only the owner's data to decrease FRR. Our template update with multi-owner data attains Equal-Error-Rate (EER) of 5.2% whereas the conventional template update method with owner's own data results in 12.0% when 10 subjects authenticate with gestures for 10 days.

Temporal Spotting of Human Actions from Videos Containing Actor's Unintentional Motions Reviewed

K. Hara, K. Nakamura, N. Babaguchi

Proc. IEEE International Conference on Multimedia and Expo (ICME2015) 2015.7

Protection and Utilization of Privacy Information via Sensing Invited

Noboru Babaguchi, Yuta Nakashima

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E98D ( 1 ) 2 - 9 2015.1

Our society has been getting more privacy-sensitive. Diverse information is given by users to information and communications technology (ICT) systems such as IC cards benefiting them. The information is stored as so-called big data, and there is concern over privacy violation. Visual information such as images and videos is also considered privacy-sensitive. The growing deployment of surveillance cameras and social network services has caused a privacy problem of information given from various sensors. To protect privacy of subjects presented in visual information, their face or figure is processed by means of pixelization or blurring. As image analysis technologies have made considerable progress, many attempts to automatically process flexible privacy protection have been made since 2000, and utilization of privacy information under some restrictions has been taken into account in recent years. This paper addresses the recent progress of privacy protection for visual information, showing our research projects: PriSurv, Digital Diorama (DD), and Mobile Privacy Protection (MPP). Furthermore, we discuss Harmonized Information Field (HIFI) for appropriate utilization of protected privacy information in a specific area.

Real-Time People Counting across Spatially Adjacent Non-Overlapping Camera Views Reviewed

R. Akai, N. Nitta, N. Babaguchi

Proc. of International Conference on Multimedia Modeling (MMM2015) 71 - 82 2015.1

Digital diorama: Privacy-preserving and intelligible sensing-based real-world content Reviewed

Naoko Nitta, Noboru Babaguchi

ITE Transactions on Media Technology and Applications 3 ( 3 ) 184 - 193 2015

This paper proposes a sensing-based real-world content called Digital Diorama, which is a three-dimensional miniature model of the dynamically changing real world created from the real-time data continuously published over the Internet by stationary cameras distributed in public spaces. Digital Diorama is designed to preserve both the privacy of the monitored persons and the intelligibility of the content by superimposing the latest background images and human icons each of which represents a monitored person in the three-dimensional model. By using the data continuously published from 10 stationary cameras installed on one floor of a shopping mall, our prototype Digital Diorama browser was able to construct and dynamically update Digital Diorama in approximately 1 fps. The results of subjective evaluations indicated that utilizing appropriate human icons can improve the intelligibility of the content while preserving the privacy of the monitored persons.

Inter-Concept Distance Measurement with Adaptively Weighted Multiple Visual Features Reviewed

Kazuaki Nakamura, Noboru Babaguchi

COMPUTER VISION - ACCV 2014 WORKSHOPS, PT III 9010 56 - 70 2015

Most of the existing methods for measuring the inter-concept distance (ICD) between two concepts from their image instances use only a single kind of visual feature extracted from each instance. However, a single kind of feature is not enough for appropriately measuring ICDs due to a wide variety of perspectives for similarity evaluation (e.g., color, shape, size, hardness, heaviness, and functions); the relationships between different concept pairs are more appropriately modeled from different perspectives provided by multiple kinds of features. In this paper, we propose extracting two or more kinds of visual features from each image instance and measuring ICDs using these multiple features. Moreover, we present a method for adaptively weighting the visual features on the basis of their appropriateness for each concept pair. Experiments demonstrated that the proposed method outperformed a method using only a single kind of visual feature and one combining multiple kinds of features with a fixed weight.

Real-Time Local Word Database Construction from Twitter Reviewed

Takuya Kamimura, Naoko Nitta, Noboru Babaguchi

2015 IEEE INTERNATIONAL CONFERENCE ON SMART CITY/SOCIALCOM/SUSTAINCOM (SMARTCITY) 299 - 306 2015

Recently, geotagged posts to social media such as Twitter have been used to automatically construct a geographical dictionary containing diverse types of local words which indicate specific locations in the real world. The existing methods typically examine the spatial locality of the usage patterns observed in the geotagged posts accumulated for a certain period of time to select the local words; however, how long the geotagged posts need to be accumulated depends on the usage frequency of the word, and additionally, some local words can indicate different locations at different times. Thus, we propose a real-time method for constructing a local word database which consistently keeps the local words and their locations up to date by iteratively adding new local words, removing old temporary local words, and updating the locations indicated by the local words. These functions are realized by adaptively recording/resetting the usage history of each word to properly examine its spatial locality and by assigning the weight for each geotag which is used to represent the locations indicated by the local words according to their temporal variations. The local word database constructed by our proposed method was verified to contain more up-todate local words and locations compared to other types of geographical dictionary constructed by experts, crowdsourcing, and from the geotagged tweets accumulated for a fixed period of time based on the performance evaluations of tweet location estimation as an example of applications utilizing the geographical dictionaries.

Camera oscillation pattern for VSLAM: Translational versus rotational Reviewed

Mohamed Heshmat, Mohamed Abdellatif, Kazuaki Nakamura, A. A. Abouelsoud, Noboru Babaguchi

2014 International Conference on 3D Imaging, IC3D 2014 - Proceedings 2014

Visual SLAM algorithms exploit natural scene features to infer the camera motion and build a map of the environment landmarks. SLAM algorithm has two interrelated processes localization and mapping. For accurate localization, we need the features location estimates to converge quickly. On the other hand, to build an accurate map, we need accurate localization. Recently, a biologically inspired approach exploits deliberate camera oscillation has been used to improve the convergence speed of depth estimate. In this paper, we explore the effect of camera oscillation pattern on the accuracy of VSLAM. Two main oscillation patterns are used for distance estimation: translational and rotational. Experiments, using static and moving robot, are made to explore the effect of these oscillation patterns on the VSLAM performance.

Dynamic feature detection using virtual correction and camera oscillations Reviewed

Mohamed Heshmat, Mohamed Abdellatif, Kazuaki Nakamura, A. A. Abouelsoud, Noboru Babaguchi

2014 International Conference on 3D Imaging, IC3D 2014 - Proceedings 2014

Visual SLAM algorithms exploit natural scene features to infer the camera motion and build a map of a static environment. In this paper, we relax the severe assumption of a static scene to allow for the detection and deletion of dynamic points. A new 'virtual correction' method is introduced which serves to detect the dynamic points by checking the re-projection error of the points before and after the virtual measurement update. It can also recover the erroneously excluded useful features, particularly the distant points which may be deleted because of the change in its position after new measurement observation. Deliberate camera oscillations are also used to improve the VSLAM accuracy and the camera observability. The simulation results showed the effectiveness of the virtual correction when combined with camera oscillation in recovering the misclassified features and detecting the dynamic features even in difficult scenarios.

Real-world event detection using Flickr images Reviewed

Naoko Nitta, Yusuke Kumihashi, Tomochika Kato, Noboru Babaguchi

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8326 ( 2 ) 307 - 314 2014

This paper proposes a real-world event detection method by using the time and location information and text tags attached to the images in Flickr. Events can generally be detected by extracting images captured at the events which are annotated with text tags frequently used only in specific times and locations. However, such approach can not detect events where only a small number of images were captured. We focus on the fact that semantically related events often occur around the same time at different locations. Considering a group of these events as an event class, the proposed method firstly detects event classes from all images in Flickr based on their similarity of the captured time and text tags. Then, from the images consisting each event class, events are detected based on their similarity of the captured locations. Such two-step approach enables us to detect events where a small number of images were captured. © 2014 Springer International Publishing.

Special Issue on Intelligent Video Surveillance for Public Security and Personal Privacy Invited

Noboru Babaguchi, Andrea Cavallaro, Rama Chellappa, Frederic Dufaux, Liang Wang

IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 8 ( 10 ) 1559 - 1561 2013.10

[Invited Speech] Protection and Utilization of Privacy Information Invited

N. Babaguchi

2013.7

[Keynote Speech] Example-Based Remixing of Multimedia Contents Invited

N. Babaguchi

1st International Workshop on Media Fragment Creation and reMIXing (MMIX'13, Co-located WS at ICME2013) 2013.7

[Invited Speech] Protection and Utilization of Privacy Information Invited

N. Babaguchi

2013.3

Content analysis for home videos Invited Reviewed

Naoko Nitta, Noboru Babaguchi

ITE Transactions on Media Technology and Applications 1 ( 2 ) 91 - 100 2013

Depth-estimation-free condition for projective factorization and its application to 3D reconstruction Reviewed

Yohei Murakami, Takeshi Endo, Yoshimichi Ito, Noboru Babaguchi

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 7727 ( 4 ) 150 - 162 2013

This paper concerns depth-estimation-free conditions for projective factorization. We first show that, using an algebraic approach, the estimation of the projective depth is avoidable if and only if the origins of all camera coordinate systems are lying on a single plane, and optical axes of the coordinate systems point the same direction that is perpendicular to the plane. Next, we generalize the result to the case where the points are possibly restricted on a plane or on a line. The result clearly reveals the trade-off between the freedom of camera motion and that of point location. We also give a least-square-based method for Euclidean reconstruction from the result of the projective reconstruction. The proposed method is evaluated through simulation from the viewpoint of computational time. © 2013 Springer-Verlag.

Efficient DC term encoding scheme based on double prediction algorithms and Pareto probability models Reviewed

Ting-Yu Ko, Chi-Jung Tseng, Hsin-Hui Chen, Jian-Jiun Ding, Noboru Babaguchi

Proceedings - IEEE International Conference on Multimedia and Expo 2013

In this paper, a new algorithm which adopts the techniques of double prediction and the Pareto probability model was applied to encode the DC term in the JPEG compression process. Conventionally, the DC term was encoded by differential coding, i.e., the difference of the DC values between the current block and the previous block. In this paper, we first use the DC terms of four adjacent blocks to predict the current DC value. We then further use the prediction error of the four adjacent blocks to estimate the variance of the prediction error of the current block. We call it the double prediction algorithm. Next, the Pareto distribution is applied to model the probability distribution of the prediction error. Simulation results show that, with the proposed algorithms, the data size required for DC terms is significantly reduced by 25% ∼ 60% and a much higher compression rate can be achieved. © 2013 IEEE.

People counting across spatially disjoint cameras by flow estimation between foreground regions Reviewed

Naoko Nitta, Takayuki Nakazaki, Kazuaki Nakamura, Ryota Akai, Noboru Babaguchi

2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2013 414 - 419 2013

Our goal is to develop a method for counting the number of people traveling over a wide area monitored by spatially disjoint multiple cameras with non-overlapping fields of view. The proposed method counts the number of people traversing across each pair of cameras' fields of view by estimating the flows between the foreground regions which have disappeared from and appeared in the camera views within a short time interval. The approach aims at resolving two problems: people in a foreground region can split and merge outside the cameras' fields of view, and the appearance variance of the same person and persons with similar appearance can lead to errors in person re-identification across cameras. The average errors of 0.184 persons in counting the number of people traveling over an area in a university campus monitored by four virtual cameras for 100 minutes have demonstrated the effectiveness of the proposed method. © 2013 IEEE.

Real-time privacy protection system for social videos using intentionally-captured persons detection Reviewed

Tatsuya Koyama, Yuta Nakashima, Noboru Babaguchi

Proceedings - IEEE International Conference on Multimedia and Expo 6 pages 2013

Most social videos, which are uploaded and shared through social networking services (SNSs), e.g., YouTube and Facebook, contain not only intentionally-captured persons (ICPs) but also non-ICPs who are unexpectedly framed in, such as passers-by. Sharing such social videos may infringe on the non-ICPs' privacy but not on the ICPs' in many cases
however, existing systems for video privacy protection simply obscure persons without distinguishing ICPs from non-ICPs. This naive obscuration may spoil the videos. Since this is a critical problem especially for social videos, in this paper, we propose a novel system for automatically generating privacy-protected videos in real-time. Our system localizes ICPs and non-ICPs using ICP detection leveraging the spatial and temporal consistency of ICPs/non-ICPs and obscures the non-ICPs. We have experimentally evaluated the performance of ICP detection and demonstrated the applicability of our system. © 2013 IEEE.

A hybrid mobile-fixed surveillance system, a new solution for public security Case study: abandoned objects' owner alert system"jointly worked"

K.Masui, M.-S.Dao, N.Babaguchi

IEICE Technical Report 2012.3

Intended human object detection for automatically protecting privacy in mobile video surveillance Reviewed

Yuta Nakashima, Noboru Babaguchi, Jianping Fan

MULTIMEDIA SYSTEMS 18 ( 2 ) 157 - 173 2012.3

With the recent popularization of mobile video cameras including camera phones, a new technology, mobile video surveillance, which uses mobile video cameras for video surveillance has been emerging. Such videos, however, may infringe upon the privacy of others by disclosing privacy sensitive information (PSI), i.e., their appearances. To prevent videos from infringing on the right to privacy, new techniques are required that automatically obscure PSI regions. The problem is how to determine the PSI regions to be obscured while maintaining enough video content to present the camera persons' capture-intentions, i.e., what they want to record in their videos to achieve their surveillance tasks. To this end, we introduce a new concept called intended human objects that are defined as human objects essential for capture-intentions, and develop a new method called intended human object detection that automatically detects the intended human objects in videos taken by different camera persons. Through the process of intended human object detection, we develop a system for automatically obscuring PSI regions. We experimentally show the performance of intended human object detection and the contributions of the features used. Our user study shows the potential applicability of our proposed system.

Abandoned Object's Owner Detection: A Case Study of Hybrid Mobile-fixed Video Surveillance System Reviewed

Minh-Son Dao, Riccardo Mattivi, Francesco G. B. De Natale, Keita Masui, Noboru Babaguchi

2012 IEEE NINTH INTERNATIONAL CONFERENCE ON ADVANCED VIDEO AND SIGNAL-BASED SURVEILLANCE (AVSS) 404 - 409 2012

In this paper, a new framework of hybrid mobile-fixed video surveillance system (HMFVSS) is introduced. The purpose of this framework is to overcome common problems of existing mobile or fixed video surveillance systems: (1) moral harassment due to unfriendly or unnaturally installed mobile sensors, and (2) blind areas due to narrow-scope moving of fixed cameras. A case study of abandoned object's owner alert system (AOOAS) is also presented to emphasize the framework's advantages. IP cameras and "Spyglass" (i.e. a mobile camera embedded on glasses) are used as fixed and mobile sensors, respectively. There are three main tasks are inherited, developed, and integrated: (1) image registration for automatically locating abandoned object, (2) common histogram based abandoned object's owner detection, and (3) faces recognition. The experimental results with careful evaluation and comparison with others shows that the proposed framework moves a step ahead in video surveillance system.

Classification based group photo retrieval with Bag of People features Reviewed

Kazuya Shimizu, Yujiro Nakai, Naoko Nitta, Noboru Babaguchi

Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, ICMR 2012 2012

This paper proposes a method for retrieving images containing a specific target person from a given image collection of group photos. This can be realized by query-by-example methods which compare the facial visual features of the target person in the given query image and of each person in the images in the image collection. However, since images are often taken under various conditions, facial appearance of the same person can vary. Since socially related people such as family and friends are often taken photos together, the people co-occurrence relations in the same images can also be a useful clue for image retrieval. Focusing on such people co-occurrence relations, we propose Bag of People (BoP) features which represent both the facial appearances of persons and their co-occurrence relations in the same images. By using the BoP features, a classifier for classifying images into two classes, images containing the target person and other images, can be trained from a small number of images labeled by user's relevance feedback. Furthermore, since the labeled images obtained by relevance feedback are much fewer than unlabeled images in the image collection, an active learning method is used to select useful images to train the classifier. When retrieving images of 24 persons in total from 550 images, after five feedback iterations, the mean average precision of 0.94 was obtained by considering the people co-occurrence relations, as against 0.69 when considering only the target person. Copyright © 2012 ACM.

Extracting Context Information from Microblog based on Analysis of Online Reviews Reviewed

Takumi Takehara, Shohei Miki, Naoko Nitta, Noboru Babaguchi

2012 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO WORKSHOPS (ICMEW) 248 - 253 2012

Recommender systems automatically determine suitable items for users. Although preferences or context of users have been widely utilized in order to evaluate the suitability of the items for users, the surrounding context have little been considered. Focusing on that many ordinary human beings voluntarily report their observations of the current situation of the world to microblogs, this paper proposes a recommender system which not only recommends suitable restaurants to users based on their preferences and context but also provides the surrounding context information reported to microblogs which will further affect the users' restaurant selection behaviors. In particular, considering that such influential surrounding context information in microblogs includes keywords related to restaurant assessment, we propose a method for automatically determining the keywords to extract the context information by analyzing online reviews, which have been gathered also from ordinary human beings over a long period of time. The experiments by using Twitter as microblogs and Tabelog, a popular online restaurant review site in Japan, to obtain online reviews, indicated that the influential context information can be extracted from Twitter with the highest recall of 93.3% by using the area-related keywords. Additionally using the restaurant-related keywords was effective in removing irrelevant information obtaining the precision of 15.9%.

DELIVERY METHOD FOR VIEWER-SPECIFIC PRIVACY PROTECTED VIDEO USING DISCRETE WAVELET TRANSFORM Reviewed

Naoya Fukuoka, Yoshimichi Ito, Noboru Babaguchi

2012 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP 2012) 2285 - 2288 2012

This paper presents a delivery method for viewer-specific privacy protected video, where "viewer-specific" implies that the way of privacy protection (e.g., box, mosaic, transparency) varies according to viewer's authority. In this method, in order to reduce the load of a delivery server, viewer-specific privacy protected videos are not produced at the delivery server, but they are produced at each viewer's terminal. The delivery server extracts and decomposes the information of human objects, and then embeds the information into background images using information hiding technique. Each viewer extracts a part of the decomposed information of human objects according to his/her authority, and then produces privacy protected video by integrating the extracted information. In this method, the discrete wavelet transform (DWT) plays a key role. The proposed method is evaluated through experiments.

MARKOV RANDOM FIELD-BASED REAL-TIME DETECTION OF INTENTIONALLY-CAPTURED PERSONS Reviewed

Tatsuya Koyama, Yuta Nakashima, Noboru Babaguchi

2012 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP 2012) 1377 - 1380 2012

Most videos taken by videographers contain intentionally-captured persons (ICPs), who are essential for what the videographers want to express in their video. This paper presents a method to detect ICPs in real-time. Whether a person in a video is an ICP or not is reflected in features such as the person's motion and camera motion, which are thus beneficial for detecting ICPs. However, estimating camera motion is computationally expensive. For real-time detection, we use samples of acceleration and angular velocity obtained from inertial sensors instead of estimating camera motion. Considering that pairwise constraints based on differences between persons' sizes also improve the detection performance, we model the ICPs using Markov random field. We experimentally evaluate the performance of our method and demonstrate that it works in real-time.

Tablet Owner Authentication Based on Behavioral Characteristics of Multi-Touch Actions Reviewed

Kumi Nakamura, Kazuhiro Kono, Yoshimichi Ito, Noboru Babaguchi

2012 21ST INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR 2012) 3431 - 3434 2012

This paper proposes a method for tablet owner authentication based on behavioral characteristics of multiple fingers' actions called multi-touch actions. The method is based on dynamic time warping, which has been commonly used for authentication using pentablet or single finger's actions, but another problem arises due to the use of multi-touch actions (e. g., identifying fingers). We also provide methods for these problems. Using proposed method, we evaluate the authentication accuracies for several types of multi-touch actions through experiments.

Indoor Positioning System Using Digital Audio Watermarking Reviewed

Yuta Nakashima, Ryosuke Kaneto, Noboru Babaguchi

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E94D ( 11 ) 2201 - 2211 2011.11

Recently, a number of location-based services such as navigation and mobile advertising have been proposed. Such services require real-time user positions. Since a global positioning system (GPS), which is one of the most well-known techniques for real-time positioning, is unsuitable for indoor uses due to unavailability of GPS signals, many indoor positioning systems (IPSs) using WLAN, radio frequency identification tags, and so forth have been proposed. However, most of them suffer from high installation costs. In this paper, we propose a novel IPS for real-time positioning that utilizes a digital audio watermarking technique. The proposed IPS first embeds watermarks into an audio signal to generate watermarked signals, each of which is then emitted from a corresponding speaker installed in a target environment. A user of the proposed [PS receives the watermarked signals with a mobile. device equipped with a microphone, and the watermarks are detected in the received signal. For positioning, we model various effects upon watermarks due to propagation in the air, i.e., delays, attenuation, and diffraction. The model enables the proposed IPS to accurately locate the user based on the watermarks detected in the received signal. The proposed IPS can be easily deployed with a low installation cost because the IPS can work with off-the-shelf speakers that have been already installed in most of the indoor environments such as department stores, amusement arcades, and airports. We experimentally evaluate the accuracy of positioning and show that the proposed IPS locates the user in a 6 m by 7.5 m room with root mean squared error of 2.25 m on average. The results also demonstrate the potential capability of real-time positioning with the proposed IPS.

Example-based video remixing Reviewed

Naoko Nitta, Noboru Babaguchi

MULTIMEDIA TOOLS AND APPLICATIONS 51 ( 2 ) 649 - 673 2011.1

A video remix is generally created by arranging selected video clips and combining them with other media streams such as audio clips and video transition effects. This paper proposes a system for semi-automatically creating video remixes of good expressive quality. Given multiple original video clips, audio clips, and transition effects as the input, the proposed system generates a video remix by five processes: I) video clip sequence generation, II) audio clip selection, III) audio boundary extraction, IV) video segment extraction, and V) transition effect selection, based on the spatial and temporal structural patterns automatically learned from professionally created video remix examples. Experiments using movie trailers of action genre as video remix examples not only demonstrate that video remixing by professionals can be imitated based on examples but also reveal that the video clip sequence generation and audio clip selection are the most important processes to improve the perceived expressive quality of video remixes.

Automatic generation of privacy-protected videos using background estimation Reviewed

Yuta Nakashima, Noboru Babaguchi, Jianping Fan

Proceedings - IEEE International Conference on Multimedia and Expo 2011

Recently, video sharing services such as YouTube and Daily-motion have become popular and many videos taken with mobile video cameras are uploaded to such a video sharing service. However, such videos can infringe on the privacy right of people in the videos because they may contain privacy sensitive information (PSI) of the people, i.e., their appearances. This strongly motivates us to develop a technique to generate privacy-protected videos. In this paper, we propose a novel system for automatic generation of privacy-protected videos based on background estimation. In most conventional techniques, objects that contain PSI are detected and obscured by, e.g., blurring. Conversely, in our system, background pixels are estimated and then substituted with intended human objects that are essential for the camera person's capture intention. We quantitatively evaluate our system to demonstrate its potential applicability. © 2011 IEEE.

Content-preserving zoom-in view generation for surveillance videos Reviewed

Kenji Watanabe, Naoko Nitta, Noboru Babaguchi

COMPUTATIONAL IMAGING IX 7873 2011

There are several zoom-in video display methods including full-zoom and fisheye view that magnify the regions of interest (ROIs). However, those methods usually discard or deform the remaining regions without considering their content. In this paper, we propose a method for generating a content-preserving zoom-in view which magnifies ROIs and at the same time preserves the content of the remaining regions. Targeting on surveillance videos, our method firstly extracts moving objects from every input frame as ROIs. Then, the importance score is calculated for each pixel in the input frame based on its content to determine where the deformation, which may cause the destruction of the content, should be avoided. Finally, a mapping problem from the input frame to the zoom-in view with respect to the importance score is formulated to deform less important regions more than the important ones. Experiments are conducted to study the effectiveness of considering the content importance. We also compare the results of our method with those of other methods, fisheye view and a method of using uniform scaling and seam carving.

Image Retrieval Considering People Co-occurrence Relations Using Relevance Feedback Reviewed

Kazuya Shimizu, Naoko Nitta, Noboru Babaguchi

MULTIMEDIA ON MOBILE DEVICES 2011 AND MULTIMEDIA CONTENT ACCESS: ALGORITHMS AND SYSTEMS V 7881 2011

The recent popularity of digital cameras allows us to take a large number of images. There is an increasing need for efficiently and accurately retrieving images containing a specific person from such image collections. While only the visual features of the specific person are used in many query-by-example retrieval methods, we focus on the fact that some people such as family or friends are more likely to appear in the same images than others and use visual features of not only the queried person but also people who have strong co-occurrence relations with the queried person to improve the retrieval performance. The relevance feedback is used to learn who co-occur with the queried person in the same images, their faces, and the strength of their co-occurrence relations. For 116 images collected from 6 persons, after five feedback iterations, the recall rate of 53% was obtained by considering the co-occurrence relations among people, as against 34% when using only features of the queried person.

Extracting intentionally captured regions using point trajectories Reviewed

Yuta Nakashima, Noboru Babaguchi

MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops 1417 - 1420 2011

When camera persons take videos with mobile video cameras, they usually have capture intentions, i.e., what they want to express in their videos, and there are intentionally captured regions (ICRs) in the video frames that are essential for the capture intentions. Extracting ICRs is thus beneficial for wide range of applications such as video summarization and video adaptation for small displays. In this paper, we present a novel method for automatically extracting ICRs. A camera person usually moves his/her camera so that ICRs can be arranged in appropriate positions in video frames
therefore, ICRs can yield specific motion. This observation indicates that such specific motion is a vital cue for extracting ICRs. The proposed method represents motion by point trajectories, which are long-term trajectories of spatially dense points in video frames, and extracts ICRs using an ICR model based on the point trajectories. We experimentally evaluate the proposed method to demonstrate its potential applicability. Copyright 2011 ACM.

Example-based video remixing support system Reviewed

Naoko Nitta, Noboru Babaguchi

MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops 563 - 572 2011

Video remixes are generally created by sequentially arranging selected video clips and mixing them with other media streams such as audio clips and transition effects. Especially, mixing music clips often effectively improves the expressive quality of video clips. This paper proposes an example-based system for supporting average users in the 3 steps in video remixing: I)video shot sequence creation, II)music clip selection, and III)audio volume adjustment. The proposed system creates a template for a video remix and gives suggestions to users on an interface such as which video clips should be selected to create a video shot sequence and which music clips should be mixed to the created video shot sequence based on professionally created video remix examples. Then, the audio volume of each video shot is automatically adjusted based on its audio content so that the sounds in the video shots and the music clips would not interfere with each other. Experiments have verified that our system was able to create a video shot sequence whose quality was improved equally as the professionally created one by mixing the selected music clips, doubling the subjective scores from 1.7 to 3.7 on a scale of 1-5. Automatic audio volume adjustment improved the subjective scores by approximately 0.5 points on average. Further, the suggestions provided on the interface was evaluated useful by 5 subjects when creating a video remix by selecting 32 video clips and 4 music clips from 265 video clips and 180 music clips. © 2011 ACM.

Example-based video remixing for home videos

Naoko Nitta, Noboru Babaguchi

Proceedings - IEEE International Conference on Multimedia and Expo 2011

Video remixes are generally created by sequentially arranging selected video clips and combining them with other media streams such as audio clips. In this paper, an example-based approach is adopted for semi-automatically creating video remixes of good expressive quality from home videos. Given multiple home video clips and audio clips, the proposed system generates a video remix by four processes: I)video clip sequence generation, II)audio clip selection, III)audio boundary extraction, and IV)video segment extraction based on professionally created video remix examples. A user interface which presents video clips according to their suitability to the examples and their perceived quality is developed so that users can efficiently and effectively select and arrange suitable video clips in video clip sequence generation. By using movie trailers of action genre as video remix examples, a 43-second video remix was created from 45-minute home videos and was subjectively evaluated better than the one created considering only the perceived quality of video clips. © 2011 IEEE.

Learning people co-occurrence relations by using relevance feedback for retrieving group photos

Kazuya Shimizu, Naoko Nitta, Noboru Babaguchi

Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR'11 2011

This paper proposes an image retrieval method which retrieves images of a specific person from group photos. Many query-by-example methods have focused only on the visual features of the queried person. However, since socially related people such as family and friends are often taken photos together, their co-occurrence relations can be useful information. Thus, we propose an image retrieval method which uses the visual features of not only the queried person but also those who co-occur with the queried person in the same images. Relevance feedback is used to learn who co-occur with the queried person, their faces, and how strong their co-occurrence relations are. When retrieving the images of 19 persons in total from 158 images, after five feedback iterations, the recall rate of 50% was obtained by considering the people co-occurrence relations, as against 33% when considering only the queried person. With human errors in giving relevance feedback, the recall rate still improved to 40%. © 2011 ACM.

Three-level privacy control for sensing-based real-world content digital diorama

Takumi Takehara, Naoko Nitta, Noboru Babaguchi

ACM International Conference Proceeding Series 17 - 20 2011

Digital Diorama, the sensing-based real-world content, can be constructed by integrating real-time information obtained from sensors monitoring the real world. In order to increase the benefits of viewers without violating the privacy of monitored persons, this paper proposes three-level privacy control over the monitored persons based on their agreement to the usage of their information obtained from sensors: privacy control with i) no agreement, ii) partial agreement. and iii) mutual agreement. i) presents only their positions to show where persons are. ii) additionally presents the information which can be automatically obtained from sensors such as age and gender to show what kinds of persons are where without disclosing the visual appearances. iii) presents their visual appearances based on their mutual agreement with specific viewers. Our evaluation indicated that the representation simulating each privacy control presented information from sensors with acceptable privacy protection. © 2011 ACM.

Fast extraction method of high-level feature using random forests from imbalanced training data Reviewed

Y. Kawai, H. Sumiyoshi, M. Fujii, M. Shibata, N. Babaguchi

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers 64 ( 12 ) 1951 - 1955 2010.12

We propose a method of using a random forest algorithm to quickly detect semantically high-level features such as specific objects. The random forest has a lower computation cost than that of the common algorithm such as a support vector machine (SVM). However, it cannot cope with training data that have a large bias in the number of negative and positive examples. We improve the conventional training algorithm to ensure sampling the data with equal probability from each class when creating bootstrap samples, which increases the classification accuracy. Experiments on the Caltech-101 dataset resulted in a recall of 64.3% and precision of 71.1%, which were comparable to those of conventional methods. The average time needed for training and for detection were reduced to one sixteenth and one twenty-seventh that of SVM, respectively.

[Invited Speech] Visual Processing for Privacy-Sensitive Information Invited

Noboru Babaguchi

2010.10

A new spatio-temporal method for event detection and personalized retrieval of sports video Reviewed

Minh-Son Dao, Noboru Babaguchi

MULTIMEDIA TOOLS AND APPLICATIONS 50 ( 1 ) 227 - 248 2010.10

In this paper, a new spatio-temporal method for adaptively detecting events based on Allen temporal algebra and external information support is presented. The temporal information is captured by presenting events as the temporal sequences using a lexicon of non-ambiguous temporal patterns. These sequences are then exploited to mine undiscovered sequences with external text information supports by using class associate rules mining technique. By modeling each pattern with linguistic part and perceptual part those work independently and connect together via transformer, it is easy to deploy this method to any new domain (e.g baseball, basketball, tennis, etc.) with a few changes in perceptual part and transformer. Thus the proposed method not only can work well in unwell structured environments but also can be able to adapt itself to new domains without the need (or with a few modification) for external re-programming, re-configuring and re-adjusting. Results of automatic event detection progress are tailored to personalized retrieval via click-and-see style using either conceptual or conceptual-visual query scheme. Experimental results carried on more than 30 hours of soccer video corpus captured at different broadcasters and conditions as well as compared with well-known related methods, demonstrated the efficiency, effectiveness, and robustness of the proposed method in both offline and online processes.

Constructing Distributed Hippocratic Video Databases for Privacy-Preserving Online Patient Training and Counseling Reviewed

Jinye Peng, Noboru Babaguchi, Hangzai Luo, Yuli Gao, Jianping Fan

IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE 14 ( 4 ) 1014 - 1026 2010.7

Digital video now plays an important role in supporting more profitable online patient training and counseling, and integration of patient training videos from multiple competitive organizations in the health care network will result in better offerings for patients. However, privacy concerns often prevent multiple competitive organizations from sharing and integrating their patient training videos. In addition, patients with infectious or chronic diseases may not want the online patient training organizations to identify who they are or even which video clips they are interested in. Thus, there is an urgent need to develop more effective techniques to protect both video content privacy and access privacy. In this paper, we have developed a new approach to construct a distributed Hippocratic video database system for supporting more profitable online patient training and counseling. First, a new database modeling approach is developed to support concept-oriented video database organization and assign a degree of privacy of the video content for each database level automatically. Second, a new algorithm is developed to protect the video content privacy at the level of individual video clip by filtering out the privacy-sensitive human objects automatically. In order to integrate the patient training videos from multiple competitive organizations for constructing a centralized video database indexing structure, a privacy-preserving video sharing scheme is developed to support privacy-preserving distributed classifier training and prevent the statistical inferences from the videos that are shared for cross-validation of video classifiers. Our experiments on large-scale video databases have also provided very convincing results.

Theoretical Analysis of the Performance of Anonymous Communication System 3-Mode Net Reviewed

Kazuhiro Kono, Shinnosuke Nakano, Yoshimichi Ito, Noboru Babaguchi

IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES E93A ( 7 ) 1338 - 1345 2010.7

This paper aims at analyzing the performance of an anonymous communication system 3-Mode Net with respect to the number of relay nodes required for communication and sender anonymity. As for the number of relay nodes, we give explicit formulas of the probability distribution, the expectation, and the variance. Considering sender anonymity, we quantify the degree of sender anonymity under a situation where some relay nodes collude with each other. The above analyses use random walk theory, a probability generating function, and their properties. From obtained formulas, we show several conditions for avoiding a situation where the number of relay nodes becomes large, and for providing high sender anonymity. Furthermore, we investigate the relationship between the number of relay nodes and sender anonymity, and give a condition for providing a better performance of 3MN.

Recoverable Privacy Protection for Video Content Distribution Reviewed

Guangzhen Li, Yoshimichi Ito, Xiaoyi Yu, Naoko Nitta, Noboru Babaguchi

EURASIP Journal on Information Security 2009 Article No. 293031 - 11 pages 2010.1

This paper presents a method which attains recoverable privacy protection for video content distribution. The method is based on discrete wavelet transform (DWT), which generates scaling coefficients and wavelet coefficients. In our method, scaling coefficients, which can be regarded as a low-resolution image of an original image, are used for producing privacy-protected image. On the other hand, wavelet coefficients, which can be regarded as privacy information, are embedded into the privacy-protected image via information hiding technique. Therefore, privacy protected image can be recovered by authorized viewers if necessary. The proposed method is fully analyzed through experiments from the viewpoints of the amount of the embedded privacy information, the deterioration due to the embedding, and the computational time. © 2009 Guangzhen Li et al.

Automated generation method for TV program trailers based on introductory text Reviewed

Yoshihiko Kawai, Hideki Sumiyoshit, Masahiro Shibata, Nobuyuki Yagi, Noboru Babaguchi

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers 64 ( 1 ) 85 - 93 2010.1

Video summarization is one of the most efficient methods for retrieval from large video archives. An automated method is described for generating TV program trailers (short video clips to advertise the program). Our method uses introductory text from an electronic program guide, which is a short description of the program highlights. We assume that the video segments corresponding to closed caption sentences with similar expressions to those of the introductory text contain appealing scenes, and our system generates a program trailer by linking these segments together. The AdaBoost algorithm is used to learn the textual characteristics of the introductory text. The proposed method was used to summarize actual TV programs to test its effectiveness.

Automatically protecting privacy in consumer generated videos using intended human object detector Reviewed

Yuta Nakashima, Noboru Babaguchi, Jianping Fan

MM'10 - Proceedings of the ACM Multimedia 2010 International Conference 1135 - 1138 2010

The growing popularity of video sharing services such as YouTube enables us to upload and share consumer generated videos (CGVs) easily, resulting in disclosure of the privacy sensitive information (PSI) of persons, i.e., their appearances. Therefore, we need a technique for automatically protecting the privacy in CGVs
however, the main problem is how to determine PSI regions automatically. In this paper, we propose a novel system for automatically protecting the privacy in CGVs. The proposed system tackles the problem of determining PSI regions by using an intended human object detector that detects human objects which the camera person wanted to capture to achieve his/her capture intention. In addition, the proposed system adopts several PSI obscuring methods such as blocking out, blurring and seam carving. We present the results of subjective evaluations of a privacy protected video in terms of the visual quality and acceptability of PSI disclosure, as well as the performance of the intended human object detector. © 2010 ACM.

Hierarchical anomality detection based on situation Reviewed

Shuichi Nishio, Hiromi Okamoto, Noboru Babaguchi

Proceedings - International Conference on Pattern Recognition 1108 - 1111 2010

In this paper, we propose a novel anomality detection method based on external situational information and hierarchical analysis of behaviors. Past studies model normal behaviors to detect anomality as outliers. However, normal behaviors tend to differ by situations. Our method combines a set of simple classifiers with pedestrian trajectories as inputs. As mere path information is not sufficient for detecting anomality, trajectories are first decomposed into hierarchical features of different abstract levels and then applied to appropriate classifiers corresponding to the situation it belongs to. Effects of the methods are tested using real environment data. © 2010 IEEE.

Face Image Retrieval across Age Variation Using Relevance Feedback Reviewed

Naoko Nitta, Atsushi Usui, Noboru Babaguchi

ADVANCES IN MULTIMEDIA MODELING, PROCEEDINGS 5916 152 - 162 2010

Given a single face image of a specific person as a query, it is very difficult to retrieve all of his/her images from a personal image collection stored for a long term clue to age-related changes in facial appearances. This paper proposes to apply relevance feedback to enhance the performance of image retrieval from the image collections with age variation. Specifically, we propose two types of update schemes: i) query expansion and ii) weight updating and show the effects of each scheme by experiments with two actual image collections. For an image collection, the recall rate improved from 40.8% to 72.5% after five iterations of relevance feedback.

EVENT TACTIC ANALYSIS IN SPORTS VIDEO USING SPATIO-TEMPORAL PATTERN Reviewed

Minh-Son Dao, Keita Masui, Noboru Babaguchi

2010 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING 1497 - 1500 2010

Recently, event detection in sports videos has been gaining some remarkable results. Unfortunately, there is a lack of useful tools for users to explore and exploit these event clips on their own demands. In this paper, a novel method using spatio-temporal patterns to analyze event tactics in sports videos is introduced. The proposed method aims to understand tactics of events such as distributions and speeds of players, or attacking/defensive formations throughout a time when such events happen without tracking objects. The major contribution of the proposed method is to model event tactics by using sequence of symbols. Each symbol represents a distribution of players in a certain period of time. Therefore, a sequence of symbols intrinsically is concerned as spatio-temporal patterns. By using these patterns, an event tactic is detected, explained, and integrated into an event video to create a visualizing abstract. This visualizing abstract is very useful to help users understand an event tactic without watching whole clip. Moreover, users could query by an example or by a text to find all events sharing the same tactic. Thorough testing with over 100 goal clips in soccer domain demonstrate the superiority of the proposed method in terms of precision recall ratios.

Discriminating intended human objects in consumer videos Reviewed

Hiroshi Uegaki, Yuta Nakashima, Noboru Babaguchi

Proceedings - International Conference on Pattern Recognition 4380 - 4383 2010

In a consumer video, there are not only intended objects, which are intentionally captured by the camcorder user, but also unintended objects, which are accidentally framed-in. Since the intended objects are essential to present what the camcorder user wants to express in the video, discriminating the intended objects from the unintended objects are beneficial for many applications, e.g., video summarization, privacy protection, and so forth. In this paper, focusing on human objects, we propose a method for discriminating the intended human objects from the unintended human objects. We evaluated the proposed method using 10 videos captured by 3 camcorder users. The results demonstrate that the proposed method successfully discriminates the intended human objects with 0.45 of recall and 0.80 of precision. © 2010 IEEE.

Digital diorama: Sensing-based real-world visualization Reviewed

Takumi Takehara, Yuta Nakashima, Naoko Nitta, Noboru Babaguchi

Communications in Computer and Information Science 81 663 - 672 2010

Many sensors around the world are consistently collecting the real-time real-world data. The data streams captured by these sensors can give us an idea of what is going on in a specific area
however, it is not easy for humans to understand their spatial and temporal relationships by just looking at them independently. This paper proposes to construct Digital Diorama, a three-dimensional view where viewers can see at a glance how people are moving around the monitored space without violating their privacy, by integrating multiple data streams captured by stationary cameras and RFID readers in real time. Digital Diorama realizes such real-world visualization with the following features: 1) view control, 2) real-time camera image superimposition, and 3) privacy control. We have demonstrated that Digital Diorama for a shopping center was able to present the current positions of persons and real-time camera images in approximately 1 frame per second. © Springer-Verlag Berlin Heidelberg 2010.

Detecting intended human objects in human-captured videos Reviewed

Yuta Nakashima, Noboru Babaguchi, Jianping Fan

2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010 33 - 40 2010

When people take videos, they always want to capture intended objects, which are essential for presenting what they want to express in their videos, and to share the intended objects with others. The concept of intended objects provide a novel perspective for video content analysis, and detecting intended objects may be beneficial for wide range of applications such as video understanding and semantics interpretation, video summarization, video adaptation, video privacy protection, and so on. In this paper, we focus on a particular type of intended objects, i.e., intended human objects, and an interesting method is developed for detecting intended human objects automatically from human-captured videos. We also investigate the correlation between intended human objects and visual attention. Our experimental results indicate that our method can successfully detect the intended human objects. © 2010 IEEE.

Modeling visual information by spatio-temporal patterns to analyze event tactic in sports video Reviewed

Keita Masui, Minh-Son Dao, Noboru Babaguchi

2010 2nd European Workshop on Visual Information Processing, EUVIP2010 198 - 203 2010

Although event detection in sports videos has been gaining some remarkable results, tools for exploring and exploiting these event clips on their own demands are far from user's expectation. In this paper, a novel method using spatio-temporal patterns to model visual information tailored to analyze event tactics in sports videos without tracking objects is introduced. The major contribution of the proposed method is to represent visual information by using a sequence of symbols. Each symbol represents a distribution of players in a certain period of time. By using these sequences, event tactics are detected, explained, and visualized to help users understand event tactics without watching whole clips. Moreover, users could query by an example or by a text to find all events sharing the same tactic. Thorough testing with over 100 goal clips in soccer domain demonstrates the superiority of the proposed method in terms of precision recall ratios. ©2010 IEEE.

Real-time user position estimation in indoor environments using digital watermarking for audio signals Reviewed

Ryosuke Kaneto, Yuta Nakashima, Noboru Babaguchi

Proceedings - International Conference on Pattern Recognition 97 - 100 2010

In this paper, we propose a method for estimating the user position where a user is holding a microphone in an indoor environment using digital watermarking for audio signals. The proposed method utilizes detection strengths, which are calculated while detecting spread-spectrum-based watermarks. Taking into account delays and attenuation of the watermarked signals emitted from multiple loudspeakers and other factors, we construct a model of detection strengths. The user position is estimated in real-time using the model. The experimental results indicate that the user positions are estimated with 1.3 m of root mean squared error on average for the case where the user is static. We demonstrate that the proposed method successfully estimates the user position even when the user moves. © 2010 IEEE.

NHK STRL at TRECVID 2009: Surveillance Event Detection and High-Level Feature Extraction Reviewed

M. Takahashi, Y. Kawai, M. Fujii, M. Shibata, N. Babaguchi, S. Satoh

Proc. of TREC Video Retrieval Evaluation (TRECVID) Workshop 2009.11

Digital Diorama: Real-Time Adaptive Visualization of Public Spaces Reviewed

T. Takehara, Y. Nakashima, N. Nitta, N. Babaguchi

Proc. of 1st International Conference on Security Camera Network, Privacy Protection and Community Safety (SPC2009) 2009.10

Security Analysis of Anonymous Communication System 3-Mode Net Against Collaborating Nodes Reviewed

K. Kono, S. Nakano, Y. Ito, N. Babaguchi

Proc. of 2009 APSIPA Annual Summit Conference(APSIPA ASC 2009) 105 - 110 2009.10

User and Device Adaptation in Summarizing Sports Videos Reviewed

Naoko Nitta, Noboru Babaguchi

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E92D ( 6 ) 1280 - 1288 2009.6

Video summarization is defined as creating a video summary which includes only important scenes in the original video streams. In order to realize automatic video summarization, the significance of each scene needs to be determined. When targeted especially on broadcast sports videos, a play scene, which corresponds to a play, can be considered as a scene unit. The significance of every play scene can generally be determined based on the importance of the play in the game. Furthermore, the following two issues should be considered: 1) what is important depends on each user's preferences, and 2) the summaries should be tailored for media devices that each user has. Considering the above issues, this paper proposes a unified framework for user and device adaptation in summarizing broadcast sports videos. The proposed framework summarizes sports videos by selecting play scenes based on not only the importance of each play itself but also the users' preferences by using the metadata, which describes the semantic content of videos with keywords, and user profiles, which describe users' preference degrees for the keywords. The selected scenes are then presented in a proper way using various types of media such as video, image, or text according to device profiles which describe the device type. We experimentally verified the effectiveness of user adaptation by examining how the generated summaries are changed by different preference degrees and by comparing our results with/without using user profiles. The validity of device adaptation is also evaluated by conducting questionnaires using PCs and mobile phones as the media devices.

A New Linguistic-Perceptual Event Model for Spatio-Temporal Event Detection and Personalized Retrieval of Sports Video Reviewed

Minh-Son Dao, Sharma Ishan Nath, Noboru Babaguichi

IMAGE ANALYSIS AND PROCESSING - ICIAP 2009, PROCEEDINGS 5716 594 - 603 2009

This paper proposes a new linguistic-perceptual event model tailoring to spatio-temporal event detection and conceptual-visual personalized retrieval of sports video sequences. The major contributions of the proposed model are hierarchical structure, independence between linguistic and perceptual part, and ability of capturing temporal information of sports events. Thanks to these advanced contributions, it is very easy to upgrade model events from simple to complex levels either by self-studying from inner knowledge or by being taught from plug-in additional knowledge. Thus, the proposed model not only can work well in unwell structured environments but also is able to adapt itself to new domains without the need (or with a few modification) for external reprogramming, re-configuring and re-adjusting. Thorough experimental results demonstrate that events are modeled and detected with high accuracy and automation, and users' expectation of personalized retrieval is highly satisfied.

Matrix-based algorithm for integrating inheritance relations of access rights for policy generation Reviewed

Kazuhiro Kono, Yoshimichi Ito, Akihito Aoyama, Hiroaki Kamoda, Noboru Babaguchi

Journal of Information Processing 17 318 - 327 2009

This paper presents a matrix-based algorithm for integrating inheritance relations of access rights for generating integrated access control policies which unify management of various access control systems. Inheritance relations of access rights are found in subject, resource, and action categories. Our algorithm first integrates inheritance relations in each category, and next, integrates inheritance relations of all categories. It is shown that these operations can be carried out by basic matrix operations. This enables us to implement the integration algorithm very easily.

Psychological study for designing privacy protected video surveillance system: PriSurv Reviewed

Noboru Babaguchi, Takashi Koshimizu, Ichiro Umata, Tomoji Toriyama

Protecting Privacy in Video Surveillance 147 - 164 2009

Abstract As video surveillance systems are widely deployed, concerns continue to grow about invasion of privacy. We have built a privacy protected video surveillance system called PriSurv. Although PriSurv protects subject privacy using image processing, criteria of controlling the subject's visual information that is privacy-sensitive should be clarified. Visual information must be disclosed by considering the trade-off between privacy and security. The level of privacy-sensitive visual information that could be disclosed to a viewer is simply called disclosable privacy in this chapter. Disclosable privacy, which deeply involves the personal sense, is affected by many factors. A sense of privacy is individual, but in some cases it might have common factors. A sense of privacy is individual, but in some cases it might have common factors. In this chapter, we analyze what factors determine and affect disclosable privacy by applying statistical analysis to questionnaire-based experimental results. These results indicate that disclosable privacy is concerned with how much a subject has feeling of closeness to a viewer and expects the viewer's responsibility. They also show that disclosable privacy differs greatly by individuals. Reflecting the obtained findings in PriSurv's design, we adapt PriSurv to reflect a personal sense of privacy. © 2009 Springer London.

Preserving topological information in sub-trajectories-based representation for spatio-temporal trajectories indexing and retrieval Reviewed

Minh-Son Dao, Ishan Nath Sharma, Noboru Babaguchi

MM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums 641 - 644 2009

Trajectories of moving objects are known as one of the most important cues for understanding semantics in video data. Although there are a lot of significant researches dealing with trajectory analysis tailored to indexing and retrieval, several problems still remain. One of them is a trade-off between whole trajectory- and sub-trajectories- based methods. The former problem is that representing a trajectory as a whole is not appropriate for detecting similar patterns of the trajectory. In contrast, the latter is that even though some key portion of two trajectories share similar patterns, the whole trajectories may be totally different. Therefore, this paper proposes a novel method to optimize such trade-off. By representing a trajectory as a combination of sequence of "word" - each word's character represents one distinct feature extracted from sub-trajectories (i.e. segments), and a topological graph of trajectory's segments, the proposed method is shift and scale invariant, can handle occlusion and distortion, and can discover similar patterns among trajectories. Thorough comparisons with well-known methods demonstrate the superiority of the proposed method in terms of precision recall ratios. Copyright 2009 ACM.

Performance Analysis of Anonymous Communication System 3-Mode Net Reviewed

Kazuhiro Kono, Shinnosuke Nakano, Yoshimichi Ito, Noboru Babaguchi

FIFTH INTERNATIONAL CONFERENCE ON INFORMATION ASSURANCE AND SECURITY, VOL 2, PROCEEDINGS 593 - 596 2009

This paper analyzes the performance of 3-Mode Net (3MN), which is a new anonymous communication system proposed in [1]. In particular, we give the probability distributions of the number of relay nodes as well as the number of encryption required for communications. The expectations and variances of these two numbers are also given. These results enable us to grasp the influence of the probabilities of mode selections in 3MN. Numerical examples are also presented to illustrate these results.

Breaking the YASS Algorithm via Pixels and DCT Coefficients Analysis Reviewed

X. Yu, N. Babaguchi

Proc. of International Conference on Pattern Recognition (ICPR2008) 2008.12

Digital Diorama: Adaptive 3D Visualization System for Indoor Environment Reviewed

R. Yamaguchi, Y. Yamamoto, N. Nitta, Y. Ito, N. Babaguchi

Proc. of International Workshop on Sensig Web 17 - 24 2008.12

NHK STRL at TRECVID 2008: High-Level Feature Extraction and Surveillance Event Detection Reviewed

Y. Kawai, M. Takahashi, M. Sano, M. Fujii, M. Shibata, N. Yagi, N. Babaguchi

Proc. of TREC Video Retrieval Evaluation (TRECVID) Workshop 1 358 - 365 2008.11

Mining Temporal Information and Web-Casting Text for Automatic Sports Event Detection Reviewed

M. S. Dao, N. Babaguchi

Proc. of IEEE International Workshop on Multimedia Signal Processing (MMSP2008) 616 - 621 2008.10

Temporal video completion by inserting another video segment Reviewed

Ryota Shoji, Naoko Nitta, Noboru Babaguchi

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers 62 ( 10 ) 1633 - 1640 2008.10

When a scene is recorded with a camcorder, there may be disruptions such as object interference or battery exhaustion resulting in missing segments in the recorded video. A temporal video completion method that inserts a video segment captured by another camera in the place of such a faulty segment is proposed. In the proposed method, a user initially uses an interface to select both faulty and preferred segments from videos captured by two different cameras. To minimize the visual difference at the transition points between the two videos, the most similar frames to those previous to and following the faulty segment are detected around both ends of the preferred segment as the start and the end frames of the inserted segment. Note that, in order to prevent artifacts as much as possible, the length of the inserted segment can be different from the length of the faulty segment. Visually plausible video completion is then achieved by smoothing the motion and color gaps between the transition frames and completing the missing regions by using spatio-temporal features of the video. Experiments were conducted to evaluate the visual plausibility of the completed videos by changing the distance between the camera and the recorded target or between two cameras.

Sensing Web Project - How to handle privacy information in sensor data - Reviewed

M. Minoh, K. Kakusho, N. Babaguchi

Proc. of 12th Intl. Conf. Information Processing and Management of Uncertainty in Knowledge-based Systems(IPMU08) 863 - 869 2008.6

_Privacy Protected Video Surveillance System Using Adaptive visual Abstraction Reviewed

K. Chinomi, N. Nitta, Y. Ito, N. Babaguchi

Proc. of Multimedia Modeling Conference (MMM2008) 144 - 154 2008.1

Isotropy-Based Steganalysis in Multiple Least Significant Bits Reviewed

X. Yu, N. Babaguchi, Y. Wang

_Security and Watermarking of Multimedia Contents X 6819 681913 - 681922 2008.1

A Fast and Effective Method to Detect Multiple Least Significant Bits Steganography Reviewed

Xiaoyi Yu, Noboru Babaguchi

APPLIED COMPUTING 2008, VOLS 1-3 1443 - 1447 2008

In this paper, we propose a fast and effective LSB steganalysis for detecting the existence of hidden message and estimating hidden message length when the embedding is performed using both of two distinct embedding paradigms in one or more than one LSB. The method is based on the analysis of image statistics and quadratic equation. Compared with weighted stego-image based method, Ker's method and Luo et.al.'s method, the detection accuracy is high. Experimental results and theoretical verification show that the proposed method is an effective method of LSB steganalysis.

An improved steganalysis method of LSB matching Reviewed

Xiaoyi Yu, Noboru Babaguchi

2008 FOURTH INTERNATIONAL CONFERENCE ON INTELLIGENT INFORMATION HIDING AND MULTIMEDIA SIGNAL PROCESSING, PROCEEDINGS 557 - 560 2008

In this paper, we propose an improved steganalysis algorithm to detect spatial domain least significant bit (LSB) matching steganography, which is much harder than the detection of LSB replacement. Firstly we propose run length based features, which are sensitive to LSB matching in spatial domain of images. At the same time, we extend Ker's features, the statistical moments, to higher orders, which are sensitive to LSB matching in transform domain. Then the improved method is constructed based on these features. Experimental results on two datasets demonstrate that this method has superior results compared with other recently proposed algorithms, and shows that the proposed method is efficient to detect the LSB matching steganography on compressed or uncompressed images.

A DISCRETE WAVELET TRANSFORM BASED RECOVERABLE IMAGE PROCESSING FOR PRIVACY PROTECTION Reviewed

Guangzhen Li, Yoshimichi Ito, Xiaoyi Yu, Naoko Nitta, Noboru Babaguchi

2008 15TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOLS 1-5 1372 - 1375 2008

This paper presents a novel scheme of a recoverable image processing for privacy protection in real-time video surveillance system. The privacy information is embedded into the video using information hiding. Thus, the original privacy information can be recoverable with secrete key if necessary. In the proposed system, the privacy information is defined as information of objects that consist of detailed data to recover the original image of objects. The scheme is based on discrete wavelet transform (DWT) which is used for generating privacy-protected low resolution image, as well as the high resolution data including privacy information. An amplitude modulo modulation based information hiding scheme is used to hide the privacy information. Experimental results have shown that the proposed system can reduce the amount of the privacy information significantly, and allows the privacy information to be revealed after being embedded in real time.

AUTOMATIC PERSONAL PREFERENCE ACQUISITION FROM TV VIEWER'S BEHAVIORS Reviewed

Makoto Yamamoto, Naoko Nitta, Noboru Babaguchi

2008 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOLS 1-4 1165 - 1168 2008

The demand for information services considering personal preferences is increasing. In this paper, we propose a system for automatically acquiring personal preferences from TV viewer's behaviors. Our system firstly extracts intervals of interest and estimates the interest degree for each extracted interval based on the temporal patterns in facial changes by Hidden Markov Models (HMMs). Then, the viewer profile is created by associating the interest degrees with the content information described in the metadata of the watched program. Experimental results have shown that the proposed methods are able to correctly estimate interest degrees for extracted intervals with a precision rate of 73.1% and a recall rate of 68.8%, and that the created viewer profiles are comparable to the actual preferences of each viewer.

RUN LENGTH BASED STEGANALYSIS FOR LSB MATCHING STEGANOGRAPHY Reviewed

Xiaoyi Yu, Noboru Babaguchi

2008 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOLS 1-4 353 - 356 2008

In this paper, we propose a steganalysis algorithm to detect spatial domain least significant bit (LSB) matching steganography, which is much harder than the detection of LSB replacement. We use the fusion of histogram of run length and histogram characteristic function to detect the LSB Matching. Experimental results on two datasets demonstrate that this method has superior results compared with other recently proposed algorithms, and shows that the proposed method is efficient to detect the LSB matching steganography on compressed or uncompressed images.

Sports event detection using temporal patterns mining and web-casting text Reviewed

Minh-Son Dao, Noburu Babaguchi

MM'08 - Proceedings of the 2008 ACM International Conference on Multimedia, with co-located Symposium and Workshops 33 - 40 2008

Event detection is one of the essential tasks by which the performance of sports video content analysis and access becomes more efficient and effective. Among internal information which are extracted from inside raw videos, the temporal information is critical to convey event meaning. In this paper, the new method for adaptively detecting event based on Allen temporal algebra and external information support is presented. The temporal information is captured by presenting events as the temporal sequences using a lexicon of non-ambiguous temporal patterns. These sequences are then exploited to mine undiscovered sequences with external text information supports by using class associate rules mining technique. By modeling each pattern with "linguistic part" and "perceptual part" those work independently and connect together via "transformer", it is easy to deploy this method to any new domain (e.g baseball, basketball, tennis, etc.) with a few changes in "perceptual part" and "transformer". Thus the proposed method not only can work well in unwell structured environments but also can be able to adapt itself to new domains without the need (or with a few modification) for external re-programming, re-configuring and re-adjusting. Experimental results that are carried on more than 30 hours of soccer video corpus captured at different broadcasters and conditions as well as compared with well-known related methods, demonstrated the efficiency, effectiveness, and robustness of the proposed method in both offline and online processes. Copyright 2008 ACM.

WEIGHTED STEGO-IMAGE BASED STEGANALYSIS IN MULTIPLE LEAST SIGNIFICANT BITS Reviewed

Xiaoyi Yu, Noboru Babaguchi

2008 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOLS 1-4 265 - 268 2008

The paper proposes a method for LSB steganalysis of images, where the secret message is embedded in a given number L of the least significant bits. The proposed estimation method is based on weighted stego-Image and no assumption of cover images is required. The method can detect the presence of stego message using LSB steganography from the case L = 1 to arbitrary L>1. The estimation formula is clean and computation complexity is low. To evaluate the proposed steganalytic method, two experiments of detection and estimation are performed. It is shown that the accuracy of detecting the existence of secret messages in images and of estimating the embedding ratio of secret messages is relatively high.

Automatic prosody labeling using multiple models for Japanese Reviewed

Ryuki Tachibana, Tohru Nagano, Gakuto Kurata, Masafumi Nishimura, Noboru Babaguchi

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E90D ( 11 ) 1805 - 1812 2007.11

Automatic prosody labeling is the task of automatically annotating prosodic labels such as syllable stresses or break indices into speech corpora. Prosody-labeled corpora are important for speech synthesis and automatic speech understanding. However, the subtleness of physical features makes accurate labeling difficult. Since errors in the prosodic labels can lead to incorrect prosody estimation and unnatural synthetic sound, the accuracy of the labels is a key factor for text-to-speech (TTS) systems. In particular, mora accent labels relevant to pitch are very important for Japanese, since Japanese is a pitch-accent language and Japanese people have a particularly keen sense of pitch accents. However, the determination of the mora accents of Japanese is a more difficult task than English stress detection in a way. This is because the context of words changes the mora accents within the word, which is different from English stress where the stress is normally put at the lexical primary stress of a word. In this paper, we propose a method that can accurately determine the prosodic labels of Japanese using both acoustic and linguistic models. A speaker-independent linguistic model provides mora-level knowledge about the possible correct accentuations in Japanese, and contributes to reduction of the required size of the speaker-dependent speech corpus for training the other stochastic models. Our experiments show the effectiveness of the combination of models.

Personalization of Video Contents Reviewed

Noboru Babaguchi

Conversational Informatics: An Engineering Approach 233 - 248 2007.10

Preliminary Experiments toward Automatic Generation of New TTS Voices From Recorded Speech Alone Reviewed

R. Tachibana, T. Nagano, G. Kurata, M. Nishimura, N. Babaguchi

Proc. Interspeech 1917 - 1920 2007.8

Estimating Intervals of Interest during TV Viewing of Specific User for Personal Preference Acquisition(jointly worked) Reviewed

Makoto Yamamoto, Hiroaki Tanimoto, Naoko Nitta, Noboru Babaguchi

The Institute of Electronics, Information and Communication Engineers J90-D ( 8 ) 2202 - 2211 2007.8

Audio-based estimation of speakers directions for multimedia meeting logs Reviewed

Yuki Yokoe, Yoshimichi Ito, Noboru Babaguchi

2007 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOLS 1-5 212 - 215 2007

This paper is concerned with an audio-based method for estimating speaker directions in meeting environment. It is well-known that cross-power spectrum phase (CSP) analysis is a very powerful tool for localizing sound sources. However, when we adopt the CSP-based method together with a circular microphone array system to estimate the speaker directions in 360-degree range (e.g. round-table discussions), the method fails to estimate the directions due to the existence of imaginary peaks of CSP coefficients. In order to circumvent the above problem, we propose a method to suppress the imaginary peaks, which uses a circular-array version of the method proposed by Nishiura and appropriate scaling around the imaginary peaks. Experimental results are also shown to demonstrate the effectiveness of the proposed method.

Determining recording location based on synchronization positions of audio watermarking

Yuta Nakashima, Ryuki Tachibana, Masafumi Nishimura, Noboru Babaguchi

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings 2 ( 2 ) II253 - II256 2007

In this paper, we propose a novel application of digital water-marking, determination of recording locations. This application enables us to determine the seat location in an auditorium, where a recording was made. Precisely measured synchronization positions of the spread-spectrum, watermarks are used for the determination. To avoid use of mismeasured synchronization positions, the algorithm, discards synchronization positions with the corresponding normalized correlation values below a threshold. The experiments with our implementation resulted in accurate determinations
almost all. of the locations can be determined within the error of 0.5 m. These experimental results successfully show the potential applicability of our application. © 2007 IEEE.

Maximum-likelihood estimation of recording position based on audio watermarking Reviewed

Yuta Nakashima, Ryuki Tachibana, Noboru Babaguchi

Proceedings - 3rd International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIHMSP 2007. 2 ( 2 ) 255 - 258 2007

In this paper, we propose a maximum-likelihood method for estimating a recording position inside several loudspeakers as a brand-new application of digital audio watermarking, which could be useful for protecting from illegal recordings of movies. An error model of times of arrival (TOAs) is utilized to estimate the recording position. The error model is constructed based on watermarking strengths, which are calculated by a watermark detection algorithm. This enables us to estimate the position if some of the TOAs are accurately measured. Experimental results indicate that the algorithm can estimate the recording position with the mean value of errors of 0.81 m.

Steganography using sensor noise and linear prediction synthesis filter Reviewed

Xiaoyi Yu, Xinshan Zhu, Noboru Babaguchi

Proceedings - International Conference on Image Processing, ICIP 2 II157 - II160 2007

This paper presents a new approach utilizing the sensor's pattern noise and linear prediction synthesis filter for steganography. The pattern noise is extracted from the images using denoising filter (for example wavelet-based filter). Then the approach introduces the linear prediction synthesis filter, whose parameters are derived from the extracted noise. After being filtered by such a filter, the secret message can be embedded by adapting the characteristics of the sensor's pattern noise. As a result, the embedding process violate little of the natural image statistics, and hence the detectability of steganalytic method is noticeably decreased. The experimental results prove the effectiveness of the new approach. © 2007 IEEE.

Privacy preserving: Hiding a face in a face Reviewed

Xiaoyi Yu, Noboru Babaguchi

COMPUTER VISION - ACCV 2007, PT II, PROCEEDINGS 4844 651 - 661 2007

This paper proposes a detailed framework of privacy preserving techniques in real-time video surveillance systems. In the proposed system, the protected video data can be released in such a way that the identity of any individual contained in video cannot be recognized while the surveillance data remains practically useful, and if the original privacy information is demanded, it can be recoverable with a secrete key. The proposed system attempts to hide a face (real face, privacy information) in a face (new generated face for anonymity). To deal with the huge payload problem of privacy information hiding, an Active Appearance Model (AAM) based privacy information extraction and recovering is proposed in our system. A quantized index modulation based data hiding scheme is used to hide the privacy information. Experimental results have shown that the proposed system can embed the privacy information into video without affecting its visual quality and keep its practical usefulness, at the same time, allows the privacy information to be revealed in a secure and reliable way.

User and device adaptation for sports video content Reviewed

Yoshimasa Takahashi, Naoko Nitta, Noboru Babaguchi

2007 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOLS 1-5 1051 - 1054 2007

Several methods for summarizing sports videos by selecting only important scenes based upon their semantic content have been proposed so far. However, there remain two problems: 1)what is important depends on each user's preferences, and 2)the summaries should be tailored for media devices that each user has. To solve these problems, we discuss user and device adaptation for sports video summarization. The proposed framework dynamically adapts the video content to fit user's preferences using user profiles which describe the preference degrees for keywords. The video content is then presented through proper media such as image or text according to the confinement of media devices. For sports videos, the framework is tested using PCs and mobile phones as the media devices.

Video completion with spatio-temporal features for video caption removal

Ryota Shoji, Naoko Nitta, Noboru Babaguchi

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers 61 ( 1 ) 91 - 97 2007

We propose a video completion method for automatically removing unnecessary objects from video. Using this method, a user selects an object from a frame on our interface, and the corresponding objects are extracted from all frames of the video. Based on estimated camera/object motions, visually plausible video completion with spatio-temporal features is achieved by repeatedly using the temporal correlation between adjacent frames and the spatial correlation between adjacent pixels in a frame. Experiments were conducted that targeted and removed video captions to evaluate processing time and visual plausibility.

Analysis of Audio-Visual Synchronous Patterns in Edited Videos - Towards an Aid for Attractive Video Editing - Reviewed

N. Nitta, N. Babaguchi

Proc. the 1st Korea-Japan Joint Workshop on Pattern Recognition(KJPR) 2006.11

Estimating Intervals of Interest During TV Viewing for Automatic Personal Preference Acquisition Reviewed

M. Yamamoto, N. Nitta, N. Babaguchi

Proc. of the 7th IEEE Pacific-Rim Conference on Multimedia(PCM2006) 615 - 623 2006.11

Estimation of recording location using audio watermarking Reviewed

Y. Nakashima, R. Tachibana, M. Nishimura, N. Babaguchi

Proc. of ACM Multimedia and Security Workshop 2006 108 - 113 2006.9

On Sensitivity Reduction Problems of Sampled-Data Systems - Performance Limitations and the Properties of Aliasing Factors - Reviewed

Y. Ito, H. Shirahama, N. Babaguchi

Proc. of International Symposium on the Mathematical Theory of Networks and Systems (MTNS2006) 1788 - 1792 2006.7

Personal identification in unconscious sensing by video and GPS positioning Reviewed

S. Nishio, N. Babaguchi, T. Akimoto, N. Hagita

Proc. of 2nd Korea-Japan Joint Symposium on Network Robot Systems 19 - 24 2006.6

Video Scene Retrieval with Symbol Sequence Based on Integrated Audio and Visual Features Reviewed

K.Morisawa, N.Nitta, N.Babaguchi

Proc. of SPIE-IS&T Electronic Imaging 6073 607307-1 - 607307-10 2006.1

Factors on the sense of privacy in video surveillance Reviewed

Takashi Koshimizu, Tomoji Toriyama, Noboru Babaguchi

Proceedings of the ACM International Multimedia Conference and Exhibition 35 - 44 2006

With the increasing demand for greater security, video surveillance technologies have recently received a lot of attention. As video surveillance cameras become ubiquitous, there are growing concerns over the cost of monitoring these systems and the possible invasion of privacy. In this paper, we discuss a system architecture of privacy-preserving video surveillance for a community that achieves a good balance between security and privacy. We call the designed system 'PriSurv'. A subset of PriSurv is implemented, and privacy protected image processing modules are installed. The privacy preserving image processing is called visual abstraction. We conducted an experiment using five abstracted images and a questionnaire to evaluate PriSurv. Analyzing the results of the experiment using factor analysis, we obtained seven factors. With a hierarchical cluster analysis, we divided the subjects into three clusters. A significant difference in the choice of images among the clusters was found using the Pearson -squared test. From this analysis, we show that the relationships between subjects and monitors affect the subjects' sense of privacy
furthermore, the subjects' sense of privacy depends on the individual person. The results of analysis supports PriSurv's design principle, which can adapt the personal sense of privacy. Copyright 2006 ACM.

TV viewing interval estimation for personal preference acquisition Reviewed

Hiroaki Tanimoto, Naoko Nitta, Noboru Babaguchi

2006 IEEE International Conference on Multimedia and Expo, ICME 2006 - Proceedings 2006 889 - 892 2006

The importance of personalized information services has been increasing. Description of personal preferences needs to be prepared beforehand to realize such services. We propose a system for automatically acquiring personal preferences from TV viewer's behaviors. Considering "when" a viewer is watching TV is highly related to the viewer's preferences, we focus on estimating the time interval during which a pre-registered viewer is watching TV. In this paper, we firstly describe the outline of the personal preference acquisition system, and address a method for estimating the TV viewing intervals based on the appearance of frontal faces. Experiments resulted in a precision rate of 97.1% and a recall rate of 70.6% on average for TV viewing interval estimation. © 2006 IEEE.

MediaTray: The Cafeteria-style Viewing Environment for Digital Contents and Its Evaluation

K.Tanaka, T.Sasaki, Y.Tonomura, T.Nakanishi, N.Babaguchi

Proceedings of 11th International Conference on Human-Computer Interaction (HCII2005) 2005.7

Meeting Recording System via Multimodal Sensing

S. Tokunaga, Y. Ito, N. Nitta, N. Babaguchi

Proc. of Japanese Society for Artificial Inteligence 2005, Workshop on Conversational Informatics 19 - 24 2005.6

[Panel Discussion] Mobile Multimedia Services Reviewed

Behzad Shahraray, Wei-Ying Ma, Avideh Zakhor, Noboru Babaguchi

Proc. of WWW2005 Special Interest Tracks and Posters 795 2005.5

Generating semantic descriptions of broadcasted sports videos based on structures of sports games and TV programs Reviewed

N Nitta, N Babaguchi, T Kitahashi

MULTIMEDIA TOOLS AND APPLICATIONS 25 ( 1 ) 59 - 83 2005.1

This paper presents a model to represent a broadcasted sports video in a semantic way and proposes a method of automatically generating semantic descriptions of significant scenes. Representation of a video should clarify the semantic content of the video as accurately as possible. Our model structurizes the video and specifies suitable semantic descriptions for video segments paying attention to the structure of both a sports game and a sports TV program. As the elements of these semantic descriptions, the proposed method tries to obtain the information about the plays and their related players from the closed-caption stream by searching key phrases. Finding the corresponding segments of the video by means of template matching for the image stream attaches these textual descriptions to the proper portion of the video. In this paper, we discuss some experimental results of our method and the potentiality for integrating these results into the standardized MPEG-7 description tools.

Automatic parsing of American football videos by intermodal collaboration based on transition rules Reviewed

Naoko Nitta, Noboru Babaguchi

IEEE International Conference on Multimedia and Expo, ICME 2005 2005 1114 - 1117 2005

This paper proposes an automatic American football video parsing method based on transition rules of an American football game. Combining the results of live scene extraction and superimposed text detection based on image features enables us to segment the video into play units of a game. Temporally associating the segmented play units with the detected superimposed texts and the closed-caption text attaches possible semantic content information to the play units. Finallly, selecting only the play units which comform to transition rules of the sports game from the obtained play unit sequence, while discarding or complementing unnecessary or insufficient play units and attached semantic content information, realizes the semantic video parsing. © 2005 IEEE.

Interactive clustering of video segments formedia structuring Reviewed

Y. Kinoshita, N. Nitta, N. Babaguchi

IEEE International Conference on Multimedia and Expo, ICME 2005 2005 630 - 633 2005

Structuring video data is necessary for its effective retrieval and summarization. In particular, collecting similar scenes from semantic aspects highly contributes to the structuring. In this paper, we propose a method of clustering the scenes with relevance feedback, which may be able to bridge the gap between the video data and its semantics. First, spatio-temporal video segments of a fixed length are clustered according to image features of each segment. Then, a user performs feedback to the results of clustering, whether each segment is relevant to the cluster it belongs to. The clustering accuracy can be improved through the interaction based on the feedback information. For diverse kinds of video streams, we investigated how the feedback should be given and demonstrated the effectiveness of the interactive clustering. © 2005 IEEE.

Playwatch: Chart-style video playback interface

Kiyoshi Tanaka, Tsutomu Sasaki, Yoshinobu Tonomura, Tadashi Nakanishi, Noboru Babaguchi

IEEE International Conference on Multimedia and Expo, ICME 2005 2005 731 - 734 2005

This paper proposes the chart-style video playback interface Play Watch
it displays a chart of semantic indices for locating video scenes. The main features of Play Watch are: 1) the user understands the distribution of the scenes because PlayWatch shows the indices in order. 2) The user can access a desired scene directly through the indices since they also act as link buttons. This paper also describes the evaluation of PlayWatch. Experiments on scene searching show that PlayWatch is effective in accessing precisely indexed scenes. ©2005 IEEE.

Video summarization for large sports video archives Reviewed

Yoshimasa Takahashi, Naoko Nitta, Noboru Babaguchi

IEEE International Conference on Multimedia and Expo, ICME 2005 2005 1170 - 1173 2005

Video summarization is defined as creating a shorter video clip or a video poster which includes only the important scenes in the original video streams. In this paper, we propose two methods of generating a summary of arbitrary length for large sports video archives. One is to create a concise video clip by temporally compressing the amount of the video data. The other is to provide a video poster by spatially presenting the image keyframes which together represent the whole video content. Our methods deal with the metadata which has semantic descriptions of video content. Summaries are created according to the significance of each video segment which is normalized in order to handle large sports video archives. We experimentally verified the effectiveness of our methods by comparing the results with man-made video summaries. © 2005 IEEE.

Automatic Video Summarization of Sports Videos Using Metadata Reviewed

Y.Takahashi, N.Nitta, N.Babaguchi

Proc. of Fifth IEEE Pacific-Rim Conference on Multimedia (PCM2004) 272 - 280 2004.12

Clustering of Video Packets Using Interactive Refinement by Relevance Feedback

Y.Kinoshita, N.Nitta, N. Babaguchi

Proc. of Fifth IEEE Pacific-Rim Conference on Multimedia (PCM2004) 626 - 633 2004.12

Video Scene Retrieval with Sign Sequence Matching Based on Audio Features Reviewed

K. Morisawa, N. Nitta, N. Babaguchi

Proc. of Fifth IEEE Pacific-Rim Conference on Multimedia (PCM2004) 121 - 129 2004.12

Embedding MPEG-7 Description in MPEG Video Data by Focusing on DCT-coefficients and Motion Vectors Reviewed

S. Taniguchi, N.Nitta, N.Babaguchi

Proc. of Pacific Rim Workshop on Digital Steganography 2004 (STEG'04) 80 - 88 2004.11

Personalized abstraction of broadcasted American football video by highlight selection Reviewed

N Babaguchi, Y Kawai, T Ogura, T Kitahashi

IEEE TRANSACTIONS ON MULTIMEDIA 6 ( 4 ) 575 - 586 2004.8

Video abstraction is defined as creating shorter video clips or video posters from an original video stream. In this paper, we propose a method of generating a personalized abstract of broadcasted American football video. We first detect significant events in the video stream by matching textual overlays appearing in an image frame with the descriptions of gamestats in which highlights of the game are described. Then, we select highlight shots which should be included in the video abstract from those detected events reflecting on their significance degree and personal preferences, and generate a video clip by connecting the shots augmented with related audio and text. An hour-length video can be compressed into a minute-length personalized abstract. We experimentally verified the effectiveness of this method by comparing man-made video abstracts.

LMI-based stability condition for 2-D discrete systems described by the Fornasini-Marchesini second model Reviewed

Y Ito, W Date, N Babaguchi

2004 47TH MIDWEST SYMPOSIUM ON CIRCUITS AND SYSTEMS, VOL II, CONFERENCE PROCEEDINGS 557 - 560 2004

This paper presents a stability criterion for 2-D discrete systems described by the Fornasini-Marchesini second model. The method presented in this paper is based on linear matrix inequalities (LMI), and hence, it is computationally tractable. In deriving the method, finite-order Fourier series approximation of the solution for frequency-dependent LMI (FDLMI), and the properties of quadratic form representation of finite-order Fourier series play key roles. From the view point of the proposed method, the existing LMI-based condition can be regarded as the one which is obtained by Fourier series approximation of order zero, and thus, it is expected that the proposed method leads to less conservative results. This is illustrated by a numerical example.

Scene retrieval with sign sequence matching based on video and audio features Reviewed

N Babaguchi, T Ishida, K Morisawa

2004 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXP (ICME), VOLS 1-3 2 1107 - 1110 2004

This paper presents a method of retrieving similar scenes from video streams with sign sequence matching. The visual and auditory streams are partitioned into fixed-length video/audio packets. Based on feature vectors extracted from these packets, sign sequences are formed. The sign sequence can be viewed as abstraction of video and audio features. DP Matching between the target and query sign sequences allows us to find scenes similar to the query in the video stream. For the purpose of efficient processing, packets, histogram based features, and sign sequences, which are the key ideas in this method, are introduced. The preliminary experimental results show that this method is promising for quick retrieval for similar scenes.

Motion estimation and detection of complex object by analyzing resampled movements of parts Reviewed

P Piamsa-nga, N Babaguchi

ICIP: 2004 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOLS 1- 5 365 - 368 2004

A moving object that has many complex moving parts is very hard to detect and its motion is not easy to estimate. In this paper, we present a new technique for motion estimation and detection of moving complex objects by analyzing the resampled motions of the parts of objects. The Kalman filter is used to track all resampled movements and the tracked routes are classified into groups that share the same fundamental movements. Our simulation show that recall of motion estimation and detection is approximately 0.8, while the computation drops exponentially.

Fault-Tolerant Control Using Time-Sharing Multirate Controllers Reviewed

H. Kawahara, Y. Ito, N. Babaguchi

Proc. of the 1st International Symposium on Systems & Human Science - For Safety, Security, and Dependability 178 - 183 2003.11

Intermodal Collaboration: A Strategy for Semantic Content Analysis for Broadcasted Sports Video Reviewed

N. Babaguchi, N. Nitta

Proc. of 2003 IEEE International Conference on Image Processing (ICIP2003) 2003.9

Effect of personalization on retrieval and summarization of sports video

Noboru Babaguchi, Kouzou Ohara, Takehiro Ogura

ICICS-PCM 2003 - Proceedings of the 2003 Joint Conference of the 4th International Conference on Information, Communications and Signal Processing and 4th Pacific-Rim Conference on Multimedia 2 ( 2 ) 940 - 944 2003

Personalization is one of the most important mechanisms to make multimedia systems easy to use. In video applications, its embodiment is to tailor video contents for a particular viewer. For this purpose, we are now developing a system of retrieving and browsing video segments, called video portal with personalization (VIPP). VTPP is characterized by 1) supporting the viewer's access to video contents and making a summarized video clip by taking his/her preference into account, and 2) acquiring the viewer's profile from his/her operations automatically. In this paper, we discuss the effect of personalization on retrieval and summarization of sports videos on VIPP.

On personalizing video portal system with metadata Reviewed

K Ohara, T Ogura, N Babaguchi

KNOWLEDGE-BASED INTELLIGNET INFORMATION AND ENGINEERING SYSTEMS, PT 2, PROCEEDINGS 2774 1062 - 1069 2003

Recently, the necessity for a system which supports an access to videos has been increasing. In addition, it is strongly desired to take a user's preference into account to realize a more flexible system. For this background, we propose a system which can automatically acquire the user's preference by monitoring the actions by the user, as well as which can provide the functions such as efficient retrieval and browsing of video segments, by taking advantage of their metadata. Furthermore we experimentally show its usefulness.

Towards abstracting sports video by highlights Reviewed

N Babaguchi

2000 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, PROCEEDINGS VOLS I-III 1519 - 1522 2000

Recently, video abstraction has become a demanding application in multimedia computing. It is defined as creating shorter video clips or video posters from the original video stream. In this paper, we present a basic approach towards abstracting sports video by highlights, dealing with American football games, Using event based indexing, we create an abstracted video clip automatically. To select the appropriate highlights of the game, an impact factor reflecting on the importance of the event is newly introduced. It was possible to make an about 5-minute clip from the 3-hour original video.

Detecting events from continuous media by intermodal collaboration and knowledge use Reviewed

N Babaguchi, S Sasamori, T Kitahashi, R Jain

IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA COMPUTING AND SYSTEMS, PROCEEDINGS VOL 1 782 - 786 1999

In this paper, we propose an event network, which is a structured representation oriented for the contents of continuous media, as well as present two methods of detecting events as the first step to construct the network. We here deal with sports TV programs, considering American football as a case study. The first method is simple intermodal collaboration: linking between visual and linguistic (closed caption) streams. Using domain knowledge about state transitions of football games, the second method attempts to extract specific visual objects including the information about contents. The experimental results indicate that the both methods are effective for event detection.

Solving contradiction in knowledge-base without interaction Reviewed

K Katsurada, M Koyama, K Ohara, N Babaguchi, T Kitahashi

1998 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS, VOLS 1-5 1546 - 1551 1998

We propose a non-interactive method for solving the contradictions caused by exceptions to ordinary rules. It is realized by 1) detecting the ordinary rules which have the instances in their bodies, and 2) converting them into the default; rules. To reduce the default rules which need much time in reasoning, we convert the minimal sets of ordinary rules to solve contradictions. Since the proposed method is executed without interaction with human, it contributes to automatic rule-base maintenance.

Generation of sketch map image and its instructions to support the understanding of geographical information Reviewed

Noboru Babaguchi, Seiichiro Dan, Tadahiro Kitahashi

Proceedings - International Conference on Pattern Recognition 3 274 - 278 1996

In this paper, we propose a method of generating a sketch map drawing and its instructions in order to support the route understanding of a human. Both are given from a structured data, called the road network, which is a graph augmented by the attributes of roads and crossings. In generating a sketch map drawing, the pictorial information about the shape and the direction of roads is modified. The sketch map drawing can be generated at any simplification level by controlling a parameter called simpleness. The instructions corresponding to the sketch map are produced by assigning appropriate terms into sentence templates. We verified that favorable results are obtained by this method. © 1996 IEEE.

REPRESENTING, UTILIZING AND ACQUIRING KNOWLEDGE FOR DOCUMENT IMAGE UNDERSTANDING Reviewed

K KISE, N BABAGUCHI

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E77D ( 7 ) 770 - 777 1994.7

This paper discusses the role of knowledge in document image understanding from the viewpoints of representation, utilization and acquisition. For the representation of knowledge, we propose two models, a layout model and a content model, which represent knowledge about the layout structure and content of a document, respectively. For the utilization of knowledge, we implement layout analysis and content analysis which utilize a layout model and a content model, respectively. The strategy of hypothesis generation and verification is introduced in order to integrate these two kinds of analysis. For the acquisition of knowledge, we propose a method of incremental acquisition of a layout model from a stream of example documents. From the experimental results of document image understanding and knowledge acquisition using 50 samples of visiting cards, we verified the effectiveness of the proposed method.

GENERATION OF SKETCH MAP DRAWING FROM VECTORIZED IMAGE Reviewed

N BABAGUCHI, K TANAKA, T KITAHASHI

ICIP-94 - PROCEEDINGS, VOL III 207 - 211 1994

Curvedness of a line picture Reviewed

Noboru Babaguchi, Tsunehiro Aibara

Pattern Recognition 20 ( 3 ) 273 - 280 1987

The shape analysis of a binary picture is of great importance for pictorial pattern recognition. In this paper, we propose a useful geometric feature parameter called curvedness. Curvedness represents which lines are dominant in a binary picture, straight or curved. Our algorithm for measuring curvedness is based on the relationship between the distance to a boundary point on each black point along each quantized direction and the mean width of a binary picture. We investigate the fundamental property of curvedness. The experimental results for Japanese Hiragana and Kanji characters show the validity of curvedness. © 1987.

　 PREV - NEXT 　

<Noboru Babaguchi>

Papers - Noboru Babaguchi