A Scheduling System Coupled with a Replica Management System for Data-intensive Applications

• MACHIDA YUYA

  Tokyo Institute of Technology

• TAKIZAWA SHIN'ICHIRO

  Tokyo Institute of Technology

• NAKADA HIDEMOTO

  National Institute of Advanced Industrial Science and Technology

• MATSUOKA SATOSHI

  Tokyo Institute of Technology

Bibliographic Information

Other Title

• レプリカ管理システムを利用したデータインテンシブアプリケーション向けスケジューリングシステム

Abstract

Existing scheduling systems for the Grid mostly handle huge I/O via a shared file system or simple staging. However, when numerous nodes access a single I/O node simultaneously, major performance degradation occurs, or in a worst case, causes I/O nodes to hang. Moreover, when a user launches a job consisting of hundreds or even thousands of tasks which share the same data set, it becomes extremely inefficient to stage essentially the same data set to each compute node after every dynamic brokering and allocation of the compute nodes. So we propose to tightly couple replica management and computation scheduling in order to reuse already replicated data effectively. We implemented a prototype system which uses a replica management system that embodies a scalable multi-replication framework, where multiple copies could be made in O(1) transfer time, and enables scheduling computation and data trasfer to single node simultaneously. The evaluation result shows our proposed technique performs superior to the traditional techniques and improves the throughput.

Journal

• IPSJ SIG Notes

  IPSJ SIG Notes 167 229-234, 2006-02-27

  Information Processing Society of Japan (IPSJ)

Details

• CRID: 1571135651934700800
• NII Article ID: 110004710289
• NII Book ID: AN10096105
• ISSN: 09196072
• Text Lang: ja
• Data Source:
  • CiNii Articles