[QueueNews] Storage Virtualization Gets Smart

QueueNews queuenews at acmqueue.com
Mon Nov 26 08:00:02 PST 2007 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
. . . . . . . . . . . . . . . . . . . . . . . .
   Queue E-Mail Newsletter
   for the Week of Nov/26/2007
. . . . . . . . . . . . . . . . . . . . . . . .
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-------------------------------------------------
      Sponsored by

      Oracle
      ACM

/----------------------------------------------------------\

Embedded Technologies

Oracle Database and Middleware in an Embeddable Package
http://www.oracle.com/go/?&Src=5951533&Act=13

\----------------------------------------advertisement------/


Latest Articles:


Storage Virtualization Gets Smart
Can a new data-management framework unlock the full potential of
storage
virtualization? The days of overprovisioned, underutilized storage
resources might soon become a thing of the past.
http://acmqueue.com/rd.php?c.509
   (scroll down to read an excerpt from this article)


The Next Big Thing
This month Kode Vicious considers the future of functional programming
and then offers up his top five protocol-design tips.
http://acmqueue.com/rd.php?c.508

/----------------------------------------------------------\

Embedded Technologies

Oracle Database and Middleware in an Embeddable Package
http://www.oracle.com/go/?&Src=5951533&Act=13

\----------------------------------------advertisement------/

New article on ACM Queue:
Storage Virtualization Gets Smart
http://acmqueue.com/rd.php?c.509
Using the infrastructure to manage the data becomes problematic as the
infrastructure itself changes over time.
by Kostadis Roussos, Network Appliance

>From the File Systems and Storage issue, vol. 5, no. 6 - September /
October 200
7

article excerpt:
Over the past 20 years we have seen the transformation of storage
from a dumb resource with fixed reliability, performance, and capacity
to a much smarter resource that can actually play a role in how data is
managed. In spite of the increasing capabilities of storage systems,
however, traditional storage management models have made it hard to
leverage these data management capabilities effectively. The net result
has been overprovisioning and underutilization. In short, although the
promise was that smart shared storage would simplify data management,
the reality has been different.

To address the real challenges
of data management and the shortcomings of traditional storage
management, we propose a new data management framework based on three
observations:

  * Data management is best performed by the
   owner of the data - the data administrator.
  * The administrator
   of the physical storage must be able to control what the data
   administrator can do.
  * Enabling the data administrator to
   control the data requires tools that abstract out the details of
   the
   storage infrastructure, because without such tools we are asking
   data
   administrators to manage storage infrastructure.

 At
 NetApp we have built such a framework and have found that it
 dramatically simplifies management of storage infrastructure by
 presenting storage management in the context of the data management
 tasks that the data administrator wants to perform.

 In this
 article we point out the business and data management trends that led
 us
 to reject traditional models for data management. We then describe the
 data management framework that we built and the first products to
 leverage that framework.

 Cost at the right service level

 We have seen a move to more shared infrastructure in an effort to
 reduce cost. That trend, along with the changing needs of data
 management, ultimately affects how data and storage management is
 actually done.

 A major IT challenge is how to provision
 functional services such as e-mail with the right data service
attached
 to that functional service at the right cost. Or to put it
differently,
 e-mail has to have the right number of copies, the CEO's e-mail must
 not
 be deleted, and the e-mail service has to tolerate and recover from a
 variety of failures without breaking the budget. Failure to do so will
 result in downtime - and that downtime, depending on the day of the
 month, can affect business.

 Dedicating physical resources to
 each functional service is fundamentally an unsustainable model. The
 cost in terms of equipment and physical space makes that impossible.
 Furthermore, existing equipment is underutilized, so the business
tries
 to get more out of it before buying new equipment.

 Leveraging
 shared resources requires some form of virtualization, leading to the
 adoption of server and storage virtualization. Server virtualization
is
 used to consolidate many underutilized physical servers onto a smaller
 number of physical servers. Storage virtualization is used to create
 storage containers that are either bigger or smaller than the
 underlying
 physical disks, thereby improving disk utilization. Like server
 virtualization, storage virtualization can do more than just improve
 utilization. Storage virtualization can be the basis for transparent
 data migration, data replication, thin provisioning, and space- and
 time-efficient backups (called snapshots).^1 Unlike server
 virtualization, the full promise of storage virtualization has not yet
 been realized because of two distinct challenges: the limitations of
 the
 underlying technology and the complex administrative handshake
required
 to leverage the features of the virtualization.

 As an example of
 the impact of these challenges, consider the e-mail application that
 requires backups. The business would like to have backups done once an
 hour. The reality is that unless those backups are done in a space-
and
 time-efficient manner, the cost is prohibitive. The storage architect,
 who has already virtualized the storage, observes that the storage
 system provides a space-efficient backup scheme that meets the
 application requirement. These backups (snapshots), which take a few
 minutes to complete, can be performed once an hour. They do not affect
 the application performance and consume only the amount of space that
 has actually changed. The storage architect therefore recommends to
the
 business and application administrators that they use these
 snapshots.

 The storage architect now runs into two distinct
 problems. The first is that the snapshots are not free because the
 storage system may have systemwide limits on how many snapshots can be
 taken. The storage architect therefore has to ensure that no single
 application administrator takes too many snapshots. The second problem
 is that application administrators typically want
 application-consistent
 snapshots, requiring coordination across the application, server, and
 storage. Coordination in practice requires coordination across
multiple
 administrative groups that understand in detail the complexity of
their
 infrastructure.

 Suppose the business can survive with backups
 performed every 12 hours and is willing to tolerate some degradation
in
 performance during those backups. Rather than deal with the complex
 coordination effort, the application administrator may be able to
 convince the business to buy more storage and use the less
 space-efficient application-based backup. This type of backup may be
 less space efficient because it is a full copy of the data followed by
 a
 compression step. With application-based backup, there is no
 coordination effort to limit the number of snapshots, nor is there any
 need for complex negotiation between the storage, server, and
 application groups. The complexity of the administration of the
storage
 infrastructure makes it worthwhile to trade that off against the
 simplicity of a less efficient mechanism. As a result of this
tradeoff,
 the application is overprovisioned in terms of capacity and
 underprovisioned in terms of data protection.

 Without effective
 storage virtualization and the ability to leverage that storage
 virtualization, we have the worst of all possible worlds. We have
 increasingly capable infrastructure that requires increasingly more
 sophisticated administrators, making it increasingly difficult to
 leverage the capabilities of that infrastructure. To compensate for
 this
 administrative complexity, we buy more storage. This, of course,
 creates
 a different challenge of how we use what we have more efficiently.
 Which
 brings us back to wanting to use virtualization...
 Read the rest of this article at acmqueue.com
 http://acmqueue.com/rd.php?c.509
 /----------------------------------------------------------\
 Join ACM A Special Offer to Join ACM for Queue Readers
 http://www.acm.org/joinacm2
 \----------------------------------------advertisement------/
 See all the latest articles and audio interviews with Queue's RSS
Feeds
 http://acmqueue.com/rd.php?s.48.7 To unsubscribe to this newsletter,
 send an email to queuenews-request at acmqueue.com with the words
 'unsubscribe' in the subject line. Change your email address
 http://www.acmqueue.com/mailman/options/queuenews Subscribe to Queue
in
 print http://www.acmqueue.com/click.php?id=30 About Queue
 http://acmqueue.com/rd.php?s.31.5 Contact Us
 http://acmqueue.com/rd.php?s.32.10 Privacy policy
 http://acmqueue.com/rd.php?s.27.9 For advertising information, contact
 advertising at acmqueue.com © 2007 ACM, Inc. All rights reserved.

More information about the QueueNews mailing list