| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

The Grid

Page history last edited by Dawn Quinn 14 years, 7 months ago

The Grid

by D. Quinn.

 


 

Figure 1 - grid

 

Historical Background

 

A Brief History

 

Just like the Web, the idea of the grid came from the scientific community, specifically those scientific communities that require high-end computing power to manage and analyze huge amounts of scientific data. Often, these science projects, referred to as e-Science, involve global collaboration with multi-disciplinary teams needing access to similar resources. "The word "grid" was borrowed from the electricity grid to imply that any compatible device could be plugged in anywhere on the Grid and be guaranteed a certain level of resources, regardless of where those resources might originate" (Heingartner, 2001). "The term "the Grid" was coined in the mid1990s to denote a proposed distributed computing infrastructure for advanced science and engineering" (Noel, 2006).  "Grid computing can trace its intellectual roots back to the operating system Multics, but its immediate ancestor is metacomputing" (GridCafe, 2008).

 

"Grid computing is a way of connecting computing resources to share their computing power" (Burne, 2007). Simply, grid computing is a service providing access to computing power from multiple locations. A grid is well suited for use within research groups; it has enhanced collaboration: The majority of research work is carried out through joint collaboration projects between research institutes.  It's common to work across sectoral boundaries (CSC, 2008), and researchers with advanced expertise work and collaborate through international grids.

 

More specifically, the grid is a concept. What drives the concept of grids is the need for supercomputing capabilities. A primary goal of the grid is to enable and simplify access to supercomputing capabilities and computational resources (Southeastern, 2007).  In fact, computational resources, specifically high performance systems, are often the first type of resource one thinks of at the mention of a grid.

 

Figure 2 - iVDgL Project map

 

How it works

 

Grid developers spend their time working on five big areas: resource sharing, secure access, resource use, the death of distance, and open standards (GridCafe, 2008).

 

  1. Resource sharing: Grid resources are owned by many different people who run different software, exist within different administrative domains, and use different systems for security and access (GridCafe, 2008). The big idea behind grid research is resource sharing.  However, multiple ownerships present challenges:  Would you share your home with a complete stranger? Would the University of Texas Brownsville share it's network clusters with Salish Kootenai College? The better question is could they?
  2. Secure access: Issues of trust underlie the issue of security. It's one thing to trust the owner of a grid resource. It's another thing to trust the grid owner with your resources.
  3. Resource use: Potentially, there are thousands of global users waiting to process jobs. Computing grids rely on middleware to allocate work efficiently and automatically, spreading users' many "jobs" among many resources, finishing them much faster (GridCafe, 2008).
  4. The death of distance:  Assuming no delays or interruptions in job processing, distance between grid resources would not be an issue. But we know that distance between resources is an issue. Currently, we experience dropped calls with cellular services. What would happen if you dropped a process in the middle of a job? In grid distance, we are referring to real-time processing. This means that high speed networking must be just that: high speed without delays or interruptions. Image that!
  5. Open standards:  A standard that is publicly available and has various rights to use associated with them (Open standard, 2008). Basically, open standards suggests greater rights to some constituencies. Which standards should be used for grid computing (GridCafe, 2008)? Developing open standards on a grid scale is challenging.

 

 

Figure 3

A two-dimensional representation of a three-dimensional holograph of water.

Image courtesy of Peter Hobson, Brunel University

www.isgtw.org/

 

What can it do?

 

Save money, make money; that's what the grid can do. Searching for images on the internet can be time consuming and a very tedious effort. There is no single grid; there are many grids, especially commercial grids.  Many companies are working on hardware and software developments to advance digital content search software using grid technology. One example is Imense Ltd, of Cambridge, England. Imense Ltd. is focused on digital search technology. MySpace, YouTube, Yahoo, Flickr and other digital search engines provide access to digital content. Using grid architecture, Immense Ltd. grid system application provides an automated way of making visual content searchable by content without any need for manual tagging while still retaining the ability to incorporate contextual information. Unlike other image search solutions, this system does not rely on image annotations or metadata, and does not require an initial example image or sketch (Venton, 2008).

 

Another example is Avanade Italy. a strategic technology consulting firm committed to helping businesses meet their technology objectives, helping reduce the time, cost, and risk associated with deploying high-impact business solutions (Avanade, 2000).  Avanade grid computing provides the power required to run complex financial simulation applications.

 

Figure 4 Global Power Access

 

How are Grids different from the Internet?

 

The Internet gives users access to content. Grids give users access to power. It's important to understand that there is no single grid; there are many grids. Think of the grid as a computing utility, much like an electrical grid. IBM's Brian Carpenter suggested "computing will become a utility just like any other utility." (Heingartner, 2001)

 

"This distributed computing environment operates as a uniform service, which looks after resource management and security independently of individual technology choices. In other words, distributed computing is treated like a public utility comparable to the electricity grid" (Carpenter, 2006).

 

Like an electrical grid, users connect to grids and use as much computing power as needed. It's seamless; users don't need to know where the power comes from or how it was produced (Burne, 2007).

 

Figure 5 Electrical grid

physicsnow.net/tip/INPHFA/vol-9/iss-5/p8.html 

 

Why do Grids matter?

 

Think about it. Grids provide combined resources of hundreds of computers to create a very powerful and fully comprehensive computing resource accessible from the comfort a single personal computer. What does this mean for the research communities? It means that very complex questions, like global warming and Alzheimer research, can be solved much faster. Grids also speed up research processes. A simulation process, like a natural disaster simulation which would normally take weeks, may only take a few hours.  For example, grid user researcher A needs to process a job needing 500 nodes for 10 hours and grid user researcher B needs to process a job needing 800 nodes for 5 hours.  The grid's vast resources can do both of these jobs simultaneously.

 

Figure 6 User Needs Processing Power

 

Who will use the Grid?

 

Eventually?  Everyone.  The housewife will hit the grid button in the kitchen to see the latest research news.  The engineer will start a complex calculation in his bedroom before going to the office so that it's done before he arrives at his lab.  The senior citizen will check his portfolio while vacationing in China.  The small town owner of a used bookstore will connect to Amazon's grid to sell his used books.  The retired farmer making snow sleds will connect his system to eBay's grid to market them around the world.  Everyone will use grids.  

 

Figure 7 Nanyan Technological University Campus Grid

 

Relevance to Education

 

The Grid and Education

 

  1. GUGrid - ARC Georgetown University Grid Initiative:  Georgetown University's Advanced Research Computing (ARC) team provides computation resources to researchers via the GUGrid initiative (Advanced Research Computing, 2006).

  2. GLOW: Grid Laboratory of Wisconsin"The Grid Laboratory of Wisconsin is a campus-wide distributed computing environment designed to meet the scientific computing needs of the University of Wisconsin-Madison (GLOW, 2003).

  3. University of Florida High-Performance Computing Center:  "The campus research grid is a network of different computing clusters around campus to enable the sharing of computational resources, presenting them as a single resource for solving large computational and data intensive problems" (University of Florida).

  4. University of Iowa Grid Research and educatiOn group @ ioWa (GROW): "The vision for GROW is to advance cyberinfrastructure-enabled computational science and engineering by broadening and strengthening connections between Grid and high performance computing community and domain sciences and engineering communities" (Grid Research and educatiOn group @ ioWa, 2006).

  5. The University of Houston Texas Learning and Computational Center (TLC2):  "The Texas Learning and Computation Center (TLC2) fosters and supports interdisciplinary research, education and training in computational sciences and engineering through centers, laboratories and individual faculty initiatives" (University of Houston).

  6. The Green Grid Project @ Dartmouth:  "The Green Grid will provide a high throughput resource for scientific and technical computing applications at Dartmouth" (Dartmouth University, 2006).

 

Figure 8 Workout In The Grid

 

The Grid and You

 

Grids offer students flexibility.  The average time to complete a doctorate is over four years.  What if you could complete all your research in 2 months and complete your doctorate studies in two years?  And what if you take a new job in Australia and need to put your doctorate on hold?  You just put your work on hold in the grid.  The possibilities are mind-boggling.

 

Glossary of Terms

Grid computing:

a) An expandable, scalable set of resources applied to solve a single or a set of problems - usually a scientific or technical problem that requires a large number of computer processing cycles.

b) The use of pools of resources onto which applications or services may be dynamically provisioned and re-provisioned to meet the goals of one or more enterprises, whilst improving both efficiency and agility.

c) Grid computing allows you to unite pools of servers, storage systems, and networks into a single large system so you can deliver the power of multiple-systems resources to a single user point for a specific purpose. To a user, data file, or an application, the system appears to be a single enormous virtual computing system.

Computational grid: A collection of computers, online instruments, data archives, and networks that are connected by a shared set of services that, when taken together, provide users with transparent access to the entire set of resources (Fox, 2001).
Data grid:

a) The storage component of a grid environment.

b) A grid computing system that deals with data — the controlled sharing and management of large amounts of distributed data (Wikipedia, 2008).

Computing fabric: Denotes an architectural pattern (such as grid architecture) applicable at multiple levels of scale, from global networks to networks on a chip.
Access grid: The set of hardware and software used to submit compute jobs.
Middleware: Software that organizes and integrates the resources in a grid. Middleware automates all the "machine to machine" (M2M) interactions that create a single, seamless computational grid.
Multics: Multiplexed Information and Computing Service
Metacomputing: Distributed supercomputing to achieve higher performance than individual supercomputers/clusters can provide.
e-Science: The term e-Science (or eScience) is used to describe computationally intensive science that is carried out in highly distributed network environments.

 

Start-up Costs

 

Grid Economics

 

Grid tool kits exist today for clients to startup business grids.  Grid services can be information services or hardware services:  information services provide access to software and data, hardware services provide access to hardware computing and storage resources.  Data for calculating grid startup costs is not readily available.  However, interest in on-demand utility computing is larger then ever before and grid commercial environments are increasing.  From readings, the commercialization of grids is complex and costly.  The big question is what are grid computing resource worth today?  In grid environments, resource value, generated from supply and demand, is dynamic.   Users need the grid resource but if the resource is not scarce, then what should be the cost?  In theory, grids costs can be linked to increased utilization, allocation flexibility, feasibility of complex computational tasks and demand.

 

Utility Pricing for Endusers

 

Amazon S3 (Simple Storage Services) and Amazon EC2 (Elastic Compute Cloud) services are available to the public now.

 

Key-players, Gurus, and Experts

 

This multimedia presentation introduces some of the leaders in grid development.

 

  • Ian FosterFoster is considered one of the founders of the international Grid community and has written many influential documents on Grid architecture and principles. He created the Distributed Systems Lab at Argonne and UofC, which has pioneered key Grid concepts, developed Globus software, the most widely deployed Grid software, and led the development of successful Grid applications across the sciences.
  • Dan Reed What aspect of the grid will be most important for tomorrow's users?   Dan A. Reed is an American computer and computational scientist, known for his contributions to high-performance computing and science policy.
  • Francine Berman:  Dr. Berman is one of the two founding Principal Investigators of the National Science Foundation's TeraGrid project.
  • Miron Livny:  Dr. Livny's research focuses on distributed processing and data management systems and data visualization environments. His recent work includes the Condor high throughput computing system, the DEVise data visualization and exploration environment and the BMRB repository for data from NMR spectroscopy.
  • Charlie Catlett:  Charlie Catlett is Chief Information Officer at Argonne National Laboratory and Division Director of Argonne's Computing and Information Systems Division. Argonne is a U.S. Department of Energy research laboratory.
  • William E. Johnston:  William E. Johnston is a Senior Scientist and Dept. Head of the US Dept. of Energy, Energy Sciences Network (ESnet) (www.es.net) in the Computational Research Division of the Computing Sciences Directorate of Lawrence Berkeley National Laboratory.
  • GridCast:  Blogging Behind The Scenes of Grid Computing.  Grid users and developers talk about grids.
  • GridTalk:  GridTalk brings the success stories of Europe's e-infrastructure to a wider audience.

 

Sample Lesson Plan

 

The idea and content for the Grid Lesson Plan is inspired by GridCafe.  The idea to use SoftChalk is mine.

 

Read First:  The Grid Lesson Plan Instructional Strategy

Grid Lesson Using SoftChalk 

 

Tutorial

 

Refer to Grid Lesson Using SoftChalk above for the tutorial.

 

Links to Grid Tutorial Sources:

 

  1. A first public demo of grid technology, brought to you by the GridCafé:  UNOSAT & Humanitarian Crisis Management Demo (a demo about disaster prevention).
  2. Grid in a flash!

 

 

 

Sources

Avanade, (2000). What We Do. Retrieved July 5, 2008, from Avanade Web site: ttp://www.avanade.com/whatwedo/index.aspx

Blatecky, Alan (2002, September 3). Grid Computing. Educause Center for Applied Research, 2002, Retrieved July 5, 2008, from http://net.educause.edu/ir/library/pdf/ERB0217.pdf

Burne, Cristy (2007, September 12). What is a grid? A crash course in grid computing. International Science Grid This Week, 41, Retrieved July 5, 2008, from http://www.isgtw.org/?pid=1000550

CSC, (2007). Research fields that utilize grids. Retrieved July 5, 2008, from CSC, the Finnish IT center for science Web site: http://www.csc.fi/english/csc/scientific_computing/CSCserves/grid/grid_fields

Daigle, Stephen L., & Voss, Brian D. (2006). IT and the Chaning Face of Research in Higher Education. Educause Center for Applied Research, 2006, Retrieved July 5, 2008, from http://connect.educause.edu/Library/ECAR/ITandtheChangingFaceofRes/40135

Dartmouth University, (2006). Green Grid Project. Retrieved July 22, 2008, from The Green Grid Project @ Dartmouth Web site: http://grid.dartmouth.edu/

Data grid. (2008, June 30). In Wikipedia, The Free Encyclopedia. Retrieved 20:39, July 22, 2008, from http://en.wikipedia.org/w/index.php?title=Data_grid&oldid=222627433

Foster, Ian (2002, July 22). What is the grid? A three point checklist. GRID today, 1, Retrieved July 5, 2008, from http://www.gridtoday.com/02/0722/100136.html

Foster, Ian. (2003). The Grid: Computing without Bounds. Scientific American, 288, Retrieved June 11, 2008, from http://pathfinder.utb.edu:3290/ehost/detail?vid=8&hid=17&sid=e2e14ef5-24d7-4f50-b147-92357f75e20a%40sessionmgr7

Foster, Ian (2002). The Grid: A New Infrastructure for 21st Century Science. physics today.org, 55, Retrieved July 5, 2008, from http://scitation.aip.org/journals/doc/PHTOAD-ft/vol_55/iss_2/42_1.shtml

Fox, Geoffrey, & Gannon, Dennis. (2001). Computational Grids. Computing in Science and Engineering, 3, Retrieved June 17, 2008, from http://pathfinder.utb.edu:2116/iel5/5992/20167/00931906.pdf

Georgetown University, (2006). GUGrid - ARC Georgetown University Grid Initiative. Retrieved July 5, 2008, from Advanced Research Computing Web site: http://gugrid.arc.georgetown.edu/

Grid@Cern, (2008). The five big ideas behind grid computing. Retrieved July 5, 2008, from GridCafe Web site: http://gridcafe.web.cern.ch/gridcafe/challenges/challanges.html

Grid@Cern, (2008). A brief history of grid computing. Retrieved July 5, 2008, from GridCafe Web Site: http://gridcafe.web.cern.ch/gridcafe/Gridhistory/ancestors.html

GROW, (2002). Grid Research and educatiOn group @ ioWa (GROW). Retrieved July 22, 2008, from GROW - Grid Research and educati)n group @ ioWa Web site: http://grow.its.uiowa.edu/dokuwiki/doku.php/grow

Heingartner, Douglas (2001, March 8). The Grid: The Next-Gen Internet?. Retrieved July 22, 2008, from Wired Web site: http://www.wired.com/science/discoveries/news/2001/03/42230

Manola, Frank (1999, March 30). Characterizing Computer-Related Grid Concepts. Retrieved July 5, 2008, from Object Services and Consulting, Inc. Web site: http://www.objs.com/agility/tech-reports/9903-grid-report-fm.html#execsumm

Noel, Christophe (2006). What is Grid? CETIC's Tentative Definition. Technology News from Walloon Research Centers, Retrieved July 5, 2008, from http://www.cetic.be/FR/Newsletter/05-2006/index.html

Open standard. (2008, June 25). In Wikipedia, The Free Encyclopedia. Retrieved 21:14, July 5, 2008, from http://en.wikipedia.org/w/index.php?title=Open_standard&oldid=221730084

Southeastern Universities Research Association, (2007). What Grids Can Do For You. Retrieved July 22, 2008, from the Grid Technology cookbook Web site: http://www.sura.org/cookbook/gtcb/index.php?topic=3&mlevel=1

TLC2, (2008). About TLC2. Retrieved July 5, 2008, from Texas Learning & Computation Center Web site: http://www.tlc2.uh.edu/About_TLC2

University of Florida, (2006). Campus Research Grid. Retrieved July 5, 2008, from High-Performance Computing Center Web site: http://www.hpc.ufl.edu/index.php?body=grid

University of Wisconsin, (2003, January 23). About GLOW. Retrieved July 5, 2008, from GLOW:: Grid Laboratory of wisconsin Web site: http://www.cs.wisc.edu/condor/glow/index.html

Venton, Danielle (2008, March 12). The business of grid computing: "What can grids do for you?". International Science Grid This Week, Retrieved July 5, 2008, from http://www.isgtw.org/?pid=1000881

Comments (10)

David Plummer said

at 10:00 pm on Jul 1, 2008

Dawn,

Looks like you're making progress. I like your graphics. Keep it up! I look forward to seeing more information on this topic.

David

Dawn Quinn said

at 10:45 pm on Jul 1, 2008

Thanks David for the encouragement.

CRod said

at 1:05 am on Jul 4, 2008

I was wondering what The Grid was all about. I agree that the graphics add to the information. I received my Texas newsletter and there is an article about using "grid computing" to fight cancer by donating idle computer time. Is that somehow related to your topic? Just wondering. Would be glad to forward the article to you if you'd like it.

CRod

David Plummer said

at 12:30 pm on Jul 5, 2008

I have the same question as CRod except I was wondering about the SETI@home project at link http://setiathome.berkeley.edu/sah_about.php. They call their system that uses idle home computer time connected via the internet as a "virtual super computer." David

Rebecca M. said

at 8:31 pm on Jul 8, 2008

Hi Dawn,

After reading your first two paragraphs, I still am a bit confused about what a grid is. For me, it is confusing to start reading how it works without having a clear understanding of what it is. It might help to explain and give examples before telling how it works or why it is good. I have read all of your material, but am still not clear as to what it is. You talk about power, sharing resources - this makes me think of electricity or something. But then you also talk about content searching. I'm sorry, but I don't get it.

Rebecca

Emily Moore said

at 11:01 am on Jul 9, 2008

Dawn,

I'm excited to see what you've done--I was wondering what the heck the "grid" was!

I like your visuals, and you've got a ton of good references. Like Rebecca, though, I need more of a grounding before I get to the nitty-gritty details.

Specifically,what *is* the grid, and who uses it to do what?

From your text, it sounds like the grid is a giant computer network (which is what the Internet is). But then you differentiate between the two. So what's the difference between the grid and the Internet? Is the grid an intranet (or group of intranets)? Also, do only certain people have access to the grid? Or is everybody able to access it, but only certain people/companies are developing software (such as search software) to exploit it?

I have a feeling that a little up-front background will make all the rest fit perfectly in place. Keep up the good work...

-Emily

Dawn Quinn said

at 5:15 pm on Jul 10, 2008

CRod -
Yes, this may be the same information I collected. Go to ~ http://cels-at-home-dev.dyndns.org/cels/

Cels@Home is based at the University of Texas in Austin.

Interesting read. I am not a "science guy", but I suspect it takes a lot of processing power to produce digital laboratory cell adhesion. If one computer can process a cell adhesion in 20 hours, just think what 20 computers could do. Send me the link so I can see if it's the same project.

Thanks again.
Dawn

Dawn Quinn said

at 5:19 pm on Jul 10, 2008

David -
Yep, same stuff. Another very interesting read. They call it "public-resource computing." An important source in this reading is ~

Foster, I. and C. Kesselman. The Grid: Blueprint for a New Computing Infrastructure. Morgan Kauffman, 1999.

Great reading. I source this in my pbwiki.
Thanks.
Dawn

Dawn Quinn said

at 5:21 pm on Jul 10, 2008

Rebeccas & Emily ~
Good point. I will rework the start so it makes more sense as you move through the article.
Thanks.
Dawn

Dawn Quinn said

at 5:24 pm on Jul 10, 2008

Everybody -
Just so you know, "work" at work has been all consuming. I am slow to produce and slow to comment. What this work represents is pieces of work put together as I do it, not necessarily in right order. I will use all your comments and make appropriate additions, modifications, deletions.
Thanks mucho.
Dawn

You don't have permission to comment on this page.