Paul Giangarra – IBM – done grid computing for govt.

 

What is Grid Computing

1)    why do I care

2)    what does it do

3)    how does it do it

Grid computing core concepts

Grid computing architecture

Grid computing impl

So what do I do now

Real world examples

 

Grid Problem: provide for flexible secure coordinated resource sharing among virtual organizations

 

Grid Challenge: create an arch and solution set based on open standards and where they exist exploit existing tech to solve this.

 

Jokingly says Grid is Posix done right. Grid allows you to virtualize resources with a set of functionality.

 

A grid is a collection of capabilities that have been virtualized to be transparently accessed.

Grid:

1)    Services Oriented Architecture

a.     Services are things that provide lots of different things, J2EE is ‘platform services’ Amazon provides a ‘book lookup service’ etc.

2)    Based on Standard, open, general-purpose protocols and interfaces

3)    Services and technologies

a.     Help coordinate and manage disparate and possibly heterogeneous resources that are not subject to centralized control

b.     Can be used to deliver no-trivial qualities of service (quality does not mean its good, it means that you know how good it is).

 

3 model and unique value propositions

1)    average computers are running at an average of 40% usage.

2)    Data virtualization is useful, security is important but the data model is good to have virtualized. Data can come from lots of different spots and be aggregated so users can function better.

3)    Resiliency – allow you to move and manage where your apps run, the grid dispatcher can move your work around to where ever it makes sense, you as app developer don’t have to know about where your app will run, the grid ‘management’ stuff will move your app to where it can run best.

Grid computing tries to build the computer looking like a single thing across the scope of the disparate pieces that make up the grid.

 

Lots of people are using grids it virtualizes license management in an enterprise. CAD runs on one machine and lots can use it because the grid makes the remote piece of computing look ‘local’.

 

Cool example of building a big OS at IBM. Lots of people had AIX boxes, when it came time to build the OS a central computer would ask around for idle machines and would farm out big chucks of work to the idle machines. When you sat down to use your computer it would kick the compile back to the scheduler. This is an example of grid computing that was happening 15 years ago.  They did not use standard grid api’s but they were doing grid computing.

 

Data Grid – you can send the work to the data, or you can get the work to the data. The Grid can be smart like processing a satellite image (4gb), you don’t want to send that whole image around the grid but if you could chunk up the image and send it out then it might make sense to distribute the work by sending the data to the work. If instead you can’t chunk the data then perhaps it makes sense to pull the work to the data. The grid can manage that based on quality of service and other stuff (I think that is what I heard).

 

What can Grid computing do for me?

1)    Infrastructure optimization

a.     workload management and consolidation – turn large compute problems into small bits that can be parallelized

2)    Increased access to data and collaboration

a.     Federation of data – inbound

b.     Global distribution – outbound

3)    Resilient / Highly Available Infrastructure

a.     Recover and failover

 

Grid computing can allow you to make use of your cycles to make your computing life easier.

 

Grids increase access to data allowing better collaboration.

 

Grid job scheduler can know that part of the job croaked then and restart the failed part.

 

Compress video is straightforward. It can more or less distributed over a lot of computers without issue. If the 2nd 10 seconds does not work then you can always restart it.

 

Web Services & Grid Protocols are synergistic. Open Grid Services Architecture.

 

Fabric – resources and access to the resources live in the fabric of the grid, file systems, process power, basic storage, etc. The resources can be queried and put together with the Fabric interfaces. You can virtually provision and build a computer and then build an application on top of it.

 

Collective – protocols and services thar are not associated with any one specific resource but rather are global in nature and capture interactions across collections of resources (i.e. the fabric).

 

App Layer – where applications live.

 

OGSA needed additional stuff from web services

1)    autonomic capabilities

2)    provisioning

3)    State management for web services

 

Lines between apps and resources are bluring

 

OSGA can be put into micro codes to manage SAN and network stuff etc, no need for OS intervention, cool stuff.

 

Cool graphics, autonomic stuff cross the whole stack

 

Globus is an open source impl of the OGSA definitions

 

OSGI = Open Grid services Infrastructure (I think this is the impl of the OSGA, i.e. Globus).

 

WSDL 2.0 will have all the OGSA extensions. Currently working on that now.

 

The extensions include stuff like this

1)    ws-addressing

2)    ws-choreography

3)    ws-attributes

4)    ws-trust

5)    ws-security

 

OGSA Structure

1)    Service management

a.     Reg and discovers

b.     Services

c.     Service orchestratation

d.     Metering and accounting

2)    Service communication

a.     Messagind and queuing serices

b.     Event services

c.     Distributed secure logging service

3)    Policy management

a.     Policy service mamanger

b.     This looks like P3P stuff

4)    Security

a.     Also looks like P3P stuff

 

These are all part of the api’s.

 

Anatomy of a grid service

1)    handle to a grid service

2)    hosting environment is on the bottom

3)    grid service lives on top of the hosting environment

4)    Consider a weather service, I can get one that is location based and I can get the handle from a service handle manager type service

5)    Service data access, lifecycle magement

6)    Service creation (factory) service discovery (registry), notification, handle management, workflow, autding etc.

 

So now we are going to build a directory service.

1)    abstract service interface remains the same despite the way the service is implemented.

2)    Our service implemented through jdbc to the ldap db, or jindi to ldap to the db, does not matter, the service is implemented however you need to, just conform to the abstract service interface.

 

Things we have to deal with in building an app in a grid env.

1)    many existing apps are monolithic or tightly coupled

2)    need to refactor apps, worry about distributed thread, datalocking, latency

3)    rehosting apps, exploit meta-os services, achieve platform independence, refactor for distributed parallel execution

4)    Need for re-hosted middleware

5)    Ability to exploit grid computing services, e.g. distributed provisioning

6)    Manage and exploit quality of service across the grid

7)    Challenge: move to and exploit services oriented architecture.

 

SOA = loose coupling and clear definition, SOA is the basis of Grid computing.

 

Needed

• J2SE 1.3.1 or later
• Servlet 2.3
• JSP 1.2

Jwsdp 1.2 has tomcat built in

Jsf-api.jar, Jsf-ri.jar, Jstl.jar, Standard.jar, Commons-beanutils.jar, Common-collections.jar, common-digester.jar, common-logging.jar

 

Five steps

1)    web.xml file must include JSF mapping

2)    copy .jar files to web-inf/lib

3)    web app must have app config file called web-inf/faces-config.xml (default name)

4)    web app must invoke FacesServlet before using JSF features

5)    jsp’s must be setup to use JSF

 

Technically you can map the faces Servlet to anything you want. Not sure that is true from my experiments.

 

Key Concepts

1)    JSF request processing model

a.     FacesServlet

b.     Lifecycle

2)    JSF Global Configuration

3)    Component model

 

Lifecycle: FacesServlet controls the lifecycle, it marshals resources and calls crucial methods. Sepcifically:

1)    get faces context

2)    get lifecycle object

3)    call the lifecycle methods

4)    releases the context

The context gives me a way to access services from the faces framework.

Lifecycle:

1)    builds or rebuilds a JSF component tree

2)    applies request values to components

3)    processes validations for new request values

4)    update model values

5)    invokes app behavior

6)    renders the response

 

Faces Config:

The default name is faces-config.xml file you can switch names and have multiple files that configure the app.

1)    Nav Rules (declartive page flow)

2)    Java Bean components

3)    Render kits

4)    Converters and Validators

5)    Custom UI Components

Components:

Graphical behaviour is managed through UI components

Basic behavior is defined in UIComponet and UIComponentBase

 

Components can use the following API’s for extended functionality

1)    Converts – change values from String to another format

2)    Listeners/Events

3)    Validators

4)    Renderers

 

Actions in JSF are meant to do stuff not manage page navigation. The page nav is managed in the faces-config.xml.

 

The event model does not do any client side events, the event model runs only on the server. You could however hack something together if you needed to.

 

Validators:

Make the data valid before it gets sent to the server.

 

Render:

Makes HTML from components.

 

Benefits of JSF include

1)    component functionality

2)    support for event-based notification on the server

3)    support for i18n and I10n

4)    XML config of things like nav

5)    Built-in integration with beans (model components)

6)    Variable representation of UI components (renderers)

7)    Easy integration with builder tools

 

Drawbacks with JSF

1)    Non-trival seup

2)    Complex processing cycle for each jsf request

3)    Performance cost to use JSF

I disagree with most of these. The setup is straightforward, the cycle is more or less transparent for most developers. The performance cost was hedged with the fact that any framework is going to cost some performance for other tradeoffs.

 

Demo of JSF:

1)    basic form elements

2)    association with model elements

3)    event notification

4)    page navigation

5)    internationalization

Tons of info about JSWSDP.

 

A discussion of common errors and the way they show up. This was really useful I think for the developer just getting started. Good idea to put into a talk.

 

Lots of cool examples that showed all the stuff discussed above.