Advanced talk on Globus tool kit 3.

 

(http://www.softwaresummit.com/2003/speakers/brown.htm)

 

GT3 Notification

1)    notification source fires events

2)    notifiction sink receives

3)    associate with some dummy service data

4)    add “ogsi:NotificationSorurce/Sink” to GWSDL

5)    sink calls addListerns on sources Service DataSource calls notifyChange to fire notifcation

6)    sink implements deliverNotification to catch

There is no tool support for the java files to declare what the GWSDL tools should do to get source and sink. However with JSR 175 you could imagine a way that this declaration could end up in java file.

 

The idea behind this is to do coarse grained notification.

 

The Service Data:

1)    structured data associated with a grid service – zero or more Service Data Elements (SDE) per instance

2)    SDEs are described by an XML grammar – java bean is generated for each SDE type (defined in an XML schema? He said .xsd file)

3)    Client calls findServiceData(“SDE1”) – careful of misspelling

4)    Helper methods extract populated Bean instance

5)    Use conjunction with notification – add payload to event.

When the interesting ‘data thing’ gets pushed along with the event. The client gets the data with the event. The other side is a pull notification where the client gets dummy data and then has to call back to the poster to get the data.

 

The SDE’s are associated with the service its self and the instances of the services

 

SDEs in every grid service

1)    GridServiceHandle

2)    FactoryLocator

3)    TerminationTime

4)    ServiceDataNames

5)    Interfaces – names of all interfaces that a service implements

 

Quick of write a grid service

1)    define all aspects of the service interface –java or gwsdl

2)    generate grid support code

3)    four mor steps

 

Grid project – example

1)    Distributed SBG Rendering

2)    RenderSourceService

a.     Farms out the svg file and a rendering area to each worker

3)    RenderWorkerService

a.     Builds a jpeg from the specified area of the svg file

4)    GUI Java Application

a.     Creates RenderSourceService via factory

b.     Triggers worker service to get work from source service

c.     Watches for status notification and finished jpegs

d.     Displays all status and tiles the sub-mages as completed

 

If I want to make my machine available I have to download Globus which takes 30 minutes or more. And there is a bug that he could not make work so he made it all local on his machine (wouldn’t it be cool if we could do that during the demo?)

 

GT3 Features

1)    factories

2)    notification

3)    service data

4)    standalone clinet

5)    could it do more, sure! This is all we need for the demo.

 

Next was code. Cool Stuff.

 

Eclipse integration is coming but not there yet.

Services are defined by interfaces, duh…

Used batik from apache to do the actual rendering. Batik gets the URL and can render from that.

 

The RenderWorker takes a renderSourceHandle which referres to the RenderSource via the handle (i.e. this is the uri to the RenderSource service) to get at its work.

 

Must have a ‘schema’ directory conisident with the code directory (foo.bar.objects, foo.bar.schema).

 

The xsd describes the methods and their arguments and their return types.

 

anyType represents ‘Object’ if you will. It’s the generic type. Used to represent the returned image from the worker service.

 

Basic steps, get a factory, use the factory to create the manager service, use the location stuff to find the created service, set the initial parameters.

 

Start some worker thereads (passing the handle to the manager) that will go back to the manager service and ask about the work to do.

 

The locator is used to find several workers and then tells them to start. They will in turn go back to the manager and ask for some work.

 

When notification happens we get the any data stuff (defined in schema) then we have to do some Corba like narrowing. To get the any type downcasted to the type we want.

 

GUI – creates a render factory instance and 16 renderers (which could be running on our machines). Distributed rendering of an image across 16 threads (which could have been on our machines). Very cool demo, can’t wait to play with the code from the CD.

Simon Nash – member of JSR 175 team

(http://www.softwaresummit.com/2003/speakers/nash.htm)

Back ground and motivation

JSR 175 specification

1)    Source syntax

2)    Processing model

3)    Runtime api

4)    Class file format

5)    Standard annotations

 

Background – java is imperative not declarative, java has declarations but only for core language constructs. The declaration of ‘the what’ limits the scope for tools to help the developer be more productive.

 

The annotations are beyond the bounds of the language, instead it fits into the language. Its not about making Java huge to handle every  thing, its about making a way for you to extend stuff so that you can declare what in your java classes (i.e. this class is a web-service). The annotation are not part of the program logic but can affect the meaning of the program (i.e. the declaration that the class is a web service might causes WSDL to be generated at deployment time). An annotation will not change the way assignment works or the way that try/catch works.

 

Semantics of the annotations can be standard or user defined. (I’m assuming that at some point we will see how to use user defined annotations. Just said we’d be getting to that…)

 

Benefits – productivity, clarity, flexibility, correctness – all the same stuff you get from something like MDA or other model based code generation. I like this but its just not new. Now that its becoming part of the language though it seems that we might be at the cusp of general acceptance of meta-data based programming (i.e. MDA). If you build bad stuff with this it will be bad. Clearly you can build bad stuff with anything, this is no difference.

 

Q: will AOP make use of this, A: probably, seems a natural fit (the join points should be able to be specified with these annotations).

 

175 – simple meta-data in source code, attached to program syntactic elements, limited range of type, stored in the class file (available at runtime), can be read at runtime.

 

Cool example of declaring a web service for a class. Interesting thing to note is that the types of the attributes for annotations is only a limited set (most noteably not Object or any arbitrary subclass, only the types defined in the JSR to be able to be used).

 

Attributes can be defaulted.

 

An example of @WebService annotation for a class then adding an @Operation on one of the operations. The @Operation is not explicitly contained in the @WebService.

There are 4 meta-annotations @Target – what can the annotation end up on, @Retention – how far to carry the annotation forward, @Inherited – declares if the annotation should be inherited, @Documented – should this stuff show up in the java doc.

 

@Retention – runtime, class or source – class is default, runtime makes them available for reflection

 

Since source is the default retention it seems that the expectation is that post processing on class files will be the norm.

 

Annotations can be used anywhere that ‘public’ can be used. They are seen as modifiers.

 

You will be able to annotate your package with a package-info.java file.

 

Processing Model:

1)    Source based

2)    Class based

 

You can integrate your annotation processors with the java compiler? There was a slide about that but it appears that its not ready yet. The way they are thinking is making a plug-in architecture for the compiler. Today what you have to do is post process the class file or process the java code (i.e. you have to parse the java…)

 

JSR 175 extends the reflection API’s to get at annotations. There is a new thing called AnnotatedElement.

 

More annotations are changed or defined in JSR’s (181, 207, 224 others as well).

Paul Giangarra – Portals

 

Third or forth generation portals are about collaboration? Missed the detail on that comment.

 

What is a portal?

1)    a place to start for your users

2)    a place to aggregate the stuff that people want to do

 

Value of Portals

1)    simplify access to resources

2)    eliminate information overload

3)    personalize for the one-to-one marketing appeal

4)    collaboration meets applications

5)    the enterprise now ‘owns’ the desktop

 

The Portal helps to aggregate the information that we want to see, personalized etc. Allows us to avoid info-overload.

 

Why Portals – employee productive, increase efficiency through automation, quicker response to customer

 

Gen0/Gen1 (-1998 to 2000) portals simple hyperlinks and search, homepages on sterids

 

Gen2 (2000 to 2001) robust app framework, collaboration, Mobile and wirelss, management framework, being deployed today

 

Gen3  (2002+) context personalization, cascading portals, web services, process integration, abstraction layers, federated search, federated portals, p2p offline support, proactive notification, mainframe support, knowledge management

 

Why are biz portals so important?

1)    eventually all web apps will be portal apps

2)    portals will be the most visible software infrastructure in a company’s e-business strategy

3)    the need for integration drives the growth and importance of portals

4)    application vendors are using portals as UI and Integration

5)    Portal frameworks…

 

Portal Planning Process – or how to get a portal into your company

1)    obtain sponsorship

2)    analyze drivers and expected benefits

3)    inventory the features desired

4)    conduct an infrastructure impact assessment

5)    select products

6)    sell internally

Basically laid out how to decide if you want to build a portal, decide what you want/need, a portal for portal sake is stupid.

 

Don’t confuse a Portal with a web site. The are not the same, the portal is window into web applications/web sites.

Portal Infrastructure

1)    knowledge arch

2)    community arch

3)    security arch

4)    tech arch

5)    dev and dep

6)    operational arch

Important for B2C portals to truly aggregate the info instead of sending you off (i.e. looking at CD players and linked off to Sony, you just lots a customer, Sony.com has a link to ‘find someone to sell this to you’).

 

Knowledge management is important in building a portal. Unstructured info is just info, if you can organize it its knowledge. We need knowledge.

 

(Paul spawned a thought: Interesting thing to think about: A development portal for open source communities. Why don’t we have something like a portal for developers. Aggregate information into knowledge, connect people in great ways that makes the community more effective.)

 

Lots of detail about objectives that you’d want to build a portal for. This was a lot like a marketing pitch of why portals are important, i.e. why do a portal.

 

Technical arch of the portal

1)    Portlet container services

a.     Content access

b.     Web clipper

c.     Search

d.     Document manager

e.     Po9rtlet data

f.      Admin

g.     Collaboration

h.     Credential vault

i.      Portlet proxy

j.      Single sign-on

2)    J2EE

3)    Page aggregation

a.     Themes and skins

b.     Transcoding

c.     JSP tag library

d.     translation

 

Base portal design – think about it, users, admin, roles etc.

Think through your page layout as well. Title bars, menu’s, hierarchies of portlets etc.

 

JetSpeed is a great spot to play with portlets.

 

Themes do most of the hard work for cross-browser support and accessibility.

 

Delta from JSP/Servlet to Portlets,

1)    portlets are very dynamic

2)    portlets can know personalization info, locale, devices etc.

 

Building a poral –tasks

1)    creating page definitions

2)    create themes ad skins

3)    developing portlets

Cool thing is that all this can happen independently.

 

JSR 168 gives us – interoperability between local portlets and portals, i.e. we can put a portlet written WLS can be deployed into WebSphere

 

WSRP – goals

1)    enable the sharing of portles (markup fragments) over the internet with a common interface

2)    enable cross vendor publishing and consuming of content

 

WSRP allows us to build a portlet and let it be vended into another portal?

A proxy portlet receives info from a web-services interface and then render that portlet in the local Portal.

 

On the portal server the generic proxy (only one, we don’t need to implement it) and it goes off via SOAP and asks for some rendering to be done. The proxy then takes the rendering from the back end and displays it. Very interesting.

Simon Philps – keynote

 

The Massively Connected Era – assumption is mostly valid, the infrastructure is something we can take for granted. When you can just connect and not worry about the syntax of XML etc. you are massively connected.

 

Rapid evolution, in just one decade the world has become massively connected. Other major changes with new inventions has changed the world a lot fast than other inventions.

 

Technology Promotes Connection – divice proliferation, growing bandwitch , SOA, many dimensions/layers of relationships

 

Moving towards Fractal Architectures – we are moving to networks of networks. (Grid Computing?) We will need an understanding of fractal dynamics in networks.

 

Massively Connected – every node is connected to everything else. Thankfully the Internet was not build like bulletin boards. Instead it was built with free open source shared, open, royalty-free standards.

 

Exponential Opportunity Growth through blah blah blah, missed what he said…

 

Interpreting Open for the Era:

1)    80’s

a.     hardware-inspired standards processes

b.     committee membership, specification

2)    90’s

a.     business-inspired standards processes

b.     consortium membership, implementation

3)    21st

a.     community-inspired standards processes

b.     community involvement, open source

Open source is more about the consequences of saving a few bucks on your purchase price, it’s a result of our massively connected time.

 

If the network were the computer,

1)    there would be no boundaries to defend, security would be very different. If we had an identity based approach, web services would be different, because we would not have to be pushing everything through port 80. Instead the security would be defined around identity instead of the boundaries.

2)    Software would become a service that would self organize etc. We would not have to know about or care about where it was running. Grid computing again…

3)    Software would be licensed on scale, not on implementation details. We don’t know exactly where/when the surge will show up so you don’t need to know about licenses (or should not need to know).

4)    System stakeholders would be enfranchised in development. The individuals involved in using (i.e. stakeholders) the software should be involved in defining the software.

The network is the computer now. Part of the reason everything is changing so quickly.

 

Open Source:

1)    it is mostly or largely a development methodology

2)    definition – commons-based peer production http://www.benkler.org/CoasesPenguin.html - 2 modes of production in the past, artisizans work, and manufacturing. Open source is the new type of production, commons-based peer kind of production. Interesting idea.

3)    Structured and controlled development

a.     Reputation-based

b.     Controlled by initiators

c.     Things based on your reputation not your status, your employer etc.

4)    High quality results commonly

5)    Remains loosely-coupled – maintains choice

a.     First and foremost is the right to fork the code, you can always take the code and make your own copy.

6)    Facilitated by commons-creating license

a.     Make the commons possible, not the most important thing, just makes it possible for the community to form and contribute

7)    Many licenses, one objective – make the community possible.

 

Open Source: Agile Development done by a community

 

Open and Proprietary – lots think these are opposite, they are not, standards are open or closed, source is community or proprietary, impl is interoperable or locked

 

Staying Open…

License is for copyright, who owns it and why – LGPL and GPL keeps the power in the hands of the copyright (and threrefore power) holder and not the community.

 

Understanding open source – deployment

The scope of free – free is not the same as zero dollars, free is about freedom not about saving money. Source is for developers, deployers don’t really care about source, they want it to work and be able to fix it if its hosed. The licenses facilitates community.

J we use the same term for our consumers as the drug industry, the first shot is free. J

 

deployment models for open-source;

1)    help your self

2)    package to produce and support

3)    customize per use and support

4)    customize to product and support

5)    aggregate to bundle and support

These are the ways that we will be using open-source in the future.

 

Research suggests that the #1 reason is choice and freedom and quality of the code. Cost is shared, not free, but controlled and shared.

 

www.sunsorce.net for details

 

Lots of other stuff about the open source stuff that Sun, openoffice.org, netbeans.org,

 

Sun Java Desktop System – complete desktop for the average user – initalliy aimed at groups. Looks really cool (it just works, idea ‘stolen’ from the mac J) Open office, mozilla etc.

 

Why sun is doing open source:

1)    shared innovation

2)    customer and partner benefits

3)    increased quality and security

4)    future direction of most software

5)    base for future evolution of best practice and unexpected value

6)    and its still great stuff even after all that

 

Sun’s Expirences – Issues

1)    open source is a big step – its for life not something you drift into

2)    source needs hosting

3)    community needs facilitating

4)    license issues outside education of most IP attorneys

5)    Developers find public exposure a shock

6)    Managers find loss of control challenging

Sun’s Experience – benefits

1)    Good returns

a.     Savings take time to appear

b.     Community is chief asset

2)    Software quality is better

a.     Fewer bugs, more quickly identified

b.     Design tracks market needs

c.     Design improved by extensive review

3)    Lock-in Avoided

a.     No single dominat force

b.     Influence of would-be marked controllers largely avoided

c.     Can always go elsewhere, no IP locks

 

Open Source is the methodology of the massively connected society

Business models of open source are better and the only way to support open source. The community is not set up for deployment, most people choose open source for freedom not because the price tag is low.

Paul Giangarra – IBM – done grid computing for govt.

 

What is Grid Computing

1)    why do I care

2)    what does it do

3)    how does it do it

Grid computing core concepts

Grid computing architecture

Grid computing impl

So what do I do now

Real world examples

 

Grid Problem: provide for flexible secure coordinated resource sharing among virtual organizations

 

Grid Challenge: create an arch and solution set based on open standards and where they exist exploit existing tech to solve this.

 

Jokingly says Grid is Posix done right. Grid allows you to virtualize resources with a set of functionality.

 

A grid is a collection of capabilities that have been virtualized to be transparently accessed.

Grid:

1)    Services Oriented Architecture

a.     Services are things that provide lots of different things, J2EE is ‘platform services’ Amazon provides a ‘book lookup service’ etc.

2)    Based on Standard, open, general-purpose protocols and interfaces

3)    Services and technologies

a.     Help coordinate and manage disparate and possibly heterogeneous resources that are not subject to centralized control

b.     Can be used to deliver no-trivial qualities of service (quality does not mean its good, it means that you know how good it is).

 

3 model and unique value propositions

1)    average computers are running at an average of 40% usage.

2)    Data virtualization is useful, security is important but the data model is good to have virtualized. Data can come from lots of different spots and be aggregated so users can function better.

3)    Resiliency – allow you to move and manage where your apps run, the grid dispatcher can move your work around to where ever it makes sense, you as app developer don’t have to know about where your app will run, the grid ‘management’ stuff will move your app to where it can run best.

Grid computing tries to build the computer looking like a single thing across the scope of the disparate pieces that make up the grid.

 

Lots of people are using grids it virtualizes license management in an enterprise. CAD runs on one machine and lots can use it because the grid makes the remote piece of computing look ‘local’.

 

Cool example of building a big OS at IBM. Lots of people had AIX boxes, when it came time to build the OS a central computer would ask around for idle machines and would farm out big chucks of work to the idle machines. When you sat down to use your computer it would kick the compile back to the scheduler. This is an example of grid computing that was happening 15 years ago.  They did not use standard grid api’s but they were doing grid computing.

 

Data Grid – you can send the work to the data, or you can get the work to the data. The Grid can be smart like processing a satellite image (4gb), you don’t want to send that whole image around the grid but if you could chunk up the image and send it out then it might make sense to distribute the work by sending the data to the work. If instead you can’t chunk the data then perhaps it makes sense to pull the work to the data. The grid can manage that based on quality of service and other stuff (I think that is what I heard).

 

What can Grid computing do for me?

1)    Infrastructure optimization

a.     workload management and consolidation – turn large compute problems into small bits that can be parallelized

2)    Increased access to data and collaboration

a.     Federation of data – inbound

b.     Global distribution – outbound

3)    Resilient / Highly Available Infrastructure

a.     Recover and failover

 

Grid computing can allow you to make use of your cycles to make your computing life easier.

 

Grids increase access to data allowing better collaboration.

 

Grid job scheduler can know that part of the job croaked then and restart the failed part.

 

Compress video is straightforward. It can more or less distributed over a lot of computers without issue. If the 2nd 10 seconds does not work then you can always restart it.

 

Web Services & Grid Protocols are synergistic. Open Grid Services Architecture.

 

Fabric – resources and access to the resources live in the fabric of the grid, file systems, process power, basic storage, etc. The resources can be queried and put together with the Fabric interfaces. You can virtually provision and build a computer and then build an application on top of it.

 

Collective – protocols and services thar are not associated with any one specific resource but rather are global in nature and capture interactions across collections of resources (i.e. the fabric).

 

App Layer – where applications live.

 

OGSA needed additional stuff from web services

1)    autonomic capabilities

2)    provisioning

3)    State management for web services

 

Lines between apps and resources are bluring

 

OSGA can be put into micro codes to manage SAN and network stuff etc, no need for OS intervention, cool stuff.

 

Cool graphics, autonomic stuff cross the whole stack

 

Globus is an open source impl of the OGSA definitions

 

OSGI = Open Grid services Infrastructure (I think this is the impl of the OSGA, i.e. Globus).

 

WSDL 2.0 will have all the OGSA extensions. Currently working on that now.

 

The extensions include stuff like this

1)    ws-addressing

2)    ws-choreography

3)    ws-attributes

4)    ws-trust

5)    ws-security

 

OGSA Structure

1)    Service management

a.     Reg and discovers

b.     Services

c.     Service orchestratation

d.     Metering and accounting

2)    Service communication

a.     Messagind and queuing serices

b.     Event services

c.     Distributed secure logging service

3)    Policy management

a.     Policy service mamanger

b.     This looks like P3P stuff

4)    Security

a.     Also looks like P3P stuff

 

These are all part of the api’s.

 

Anatomy of a grid service

1)    handle to a grid service

2)    hosting environment is on the bottom

3)    grid service lives on top of the hosting environment

4)    Consider a weather service, I can get one that is location based and I can get the handle from a service handle manager type service

5)    Service data access, lifecycle magement

6)    Service creation (factory) service discovery (registry), notification, handle management, workflow, autding etc.

 

So now we are going to build a directory service.

1)    abstract service interface remains the same despite the way the service is implemented.

2)    Our service implemented through jdbc to the ldap db, or jindi to ldap to the db, does not matter, the service is implemented however you need to, just conform to the abstract service interface.

 

Things we have to deal with in building an app in a grid env.

1)    many existing apps are monolithic or tightly coupled

2)    need to refactor apps, worry about distributed thread, datalocking, latency

3)    rehosting apps, exploit meta-os services, achieve platform independence, refactor for distributed parallel execution

4)    Need for re-hosted middleware

5)    Ability to exploit grid computing services, e.g. distributed provisioning

6)    Manage and exploit quality of service across the grid

7)    Challenge: move to and exploit services oriented architecture.

 

SOA = loose coupling and clear definition, SOA is the basis of Grid computing.

 

Needed

• J2SE 1.3.1 or later
• Servlet 2.3
• JSP 1.2

Jwsdp 1.2 has tomcat built in

Jsf-api.jar, Jsf-ri.jar, Jstl.jar, Standard.jar, Commons-beanutils.jar, Common-collections.jar, common-digester.jar, common-logging.jar

 

Five steps

1)    web.xml file must include JSF mapping

2)    copy .jar files to web-inf/lib

3)    web app must have app config file called web-inf/faces-config.xml (default name)

4)    web app must invoke FacesServlet before using JSF features

5)    jsp’s must be setup to use JSF

 

Technically you can map the faces Servlet to anything you want. Not sure that is true from my experiments.

 

Key Concepts

1)    JSF request processing model

a.     FacesServlet

b.     Lifecycle

2)    JSF Global Configuration

3)    Component model

 

Lifecycle: FacesServlet controls the lifecycle, it marshals resources and calls crucial methods. Sepcifically:

1)    get faces context

2)    get lifecycle object

3)    call the lifecycle methods

4)    releases the context

The context gives me a way to access services from the faces framework.

Lifecycle:

1)    builds or rebuilds a JSF component tree

2)    applies request values to components

3)    processes validations for new request values

4)    update model values

5)    invokes app behavior

6)    renders the response

 

Faces Config:

The default name is faces-config.xml file you can switch names and have multiple files that configure the app.

1)    Nav Rules (declartive page flow)

2)    Java Bean components

3)    Render kits

4)    Converters and Validators

5)    Custom UI Components

Components:

Graphical behaviour is managed through UI components

Basic behavior is defined in UIComponet and UIComponentBase

 

Components can use the following API’s for extended functionality

1)    Converts – change values from String to another format

2)    Listeners/Events

3)    Validators

4)    Renderers

 

Actions in JSF are meant to do stuff not manage page navigation. The page nav is managed in the faces-config.xml.

 

The event model does not do any client side events, the event model runs only on the server. You could however hack something together if you needed to.

 

Validators:

Make the data valid before it gets sent to the server.

 

Render:

Makes HTML from components.

 

Benefits of JSF include

1)    component functionality

2)    support for event-based notification on the server

3)    support for i18n and I10n

4)    XML config of things like nav

5)    Built-in integration with beans (model components)

6)    Variable representation of UI components (renderers)

7)    Easy integration with builder tools

 

Drawbacks with JSF

1)    Non-trival seup

2)    Complex processing cycle for each jsf request

3)    Performance cost to use JSF

I disagree with most of these. The setup is straightforward, the cycle is more or less transparent for most developers. The performance cost was hedged with the fact that any framework is going to cost some performance for other tradeoffs.

 

Demo of JSF:

1)    basic form elements

2)    association with model elements

3)    event notification

4)    page navigation

5)    internationalization

Tons of info about JSWSDP.

 

A discussion of common errors and the way they show up. This was really useful I think for the developer just getting started. Good idea to put into a talk.

 

Lots of cool examples that showed all the stuff discussed above.