Document control

AreaEGI Federation Operations
Procedure status

FINAL

OwnerAlessandro Paolini 
ApproversOperations Management Board
Approval status

APPROVED

Approved version and date

v8,  

Statement

A procedure for integrating new cloud management framework (Cloud platform) or middleware (Compute, Storage, etc.) in the EGI Production Infrastructure.

Next procedure reviewupon request

Procedure reviews

The following table is updated after every review of this procedure.

DateReview bySummary of resultsFollow-up actions / Comments

 

Alessandro Paolini copy from PROC19 in EGI Wiki. Updated some links and information.




Table of contents

Overview

To assure production quality of the EGI Infrastructure, every cloud management framework (Cloud platform) or middleware piece (Compute, Storage, etc.) supported by Production Resource Centres needs to fulfil certain requirements. The goal of this procedure is to assure EGI Infrastructure compliance.

Definitions

Please refer to the EGI Glossary for the definitions of the terms used in this procedure.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", “MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Entities involved in the procedure

  • Technology Provider (TP): person representing or leading Technology Provider team
  • EGI Operations (EGIOps)
  • Operations Centre (OC)
  • Resource Centre (RC)
  • Operations Management Board: EGI operations policy board

Prerequisites

Before sending a request:

  • OC has to have
    • the support of TP with effort to integrate with EGI Infrastructure (information system, accounting, monitoring etc), provide support via GGUS and maintain software via UMD
    • one or more RC available to deploy the new platform
  • TP has to have
    • effort to integrate with EGI Infrastructure (information system, accounting, monitoring etc), provide support via GGUS and maintain software via UMD
    • the support of one or more OC, with one or more RC available to deploy the new platform and the integration-software developed by the TP

Triggers

Steps

Request submission and validation

The request can be send by:

  1. Operations Centre
  2. EGI Operations
  3. Technology Provider

Resource Centres can also request integration of new cloud management framework or middleware. Such request should be first approved by Operations Centre, it belongs to. In such case OC is responsible to create a ticket on behalf of RC.

StepAction onAction
1ApplicantOpens a GGUS ticket to Operations to start the process.
Subject: Request for integration of XXX to EGI Production Infrastructure (PROC19)

Dear Operations,

We would like to request for starting procedure of integrating XXX to EGI Production Infrastructure:
PROC19 Integration of new cloud management framework or middleware stack in the EGI Infrastructure

Prerequisite data:
* name of Technology Product:
* Technology Provider (person representing or leading the team) contact details(name, email):
* customers of the Product (eg. user community, Operations Centre):
* motivation:


Best Regards
XXX
2EGIOps

Operations contacts the OMB to request the approval of the request.

Functional requirements

Functional requirements for new product to be integrated:

  • support VO concept
  • support X.509 certificates and/or federated identities (i.e. be compatible with EGI Check-in)

Integration steps

Integration covers following areas (where possible steps can be done in parallel):

#ResponsibleActionAdditional temporary comments
0aEGIOpsWhen Approved, EGIOps and TP should agree on Underpinning Agreement (UA)agree on Corporate-level Technology Provider Underpinning Agreement or on a customised version
0bEGIOps

Set up an integration Task force for given Technology Product composed of:

  • Technology Provider representative
  • Operations tools representative
  • NGI representatives (wanting to deploy Technology Product) with Pilot Site
  • EGI Operations representative
  • User communities representative (interested in deployment of Technology Product)
  • EGI Security team representative
  • UMD representative

Configuration Management

#ResponsibleActionAdditional temporary comments

1a

GOCDB 

Add new service types agreed within Task Force.


1bPilot SiteDeploy technical service instance and register in GOCDB.

Information System

#ResponsibleActionAdditional temporary comments
2aTechnology Provider

Develop software for integration with BDII.


Analyse the use cases for deciding if the new technology has to be published in the BDII or not, and the relevant set of information to publish.

  • Must the new technology be published in the BDII?
    • it has to be created the information providers
  • Is it necessary any modification to the Glue Schema for properly publishing the new technology information?
    • any modification to the Glue Schema has to be discussed with the Glue Working Group
2bPilot SiteDeploy software for integration with BDII and documentation.
2c  EGI Operations
Verify integrationAlessandro Paolini, Enol Fernandez, Baptiste Grenier, Operations checks documentation

Monitoring

#ResponsibleActionAdditional temporary comments
3aTechnology ProviderDevelop nagios probe with support from SAM team and documentation.ARGO Guidelines for monitoring probes
3bARGO, EGI Ops

Check probe, verify results, add to SAM release.

Add test to ARGO_MON profile.

PROC06 and PROC07
3cARGO, EGI OpsDeploy probe in production and publish documentation.Operations checks documentation
3dARGO, EGI Opsif the new technology needs to be monitored by secmon and pakiti, add the related tests in the SEC_MONITOR profile.Operations verify that the security tests are properly executed

Operations (ROD) Dashboard

#ResponsibleActionAdditional temporary comments
4EGI Ops Add test to Operations profile (PROC06 Setting a Nagios test status to OPERATIONS)

Support

#ResponsibleActionAdditional temporary comments
5aTechnology Provider

Declare Quality of Support for 3rd level Support Unit (SU) and name of SU

FAQ GGUS-New-Support-Unit


5bGGUS  Create Support Unit under "Product Teams" category

Accounting

#ResponsibleActionAdditional temporary comments
6aTechnology Provider

Develop software for integration with APEL


Define integration and what data should be published.

  • if the new technology is using computing or storage services for which accounting data are already collected, there is no need of new parser/software for integration with APEL
6bAPEL Validate integrationEGI Operations checks documentation
6cEGI Accounting Portal Display data

UMD

#ResponsibleActionAdditional temporary comments
7aTechnology ProviderEnsure software developed for the integration of the new Technology Product satisfies UMD Minimal Requirements

Request the inclusion into UMD; see here the information to provide


7bEGI Ops (UMD representative)Technology Provider info is added in Technology Providers List and UMD Product ID card
7cEGI Software provisioning TeamApplies the UMD Software Provisioning process to assess the quality of the new product
7dEGI Ops (UMD representative)Once confirmed a successful provisioning (step 11c) includes the new product/products into an UMD release and makes it available to the production infrastructure, in the UMD repositories

VM image Marketplace

#ResponsibleActionAdditional temporary comments
8aTechnology ProviderImplement subscription to VM image lists from EGI MarketPlace and create documentation.
8bPilot SiteAdd service endpoint to GOCDB (type: eu.egi.cloud.vm-metadata.vmcatcher)
8cNGI/EGI OpsCheck eu.egi.cloud.vm-metadata.vmcatcher is passing  https://cloudmon.egi.eu/nagios/ 
8dEGI Cloud VM Image Management SUValidate integrationOps support check documentation

Documentation

#ResponsibleActionAdditional temporary comments
9aEGI OpsUpdate relevant documentation
9bTechnology ProviderDevelop documentation for users and admins where missing
9cEGI OpsValidate Documentation

Resource Allocation

#ResponsibleActionAdditional temporary comments
10Resource Allocation

Add new access method in e-GRANT(if needed)

Define if the middleware is a new way of accessing resources

e-GRANT was dismissed. To evaluate if similar steps are necessary for the AoD service or EGI Marketplace

Security

#ResponsibleActionAdditional temporary comments
11aTechnology Providercomplete the EGI SVG Software Security ChecklistA brief written response to Chair of SVG (Linda.Cornwall <AT> stfc.ac.uk) is requested
11bSecurity teamProvide recommendations based on provided input
11cTechnology ProviderImplement recommendations
11dSecurity TeamValidate implementation of recommendations

The Announcement

EGI Ops announces the availability of new product to OMB and includes the announcement in the monthly EGI Broadcast to communicate the availability of the new product to NGIs, VOs, RCs managers.