CMS analysis cluster on-demand

The LHC  Community

The video shows the submission of a TOSCA template to the PaaS orchestrator. The TOSCA describes a Mesos cluster with Marathon application framework. The application is a HTCondor Batch system plus ancillary services such as Squid Proxy, TTS Interface. The submission is performed using ORCHENT client. The same client is also used to check the status of Mesos cluster creation.
Waiting for cluster creation, a CMS Data Analysis Workflow has been submitted to CMS Workload Management System (HTCondor Global Pool). Once the cluster will be up&runnig and the HTCondor_startd process will joins the HTCondor global Pool, the submitted jobs will start running.
On cluster ready a check on the actually running Dockers process is performed. So after login to the Mesos cluster, running processes are analysed in order to check that a  CMS Data Analysis job is executing (a check on cmsRun executable is performed).
As a proof that jobs are real and actually communicate with the rest of CMS computing infrastructure, a status check from CMS Dashboard web interface is also performed.
A specific walkthrough is finally done to verify that a X.509 is obtained from INDIGO-TTS. For that purpose a check on DN used by condor_startd to join the pool is shown.


INDIGO Service used

PaaS Orchestrator