Overview  Package   Class  Use  Tree  Deprecated  Index  Help 
 PREV CLASS   NEXT CLASS FRAMES    NO FRAMES    
SUMMARY: NESTED | FIELD | CONSTR | METHOD DETAIL: FIELD | CONSTR | METHOD

fr.ifremer.isisfish.simulator.launcher
Class SSHSimulatorLauncher

java.lang.Object
  extended by fr.ifremer.isisfish.simulator.launcher.SSHSimulatorLauncher
All Implemented Interfaces:
SimulatorLauncher
public class SSHSimulatorLauncher
extends Object
implements SimulatorLauncher

Use a remote simulation server. Upload zip simulation file on server and launch simulation on that file. Isis-Fish must be installed on remote server. Caparmor file layout ($i = plan/as increment) :

All $ISIS-TMP/simulation-$id-* files are deteled after result download. Special case : where $shortid is id of parent job (without increment), they are not deleted after result download.

Version:
$Revision: 3460 $ Last update : $Date: 2011-10-06 22:08:09 +0200 (Thu, 06 Oct 2011) $ By : $Author: chatellier $
Author:
chatellier
See Also:
JSch

Nested Class Summary
protected static class SSHSimulatorLauncher.ControlProgressMonitor
          Redefine a custom progress monitor that update control.
 
Field Summary
protected  freemarker.template.Configuration freemarkerConfiguration
          Freemarker configuration
protected static org.apache.commons.logging.Log log
          Class logger
protected static String QSUB_SCRIPT_TEMPLATE
          Freemarker qsub template.
protected static com.jcraft.jsch.Session sshSession
          Opened session to ssh service.
protected static com.jcraft.jsch.Session sshSftpSession
          Opened session to sftp service.
 
Constructor Summary
SSHSimulatorLauncher()
          Constructor.
 
Method Summary
protected  void clearSimulationFiles(com.jcraft.jsch.Session session, SimulationControl control)
          Remove all $ISIS-TMP/simulation-$id-* files on caparmor.
protected  void closeSSHSession(com.jcraft.jsch.Session session)
          Close ssh session.
protected  File downloadResultsArchive(com.jcraft.jsch.Session session, SimulationControl simulationControl, String md5sum)
          Download simulation zip results.
protected  File downloadResultsMD5File(com.jcraft.jsch.Session sshSession, String simulationId)
          Download remote simulation md5 control file and store its content into temp file.
protected  File downloadSimulationFile(com.jcraft.jsch.Session sshSession, String simulationId, String fileName)
          Download remote simulation control file and store its content into temp file.
 int getCheckProgressionInterval()
          Retourne l'intervalle de temps a utiliser entre deux verification de progresssion.
protected  File getLaunchSimulationScriptFile(String simuationId, String simulationZip, boolean standaloneZip, String simulationResultZip, String preScriptPath, boolean multipleSimulationScript)
          Retourne un fichier temporaire contenant le script de lancement de simulation.
protected  String getRemoteResultArchivePath(String simulationId)
          Get remote simulation zip path.
protected  String getRemoteTempDirectory()
          Get remote directory absolute path.
protected  String getSimulationScriptLaunchContent(String templateName, String simuationId, String simulationZip, boolean standaloneZip, String simulationZipResult, String preScriptPath, boolean multipleSimulationScript)
          Utilise freemarker pour recuperer le contenu du script.
 SimulationStorage getSimulationStorage(SimulationService simulationService, SimulationControl control)
          Retourne le storage où est stockée la simulation.
protected  com.jcraft.jsch.Session getSSHSession()
          Get opened ssh session or try to open a new one.
protected  void initFreemarker()
          Init freemarker configuration.
 int maxSimulationThread()
          Retourne le nombre maximal de thread de simulations simultanées supporté.
protected  void message(SimulationControl control, String message)
          Display message both in commons-logging and control text progress.
protected  com.jcraft.jsch.Session openSSHSession()
          Connect to remote server throw SSH, and return session.
protected  void putSshKeyOnRemoteServer(com.jcraft.jsch.Session session, File sshKey)
          Add ssh key into $HOME/.ssh/authorized_keys file.
protected  void sendStartSimulationRequest(com.jcraft.jsch.Session session, String simulationId, String scriptRemotePath, int lastSimulationNumber)
          Add script in remote qsub queue.
protected  void sendStopSimulationRequest(com.jcraft.jsch.Session session, String simulationId)
          Send qdel request on job.
 void simulate(SimulationService simulationService, SimulationItem simulationItem)
          Execute une simulation.
 void simulationStopRequest(SimulationJob job)
          Called if a started simulation has been ask to stop.
protected  void startSimulation(SimulationItem simulationItem, String simulationid, String simulationRemoteZipPath, String remoteResultZip, String simulationPreScriptPath)
          Start simulation if necessary.
 String toString()
           
 void updateControl(SimulationService simulationService, SimulationControl control)
          Met à jour la progression.
protected  String uploadPreScriptIfNecessary(com.jcraft.jsch.Session session, String simulationId, String simulationPreScript)
          Upload pre script on remote server.
protected  void uploadSimulation(com.jcraft.jsch.Session session, String remoteDirectory, String remoteSimulationZipPath, File simulationFile)
          Perform simulation upload.
protected  String uploadSimulationIfNecessary(com.jcraft.jsch.Session session, SimulationItem simulationItem, String simulationid, File simulationFile)
          Upload simulation if necessary and always return the remote simulation zip path to use.
protected  String uploadSimulationScript(com.jcraft.jsch.Session session, String simulationid, File simulationScript)
          Upload script on remote server.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

log

protected static org.apache.commons.logging.Log log
Class logger

freemarkerConfiguration

protected freemarker.template.Configuration freemarkerConfiguration
Freemarker configuration

QSUB_SCRIPT_TEMPLATE

protected static final String QSUB_SCRIPT_TEMPLATE
Freemarker qsub template.

See Also:
Constant Field Values

sshSession

protected static com.jcraft.jsch.Session sshSession
Opened session to ssh service. Stored in static context to not reask passphrase at each connection.

sshSftpSession

protected static com.jcraft.jsch.Session sshSftpSession
Opened session to sftp service. Stored in static context to not reask passphrase at each connection.

Constructor Detail

SSHSimulatorLauncher

public SSHSimulatorLauncher()
Constructor. Init freemarker.

Method Detail

initFreemarker

protected void initFreemarker()
Init freemarker configuration.

message

protected void message(SimulationControl control,
                       String message)
Display message both in commons-logging and control text progress.

Parameters:
control - control
message - message to display

maxSimulationThread

public int maxSimulationThread()
Description copied from interface: SimulatorLauncher
Retourne le nombre maximal de thread de simulations simultanées supporté.

Specified by:
maxSimulationThread in interface SimulatorLauncher
Returns:
un nombre superieur a 0

getCheckProgressionInterval

public int getCheckProgressionInterval()
Description copied from interface: SimulatorLauncher
Retourne l'intervalle de temps a utiliser entre deux verification de progresssion.

Specified by:
getCheckProgressionInterval in interface SimulatorLauncher
Returns:
intervalle (en secondes);

toString

public String toString()
Overrides:
toString in class Object

simulationStopRequest

public void simulationStopRequest(SimulationJob job)
                           throws RemoteException
Called if a started simulation has been ask to stop. Tell launcher to stop simulation if possible. Try to send a qdel command.

Specified by:
simulationStopRequest in interface SimulatorLauncher
Parameters:
job - job to stop
Throws:
RemoteException - Si pour l'execution de la simulation on utilise des resources distantes (serveurs) et que ceux si non pas pu etre contacte.

simulate

public void simulate(SimulationService simulationService,
                     SimulationItem simulationItem)
              throws RemoteException
Execute une simulation. Dans le cas de ssh:

getSimulationStorage

public SimulationStorage getSimulationStorage(SimulationService simulationService,
                                              SimulationControl control)
                                       throws RemoteException
Retourne le storage où est stockée la simulation. Se connecte au serveur distant et télécharge les résultats de la simulation. Simulation must have been downloaded with updateControl(SimulationService, SimulationControl) before calling this method.

Specified by:
getSimulationStorage in interface SimulatorLauncher
Parameters:
simulationService - le SimulationService qui a initie la simulation
control - le control de la simulation
Returns:
le storage contenant la simulation qui vient d'etre faite ou null en fait au lieu de faire une seul simulation, plusieurs ou ete faite par exemple tout un plan de simulation Cette classe est tout de meme responsable du stockage de chaque simulation dans des storages.
Throws:
RemoteException - Si pour l'execution de la simulation on utilise des resources distantes (serveurs) et que ceux si non pas pu etre contacte.

updateControl

public void updateControl(SimulationService simulationService,
                          SimulationControl control)
                   throws RemoteException
Met à jour la progression. Se connecte au serveur distant et télécharge le fichier de control. Injecte ensuite ce fichier dans le SimulationControl. Essaye aussi de telecharger le fichier md5 de la simulation, et, s'il est present, l'archive de résultat. Supprime tous les fichiers de la simulations apres avoir télécharger les résultats.

Specified by:
updateControl in interface SimulatorLauncher
Parameters:
simulationService - le SimulationService qui a initie la simulation
control - le control de la simulation
Throws:
RemoteException - Si pour l'execution de la simulation on utilise des resources distantes (serveurs) et que ceux si non pas pu etre contacte.

getSSHSession

protected com.jcraft.jsch.Session getSSHSession()
                                         throws com.jcraft.jsch.JSchException
Get opened ssh session or try to open a new one. This method must synchronized.

Returns:
opened ssh session.
Throws:
com.jcraft.jsch.JSchException

openSSHSession

protected com.jcraft.jsch.Session openSSHSession()
                                          throws com.jcraft.jsch.JSchException
Connect to remote server throw SSH, and return session.

Returns:
valid opened session
Throws:
com.jcraft.jsch.JSchException

closeSSHSession

protected void closeSSHSession(com.jcraft.jsch.Session session)
Close ssh session.

Parameters:
session - session to close

putSshKeyOnRemoteServer

protected void putSshKeyOnRemoteServer(com.jcraft.jsch.Session session,
                                       File sshKey)
                                throws com.jcraft.jsch.JSchException
Add ssh key into $HOME/.ssh/authorized_keys file. Just connect and do an "echo xx >> .ssh/authorized_keys"

Parameters:
session - opened session
sshKey -
Throws:
com.jcraft.jsch.JSchException

uploadSimulationIfNecessary

protected String uploadSimulationIfNecessary(com.jcraft.jsch.Session session,
                                             SimulationItem simulationItem,
                                             String simulationid,
                                             File simulationFile)
                                      throws SSHException
Upload simulation if necessary and always return the remote simulation zip path to use.

Parameters:
session - already open valid ssh session
simulationItem - simulation item
simulationid - simulation id
simulationFile - simulation file to upload
Returns:
remote file path or null if errors
Throws:
SSHException - if upload fail

uploadSimulation

protected void uploadSimulation(com.jcraft.jsch.Session session,
                                String remoteDirectory,
                                String remoteSimulationZipPath,
                                File simulationFile)
                         throws SSHException
Perform simulation upload. Create remote temp directory if not exists.

Parameters:
session - already open valid ssh session
remoteDirectory -
remoteSimulationZipPath -
Throws:
SSHException

downloadResultsArchive

protected File downloadResultsArchive(com.jcraft.jsch.Session session,
                                      SimulationControl simulationControl,
                                      String md5sum)
                               throws SSHException,
                                      IOException
Download simulation zip results. MD5 control check sum if done, return null, if checkSum fail. File if configured to auto delete at JVM shutdown.

Throws:
SSHException - if download fail (can happen if remote file doesn't exist
IOException - if download fail (can happen if remote file doesn't exist

downloadSimulationFile

protected File downloadSimulationFile(com.jcraft.jsch.Session sshSession,
                                      String simulationId,
                                      String fileName)
                               throws IOException,
                                      SSHException
Download remote simulation control file and store its content into temp file.

Parameters:
sshSession - valid opened ssh session
simulationId - id de la simulation
fileName - nom du fichier a telecharger
Returns:
downloaded temp file (file have to be manually deleted)
Throws:
IOException
SSHException - if remote file doesn't exists

downloadResultsMD5File

protected File downloadResultsMD5File(com.jcraft.jsch.Session sshSession,
                                      String simulationId)
                               throws IOException,
                                      SSHException
Download remote simulation md5 control file and store its content into temp file.

Parameters:
sshSession - valid opened ssh session
simulationId - id de la simulation
Returns:
downloaded temp file (file have to be manually deleted)
Throws:
IOException
SSHException - if remote file doesn't exists

clearSimulationFiles

protected void clearSimulationFiles(com.jcraft.jsch.Session session,
                                    SimulationControl control)
                             throws IOException,
                                    SSHException
Remove all $ISIS-TMP/simulation-$id-* files on caparmor.

Parameters:
session - valid opened ssh session
control - simulation control
Throws:
IOException
SSHException

uploadSimulationScript

protected String uploadSimulationScript(com.jcraft.jsch.Session session,
                                        String simulationid,
                                        File simulationScript)
                                 throws SSHException
Upload script on remote server.

Parameters:
session - valid opened ssh session
simulationScript - file to upload
Throws:
SSHException - if upload fail

getRemoteResultArchivePath

protected String getRemoteResultArchivePath(String simulationId)
Get remote simulation zip path.

Parameters:
simulationId - simulation id

uploadPreScriptIfNecessary

protected String uploadPreScriptIfNecessary(com.jcraft.jsch.Session session,
                                            String simulationId,
                                            String simulationPreScript)
                                     throws SSHException,
                                            IOException
Upload pre script on remote server. Return path if uploaded or null if no upload needed.

Parameters:
session - valid opened ssh session
simulationId - simulation id
simulationPreScript - script content
Throws:
SSHException - if upload fail
IOException - if upload fail

startSimulation

protected void startSimulation(SimulationItem simulationItem,
                               String simulationid,
                               String simulationRemoteZipPath,
                               String remoteResultZip,
                               String simulationPreScriptPath)
                        throws Exception
Start simulation if necessary. Current simulation can be started later with a PBS multi job.

Parameters:
simulationItem - simulation item (needed for additionnal info, simulation, number, indenpendant, etc...)
simulationid - simulation id
simulationRemoteZipPath - simulation preparation (input) zip path
remoteResultZip - simulation result (output) zip path
simulationPreScriptPath - simulation prescript
Throws:
Exception

getLaunchSimulationScriptFile

protected File getLaunchSimulationScriptFile(String simuationId,
                                             String simulationZip,
                                             boolean standaloneZip,
                                             String simulationResultZip,
                                             String preScriptPath,
                                             boolean multipleSimulationScript)
                                      throws IOException
Retourne un fichier temporaire contenant le script de lancement de simulation. Le fichier temporaire est configuré pour se supprimer tout seul.

Parameters:
simuationId - id de la simulation
simulationZip - zip de la simulation
standaloneZip - standalone simulation zip
preScriptPath - simulation pre script path (can be null)
multipleSimulationScript - if true build a multijob simulation script
Returns:
un Fichier temporaire ou null en cas d'exception
Throws:
IOException - if can't build script

getSimulationScriptLaunchContent

protected String getSimulationScriptLaunchContent(String templateName,
                                                  String simuationId,
                                                  String simulationZip,
                                                  boolean standaloneZip,
                                                  String simulationZipResult,
                                                  String preScriptPath,
                                                  boolean multipleSimulationScript)
                                           throws IOException
Utilise freemarker pour recuperer le contenu du script. Remplace aussi la variable $simulation du template.

Parameters:
templateName - url du template
simuationId - id de la simulation
simulationZip - zip de la simulation
standaloneZip - standalone simulation zip
simulationZipResult - zip resultat de la simulation
preScriptPath - simulation pre script path (can be null)
multipleSimulationScript - if true build a multijob simulation script
Throws:
IOException - if can't get script content

sendStartSimulationRequest

protected void sendStartSimulationRequest(com.jcraft.jsch.Session session,
                                          String simulationId,
                                          String scriptRemotePath,
                                          int lastSimulationNumber)
                                   throws SSHException
Add script in remote qsub queue.

Parameters:
session - valid opened session
simulationId - simulation id (short version for a multiple job)
scriptRemotePath - remote script path
lastSimulationNumber - if >=0 start a multiple pbs job form 0 to lastSimulationNumber
Throws:
SSHException - if call fail

sendStopSimulationRequest

protected void sendStopSimulationRequest(com.jcraft.jsch.Session session,
                                         String simulationId)
                                  throws SSHException
Send qdel request on job.

Parameters:
session - valid opened session
simulationId - simulation id
Throws:
SSHException - if call fail

getRemoteTempDirectory

protected String getRemoteTempDirectory()
Get remote directory absolute path. Don't use File.separator here, caparmor is always unix.

Returns:
remote temp directory path
Overview  Package   Class  Use  Tree  Deprecated  Index  Help 
 PREV CLASS   NEXT CLASS FRAMES    NO FRAMES    
SUMMARY: NESTED | FIELD | CONSTR | METHOD DETAIL: FIELD | CONSTR | METHOD
Copyright © 1999-2012 CodeLutin. All Rights Reserved.