Skip to content

Jenkins Jagger Easy Deploy how it works

Jump to bottom
amikryukov edited this page Mar 13, 2013 · 3 revisions

What actually going on in plug-in easy deploy step.

The main idea of it is to create deploying script automatically.

There are 3 main steps in script:

  • deploy each Jagger node
  • start nodes
  • collect results and logs

Deployment

Deployment of each node is similar, so let us consider Master node deployment.

Deploying part implies:

  • Deleting (if exists) path 'runned_jagger' in user directory. Creating new.

      $ ssh master.name@master.address "rm -rf runned_jagger"
  $ ssh master.name@master.address "mkdir runned_jagger"

  • Sending Jagger suite.

      $ scp JAGGER_TEST_SUITE.zip master.name@master.address:runned_jagger

  • Unzip Jagger suite.

      $ ssh master.name@master.address "unzip runned_jagger\JAGGER_TEST_SUITE.zip -d runned_jagger" > \dev\null

  • Killing previous Jagger process.

      $ ssh master.name@master.address "runned_jagger\stop.sh"
  $ ssh master.name@master.address "runned_jagger\stop_agent.sh"

Starting Jagger Nodes

Starting goes through start scripts (start.sh,start_agent.sh) but with specific properties.

Starting Master Node

Master starts after starting Kernels and Agents. Plugin waits while Master finished.

$ ssh master.name@master.address "cd runned_jagger; ./start.sh ./enviroment.properties '
    -Dchassis.conditions.min.agents.count=1 
    -Dchassis.conditions.min.kernels.count=1 
    -Dchassis.storage.rdb.client.driver=com.mysql.jdbc.Driver 
    -Dchassis.storage.rdb.client.url=jdbc:mysql://address:port/name_of_db 
    -Dchassis.storage.rdb.username=user.name 
    -Dchassis.storage.rdb.password=user.password 
    -Dchassis.storage.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect 
    -Dchassis.roles=MASTER,COORDINATION_SERVER,HTTP_COORDINATION_SERVER 
    -Dchassis.coordinator.zookeeper.endpoint=master.address:2181 
    -Dchassis.storage.fs.default.name=hdfs://master.address/ 
    -Dadditional.property=some.overriden.value 
    -Danother.additional.property=some.concrite.value 
 '"

Let us look deeper, and handle this command line by line.

 ssh master.name@master.address "cd runned_jagger; ./start.sh ./enviroment.properties '

Nothing but starting start.sh script with environment.properties that was specified by the user in plug-in, or default environment properties if last was not specified. If Ssh Key Path field defined, then -i ssh option takes place with the specified path. If Set JAVA_HOME field defined, there is an additional step in ssh command. actually

 ssh master.name@master.address "cd runned_jagger; export SPECIFIED_JAVA_HOME; ./start.sh ./enviroment.properties '

 -Dchassis.conditions.min.agents.count=1

Number of Agents specified in the plugin.

 -Dchassis.conditions.min.kernels.count=1

Number of Kernels specified in the plugin.

 -Dchassis.roles=MASTER,COORDINATION_SERVER,HTTP_COORDINATION_SERVER

Services that will start on the given node. Here we can see Master node. If an external data base has not been declared RDB_SERVER role adds to chassis.roles property.

 -Dchassis.coordinator.zookeeper.endpoint=master.address:2181 
 -Dchassis.storage.fs.default.name=hdfs://master.address/

Point to Master node because it has COORDINATION_SERVER role at the same time. It should be noted that by default this property point to Master node with default port. If you want to change port, you should override chassis.coordinator.zookeeper.endpoint with Additional Properties and clientPort with Additional Properties field in plug-in or edit the properties file that pointed in plug-in (more about properties).

 -Dchassis.storage.rdb.client.driver=com.mysql.jdbc.Driver 
 -Dchassis.storage.rdb.client.url=jdbc:mysql://address:port/name_of_db 
 -Dchassis.storage.rdb.username=user.name 
 -Dchassis.storage.rdb.password=user.password 
 -Dchassis.storage.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect

RDB Properties that comes from Use External Data Base field. If this field not specified the same properties but with H2 database will take place.

 -Dadditional.property=some.overriden.value 
 -Danother.additional.property=some.concrite.value

This is properties from Aditional Properties field in plug-in. It comes after any other properties that give you the oportunety to override any properties that specifies above.

Starting Kernel

Kernel starts and waits for Master node.

 ssh -f -i ssh.key.path.kernel kernel.name@kernel.address "cd runned_jagger; ./start.sh ./enviroment.properties '
    -Dchassis.roles=KERNEL 
    -Dchassis.storage.rdb.client.driver=com.mysql.jdbc.Driver 
    -Dchassis.storage.rdb.client.url=jdbc:mysql://address:port/name_of_db 
    -Dchassis.storage.rdb.username=user.name 
    -Dchassis.storage.rdb.password=user.password 
    -Dchassis.storage.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect 
    -Dchassis.coordinator.zookeeper.endpoint=master.address:2181 
    -Dchassis.storage.fs.default.name=hdfs://master.address/ 
    -Dadditional.property=some.overriden.value 
    -Danother.additional.property=some.concrite.value 
' > /dev/null 2>&1"

As we can see Kernel starts pretty much like Master. Except some moments.

 ssh -f -i ssh.key.path.kernel kernel.name@kernel.address "cd runned_jagger; ./start.sh ./enviroment.properties '

-f option execute command in daemon mode. So script continue to execute after Kernel starts.

    -Dchassis.roles=KERNEL

As this node is Kernel only KERNEL role specify.

' > /dev/null 2>&1"

It means that we do not want to store or see a logout anywhere. It is not necessary. Passing logs to jnekins will take place anyway.

Starting Agent

Agent starts and waits for Master node.

ssh -f -i agent.ssh.key.path agent.name@agent.address "cd runned_jagger; export path/to/java; ./start_agent.sh '
  	JAVA_OPTIONS 
  	-Dchassis.coordination.http.url=http://master.address:8089 
  	-Djmx.enabled=true 
  	-Djmx.services=localhost:9985,localhost:9754 
  	-Dadditional.property=some.overriden.value 
  	-Danother.additional.property=some.concrite.value 
' > /dev/null 2>&1"

Additional properties adds at the end like with Starting Master node. Line by Line.

ssh -f -i agent.ssh.key.path agent.name@agent.address "cd runned_jagger; export path/to/java; ./start_agent.sh '

Starts in daemon mode. Just execute start_agent.sh script with specific properties.

  	-Dchassis.coordination.http.url=http://master.address:8089

Points to Master node with default port. If you want to change port, you should override chassis.coordination.http.url with Additional Properties and chassis.coordination.http.port with Additional Properties field in plug-in or edit properties file that pointed in plug-in (more about properties).

  	-Djmx.enabled=true

Enable JMX or not. Taking from Use JMX field. If it is false, then monitoring information would not be collecting.

  	-Djmx.services=localhost:9985,localhost:9754

If Use JMX field checked then jmx.services property should take place. Ports taking from Port(s) field.

' > /dev/null 2>&1"

The same meaning as in Starting Kernel.

Results

This part consists of next steps:

  • Copy reports

      scp master.name@master.address:"runned_jagger\*.xml runned_jagger\*.pdf runned_jagger\*.html" result

It copies every xml,pdf,html file that appears in runned_jagger path on the master machine to result path in the jenkins job workspace.

  • Copy logs

      scp master.name@master.address:runned_jagger\*.log* result

Copy logs from node, zip log file(s) into one file. It comes for every jagger node, so all logs appear in result path in jenkins job workspace.

  • Stop Agents

      ssh -i agent.ssh.key.path agent.name@agent.address "runned_jagger\stop_agent.sh"

Same as in deploying part.

  • ZIP results

      cd result
  zip -9 report.zip *.pdf *.html *.xml

On this step every pdf,xml,html file zips in one file.

Clone this wiki locally