Provide a list of Storm topologies to manage

This is a reference part. Refer to the associated overview for a more synthetical view.


Each item of the list has the following attribute:

Name req? Description
name yes Name of the topology.
launching_cmd yes The command to launch this topology. May be 'storm jar ...' for simple case. A launching script for more complex one.
launching_dir no Will cd in this folder before launching 'launching_cmd' Must be an absolute path.
Default: ~
wait_time_secs no The wait_time in seconds provided to the `kill' command (Delay between spouts deactivation and topology destruction)
Default: 30.
timeout_secs no Timeout before raising an error value when waiting a target state.
Default: Value set by storm_relay:default_timeout_secs
ranger_policy no Definition of Apache Ranger policy bound to this topology. Parameters are same as storm_ranger_policies except than topologies should not be defined.
The policy name can be explicitly defined. Otherwise, a name will be generated as "_<topology>_".
See example below for more information
no_remove no Bolean. Used only for eventual associated ranger policy. Setting to yesprevent this policy to be remove when HADeploy will be used in REMOVE mode.
Default: no.
when no Boolean. Allow conditional deployment of this item.
Default True

Asynchronous launch

To preserve launching time, HADeploy launch all topologies in a simultaneous way, by using some asynchronous procesing.

This can lead to difficulty to debug, as failing launch des not raise any error, except a timeout after quite long time. More on that here


- name: storm1
  launching_dir: ${basedir1}
  launching_cmd: "storm jar ./storm1-${storm1_version}-uber.jar com.mydomain.storm1.ClusterTopology"
  wait_time_secs: 10
- name: storm2
  launching_dir: ${basedir2}
  launching_cmd: "./"
  wait_time_secs: 15
- name: storm2
  launching_dir: /opt/storm2
  launching_cmd: "./"
  wait_time_secs: 10
    audit: yes
    enabled: yes
    - users:
      - stormrunner
      - 'fileDownload'
      - 'killTopology'
      - 'rebalance'
      - 'activate'
      - 'deactivate'
      - 'getTopologyConf'
      - 'getTopology'
      - 'getUserTopology'
      - 'getTopologyInfo'

The launching cmd or script

Here is some requirement about the launching command or script.

For kerberos: client_jaas.conf

In a kerboros secured context, the storm command use a configuration file to define how it will authenticate against the Storm server.

For HADeploy usage, this file must be configured to use the current ticket in the local cache. Unfortunately, this is not always the case. Sometime, it may be configured to use another principal and keytab.

If this is the case, we suggest to explicitly provide such a configuration file: For example, in an Hortonworks context, one can write:

storm -c jar ./myjar.jar com.mydomain.mytopology.ClusterTopology  $1

Here is the content of a client_jaas.conf we can use:

StormClient { required