Is it possible (and how) to specify a shell script somewhere which will be executed each time a new node is added to Ambari cluster?
I'm using HDP Ambari for that and I would like to add some symbolic links when setup of new node is completed, but I want to automatize that so that I (or someone else) don't forget it.
There is no functionality that currently exists that will enable you to execute a script when a node is added to the cluster. What you're asking for is a custom hook. You would have to look through the Ambari source code and see if you can define a custom hook for the stack. There are a few hooks provided in each stack, for examples see: https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks
Related
I'm trying to test a kafka stream on jmeter using the pepper box config, but each time I try adding java request parameters it goes back to the default parameters without saving the ones I have added. I have tried the recommendations on here of adding the underscore, so _ssl.enabled, but the params are still disappearing. Any recommendations? Using jmeter5.3 and pepper-box1.0
I believe you need to put your SSL properties to the PepperBoxKafkaSampler directly, there are pre-populated placeholders which you can change and the changes persist.
The same behaviour is for Java Request Defaults
It might be the case your installation got corrupt somehow or there is a conflict with another JMeter Plugin, check jmeter.log file for any suspicious entries
In the meantime you may find Apache Kafka - How to Load Test with JMeter article useful
I had the same issue. I got around this issue by cloning the pepperbox repository https://github.com/GSLabDev/pepper-box and made changes to the PepperBoxKafkaSampler.java file, updated the setupTest() method with your props. You can also add the parameters making use of the .addArgument() method (used in PepperBoxKafkaSampler.java) to make the parameters available in jmeter.
Rebuild the repo using maven mvn clean install replace the old pepperbox jar in jmeter/lib/ext with your new built jar.
I am creating AWS EMR using cloudformation template. I need to run the steps parallel. For that I am trying to change the YARN Scheduler from FIFO to fair / capacity scheduler.
I have added:
yarn.resourcemanager.scheduler.class : 'org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler'
Do I need to add FairScheduler.xml file in conf.empty folder? If so, can you please share the xml file.
and if I want to add fairscheduler.xml through cloudformation template, then do I need to use bootstrap for it? if so could you provide me the bootstrap file please.
Looks like even though after changing the scheduler, EMR won't allow to run jobs concurrently.
You can configure your cluster by specifying the configuration in cloud-formation scripts.
This a example to configure
- Classification: fair-scheduler
ConfigurationProperties:
<key1>: <value1>
<key2>: <value2>
- Classification: yarn-site
ConfigurationProperties:
yarn.acl.enable: true
yarn.resourcemanager.scheduler.class: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler
Please follow these -
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-configuration.html
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps.html
EMR recently allows you to run multiple steps in parallel -
https://aws.amazon.com/about-aws/whats-new/2019/11/amazon-emr-now-allows-you-to-run-multiple-steps-in-parallel-cancel-running-steps-and-integrate-with-aws-step-functions/
I am trying to make a Django server on AWS. My django app depends on some mathematical python libraries like numpy, scipy, sklearn etc. However there is an issue for which I need to this after every deployment
sudo nano /etc/httpd/conf.d/wsgi.conf
---------------------------------------
add this line in the file
WSGIApplicationGroup %{GLOBAL}
---------------------------------------
sudo /etc/init.d/httpd reload
Basically I need "WSGIApplicationGroup %{GLOBAL}" in my wsgi.conf file otherwise I get 504. I am using a Custom AMI built on top of Amazon Linux 2014 and I am using EB CLI for deployment. However whenever I deploy the wsgi.conf is reset and it does not contain the line that I have added previously and I need to manually SSH into the EC2 instance and do this task myself. It gives a overhead on every deployment and its also not feasible once we scale up (cloning or creating instances also resets it). So is there a way that this will be automatically done after every deployment ?
The content of the wsgi.conf is fixed, so basically I can make a script easily to create it but the issue is how to trigger the script automatically ?
PS:I am new to AWS
You need to use AWS Elastic Beanstalk feature called .ebextensions: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html
In your case you can't use Files or Commands sections, because:
The commands are processed in alphabetical order by name, and they run
before the application and web server are set up and the application
version file is extracted.
You need to use Container_commands section:
They run after the application and web server have been set up and the
application version file has been extracted, but before the
application version is deployed.
Example .ebextensions/01wsgi.config (not tested :-))
container_commands:
apache_reload:
command: |
echo "WSGIApplicationGroup %{GLOBAL}" >> /etc/httpd/conf.d/wsgi.conf
/etc/init.d/httpd reload
Feel free to tweak my example as you want, for example you can copy your temporary wsgi.conf file somewhere and then replace original in Container_commands section.
I use jboss fuse 6.0.0 in windows and start the container using the bin/fuse.bat. The etc/users.properties is modified to add the line admin=admin,admin.
At first the admin command acts as normal. I have admin:list showing all the containers, admin:create to create the child containers.
Then I followed the instructions of
https://access.redhat.com/site/documentation/en-US/JBoss_Fuse/6.0/html/Getting_Started/files/Deploy-Fabric-Create.html
and create a fabric use the command fabric:create --clean. After that the admin command is gone! I get Command not found: admin:list, and I can no longer list the child containers created by admin:create. The fabric:container-list command only enumerates the containers created by the fabric:container-create-child command.
Does any one experienced this problem before? Is it normal? How can I get the admin commands back?
This is expected, when you create fabric, then fabric is managing the containers. So you should use fabric commands to create/manage your containers.
I am trying to get ActiveMQ server running on a RaspberryPI Debian Squeeze box and all appears to be installed correctly but when I try and start the service I am getting the following...
root#raspberrypi:/var/www/activemq/apache-activemq-5.7.0# bin/activemq
INFO: Loading '/etc/default/activemq'
INFO: Using java '/usr/jre1.7.0_07/bin/java'
/usr/jre1.7.0_07/bin/java: 1: /usr/jre1.7.0_07/bin/java:ELF0
4°: not found
/usr/jre1.7.0_07/bin/java: 2: /usr/jre1.7.0_07/bin/java: Syntax error: "(" unexpected
Tasks provided by the sysv init script:
restart - stop running instance (if there is one), start new instance
console - start broker in foreground, useful for debugging purposes
status - check if activemq process is running
setup - create the specified configuration file for this init script
(see next usage section)
Configuration of this script:
The configuration of this script can be placed on /etc/default/activemq or /root/.activemqrc.
To use additional configurations for running multiple instances on the same operating system
rename or symlink script to a name matching to activemq-instance-<INSTANCENAME>.
This changes the configuration location to /etc/default/activemq-instance-<INSTANCENAME> and
$HOME/.activemqrc-instance-<INSTANCENAME>. Configuration files in /etc have higher precedence.
root#raspberrypi:/var/www/activemq/apache-activemq-5.7.0#
It looks like there is an error somewhere but I am a fairly newbie at this and don't know where to look.
Just adding an answer since,becoz as per the documentation , the command is wrong
to start activemqm, use
Navigate to [installation directory/bin] and run ./activemq start or simple bin/activemq start
if you want to see it live in a window use bin/activemq console
To stop, you have to kill the process
The default ActiveMQ "getting started" link sends u here : http://activemq.apache.org/getting-started.html which is the "Getting Started Guide for ActiveMQ 4.x".
If you scroll down main documentation page, you will find this link with the proper commands :
http://activemq.apache.org/version-5-getting-started.html