Anno nuovo, configuration management nuovo…

Buon anno a tutti i lettori di devopsrecipes.info ūüôā

Qualche giorno fa, approfittando dell’apparente calma lavorativa tipica delle feste natalizie, ho deciso di studiare GO (https://golang.org/).

Molto figo, ma soprattutto, usare i puntatori mi ha fatto tornare indietro nel tempo quando “smanettavo” pesantemente¬†in C.

Quel che √® uscito fuori, tra un p√≤ i programmi pieni di “foo” e “foobar” senza senso, √® stato Congruit. Un nuovo configuration management tool scritto in GO ma che opera prettamente in Bash…

Link al repo Github

logomakr_9879lg

Naturalmente √® molto minimale ma ha un non so che di “Bash” che lo colora molto di Sys-admin.

Diciamo che nell’ottica Devops √® un p√≤ pi√Ļ Ops. Pull request al repo sono benvenute ūüėČ

Saluti

Advertisements

A simple recipe for MongoDB clusters

Hi everybody!

my task of today, was to configure a MongoDB¬†with redundancy and high availability…

I decided to write my own Chef cookbook to configure “replica”

Below the most important parts:

Install MongoDB packages

Screen Shot 2016-10-14 at 21.13.03.png

Use template for the main configuration file

screen-shot-2016-10-14-at-21-15-04

Enable Linux service of MongoDB at the boot

screen-shot-2016-10-14-at-21-15-57

What about the replica’s configuration?

I suggest to use a custom LWRP that execute rs.initiate() to declare the replica set, rs.add(); to add primary and secondary servers, rs.addArb() to add arbiter servers. You can loop over hashes like this:

foobar => { “secondaries” => [“mynode01:27017″,”mynode02:27017”]}

and a Chef provider like this

screen-shot-2016-11-09-at-21-38-31

I need to test my cluster, so let’s prepare a Vagrant file like this in order to manage¬†all virtual machines¬†concurrently

screen-shot-2016-10-14-at-21-23-16

Finally, Haproxy as reverse proxy and load balancer! Use autodiscovery (https://github.com/hw-cookbooks/haproxy) in order to find automatically the backends and “health check” to point to the right node after a new primary server’s promotion.

screen-shot-2016-10-14-at-21-25-20

 

Ad maiora!

Cooking on Windows Sever..

Hi guys!

I have to manage Windows Server 2008 using Chef..

First of all, I need to choose the strategy for bootstrapping the new node and I am going to use an unattended bootstrap..

Configuring network card to talk with 33.33.33.10 (my Chef server)

Screen Shot 2016-08-04 at 18.19.09

Download and install  chef-client for Windows from https://downloads.chef.io/chef-client/windows/

Screen Shot 2016-08-04 at 15.41.06.png

Copy client.rb and validation.pem to c:\chef and launch chef-client from the command line.

This my new Windows node!

Screen Shot 2016-08-04 at 18.09.15

Let’s prepare a simple cookbook named mywindows:

knife cookbook create mywindows

Let’s insert¬†the following resources in recipes/default.rb

#Modify the hosts file
template "C:\\Windows\\System32\\drivers\\etc\\hosts" do
  source "hosts.erb"
  action :create
end

 

#Restart an array of services
[ "Dnscache" , "UxSms" ].each do |s|
 service s do
    action :restart
   end
end

 

#Update Group Policy
execute 'gpupdate' do
  command "gpupdate.exe"
end

 

#Create a key into the registry
windows_registry 'HKCU\Software\Test' do
  values 'MySuperKEy' => Time.now
  action :create
end

 

#Install Putty
windows_package 'Putty' do
  source 'C:\\putty-0.60-installer.exe'
  installer_type :inno
  action :install
end

 

#Enable a Windows Feature
windows_feature "WindowsServerBackup" do
  action :install
end

Run chef-client and all works fine!

Screen Shot 2016-08-04 at 18.34.08.png

Bye!

 

Install Chef Server on Suse Linux Enterprise 11

Hi Folks!

Today I dealt with a problem… and I found a solution because Chef is a great tool!

At moment there is not an RPM for Suse Linux available from the official website, but this does not matter ūüôā

Problem: Install Chef Server, Chefdk, Chef-manage into a Suse Linux Enterprise 11 virtual machine without installing the rpm packages of RHEL systems.

Screen Shot 2016-08-02 at 19.02.17.png

This is what you can do:

  1. Dowload the following packages:
    • chef-server-core-12.8.0-1.el6.x86_64.rpm,
    • chefdk-0.16.28-1.el6.x86_64.rpm,
    • chef-manage-2.4.1-1.el6.x86_64.rpm
  2. Extract all stuff from RPM with:
    • rpm2cpio ¬†chef-manage-2.4.1-1.el6.x86_64.rpm ¬† | cpio -idmv

  3. Move content of the extraction to the correct folders: /opt/{chef,chef-manage,opscoode}
  4. Set PATH=”/opt/opscode/bin:/opt/chefdk/bin/:/data/opt/chef-manage/bin:$PATH” in your profile login script
  5. chef-server-ctl reconfigure
  6. chef-manage-ctl reconfigure
  7. again chef-server-ctl reconfigure

At the end all services are up and running

Screen Shot 2016-08-02 at 19.12.35.png

and my workstation too ūüôā

Chef Automate – Installation guide

Hi guys!

let’s take a look of Chef Automate.. In this post we will se how to install it quickly.

Screen Shot 2016-07-28 at 12.38.39

I will install it through Vagrant but you can use my cookbook into a Chef Server.


Requirements:

  1. a Chef Server. Change¬†default[‘chef_automate’][‘chef_server’][‘url’] with the correct IP
  2. an user’s key (client.pem) of a member of your¬†Chef Server Organization. Change¬†default[‘chef_automate’][‘key’][‘base’] and¬†default[‘chef_automate’][‘key’][‘name’] with your values
  3. a Virtualbox private network 192.168.56.0 (or you can set a port forwarding into the Vagrantfile in order access to the webserver through  http://127.0.0.1)
  4. a delivery.license file. Put it into the cookbook directory. You can see it on /vagrant into the guest vm.

Start the provision..

  1. git clone https://github.com/lucky-sideburn/chef_automate.git
  2. vagrant up
  3. https://automate-box01/e/umbrella_corporation/ or use your preferred internal IP or use port forwarding to 127.0.0.1
  4. Select your enterprise

Screen Shot 2016-07-28 at 19.47.56

5. Enjoy!

Screen Shot 2016-07-28 at 20.21.16


 

Thanks!

Eugenio Marzo – Devops Engineer @Sourcesense


sourcesenseLogo266

Build chef LWRP and manage OpenSSH server banner with Chef

Hi guys,
in this article we will se how to build a small LWRP Chef cookbook..The final result will be:

ssh_banner_banner “banner” do
banner_file _banner_file
sshd_config_file node[‘ssh_banner’][‘sshd_config_file’]
paranoic_mode true
action :create
notifies :restart, “service[sshd]”
end


If “paranoic mode” is true, chef will change configuration file and restart sshd, but after 20 seconds (by default) it will do a rollback of configuration

You can try it use Vagrant and Virtualbox..

1. clone git repo from github:

  git clone https://github.com/EugenioMarzo/cookbook-ssh-banner.git

2. show the new banner to copy:

 cat files/default/chef_ssh_banner

3. start vagrant virtual machine:

  vagrant up

4. once the deploy is completed:

Screen Shot 2014-09-01 at 16

Let’s see how to create a simple LWRP:

1. Declare variables in resources/banner.rb

actions :create, :delete

default_action :create

attribute :sshd_config_file, :kind_of => String

attribute :banner_file, :kind_of => String

attribute :paranoic_mode

2. create an action in providers/banner.rb.. Let’s see the :delete function :

action :delete do
#check if ssh banner file is present
check_banner_file new_resource.banner_file
#check if paranoic mode is enabled
paranoic_mode

if ::File.open(new_resource.sshd_config_file).grep(/Banner\ .*/).size >= 1
Chef::Log.info(“Deleting SSH Banner..”)
execute ” sed -i s/Banner\\\ .*//g #{new_resource.sshd_config_file}”

#the next function will inform that the state is changed, an action has been done. This is important because after this will be executed a notify action like a sshd restart

new_resource.updated_by_last_action(true)
else
Chef::Log.info(“SSH Banner not found … doing nothing..”)
new_resource.updated_by_last_action(false)

end

end

3. use it in a recipe.. Delete a banner:

ssh_banner_banner “banner” do
banner_file _banner_file
sshd_config_file node[‘ssh_banner’][‘sshd_config_file’]
paranoic_mode false
action :delete
notifies :restart, “service[sshd]”
end

4. for adding a banner use:

ssh_banner_banner “banner” do
banner_file _banner_file
sshd_config_file node[‘ssh_banner’][‘sshd_config_file’]
paranoic_mode false
action :create
notifies :restart, “service[sshd]”
end

Setting up Wildfly8 Cluster in 5 minutes with Chef and Vagrant

Hi! This is my first post of this blog and I would like start with my last cookbook “wildfly-clu”. ¬† We will create a simple Wildlfy cluster(domain mode) composed by 3 servers. For this test I will use CentOS release 6.3 (Final).

Final result:

You can reach the app directly from the nodes:
http://myserver1:8080/helloworld/
http://myserver2:8080/helloworld/
http://myserver3:8080/helloworld/

WILDLFYCONS

haproxy

 

Quick HowTo:
check if the vagrant-berkshelf plugin is installed. If not, launch ” vagrant plugin install vagrant-berkshelf”
git clone https://github.com/EugenioMarzo/cookbook-wildfly-clu
cd wildfly-clu
vagrant up

Detailed Description:

Node1 (myserver1) => Domain controller Р Application Server РReverse proxy

Node2 (myserver2) => Slave – Application Server

Node3 (myserver3) => Slave – Application Server

Prerequisites:  Virtualbox 4.3.10 ,Vagrant 1.4.3 , Ruby 1.9.3,Git

Let’s start..

1. Configure your /etc/hosts in order to resolve locally the name of all VMs
33.33.33.11 myserver1
33.33.33.13 myserver3
33.33.33.12 myserver2

2.  Clone the cookbook
`git clone https://github.com/EugenioMarzo/cookbook-wildfly-clu`

3. Quick overview of Vagrantfile
Vagrant can configure multiple virtual-machines. An example of ./mycookbook/Vagrantfile:
#This will be the configuration for myserver1. Being a cluster you will have the same configuration for myserver2 and myserver3

config.vm.define "myserver1" do |myserver1|

myserver1.vm.hostname = “myserver1”
myserver1.vm.network :private_network, ip: “33.33.33.11”
myserver1.vm.network :public_network
myserver1.vm.provision :chef_solo do |chef|
chef.json = {
:java => {:jdk_version => “7”}

}

chef.run_list = [
“recipe[java]”, ¬†#to install java
“recipe[wildfly-clu::default]”, ¬†#install wildfly
“recipe[wildfly-clu::logs]”, #log rotation
“recipe[wildfly-clu::domain]” # when is in run_list configure the domain mode

]
end
end

3. show the VMs configured in Vagrantfile

check if the vagrant-berkshelf plugin is installed. If not, launch ” vagrant plugin install vagrant-berkshelf”
root@myclient1:~/vagrantlab/wildfly-clu# vagrant status
Current machine states:

myserver1 not created (virtualbox)
myserver2 not created (virtualbox)
myserver3 not created (virtualbox)

 

4. a quick overview of the most important attributes of the cookbook

Version and URL of Wildfly8
default['wildfly-clu']['wildfly']['version'] = "8.0.0"
default['wildfly-clu']['wildfly']['url']="http://download.jboss.org/wildfly/8.0.0.Final/wildfly-8.0.0.Final.tar.gz"

#######################################################################
## Set the following variable to true if you want use the domain mode.
default[‘wildfly-clu’][‘mode’][‘domain’] = true
##
#######################################################################

#if you create this file the recipe will not change domain.xml,host.xml and mgmt-******.properties after the first installation
default[‘wildfly-clu’][‘wildfly’][‘lock’] = “/usr/local/#{node[‘wildfly-clu’][‘name’]}/conf.lock”

The cluster schema

default[‘wildfly-clu’][‘cluster_schema’] = {
“myserver1” => { :role => “domain-controller” ,
:ip => “33.33.33.11”,
:port_offset => “0” },
“myserver2” => { :role => “slave” ,
:ip => “33.33.33.12” ,
:master => “myserver1” ,
:port_offset => “0”},
“myserver3” => { :role => “slave” ,
:ip => “33.33.33.13” ,
:master => “myserver1” ,
:port_offset => “0” }

}

#set this to true in order to deploy an helloworld application
default[‘wildfly-clu’][‘wildfly’][‘deploy_hello_world’] = true

#set this to true in order to configure an haproxy with the slaves declared in the cluster_schema
default[‘wildfly-clu’][‘wildfly’][‘haproxy’] = true

#DEFAULT java options to use in all slaves and the master for run the application.
default[‘wildfly-clu’][‘java_opts’] = {
“heap-size” => “64m”,
“max-heap-size” => “64m”,
“permgen-size” => “64m”,
“max-permgen-size” => “64m” }

5. Setting up the test environment with Vagrant

root@myclient1:~/vagrantlab/wildfly-clu# vagrant up

Bringing machine ‘myserver1’ up with ‘virtualbox’ provider…
Bringing machine ‘myserver2’ up with ‘virtualbox’ provider…
Bringing machine ‘myserver3’ up with ‘virtualbox’ provider…

#downloading the virtualbox machine used as template
[myserver1] Importing base box ‘Berkshelf-CentOS-6.3-x86_64-minimal’…
Progress: 90%

[myserver1] Available bridged network interfaces:
1) eth0
2) virbr0
3) lxcbr0
4) virbr1
#Choose 1 if you want bridge the network cards of the VM to eth0

[myserver1] Booting VM…
[myserver1] Waiting for machine to boot. This may take a few minutes…
[myserver1] Machine booted and ready!
[myserver1] Configuring and enabling network interfaces.

Through Chef-Solo will be configured the VMs just created. Below the most important steps
Running chef-solo…
[2014-04-03T13:25:55+00:00] INFO: *** Chef 10.14.2 ***
[2014-04-03T13:26:01+00:00] INFO: Run List is

, recipe[wildfly-clu::default], recipe[wildfly-clu::logs], recipe[wildfly-clu::domain]]
[2014-04-03T13:26:08+00:00] INFO: package[java-1.7.0-openjdk] installing java-1.7.0-openjdk-1.7.0.51-2.4.4.1.el6_5 from upda
[2014-04-03T13:26:35+00:00] INFO: package[java-1.7.0-openjdk-devel] installing java-1.7.0-openjdk-devel-1.7.0.51-2.4.4.1.el6_5 from updates repository

#Downloading Wildfly..
[2014-04-03T13:26:46+00:00] INFO: user[wildfly] created
[2014-04-03T13:27:00+00:00] INFO: remote_file[wildfly] updated
[2014-04-03T13:27:00+00:00] INFO: remote_file[wildfly] owner changed to 502
[2014-04-03T13:27:00+00:00] INFO: remote_file[wildfly] group changed to 503
[2014-04-03T13:27:00+00:00] INFO: remote_file[wildfly] mode changed to 775
[2014-04-03T13:27:00+00:00] INFO: remote_file[wildfly] sending run action to bash[wildfly_extract] (immediate)
[2014-04-03T13:27:01+00:00] INFO: bash[wildfly_extract] ran successfully
[2014-04-03T13:27:01+00:00] INFO: bash[wildfly_extract] sending create action to link[/usr/local/wildfly] (immediate)
[2014-04-03T13:27:01+00:00] INFO: link[/usr/local/wildfly] created
[2014-04-03T13:27:01+00:00] INFO: link[/usr/local/wildfly] sending create action to template[/etc/default/wildfly.conf] (immediate)

# Copy configuration read by init script
[2014-04-03T13:27:01+00:00] INFO: template[/etc/default/wildfly.conf] updated content

#Copy configuration for domain mode
[2014-04-03T13:27:06+00:00] INFO: template[/usr/local/wildfly/domain/configuration/domain.xml] mode changed to 775

#installing Haproxy
[2014-04-03T13:27:06+00:00] INFO: package[haproxy] installing haproxy-1.4.24-2.el6 from base repository

#deploy Hello world!
[2014-04-03T13:27:15+00:00] INFO: cookbook_file[helloworld.war] sending run action to bash[deploy_helloworld] (delayed)
[2014-04-03T13:27:19+00:00] INFO: bash[deploy_helloworld] ran successfully
[2014-04-03T13:27:19+00:00] INFO: Chef Run complete in 77.838215277 seconds

The same operation will be for each virtual machines described in Vagrantfile.

 

6. a quick overview inside the virtual machines:

#use it to connect via SSH
root@myclient1:~/vagrantlab/wildfly-clu# vagrant ssh myserver1

[vagrant@myserver1 ~]$ sudo su
[root@myserver1 vagrant]# cat /etc/redhat-release
CentOS release 6.3 (Final)

[root@myserver1 vagrant]# /etc/init.d/wildfly status
wildfly is running (pid 5032)

#cron job for rotate logs
[root@myserver1 vagrant]# cat /var/spool/cron/root
# Chef Name: Wildfly log rotation 0
0 0 * * * find /usr/local/wildfly/domain/log -name ‘*’ -a ! -name ‘*.gz’ -mtime +1 -a ! -name ‘console.log’ -a ! -name ‘boot.log’ -exec gzip ‘{}’ ;
# Chef Name: Wildfly log rotation 1
0 0 * * * find /usr/local/wildfly/domain/log -name ‘*.txt.gz’ -mtime +30 -exec rm -f ‘{}’ ;

[root@myserver1 vagrant]# cat /etc/default/wildfly.conf
export JBOSS_USER=wildfly
export JBOSS_HOME=/usr/local/wildfly
export JBOSS_CONSOLE_LOG=/usr/local/wildfly/domain/log/console.log
export JBOSS_MODE=”domain”

Posted 3th April by Eugenio Marzo