Uh oh, automated test and bad assumptions

So I'm looking at a test that is telling me it Passed, but this test is looking for a docker container and well, no containers are RUNNING yet!  WTF?

This one Returns 0, but we assert we want a 0

assert $r 0 "Should not see db connection warnings"
======================================

root@ip-172-31-40-182:/tmp/HUB4.4.0# sudo /usr/bin/docker logs 40e0533b4f76 2>&1 | grep  "WARN <HIDDEN>- Unable to manage connection" | wc -l

0

======================================

when I strip out 2>&1 we see there's a failure that's been being swallowed:

root@ip-172-31-40-182:/tmp/HUB4.4.0# sudo /usr/bin/docker logs 40e0533b4f76 | grep  "WARN <HIDDEN>- Unable to manage connection" | wc -l

Error: No such container: 40e0533b4f76

0

======================================

Time to refactor!

As always, what I blog about are my views and opinions and only mine, and are never the views or opinions of my past | present | future employers.

From Demo to Deployed in 1 week!

Last week I posted about a cool, impactful, project I worked on and demo'd to the company at our weekly Engineering demo's.  I also posted this about setting up a bastion host so that anyone can  clone the project in git and run it themselves.

The bastion host setup was in preparation to go from demoWare to making the demo work within the Kippernetes CI/CD framework getting my shit merged to master - which it is now!  😎

What was amazing to me is not just the work I did, hell I only got to the 1 yard line, it was folks on my team like +jay vyas and Alan Bradley who took my work over the goal line.  Could I have done it?  Yes, would we have done the work in 1 week?  Maybe, maybe not.  Team effort is needed to succeed, and for me to learn and improve.  Sometimes that's a PR and code review, other times it's just everyone doing what they do best to get it done.

I was so damn happy when I saw the first real CI/CD test get kicked off and work!

See that?  Failing: 0


What's even cooler is today I found that some of the docker-compose tests were failing - and it was because there was a real bug that had been checked in last night!

This really re-enforces a couple things for me:

1. I actually do have the skill set to do DevOps, and my request to transfer to a DevOps role is the right thing for me
2. Automate everything you can, and push back on manually testing beyond doing so as a first pass and as a means to an end of automating said manual test.  Now don't get me wrong, there are some tests that either can't be automated or automating those tests is so complex that there is little to no ROI

More cool shit coming!

Hit me with any comments below!

As always, what I blog about are my views and opinions and only mine, and are never the views or opinions of my past | present | future employers.

Vagrant - now we're talking infra as Code!!

Two weeks ago I'm in an Infrastructure team meeting and we get some news; people are clamoring for a docker-compose ephemeral lab for R&D and Test Engineering to work on and test the Black Duck Software Hub.

Damn, the team's been focusing on more Cloud Native deployments like Kubernetes, and OpenShift - docker-compose wasn't even in our peripheral vision.  Shit!  Okay time for me to make my move.  One of our team's technical leaders is +jay vyas and he asked if I could take ownership of the docker-compose work.  Of course I will.  We decided that I can probably get this done in about a week using Vagrant by HashiCorp. 

So last Tuesday I 'brew install vagrant' and set off to change the world!

So in three days I went from never having spun up any infra as software to a demo to the entire engineering team on Friday.

Doc Vyas helped me with a some seed code and I ran with that to get started. By the end of that 1st day I already had a Vagrant recipe that spun an Ubuntu 16.04 running in EC2.  Hell yea!

Day two I set forth to write some solid bash script that would:
update_os
inst_packs #installs the proper packages I need
add_gpgkey
add_stable_repo
update_packg_index
inst_docker
hello_world
inst_compose
start_hub
hub_ready

These are literally the functions I ended up with, self documenting - w00t!
The most complex code in this whole shebang is this:

=======================================
# Smoke test:  Is HUB ready and healthy
  function hub_ready() {
    TRIES=0
    while [[ $(sudo docker ps -a | grep 'blackduck' | grep "(healthy)" | wc -l) -lt $NUM_CONTAINERS ]] && [[ $TRIES -lt 120 ]] ; do
      sudo docker ps -a;
      sleep 1;
        let TRIES+=1
    done
  }

========================================
I'm just checking that all the hub docker containers are running and healthy, in 120 ticks of the clock.

So once I got all the things working on my local I committed and pushed, and was merged upstream  to our cool infrastructure project over on github:  https://github.com/blackducksoftware/kippernetes

My demo project is in /hack/compose-vagrant-up
and I'm actively working on wiring the demo into the Kippernetes CI/CD, with a goal of completing that work before Christmas break.
I've been blogging a bit about a test bastion host I've built in EC2 to run through the setup and configuration I'll need to do to the upstream Kippernetes bastion host to wire in my demoware.  You can check that out here https://tinyurl.com/sheppvagrantup01

Vagrant up!

As always, what I blog about are my views and opinions and only mine, and are never the views or opinions of my past | present | future employers.

Setting up a CentOS Bastion host for Vagrant

Well I'm up against it, I need to get my Vagrant recipe out of hack this shit together mode, and productize it a bit.
First I need to make it so I'm not 'vagrant up' from my MacBook, this doesn't work well when I need to go home and close my Mac yet keep Vagrant up instances running in EC2.  So what shall I do?  Yea, I could use TMUX and just jump back into a session, but that's lame.

Well how about a bastion host in EC2 that has Vagrant installed so I can push my bits and run them out of EC2?  Sounds cool AF - Let's try it out!

I've got a CentOS 7.x running using an official CentOS.org image off AWS:

Okay and we'll go ahead and use the t2.medium image.

Now that the instance is up let's update it:

Great!  I've gone ahead and downloaded the 64 bit version of Vagrant for CentOS, and using Filezilla uploaded to the AWS instance.  Time to install Vagrant

Cool, now we're cooking!

Now I need to get my CentOS into a DevOps state :)

I suggest running 'yum update' 

Install pre-reqs:

yum install -y ruby, gcc

Now lets install the vagrant aws plugin:

vagrant plugin install vagrant-aws

We need to get Vagrant onto the CentOS box. I'm not going to tell you how to do that, I personally DL to my Mac and then use FileZilla to push the bits to the bastion. Others may want to just curl or wget the bits, do what makes you happy - just stay in your own lane and out of mine. ;)

Vagrant strongly suggests to not install via package managers, as these don't always have the most updated bits, dependencies, etc. So just save yourself the time & hassle and DL and install from the rpm.

Grab Vagrant's CentOS rpm here: https://www.vagrantup.com/downloads.html

Okay, the bits are on the CentOS bastion host, lets install it: rpm -Uvh .rpm

Once the install is complete make sure it is installed: vagrant --version

Kewl, we're ready to rock and roll!

Have fun DevOps peeps!  Vagrant Up!

As always, what I blog about are my views and opinions and only mine, and are never the views or opinions of my past | present | future employers.