Microservices & Containers: Insights from the Experts

Microservices webinar cover

Last month, Sysgain's own Parthu Kishen participated in a webinar hosted by Logentries on Microservices and Containers. Specifically, the webinar addressed the overall usage of microservices and containers, and focused on answering questions relating to how microservices and containers are used, what the benefits of using them are, and what their most notable challenges are. The webinar featured knowledgeable experts from Logentries, XebiaLabs, Cohaesus, and Sysgain, who provided their own insights and experiences as they related to microservices and containers.

We are going to share Parthu Kishen's key takeaways of the discussion here, in this blog post, so that you may learn more about microservices and containers, and how they can help you achieve your business goals.

If you're interested in seeing the whole session, with input and questions from all hosts, we encourage you to watch the recording here, on Logentries' web site.

Microservices and containers are quickly growing in popularity, mostly due to the fact that they speed up the delivery process, while also simplifying the management of it. Containerized systems and microservices are being used to replace the "monolithic" applications previously (or currently) used by businesses to deliver key business features.

How exactly are microservices and containers being used? How can you benefit from them?

Read on, and find out!

Why did your team decide to start using containers? How are you using containers today, and what is your long-term plan?

Parthu Kishen: We build all our infrastructure as code, from Infrastructure provisioning and management using AWS CloudFormation to Configuration management using Chef. Our team at Sysgain started using containers in our development and testing framework for Infrastructure as Code, and for Chef cookbooks to test them and make sure they were converging properly.

We also noticed how other organizations were realizing that they have these monolithic applications that - as they grow in complexity - make it increasingly difficult to accelerate rate of change of delivering required features to customers without increasing risk, cost, and/or timeline, and so we decided to implement this at Sysgain to avoid such issues. Also, having been "battle-tested" at companies like Amazon, Facebook, and Twitter, I feel there is a large focus these days on using microservices to build solutions for this era.

"We see a big need there, and as a services company we saw a lot of value in providing this to our customers, so we started building out our architecture, our frameworks, and our internal tools and services around using microservices and Docker - as well as AWS Lambda. That's primarily what drove adoption within the company, and the need is definitely there with our customers."

Recently, we did a project where AWS Lambda was used heavily. AWS Lambda is a compute service from Amazon that can be deployed in milliseconds and lives for a few seconds before terminating. Lambda was helpful because there was no need to worry about infrastructure or scaling it, and with minimal configuration we could focus on application code and building out microservices, thereby delivering value very quickly to our customers.

"We see great value in both AWS Lambda [which is like Container-as-a-Service, if you will] and Docker; side-by-side, not just in dev but also in production environments."

Do you think there will be more adoption of containers going forward?

Parthu: I think we're seeing an evolution at work. Initially, enterprises have/had datacenters where it would take a few months to get everything set up. One could then deploy their applications on this infrastructure.

With cloud, the infrastructure is available on demand and the application deployment has moved more towards images and AMIs, where the Operations teams expect you to build an AMI needed to deploy into production, like Netflix. Others who completely treat infrastructure as code, bootstrap the instances with required software and configuration in a dynamic fashion.

AMIs or dynamic bootstrapping take time, which may not be the best use case for spinning up things very quickly, which is where we see the path of evolution leading next in terms of Docker containers, where you could spin up application services in seconds. Applications (or rather: services/microservices) and all configurations related to them are containerized, and these can be launched on VM as needed. The underlying infrastructure still has to scale as needed.

AWS Lambda is a parallel option you have where you can spin up a service in a few milliseconds, and it will run only for a lifetime of a few seconds. Rather than orchestrating, scheduling and managing containers, this is another option made availble by AWS and can be thought of as "Container as a service".

There is less flexibility in terms of OS type, hardware choices and how long the functions lives (max : 1 min), but these might change in the future based on the need.

"So, I think we'll see both of them exist together for different use cases; both are pretty strong and compelling technologies."

For people who want to start using these technologies, what's the best way to go about it?

Parthu: The best way to learn anything is to go "hands-on." There are a lot of good resources for this, such as talks on YouTube about microservices, and how you should migrate to microservices as well. For containers - specifically Docker - there is a lot of documentation available on their new website. I think that's a good place to learn.

If there are any questions, we can step in and help as well, with strategies and proof-of-concepts to help get things going.

"One big piece everybody should look at is continuous delivery of such services. Even if you build microservices, if you don't have a continuous delivery plan for that then you are not achieving the end goal.

It's as important as building your microservices -- to have a good DevOps methodology and tooling and process to get it all the way from your developer laptop to production and achieve continuous delivery.

If you are doing multiple releases a day, I think that's where - and make the changes pretty small, so that you reduce the risk - you get a lot of value from these services."

Operationalizing and monitoring containers and containerized applications/services:

Parthu: Microservices and containers are great, but with all of these distributed services there are all of these pain points that arise. How do you distribute it? How do you aggregate all the logs, centralize information, and make sense of it?

"The need for having a centralized information gathering system is critical. That's one of the big challenges, I think, you face as companies try to adopt. There are many other challenges of course, with things like security, but many of those are being addressed by the frameworks that are available."

With the way things are going, in the next few years I expect to see lots more maturity with best practices and production-ready tools -- there will soon be a common way of doing things.

Lastly, What other key processes or tools have you put in place to help you successfully roll out and manage containers?

The experts ended this discussion with listing some of the most useful tools they used in their own work with managing microservices and containers. Here are some of the tools Parthu named:

Parthu Kishen's ending quote leaves us with some food for thought:

"Having a continuous delivery pipeline is the key. Without that, your (micro)services won't get out at the needed pace."