Microservices Architecture Design and Best Practices 

What is Microservices Architecture?

A microservices architecture style is an approach for developing small services each running in its process. It enables the continuous delivery/deployment of large, complex applications. It also allows an organization to evolve its technology stack.

Why Microservices Architecture?

Microservices came in a picture of building systems that were too big. The idea behind microservices is that there are some applications that can easily build and maintain when they are broken down into smaller applications that work together. Each component is continuously developed and separately managed, and the application is then merely the sum of its constituent elements. Whereas in traditional “monolithic” application which is all developed all in one piece.

Microservices Architecture Design

Distributed architecture

All the services communicate with the API gateway through REST or RPC. These services can be deployed as multiple instances, and the requests can be distributed to these instances.for

Separately deployed components

Each component is deployed separately. If one component needs changes, others don’t have to deploy again.

Service components

Services components communicate with each other via service discovery

Bounded by contexts

It encapsulates the details of a single domain, and define the integration with other domains. It is about implementing a business capability.

Benefits of Adopting Microservices Architecture Design

• Asynchronicity.
• Integration & Disintegration.
• Complected Deployments.
• Evolutionary Architecture.
• Components are deployed.
• Features are released.
• Applications consist of routing.
• Easier to understand the code — It is easy to distinguish one small service and flow of the whole service rather than one big codebase.
• Fast Software delivery — Each service can be developed by different developers and in many different languages.
• Efficient debugging — Don’t have to jump through multiple layers of an application and in essence better fault isolation.
• Reusable — Since it is an independent service it can be used in other projects also.
• Scalability
• Horizontal scaling
• Workload partitioning
• Don’t have to scale the whole project. We only need to scale up that component that needs to scale up.
• Deployment — Need only to deploy that service which has been changed not the whole project again.

Characteristics of Microservices Architecture Design

• Small in size
• Messaging enabled
• Bounded by contexts
• Autonomously developed
• Independently deployable
• Decentralized
• Built and released with automated processes

Continue Reading: XenonStack/Insights

Overview

Prerequisites for Rocket Chat Deployment on Kubernetes

• Kubernetes
• Shared Persistent Storage - Option are GlusterFS, Ceph FS, AWS EBS, AzureDisk etc.
• Rocket Chat Application
• Dockerfile
• Container-Registry
• Kubectl

Step 1 - Create a Rocket Chat Container Custom Image

Step 2 - Build Rocket Chat Docker Custom Image

Step 3 - Create a MongoDB Container Custom Image

Step 4 - Building a MongoDB Docker Custom Image

Step 5 - Adding Container Registry to Docker Daemon

Step 6 - Pushing Custom PostgreSQL Container Image to Container Registry

Step 7 - Create a Storage Volume (Using GlusterFS)

Step 8 - Deploy MongoDB on Kubernetes

• Docker Image: We have created a Docker Image for MongoDB in Step 4 and pushed to docker hub or private docker registry.

Overview

Prerequisites are mentioned below

• Kubernetes
• Kubectl
• Shared Persistent Storage - Option are GlusterFS, Ceph FS, AWS EBS, AzureDisk, etc.
• Python Application Source Code
• Dockerfile
• Container-Registry

Dockerfile

FROM python:2.7
MAINTAINER XenonStack

# Creating Application Source Code Directory
RUN mkdir -p /usr/src/app

# Setting Home Directory for containers
WORKDIR /usr/src/app

# Installing python dependencies
COPY requirements.txt /usr/src/app/
RUN pip install --no-cache-dir -r requirements.txt

# Copying src code to Container
COPY . /usr/src/app

# Application Environment variables
ENV APP_ENV development

# Exposing Ports
EXPOSE 5035

# Setting Persistent data
VOLUME ["/app-data"]

# Running Python Application
CMD ["python", "wsgi.py"]

Building Python Docker Image

$ docker build -t <name of your python application>:<version of application> .

Publishing Container Image

• Adding Container Registry to Docker Daemon

$ docker version
Client:
 Version:   17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:     Mon Mar 27 17:14:09 2017
 OS/Arch:   linux/amd64 (Ubuntu 16.04)

$ sudo nano /etc/docker/daemon.json

{
 "insecure-registries": ["<name of your private registry>"]
}

$ sudo systemctl daemon-reload
$ sudo service docker restart

$ docker info

Insecure Registries:

 <your container registry name>
 127.0.0.0/8

• Pushing container Images to Registry

#!/bin/bash
pip install --upgrade --user awscli

mkdir -p ~/.aws && chmod 755 ~/.aws

cat << EOF > ~/.aws/credentials
[default]
aws_access_key_id = XXXXXX
aws_secret_access_key = XXXXXX
EOF

cat << EOF > ~/.aws/config
[default]
output = json
region = XXXXX
EOF

chmod 600 ~/.aws/credentials

ecr-login=$(aws ecr get-login --region XXXXX)
$ecr-login

Overview

Prerequisites  

• Kubernetes - Kubernetes is an open source platform that automates container operations and Minikube is best for testing Kubernetes.
  
  

• Kubectl - Kubectl is command line interface to manage Kubernetes cluster either remotely or locally. To configure kubectl in your machine follow this link.

• Shared Persistent Storage - Shared Persistent Storage is permanent storage that we can attach to the Kubernetes container so that we don`t lose our data even container died. We will be using GlusterFS as a persistent data store for Kubernetes container applications.

• .NET Application Source Code - Application Source Code is source code that we want to run inside a kubernetes container.

• Dockerfile - Dockerfile contains a bunch of commands to build .NET application.

• Container-Registry - The Container Registry is an online image store for container images.

• Private Docker Hub
• AWS ECR
• Docker Store
• Google Container Registry

Create a Dockerfile

FROM microsoft/dotnet:1.1-sdk
# Setting Home Directory for application 
WORKDIR /app 

# copy csproj and restore as distinct layers
COPY dotnetapp.csproj .
RUN dotnet restore

# copy and build everything else
COPY . .
RUN dotnet publish -c Release -o out

EXPOSE 2223

ENTRYPOINT ["dotnet", "out/main.dll"]

Building .NET Application Image

$ docker build -t <name of your application>:<version of application> .

Publishing Container Image

Creating Deployment Files for Kubernetes

• Deployment File

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: <name of application>
  namespace: <namespace of Kubernetes>
spec:
  replicas: <number of application pods>
  template:
 metadata:
   labels:
    k8s-app: <name of application>
 spec:
   containers:
   - name: <name of application>
     image: <image name >:<version tag>
     imagePullPolicy: "IfNotPresent"
     ports:
      - containerPort: 2223

• Service File

apiVersion: v1
kind: Service
metadata:
  labels:
 k8s-app: <name of application>
  name: <name of application>
  namespace: <namespace of Kubernetes>
spec:
  type: NodePort
  ports:
  - port: 2223
  selector:
 k8s-app: <name of application>

Running .NET Application on Kubernetes

$ kubectl create -f <name of application>.deployment.yml
$ kubectl create -f <name of application>.service.yml

Verification

$ kubectl get po --namespace=<namespace of kubernetes> | grep <application name>

Result of above command

Introduction To DevOps

DevOps Features

• Maximize speed of delivery of the product.
• Enhanced customer experience.
• Increased time to value.
• Enables fast flow of planned work into production.
• Use Automated tools at each level.
• More stable operating environments.
• Improved communication and collaboration.
• More time to innovate.
  
  
  
  
  

  DevOps Consists of 5 C’s

  
  
  DevOps practices lead to high productivity, lesser bugs, improved communication, enhanced quality, faster resolution of problems, more reliability, better and timely delivery of software.
  
  
• Continuous Integration
• Continuous Testing
• Continuous Delivery
• Continuous Deployment
• Continuous Monitoring
  
  
  
  
  

1. Continuous Integration 

2. Continuous Testing 

3. Continuous Delivery 

4. Continuous Deployment 

5. Continuous Monitoring 

Key Technologies and Terminologies In DevOps

6. Microservices

Advantages Of Microservices

• Microservices can be developed in variable programming languages.
• Errors in any module or microservices can easily be found out, thus saves time.
• Smaller modules or microservices are easier to manage.
• Whenever any update required, it can be immediately pushed on that particular microservices, otherwise, the whole application needs to be updated.
• According to client need, we can scale up and down particular microservice without affecting the other microservices.
• It also leads to increase in productivity.
• If any one module goes down, the application remains largely unaffected.

Disadvantages Of Microservices

• If any application involves the number of microservices, then managing them becomes a little bit difficult.
• Microservices leads to more memory consumption.
• In some cases, testing microservices becomes difficult.
• In production, it also leads to complexity of deploying and managing a system comprised of different types of services.

7. Containers

Advantages Of Containers

• Wastage of resources like RAM, Processor, Disc space are controlled as now there is no need to pre-locate these resources and are met according to application requirements.
• It’s easy to share a container.
• Docker provides a platform to manage the lifecycle of containers.
• Containers provide consistent computation environment.
• Containers can run separate applications within a single shared operating system.

8. Container Orchestration

• Cluster Management - Developer’s task is limited to launch a cluster of container instances and specify the tasks which are needed to run. Management of all containers is done by Orchestration.

• Task Definitions - It allows the developer to define task where they have to specify the number of containers required for the task and their dependencies. Many tasks can be launched through single task definition.

• Programmatic Control - With simple API calls one can register and deregister tasks, and launch and stop Docker containers.

• Scheduling - Container scheduling deals with placing the containers from the cluster according to the resources they need and the availability of requirements.

• Load Balancing - Helps in distributing traffic across the containers/deployment.

• Monitoring - One can monitor CPU and memory utilization of running tasks and also gets alerted if scaling is needed by containers.

Tools used for Container Orchestration

• Amazon ECS - Amazon ECS is yet another product from Amazon Web Services that provides the runtime environment for Docker Containers and provide orchestration. It allows running Dockerized applications on top of Amazon’s Infrastructure.

• Azure Containers Service - Azure Container Service product is by Microsoft allowing similar functionalities. It has very good support for .NET ecosystem.

• Docker Swarm - It’s an open source tool, part of Docker’s landscape. With this tool, we can run multiple docker engines as a single virtual Docker. This is Dockers own containers orchestration Tool. It consists of the manager and worker nodes that run different services for orchestration. Managers that distributes tasks across the cluster and worker node run containers assigned by managers.

• Google Container Engine - Google Container Engine allow us to run Docker containers on Google Cloud Platform. It schedules the containers into the cluster and manages them as per the requirements were given. It is built on the top of Kubernetes i.e. an open source Containers Orchestration tool.
  
  
  Continue Reading About Latest DevOps Trends At: XenonStack.com/Blog

What is PostgreSQL?

• It runs on all major operating systems, including Linux, UNIX (AIX, BSD, HP-UX, SGI IRIX, MacOS, Solaris, Tru64), and Windows.

• It is fully ACID compliant, has full support for foreign keys, joins, views, triggers, and stored procedures (in multiple languages)

• It includes most SQL:2008 data types, including INTEGER, NUMERIC, BOOLEAN, CHAR, VARCHAR, DATE, INTERVAL, and TIMESTAMP.

• It also supports storage of binary large objects, including pictures, sounds, or video.

• It has native programming interfaces for C/C++, Java, .Net, Perl, Python, Ruby, Tcl, ODBC, among others, and exceptional documentation.

Prerequisites

• Kubernetes Cluster

• GlusterFS Cluster

FROM ubuntu:latest
MAINTAINER XenonStack

RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8

RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ xenial-pgdg main" > /etc/apt/sources.list.d/pgdg.list

RUN apt-get update && apt-get install -y python-software-properties software-properties-common postgresql-9.6 postgresql-client-9.6 postgresql-contrib-9.6

RUN /etc/init.d/postgresql start &&\
 psql --command "CREATE USER root WITH SUPERUSER PASSWORD 'xenonstack';" &&\
 createdb -O root xenonstack

RUN echo "host all  all 0.0.0.0/0  md5" >> /etc/postgresql/9.6/main/pg_hba.conf

RUN echo "listen_addresses='*'" >> /etc/postgresql/9.6/main/postgresql.conf

# Expose the PostgreSQL port
EXPOSE 5432

# Add VOLUMEs to allow backup of databases
VOLUME  ["/var/lib/postgresql"]

# Set the default command to run when starting the container
CMD ["/usr/lib/postgresql/9.6/bin/postgres", "-D", "/var/lib/postgresql", "-c", "config_file=/etc/postgresql/9.6/main/postgresql.conf"]

$ docker build -t dr.xenonstack.com:5050/postgres:v9.6

$ gluster volume create postgres-disk replica 2 transport tcp k8-master:/mnt/brick1/postgres-disk  k8-1:/mnt/brick1/postgres-disk
$ gluster volume start postgres-disk
$ gluster volume info postgres-disk

• Docker Image: We have created a Docker Image for Postgres in Step 2

• Persistent Shared Storage Volume: We have created a Persistent Shared Storage Volume in Step 3

• Deployment & Service Files: Next, we will create Deployment & Service Files

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: postgres
  namespace: production
spec:
  replicas: 1
  template:
 metadata:
   labels:
    k8s-app: postgres
 spec:
   containers:
   - name: postgres
     image: dr.xenonstack.com:5050/postgres:v9.6
     imagePullPolicy: "IfNotPresent"
     ports:
     - containerPort: 5432
     env:
     - name: POSTGRES_USER
       value: postgres
     - name: POSTGRES_PASSWORD
       value: superpostgres
     - name: PGDATA
       value: /var/lib/postgresql/data/pgdata
     volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: postgredb
   volumes:
     - name: postgredb
       glusterfs:
         endpoints: glusterfs-cluster
         path: postgres-disk
         readOnly: false