XenonStack

Wednesday, 30 May 2018

5/30/2018 04:50:00 pm

Continuous Delivery Pipeline for Deploying Microservices Based Scala Application on Kubernetes

Overview

Running Containers at any real-world scale requires container orchestration, and scheduling platform like Docker Swarm, Apache Mesos, AWS ECS but the most popular out of it is Kubernetes. Kubernetes is an open source system for automating deployment and management of containerised applications.

In this post, We’ll share the process how you can Develop and Deploy Microservices based PHP Laravel Application on the Container Environment - Docker and Kubernetes and adopt DevOps in existing PHP Applications.

Prerequisites For Deploying Laravel Application on Kubernetes

To follow this guide you need -

Kubernetes
Kubectl
Scala Application Source Code
Dockerfile
Container-Registry

Kubernetes

It is an open source platform that automates container operations, and Minikube is best for testing Kubernetes.

Kubectl

Kubectl is command line interface to manage Kubernetes cluster either remotely or locally. To configure kubectl on your machine follow this link.

Shared Persistent Storage

Shared Persistent Storage is permanent storage that we can attach to the Kubernetes container so that we don`t lose our data even when container dies. We will be using GlusterFS as the persistent data store for Kubernetes container applications.

Scala Application Source Code

Scala Application Source Code is source code that we want to run inside a Kubernetes container.

DockerFile

Dockerfile contains a bunch of commands to build PHP Laravel application.

Container Registry

The Registry is an online image store for container images.

Below mentioned options are few most popular registries.

CONTINUE READING:XenonStack/Blog

XenonStack 0

Tuesday, 22 May 2018

5/22/2018 05:25:00 pm

Persistent Storage Solution For Containers - Docker & Kubernetes

Overview of Persistent Storage

Persistent Storage is a critical part in order to run stateful containers. Kubernetes is an open source system for automating deployment and management of containerized applications.

There are a lot of options available for storage. In this blog, we are going to discuss most widely used storage which can use on on-premises or in a cloud-like GlusterFS, CephFS, Ceph RBD, OpenEBS, NFS, GCE Persistent Storage, AWS EBS, NFS & Azure Disk.

Prerequisites for Implementing Storage Solutions

To follow this guide you need -

Kubernetes
Kubectl
DockerFile
Container Registry
Storage technologies that will be used -
- OpenEBS
- CephFS
- GlusterFS
- AWS EBS
- Azure Disk
- GCE persistent storage
- CephRBD

Kubernetes

Kubernetes is one of the best open-source orchestration platforms for deployment, autoscaling and management of containerized applications.

Kubectl

Kubectl is a command line utility used to manage kubernetes clusters either remotely or locally. To configure kubectl use this link.

Container Registry

Container Registry is the repository where we store and distribute docker images. There are several repositories available online we have DockerHub, Google Cloud,Microsoft Azure, and AWS Elastic Container Registry (ECR).

Container storage is ephemeral, meaning all the data in the container is removed when it crashes or restarted. Persistent storage is necessary for stateful containers in order to run applications like MySQL, Apache, PostgreSQL etc. so that we don’t lose our data when a container stops.

3 Types of Storage

Block Storage - It is most commonly used storage and is very flexible. Block storage stores chunks of data in blocks. A block is only identified by its address. It is mostly used for databases because of its performance.
File Storage - It stores data as files, each file is referenced by a filename and has attributes associated with it. NFS is the most commonly used file systems. We can use file storage where we want to share data with multiple containers.
Object Storage - Object storage is different from file storage and block storage. In object storage data is stored as an object and is referenced by object ID. It is massively scalable and provides more flexibility than block storage but performance is slower than block storage. Most commonly used object storage are Amazon S3, Swift and Ceph Object Storage.

7 Emerging Storage Technologies

OpenEBS Container Storage

OpenEBS is a pure container based storage platform available for Kubernetes. Using OpenEBS, we can easily use persistent storage for stateful containers and the process of provisioning of a disk is automated.

It is a scalable storage solution which can run anywhere, from cloud to on-premises hardware.

Ceph Storage Cluster

Ceph is an advanced and scalable Software-defined storage which fits best with the needs of today’s requirement providing Object Storage, Block Storage and File System on a single platform.

Ceph can also be used with Kubernetes. We can either use CephFS or CephRBD for persistent storage for kubernetes pods.

Ceph RBD is the block storage which we assign to pod. CephRBD can’t be shared with two pods at a time in read-write mode.
CephFS is a POSIX-compliant file system service which stores data on top Ceph cluster. We can share CephFS with multiple pods at the same time. CephFS is now announced as stable in the latest Ceph release.

GlusterFS Storage Cluster

GlusterFS is a scalable network file system suitable for cloud storage. It is also a software-defined storage which runs on commodity hardware just like Ceph but it only provides File systems, and it is similar to CephFS.

Glusterfs provides more speed than Ceph as it uses larger block size as compared to ceph i.e Glusterfs uses a block size of 128kb whereas ceph uses a block size of 64Kb.

AWS EBS Block Storage

Amazon EBS provides persistent block storage volumes which are attached to EC2 instances. AWS provides various options for EBS, so we can choose the storage according to requirement depending on parameters like number of IOPS, storage type(SSD/HDD) etc.

We mount AWS EBS with kubernetes pods for persistent block storage using AWSElasticBlockStore. EBS disks are automatically replicated over multiple AZ’s for durability and high availability.

GCEPersistentDisk Storage

GCEPersistentDisk is a durable and high-performance block storage used with Google Cloud Platform. We can use it either with Google Compute Engine or Google Container Engine.

We can choose from HDD or SSD and can increase the size of the volume disk as the need increases. GCEPersistentDisks are automatically replicated across multiple data centres for durability and high availability.

We mount GCEPersistentDisk with kubernetes pods for persistent block storage using GCEPersistentDisk.

Azure Disk Storage

An Azure Disk is also a durable and high-performance block storage like AWS EBS and GCEPersistentDisk. Providing the option to choose from SSD or HDD for your environment and features like Point-in-time backup, easy migration etc.

An AzureDiskVolume is used to mount an Azure Data Disk into a Pod. Azure Disks are replicated within multiple data centres for high availability and durability.

Network File System Storage

NFS is the one of the oldest used file system providing the facility to share single file system on the network with multiple machines.

There are several NAS devices available for high performance or can we make our system to be used as NAS. We use NFS for persistent storage for pods and data can be shared with multiple instances.

Deployment of Storage Solutions

Now we are going to walk through with the deployments of storage solutions described above. We are going to start with Ceph.

Continue Reading:XenonStack/Blog

XenonStack 0

Tuesday, 8 May 2018

5/08/2018 06:51:00 pm

Test Driven & Behavior Driven Development in Python

Overview

Test Driven Development (TDD) is a great approach for software development. TDD is nothing but the development of tests before adding a feature in code.

This approach is based on the principle that we should write small codes rather than writing long codes. In TDD, whenever we want to add more functionality in our codes, we first have to write a test for that. After that, we add new functionality with small code lines combined and then test it with our test. This approach helps us to reduce the risk of encountering significant problems at the production level.

Test Driven Development (TDD)

Test Driven Development is an approach in which we build a test first, then fail the test and finally refactor our code to pass the test.

Test Driven Development (TDD) Approach

As the name suggests, we should first add the test before adding the functionality in our code. Now our target is to make the test pass by adding new code to our program. So we refactor our code to pass the written test. This uses the following process -

Write a failing unit test
Make the unit test pass
Repeat

Test Driven Development (TDD) Process Cycle

As shown in the flow

First, add tests for the functionality.
Next, we run our test to fail.
Next, we write code according to the error we received.
Then we run the tests again to see if the test fails or passes.
Then refactor the code and follow the process again.

Benefits of Test Driven Development (TDD)

Now the question arises why should one opt TDD approach. Practicing TDD brings lots of benefits. Some of the benefits are listed below -

In TDD we build test before adding any new feature to it, that means in TDD approach our entire code is covered under the test. That’s a great benefit of TDD as compared to the code which has no test coverage.
In TDD one should have a specific target before adding new functionality. This means before adding any new functionality one should be clear about its outcome.
In an application, one method depends on the other. When we write tests before the method that means we should have clear thoughts about the interfaces between the methods. That allows us to integrate our method with the entire application efficiently and help in making our application modular too.
As the entire code is covered by the test that means our final application will be less buggy. This is a big advantage of the TDD approach.

Acceptance Test Driven Development (ATDD)

ATDD is short for Acceptance Test Driven Development. In this process, a user, business manager and developer, all are involved.

First, they discuss what the user wants in his product; then business manager creates sprint stories for the developer. After that, the developer writes tests before starting the projects and then starts coding for their product.

Every product/software is divided into small modules, so the developer codes for the very first module then test it and sees it getting failed. If the test passes and the code are working as per user requirements, it is moved to the next user story; otherwise, some changes are made in coding or program to make the test pass.

This process is called Acceptance Test Driven Development.

Behavior Driven Development (BDD)

Behavior Driven testing is similar to Test Driven Development, in the way that in BDD also tests are written first and tested and then more code is added to it to make the test pass.

The major area where these two differ is that tests in BDD are written in plain descriptive English type grammar.

Tests in BDD aim at explaining the behaviour of the application and are also more user-focused. These tests use examples to clarify the user requirements in a better way.

Features of Behavior Driven Development (BDD)

The major change is in the thought process which is to be shifted from analyzing in tests to analyzing in behaviour.
Ubiquitous language is used; hence it is easy to be explained.
BDD approach is driven by business value.
It can be seen as an extension to TDD; it uses natural language which is easy to understand by non-technical stakeholders also.

You May also Love to Read Test Driven Development in Scala

Behavior Driven Development (BDD) Approach

We believe that the role of testing and test automation TDD(Test Driven Development) is essential to the success of any BDD initiative. Testers have to write tests that validate the behaviour of the product or system being generated.

The test results formed are more readable by the non-technical user as well. For Behavior Driven Development to be successful, it becomes crucial to classify and verify only those behaviours that give directly to business outcomes.

Developer in the BDD environment has to identify what program to test and what not to test and to understand why the test failed. Much like Test Driven Development, BDD also recommends that tests should be written first and should describe the functionalities of the product that can be suited to the requirements.

Behavior Driven Development helps greatly when building detailed automated unit tests because it focuses on testing behaviour instead of testing implementation. The developer thus has to focus on writing test cases keeping the synopsis rather than code implementation in mind.

By doing this, even when the requirements change, the developer does not have to change the test, input and output to support it. That makes unit testing automation much faster and more reliable.

Though BDD has its own sets of advantages, it can sometimes fall prey to reductions. Development teams and Tester, therefore, need to accept that while failing a test is a guarantee that the product is not ready to be delivered to the client, passing a test also does not mean that the product is ready for launch.

It will be closed when development, testing and business teams will give updates and progress report on time. Since the testing efforts are moved more towards automation and cover all business features and use cases, this framework ensures a high defect detection rate due to higher test coverage, faster changes, and timely releases.

Benefits of Behavior Driven Development (BDD)

It is highly suggested for teams and developers to adopt BDD because of several reasons, some of them are listed below -

BDD provides a very accurate guidance on how to be organizing communication between all the stakeholders of a project, may it be technical or non-technical.
BDD enables early testing in the development process, early testing means lesser bugs later.
By using a language that is understood by all, rather than a programming language, a better visibility of the project is achieved.
The developers feel more confident about their code and that it won’t break which ensures better predictability.

Test Driven Development (TDD) with Python

Here we exploring the Test-driven development approach with Python. Python official interpreter comes with the unit test module

Python Unit Testing

These are the main methods which we use with python unit testing

Method	Checks That
assertEqual(a, b)	a == b
assertNotEqual(a, b)	a != b
assertTrue(x)	bool(x) is True
assertFalse(x)	bool(x) is False
assertIs(a, b)	a is b
assertIsNot(a, b)	a is not b
assertIsNone(x)	x is None
assertIsNotNone(x)	x is not None
assertIn(a, b)	a in b
assertNotIn(a, b)	a not in b
assertIsInstance(a, b)	isinstance(a, b)
assertNotIsInstance(a, b)	not isinstance(a, b)

You May also Love to Read Test Driven & Behavior Driven Development in Java

Test Driven Development (TDD) in Python with Examples

We are going to work on an example problem of banking. Let's say that our banking system introduced a new facility of credit money. So we have to add this to our program.

Following the TDD approach before adding this credit feature, we first write our test for this functionality.