XenonStack

A Stack Innovator

Post Top Ad

Showing posts with label python. Show all posts
Showing posts with label python. Show all posts

Thursday, 7 September 2017

9/07/2017 04:24:00 pm

Deploying Python Application on Docker & Kubernetes


Overview


In this post, We’ll share the process how you can Develop and Deploy Python Application using Docker and Kubernetes and adopt DevOps in existing Python Applications.

Prerequisites are mentioned below


To follow this guide you need

Kubernetes is an open source platform that automates container operations, and Minikube is best for testing kubernetes in a local environment.

Kubectl is command line interface to manage kubernetes cluster either remotely or locally. To configure kubectl in your machine follow this link.

Shared Persistent Storage is permanent storage that we attach to the kubernetes container. We will be using cephfs as a persistent data store for kubernetes container applications.

Application Source Code is source code that we want to run inside a kubernetes container.

Dockerfile contains all the actions that are performed to build python application.

The Registry is an online image store for container images.

Below mentioned options are few most popular registries.

2. AWS ECR

Dockerfile


The Below mentioned code is sample docker file for Python applications. In which we are using python 2.7 development environment.


FROM python:2.7
MAINTAINER XenonStack

# Creating Application Source Code Directory
RUN mkdir -p /usr/src/app

# Setting Home Directory for containers
WORKDIR /usr/src/app

# Installing python dependencies
COPY requirements.txt /usr/src/app/
RUN pip install --no-cache-dir -r requirements.txt

# Copying src code to Container
COPY . /usr/src/app

# Application Environment variables
ENV APP_ENV development

# Exposing Ports
EXPOSE 5035

# Setting Persistent data
VOLUME ["/app-data"]

# Running Python Application
CMD ["python", "wsgi.py"]

Building Python Docker Image

The Below mentioned command will build your application container image.

$ docker build -t <name of your python application>:<version of application> .

Publishing Container Image


To publish Python container image, we can use different private/public cloud repository like Docker HubAWS ECRGoogle Container RegistryPrivate Docker Registry.

  • Adding Container Registry to Docker Daemon
If you are using docker registry other than docker hub to store images, then we need to add that container registry to our local docker daemon and kubernetes Docker daemons.

You must have following things to follow next steps.

$ docker version
Client:
 Version:   17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:     Mon Mar 27 17:14:09 2017
 OS/Arch:   linux/amd64 (Ubuntu 16.04)


Now we need to Create a “daemon.json” in below-mentioned location

$ sudo nano /etc/docker/daemon.json


And add the following content to it.

{
 "insecure-registries": ["<name of your private registry>"]
}


Now Run the following commands to reload systemctl and restart docker daemon.

$ sudo systemctl daemon-reload
$ sudo service docker restart


To verify that your container registry is added to local docker daemon, use the below-mentioned steps.

$ docker info


In output of above, you get your container registry like this

Insecure Registries:
 <your container registry name>
 127.0.0.0/8

  • Pushing container Images to Registry

I'm using AWS ECR for publishing container images.

You must have an AWS account with Amazon ECR permissions. Create AWS ECR repository using a below-mentioned link.

After creation, you will get registry URL, username, and password from own AWS cloud.

Here is a shell script that will add your AWS credentials for Amazon ECR in your local system so that you can push images to AWS ECR.

#!/bin/bash
pip install --upgrade --user awscli

mkdir -p ~/.aws && chmod 755 ~/.aws

cat << EOF > ~/.aws/credentials
[default]
aws_access_key_id = XXXXXX
aws_secret_access_key = XXXXXX
EOF

cat << EOF > ~/.aws/config
[default]
output = json
region = XXXXX
EOF

chmod 600 ~/.aws/credentials

ecr-login=$(aws ecr get-login --region XXXXX)
$ecr-login

Now we need to retag python application image and push them to docker hub container registry.

To Retag application container image

$ docker tag <name of your application>:<version of your application> <aws ecr repository link>/<name of your application >:<version of your application>

Continue Reading The Full Article At - XenonStack.com/Blog

Friday, 10 February 2017

2/10/2017 03:45:00 pm

Building Serverless Microservices With Python


Serverless Computing is Exploding


As we move to the different models of production, distribution, and management when it comes to applications, it only makes sense that abstracting out the, behind the scenes processes should be handled by third parties, in a move towards further decentralization.

And that’s exactly what serverless computing does – and startups and big companies are adopting this new way of running applications.

In this post, we will discover answers to questions:

What Serverless is all about and how does this new trend affect the way people write and deploy applications?

Serverless Computing


"Serverless” denotes a special kind of software architecture in which application logic is executed in an environment without visible processes, operating systems, servers or virtual machines.

It’s worth mentioning that such an environment is actually running on the top of an operating system and use physical servers or virtual machines, but the responsibility for provisioning and managing the infrastructure entirely belongs to the service provider.

Therefore, a software developer focus more on writing code.

Serverless Computing Advances the way Applications are Developed


Serverless applications will change the way we develop applications. Traditionally a lot of business rules, boundary conditions, complex integrations are built into applications and this prolongs the completion of the system as well as introduces a lot of defects and in effect, we are hard wiring the system for certain set of functional requirements.

The serverless application concept moves us away from dealing with complex system requirements and evolves the application with time. It is also easy to deploy these microservices without intruding the system.

Below figure shows how the way of application development changed with time.

Monolith- A monolith application puts all its functionality into a single process and scale by replicating the monolith on multiple servers.

Microservice- A microservice architecture puts each functionality into a separate service and scale by distributing these services across servers, replicating as needed.

FaaS- Distributing Microservices further into functions which are triggered based on events.

Monolith => Microservice => FaaS


building serverless microservices with python


Let’s get started with the deployment of a Serverless Application on NexaStack.To create a function, you first package your code and dependencies in a deployment package. 

Then, you upload the deployment package on our environment to create your function.
  • Creating a Deployment Package
  • Uploading a Deployment Package

You May also Like: Building Serverless Microservices With Java

Database Integration For Your Application


  • Install MongoDB and configure it to get started.
  • Create Database EmployeeDB
  • Create table Employee
  • Insert some records into the table for the demo.
  • Write a file “config.py” to setup configuration on the serverless architecture as shown below.


building serverless microservices with python

 

Continue Reading the full Article at: XenonStack.com/Blog

Monday, 6 February 2017

2/06/2017 11:16:00 am

Building Serverless Microservices With JAVA


Serverless Architecture


The phrase “serverless” doesn’t mean servers are no longer required. It solely proposes that developers no longer have to think that much about them.

Going serverless lets developers shift their focus from the server level to the task level which is writing codes.


serverless microservices with java


 

What it means to have servers?


First, let’s talk about what it means to have servers (virtual servers) providing the computing power required by your application. Owning servers comes with responsibilities -
  • Managing how the primitives (functions in the case of applications, or objects when it comes to storage) map to server primitives (CPU, memory, disk etc.).
  • Own provisioning (and therefore paying) for the capacity to handle your application’s projected traffic, independent of whether there’s actual traffic or not.
  • Own managing reliability and availability constructs like redundancy, failover, retries etc.

Advantages of going Serverless


Why should one move to serverless architecture can be adequately described through its benefits.

  • PaaS and Serverless - A user of traditional PaaS have to specify the amount of resources—such as dynos for Heroku or gears for OpenShift—for the application. The Serverless platform will take care of finding a server where the code is to run and to scale up when necessary.
  • Lower operational and development costs - The containers used to run these functions are decommissioned as soon as the execution ends. And the execution is metered in units of 100 ms, You don't pay anything when your code isn't running.
  • Fits with microservices, which can be implemented as functions.

Serverless architectures refer to applications that significantly depend on third-party services (knows as Backend as a Service or "BaaS") or on custom code that's run in ephemeral containers (Function as a Service or "FaaS").

But there are cons related to moving your application to FaaS which is discussed in our next post: Building Serverless Microservices with Python

Simplest way of thinking about FaaS is that it changes thinking from "build a framework to sit on a server to react to multiple events to "build/use micro-functionality to react to a single event."

How to migrate to a Microservices Architecture?


In a simple definition, Microservices are independently scalable, independently deployable systems that communicate over some protocols HTTP (XML, JSON), Thrift, Protocol Buffers etc.

Microservices are Single Responsibility Principle at code base level.

Below are some of the factors that can be followed to build Microservices:
  • One code per app/service: There is always a one-to-one correlation between the codebase and the service.
  • Explicitly declare and isolate dependencies: This can be done by using packaging systems.
  • Use environment variables to store configurations.
  • Strictly separate build, release and run stages.
  • Treat logs as event streams. Route log event stream to analysis system such as Splunk for log analysis.
  • Keep development, staging, and production as similar as possible.


Microservices Architecture: Benefits


Microservices Architectures have lots of very real and significant benefits:
  • Systems built in this way are inherently loosely coupled
  • The services themselves are very simple, focusing on doing one thing well
  • Multiple developers and teams can deliver independently under this model
  • They are a great enabler for continuous delivery, allowing frequent releases whilst keeping the rest of the system available and stable
In this post, we will implement a Nexastack function which integrates with a database(MongoDB used here).

We are going to implement this new function in Java using Spring Framework. So, Let’s get started -

Employee Service


We are going to build an Employee Service consisting of a function to show Employees information from the database.

For Demo purpose we are here implementing one function “GetEmployee”.
serverless microservices with java

1. Setting up MongoDB Instance 


  • Install MongoDB and configure it to get started.
  • Create Database EmployeeDB
  • Create table Employee
  • Insert some records into the table for demo.
  • Write a file “config.properties” to setup configuration on the serverless architecture


serverless microservices with java


 Continue Reading the full article at: XenonStack.com/Blog