For the past one month , I have been experimenting with one of the promising serverless framework for creating serverless functions.  With the modern applications moving to cloud with microservices this framework becomes very handy to create/manage your microservices in the form of functions.

What do we need microservices?​

When I started my career as a developer most of the applications that I worked on are with three Tier architecture and most of the companies built the applications with monolithic architecture even before cloud platforms were existed. With the modern technologies everyone is decomposing the business functionalities of the application into several micro services to avoid single point of failure. Assume Uber as an application and their core functionalities such as Registration,Payment,Email notifications, Push notifications could be broken down to several microservices in order to avoid any downtime. With monolithic architecture single point of failure would cause the entire application to shut down. To understand in detail, look at the following diagram

• 
• 

Serverless != No server​

Many of us have the common understanding of serverless as its about without a server. None can be executed/hosted without having a server. It’s just that the fack that you will have no way to actually see the server which executes your code. As a developer with serverless you do not have to worry about managing servers as it will be automatically handled. Serverless becomes handy because of the following reasons

• Reduce time-to-market
• Easier deployment
• Scale automatically
• Focus on business logic
• Cost reduction

Serverless is used for mainly event event driven architecture where functions has an endpoint that triggers something. Say for example, Trigger a notification once the file is uploaded.

Serverless and Microservices are great couple together. You should choose serverless when your functions/services are,

• Stateless
• Short Job
• Event-driven stuff, e.g. Time-based / webhook
• Simple application with less dependencies

OpenFaaS – Serverless Functions Made Simple​

There are so many frameworks out there to build applications with serverless out of them OpenFaas stands out as its not vendor locked and you can use it for both On prem as well as in any of the cloud platform. It is very simple and need few commands to get your functions deployed anywhere. It can be exposed with Docker Swarm or Kubernetes to the outside world.

Following are the reasons if you ever want to choose OpenFaas,

• Anything can be a function
• Leverage existing skills in teams
• Avoid vendor lock-in
• Run anywhere - cloud or on-prem
• 13,000 stars on github with large contributors

OpenFaaS Architecture​

There are main two components that you should get to know before getting started with OpenFaas.

Function Watchdog​

As the name indicates watchdog is responsible to convert http messages to stdin then it will be passed to functions and stdout vice versa.Any docker image could be turned to serverless by adding function watchdog

API Gateway / UI Portal​

As you have heard from AWS API gateway, it does the similar job here as well. It provides an external route into your functions and collects Cloud Native metrics through Prometheus. It also scale functions according to demand by altering the service replica count in the Docker Swarm or Kubernetes API. It also provides the UI to invoke functions in your browser and create new ones as needed.

Faas-CLI​

The command line interface helps you to deploy your functions or quickly create new functions from templates  in any programming language you prefer.

Setup OpenFaas on Kubernetes API​

I am a fan of Microsoft Azure, I will be providing the steps to setup OpenFaas on Azure Kubernetes. I have done this at a workshop and you can find 10 steps to build a kubernetes cluster and then all steps can be done simply by hitting few kubectl and helm commands.

Step 1: Launch Azure Cloud shell with https://shell.azure.com/bash

Step 2: Create a resource group

az group create --name aksColomboRG --location eastus

Step 3: Create AKS cluster

az aks create  --resource-group aksColomboRG  --name openFaasCluster --node-vm-size Standard_A2_v2   --node-count 1 --enable-addons monitoring --generate-ssh-keys

Step 4: Connect to the cluster

az aks get-credentials --resource-group aksColomboRG --name openFaasCluster --admin -a  --overwrite-existing

Step 7: List the cluster nodes

kubectl get all -n kube-system

Kubectl is a command line interface for running commands against Kubernetes clusters.

Step 8: Install and Init Helm

helm init –upgrade

Helm fills the need to quickly and reliably provision container applications through easy install, update, and removal

Step 9: Install OpenFaas

git clone https://github.com/openfaas/faas-netes

Step 10: Create namespace OpenFaas

kubectl create ns openfaaskubectl

Step 11: Create second namespace for OpenFass Functions

kubectl create ns openfaas-fn

Step 12 : Check you have a tiller pod in ready state

kubectl -n kube-system get po

Step 13: Manually start the tiller

kubectl logs --namespace kube-system tiller-deploy-66cdfd5bc9-46sxv

When a user executes the Helm install command, a Tiller Server receives the incoming request and installs the appropriate package

Step 14: Resolve cannot list configmaps in the "kube-system"

kubectl create serviceaccount --namespace kube-system tiller

Step 16: A Helm chart for OpenFaaS is included in the cloned repository. Use this chart to deploy OpenFaaS into your AKS cluster.

helm repo add openfaas https://openfaas.github.io/faas-netes/

helm upgrade --install --namespace openfaas --set functionNamespace=openfaas-fn --set async=true --set serviceType=LoadBalancer openfaas openfaas/openfaas

Step 17: See OpenFaas live

kubectl get all -n openfaas

and you should copy the service/gateway-external url with the port and paste it in the browser. You should see OpenFaas live.

Well, that’s all about for this post, I will write another post about how to execute functions and how to build your custom function in the coming days.

You can find the workshop slides from here. Keep watching :)

I have been working with couple of applications built with CosmosDB and one of the things that surprised me was one cannot clear all documents in a collection from the Azure web portal or using the Storage Explorer. As I was struggling to do this while doing some tests on the application I decided to write a blog on the solution I used. There are two ways to achieve the same

• Using a stored procedure
• Using Cosmosdb SDK

Using Cosmosdb SDK:​

I came up with a script in Node which can be done with any of the programming languages such as C#,Python supported by the SDK

Let’s go through the steps:

Step 1:

Open VScode and Create a file named cosmosdb_helper.js

Step 2:

Let’s install the necessary packages needed.

Install documentdb javascript sdk with the following command,

npm i documentdb

and you will see the output as follows

Let’s install require to handle the dependencies with the following command,

npm i require

and you will see the output as follows,

Step 3: 

Let's do some coding. You will be able to understand the following code with the comments added on each line,

https://gist.github.com/sajeetharan/8efe2c9424dfc89d1f58b34627858944

Suppose if you have partitionKey created with your collection, you need to pass queryoptions with the partitionKey in selectAll as well as deletDocument as follows,

https://gist.github.com/sajeetharan/d5302257d3b5e54a33e5601b215decf1

Step 4:

Let’s run the script and see the output,

You can run the helper script as follows,

if you want to list all documents in the collection,

node cosmosdb_helper.js selectAll

which will list the output of all documents in the collection.

If you want to delete all documents within a collection, you can run the script as,

node cosmosdb_helper.js deletAll

which will remove all documents in the collection.

Using a stored procedure:​

As mentioned above, 2nd way is to use the stored procecure given by Microsoft employee as mentioned here.

Hope the helper script will help someout out there in order to delete all documents in a collection. You can get the whole code from Cosmosd_Helper

Traditional Architecture :​

In a traditional Application with the normal approach, transactional use-cases usually involve persisting data in a few SQL tables or in a NOSQL database. When the changes are performed on the object the database is updated to match the new state.

The traditional approach works well in case if you do not need to know the changes that object has gone through, but in modern systems customers always comes up with a requirement to get the log of changes that particular entity has gone through. With the traditional approach, there is no way of knowing what the user had in the object before changing it, or at which point of time the contents changed. We can still solve this with the traditional way by storing the extra information about the modifications but the solution becomes more complex.

For example in traditional approach,

https://gist.github.com/sajeetharan/2d9921571c67f7038ec5a4053882b85f

Which will create an entry for each insert in the SQL database as follows,

The current state is saved in a relational database. We load the object, change it and save  it back.

EventSourcing Architecture :​

In the eventsourcing solution, we look at the problem as a sequence of events that occur and save the occurrence of events as it is. The events contains all details about what actually happened at particular point of time. These are historical information and once it is saved it should not be modified.

https://gist.github.com/sajeetharan/825ec83fd780b7670146649bf6d4a0ce

All events for a certain product are stored. Their data and sequence define the current state of the product. Event is the easiest way to remember what happened at a certain time. Event sourcing comes with an advantage of having audit trail by itself and to get full understanding of what the system is doing.

Event Sourcing Architecture with AzureCosmosdb and EventHub

To implement event sourcing in your application, Microsoft azure provides the following services to  full fledged solution and we will discuss in this blog.

Lets look at the diagram below,

Application 1 stores the data in the traditional database and your customer needs the changes that has been done on the product. The above architecture will easily fulfill the requirement with the event sourcing.

Components involved in the architecture as follows,

Azure EventHub​

Azure Eventhub is a managed service to receive and process millions of events per second. It is intended to handle event based messaging in huge scale. This could be used in an product if you have devices application publishing events and send them to eventhub. It will create a stream of all these events which can be read by different applications in different ways. Eventhub provides interfaces such as AMQP and HTTP to make it easy to send messages to it. In Eventhub we can define consumer groups which lets us to read stream of events. We can decide on consumer group based on the number of receivers(applications)

CosmosDB​

Azure Cosmos DB is a globally-distributed, multi-model database as a service build for low latency and elastic scalability.  It supports the following options to store the data and it is highly available from anywhere in the world,

• Key-value
• Column-family
• Document: MONGO or SQL
• Graph

I will be not going in detail as there are enough blogs to get started with CosmosDB. In the above architecture there will be millions of events created after each update hence we need to store them in the cosmosdb with the state of the object. This way brings a lot of benefits. First, the event store with cosmosdb becomes your canonical source of truth that describes the updates applied to your domain in an unbiased form.

Implementation:

Application 1:

Whenever user updates an object in the application1, there will be notification message sent to the EventHub with an ID (unique id for each message) that something has happened on application. We could make use of epoch timestamp with 8 digits to make sure it is a unique one. A sample payload would look like,

{"MessageId": 1547632386819}

Note:  As Eventhub can have a message of maximum size 256k it is always better to have minimum size of message.

Once the notification is sent, the state of the object is stored in the eventstore(cosmosdb).

Application 2:

Application 2 will have an EventHub receiver which runs on the background which will subscribe to the EventHub and get the latest message. Once the id is retrieved by the receiver, it can request the eventstore with the id and get all the changes prior to the id as follows,

https://gist.github.com/sajeetharan/c34965a606c8afff9d02f2a3a17522bf

which will create the documents in Cosmosdb as,

With the above approach ensures that all changes to product are stored as sequence of events. When we look at broader picture, it also ensures that all changes to application state are stored.

This is the simple architecture diagram to implement event sourcing in your application. One of the very good pattern to implement event sourcing is by using CQRS(Command Query Responsibility Segregation).

Lets look at the etail implementation with the code in the upcoming blogs. Hope this will help someone out there to implement event sourcing in your application if you are using Azure platform.

The wait is over and Angular 7 is out, finally time has come to write about some tips to build fast angular application. In this blog i will share my experience with practices that you need to follow when you are starting a new project.

There are three primary things important when you are starting to build an application with Angular.

1. Project Architecture
2. Application Infrastructure
3. Conventions,Formating and Tooling

CONVENTIONS,FORMATTING AND TOOLING​

• Always place the Angular application outside your backend code​

Keep the application in its own repository so that can be deployed/version separately. It will also help in tooling. One key rule is to treat it as a real application.

• Use VS Code for best experience​

Encourage your team to switch to VS Code which has immense support for development with Angular. It has strong support for extensions used in Angular projects and well-regarded by front end community.

• Use Tooling​

It will kill all the disagreements. Use VSCode extensions such as TSLint, Angular Language ervice, Prettier, EditorConfig and SCSS intellisence.

• Naming and Syntax Conventions​

Always use Singular for Modules and Singular or Plural for  components/ services/ directives and pipes

• Use Angular style commit messages​

Examples :

'feat(notification.service): add display param'

'refactor(order models): rename couponId to couponCodeId

• Use Codelyzer to do statistical analysis on Angular/Typescript code​

It's a set of tslint rules for static code analysis of Angular TypeScript projects. If you are doing continuous deployment configure with your pipeline.

• Follow consistent structure​

If you are using Angular-Cli always follow the same pattern to generate the necessary files so that it will be consistent throughout the application.

• Absolute path for ES Modules​

Always use absolute path that would help in refactoring(moving files around or renaming) and very easy to organize files.

• Do not use null and assign default values​

In the templates always use safe navigation type operator which can help in preventing the following annoying errors "cannot read property "name" of undefined"

• Choose Intelligent defaults and be consistent​

'' for string - declare string with default value ""

0 for number - declare number with default value 0**[]** for arrays - declare array with default value []

• Build small reusable components​

Don't make your component code go more than 300 lines. Split the pieces that can be reused in a component and make them as a new component. The component should be made as dumb as possible. It should not dependent on any inputs or outputs provided, it should work simple. General rule of thumb is to make the last child in the component tree to be the dumbest. Reusable components reduce the duplication of code and you can make changes easily.

Components should deal only with the presentation logic Don't have logic other than the presentation logic. Components are designed for presentational purposes and only should focus on what the view should do. Business logics should be separated out to services/methods from the presentation/view logic.

• Use trackBy in ngFor Loops​

DOM manipulations are always expensive, immutable practices always generate a new collection which will result in bad performance. When your array changes, Angular will be rendering the whole DOM tree, when you use trackBy, it will know which elements has changed and will make changes only to those particular elements.

• const vs let​

Make use of let and const wherever its appropriate. It will help a lot in identifying issues when a value is reassigned to a constant accidentally with a compile error and it improves the readability of the code.

• Pipable Operators​

With Angular version above 5.5 , you can use pipeable operators which are tree-shakable (only the code need to execute will be included when they are imported) and it will be easy to identify unused code in the component files.

• Subscribe in template​

Rather than subscribe in service or async popes unsubscribe themself automatically it will make the code simpler by stopping the need to manually manage subscriptions which could cause a memory leak. When using subscribe, the risks can be eliminated by using a lint rule to detect unsubscribed observables.

• Clean up subscriptions​

When you subscribe in the component to observables, make sure you unsubscribe from them completely with operators like take,takeUntil,Unsubscribe etc

• Use appropriate Operators​

switchMap: when you want to ignore the previous emissions when there is a new emission mergeMap: when you want to concurrently handle all the emissions

concatMap: when you want to handle the emissions one after the other as they are emitted

exhaustMap: when you want to cancel all the new emissions while processing a previous emisssion

• Stop using any:type everything in the code​

Always declare variables or constants with a type other than any. This is the advantage that you have with Typescript when you have good typings in your application which makes refactoring easier and safe and also avoid unintented issues.

• Use lint rules as you need​

TSLINT has various options built-in already like no-any, no-magic-numbers, no-console, etc that you can configure in your tslint.json to enforce certain rules in your code base

• Dont Repeat Yourself​

One of the common mistake that i did as a developer was copy paste the same code in all components. Do not repeat or have the same code in different places in the code base. Extract the repeating code and make them as a generic method which could be used in different components.

• Avoid Logic in component templates​

Place the logic in the component file rather than on the template such as && condition since it cannot be possible to unit test also it is prone to more bugs when changing template code

APPLICATION INFRASTRUCTURE​

• Lazy load everything​

When you are building application with large number of modules always do lazy loading which could improve your application performance by large margin. Angular CLI makes this easy and helps break up your app into logical bundles. With Lazy loading users only pay for what they want. For example, Sensitive (admin only) code will not be downloaded for users that don't have access.

• Analyze your bundle​

If you are using any bundling mechanism always analyze the size of the bundle generated. You can use webpack-bundle-analyzer for example and you can improve the performance there on.

Install source map explorer

• npm install -g source-map-explorer

Build with source map

• ng build --prod -sm

Inspect your bundle

• source-map-explorer dist/vendor*.js

• Use Debug.Service.ts to track errors​

It is always good to have a common debug service to assist with the development. This service could replace the calls to console.log.  You can use Visual studio extension codelensto track down the calls with console.log.This service can be toggled at run time with local storage value and needs to be switched off with production builds.

• RXJS Operators​

When using it becomes handy to handle so many operations with Rxjs operators. Always remember to include/import only the things you need. Also make sure to add noUnusedLocals in the tsconfig.

• Use ES Modules for helper functions​

With ES Modules it is really easy to import only available when thy are needed.

• Keep environment values in environment files​

It becomes really easy and very helpful to manage environment values when it comes to continuous improvement and continuous deployment.

• Avoid Base classes / Inheritance​

Even though Angular is build with Typescript, many of the developers tend to use services from a base class. This should be only if necessary. It will result in restricting the flexibility as when your app grows and use cases changes.

Also create a utility.services.ts to contain all the base helper services. For ex: debug.service.ts , notification.service.ts  session.service.ts can be placed within utility.service.ts which helps in preventing app wide changes to base constructor.

• Use Obseravable/State Management Patterns​

It is really important to follow Redux patern (RxJS) to have a better state management in application. Also in components use ngUnsubscribefor complex Observable management in Components. Use shareReplayoperator and/or async pipe for simpler cases

• More things to consider on Application​

Always consider browser caching and application versioning. Test the update process and the experience across browsers. Always figure out user experience and continuously work on it. Use global error handler to store report errors to API and do not access Document or Window Objects manually.

PROJECT ARCHITECTURE​

• One of the primary thing that you need to consider before starting a project is the architecture on how to build flexible,simple,fast application. It needs lot of planning and consistency to get the basement correct. Building things is really hard enough. When it comes to building Angular application i would recommend to follow the following Guidelines
• One best place to start with good practices is by following the recommended style guide. You need to take what works for your team and skip what does not work. Also try to learn from others mistakes or any other projects that you've already worked with.
• While designing module,s it is really important to know about how to structure your modules and what should go under Feature/Core/Shared Modules.

• Keep a flat file structure as long as possible which means you should not add hierarchy with less than 20 files and you can always move files as the app grows larger.
• Maintain your application version using the package.json which could be embedded in your app.
• Use package management tooling to guarantee reproducible dev environment and builds.
• Set custom host for the application by changing the default url.
• Use proxies if you are integrating with an API.
• Lets look at what you can do while implementing the above architecture in your application

Follow this repository in order to get started

The above are the some of the most important practices that you need to follow when you are starting a  new project with Angular. Hope this helps someone out there.

After building my first production ready application with ASP.NET Core 2.1 and tested locally when i tried to deploy for the first time on Azure, I was stuck with the following page.

Since many different problems can cause this error page, I would strongly recommend the following in order to determine the root cause quickly and easily, without meddling with Azure.

You can enable extremely helpful developer friendly error messages at startup by setting the .UseSetting("detailedErrors", "true") and .CaptureStartupErrors(true) actions in your Program.cs file.

https://gist.github.com/sajeetharan/a186641a8f7c7e00159e190f1fe97d80

With the above settings publish your application to azure. Once you identify the root cause and resolve your issue, These above settings should be removed as soon as your troubleshooting is complete so as not to expose your application to malicious attacks.

Hope this helps someone out there.