When you are involving in an architectural discussion which involves Azure's cosmosdb, a mandatory question that you get is "Is not Azure Cosmos DB very expensive?". Based on the fact that Cosmos DB is evolving very fast, there are lot of customers stepping in to use the service in their architecture. To understand one thing Cosmos DB is not priced based on the usage the pricing is based on what you reserve. One of the best serverless example we could consider here is renting out a car rather than managing and hailing it. Key point here is you pay for what you reserve, which is the capacity, which is refered in terms as Request Units (RUs). Any customer consuming the Cosmos DB serivce will be paying for the RUs as well as the space.

There have been many questions asked on forums,discussions on how to scale up/down cosmos DB Request units. Being a fan of two major services in Azure, i decided to write on how to scale Cosmos DB with Azure function.Azure Functions and Cosmosdb services are getting more closer and closer together in the recent times. One of the case study that we can consider here is whenever you are experiencing throttling(429) due to the burst of high traffic for a period of time, you will be increasing the Request Units(RUs) in the portal to handle it. Which is sort of a pain to handle it manually and the resulting cost will be very high if you've forgotten to scale it down. Let's see how to autoscale using Azure function to mitigate this issue. It involves 2 steps ,

• Create an Azure function to scale throughput on a collection and publish
• Connect the function to CosmosDB alert using an HTTP webhook.

The following solution will help you only to scale up , but the same function can be used to sacle it down if you pass a negative value for the CosmosDB_RU attribute.

Step 1: Let's create the Function and Publish​

Step 1: Open Visual Studio 2019

Click File->New Azure Functions from the templates available and give a name for your function and click ok. I have given the name as Cosmos_scale

I will be using Azure function 2.x , so you will be taken to a new window where select Trigger type as HttpTrigger and select Authorization level as Function and no need for Storage Account

Once the project is created, rename the default Function1.cs with your name, in this case it will be Cosmosscale.cs. Let's get into the actual implementation of the function

Step 2: Add Microsoft.Azure.Documents.Core Nuget Package to the solution

In order to communicate with the Cosmos DB account and make use of the operations let's add the Nuget package Microsoft.Azure.Documents.Core package.

Step 3: Add Microsoft.Extensions.Configuration to the solution

In order to connect to Cosmos DB we need to get the connection string and the key from the appsettings, lets add Microsoft.Extensions.Configuration. For Azure Functions v2, the ConfigurationManager is not supported and you must use the ASP.NET Core Configuration system:

Let's understand the logic here,

As a first step, lets create the Client to connect to Cosmos DB

https://gist.github.com/sajeetharan/788a6acf99416dde8e6ebd652c2b3ed2

Get the connection self link

https://gist.github.com/sajeetharan/80c87e46cf2969eb11c3c4971f6759ac

As we have already created the account and the collection, lets get the current offer of it

https://gist.github.com/sajeetharan/a3d2d5357791f50276585aa2a04461c1

Let's get the current throughput count

https://gist.github.com/sajeetharan/38302ad7f17b4844afdfd6fd0162e776

Set the new offer with the throughput increment added to the current throughput

https://gist.github.com/sajeetharan/d38fbb251e9e57177a5e68c5735a0fef

That's it, additional step would be to handle the failure and return the response back.

This is how the whole function would look like,

https://gist.github.com/sajeetharan/06544e38bb460df5de44cbaef9b05b43

Step 4: Add the config values to local.settings.json

Now we need to add the values to "local.settings.json". These values will be used to test the function locally before deploying it to Azure.

The setting key "CosmosDB_RU" is to increase the RU by 100, and if you want to decrease you can set a negative value say "-100".

You can get these values from the portal by navigating to the Cosmos DB account.

Step 5: Check the function with Postman

Now we have setup and created the function locally. To test the app locally, click on the run button. Using PostMan send a GET or Post request by using the url

http://localhost:7071/api/Cosmosscale

If you have followed the steps and set up everything correctly, you will be seeing he following response in the console. LogInformation messages will specify the current and the provisioned throughput.

Now we have successfuly tested the autoscaling function in local. Let's publish the Function.

Publish the Function on the portal​

In this ste, lets deploy the function through the portal to the new function app.

Navigate to the Azure portal and provision a Function App with the default settings.

Click on “Function app settings” on your Function App’s homepage, then click on “Manage application settings”. Add the values in the table below to Application settings. The advantage of Adding values to Application settings allows the function’s manager to edit the values later.

Key	Value
CosmosDB_Uri	https://<uri>.documents.azure.com:443/
CosmosDB_appKey	<primarykey>
CosmosDB_DatabaseId	<database_name>
CosmosDB_ContainerId	<container name>
CosmosDB_RU	<RU increment/decrement as integer>

Let's publish the function app using Visual Studio.

Right click on the project file > Publish… > Select Existing > Publish > Select the Function App we provisioned in the previous step and click ok.

Now we have successfuly deployed the function to Azure. Let's do the final step

Test the function on Azure by navigating to the function, in the portal blade, and clicking run. We should see the following output if the function succeeds.

If it does not work, make sure you have entered the configuration correctly in the app settings.

You can access the full source code from here.

As this function can be invoked periodically, you can ammend the logic to scale up/down RUs based on time/month/year etc.

Now you can use this function url as a webhook and can be called from anywhere to scale up/down automatically. Hope this will help someone out there to manage the consumption and reduce the cost.

Road traffic is a very classic and burning problem in Sri Lanka and in most of the Asian countries. Personally I have to spend 2 hours on the road everyday by just stuck in the traffic and I assume the same for other people who gets stuck for many hours with no way out.

I was thinking of implementing a solution through various ways with PaaS provided by Azure, this blog focus on one of the solution with Azure by using various services such as IOT hub,Functions,Cosmosdb,Powerbi and bot framework

Overall Architecture:​

Components used:​

• Azure IOT hub
• Azure Functions
• Azure cosmosdb
• Power BI

How it works?​

IOT sensors can be placed in heavy traffic areas to monitor traffic level along the road. Everyone can access the data using Facebook messenger bot and subscribe on specific road/area. When there is heavy traffic, push notification will be sent to subscribers allow them to avoid that area and redirected to some other road. Also a notification will be send to the traffic police to take over the control.

How to build?​

The high level architecture includes 4 steps.

• IOT sensors placed over the areas will send data to the Azure IOT hub.
• Azure IoT Hub will be configured to trigger Azure Functions to Store data into Cosmos DB  and also send notification to Facebook messenger bot(This could be replaced with LINE,Telegram,Skype etc) subscribers.
• Facebook messenger bot connects to Azure Functions which will acts as serverless bots over HTTPS and  Azure Functions to process message from Facebook messenger users and reply back
• PowerBI connects to Cosmos DB and then Visualize Traffic level on map in real time and could be displayed on the control room.

I will be implementing this POC and publish the code in my github repository in the coming days. In the meantime if you have any suggestions feel free to comment below.

I was at the Global Azure Bootcamp recently concluded last week, one of the participant came and asked me, "Hey what is Cosmos DB" I casually responded “Well, that’s Microsoft’s globally distributed, massively scalable, horizontally partitioned, low latency, fully indexed, multi-model NoSQL database". The immediate question came after that was whether it supports hybrid applications. In this blog I will be explaining how to leverage the features of cosmos db when you're building hybrid applications.

Ionic framework is an open source library of mobile-optimized components in JavaScript, HTML, and CSS. It provides developers with tools for building robust and optimized hybrid apps which works on Android,IOS and web.  Ionic comes with very native-styled mobile UI elements and layouts that you’d get with a native SDK on iOS or Android.

Let's see how to build Hybrid application with Ionic and Cosmosdb

PreRequisites:​

You need to have npm and node installed on your machine to get started with Ionic.

Step 1: Make sure you've installed ionic in your machine with the following command,

ionic -v

Step 2: Install Ionic globally

if it's not installed, install it by using the command

npm i -g ionic

Step 3: Create a new ionic project

Ionic start is a command to create a new ionic project. You pass in the directory name to create, and the template to use. The template can be a built-in one (tabs, blank) or can point to a GitHub URL.

The aim is to create a simple ToDo application which displays lists of tasks, and user should be able to add new tasks.

Lets create a blank project with the command

 ionic start cosmosdbApp tabs

You will see a new project getting created.

Step 4: Run the ionic app

You can run the app with a simple command by navigating to the folder and then run

Ionic serve

This starts a local web server and opens your browser at the URL, showing the Ionic app in your desktop browser. Remember, ionic is just HTML, CSS and JavaScript!

It will open the application in the default browser with the default app.

If you're stuck at any point you can refer to my slides on How to get started with Ionic and follow the steps.

https://slides.com/sajeetharansinnathurai/ionicnsbm#/10

Step 5: Create the cosmosdb account on azure portal,

We need to create an cosmosdb account and use the Endpoint and SecretKey in our application. You can follow the

Blog on how to create it from the azure portal

https://sajeetharan.wordpress.com/2018/03/26/wear-out-the-features-of-azure-cosmosdb-with-aspnetcore-application/

Create a database named "ToDoList" and containeras "Items".

Once created, lets go back to the application.

Step 6: Lets add Cosmosdb SDK to the project

To consume the Cosmosdb service in the application we need to add the Azure Cosmos Javascript SDK to the project. This can be done with the command

npm i @azure/cosmos

Step 7:  You need to create an interface/model which will be used to save/retrieve the objects from the cosmosdb

https://gist.github.com/sajeetharan/a1dff67c87dce10bc2220aa0e9c550c3

Step 8: You need to add two pages home and todo page , home to display the lists of items inside the containerand todo page to add a new item. These two pages can be generated insdie the module with the command,

ionic g page home

Ionic g page todo

https://gist.github.com/sajeetharan/85229d091990d149b9c09d71f3387287

Step 9: We need to add a service to embed the logic to communicate with the cosmosdb you can create a new service inside the module as,

ionic g service cosmos

https://gist.github.com/sajeetharan/d7336abce15fa4d10ffd9d806b819462

As we need to make use of the available methods inside the @azure/cosmos You can import cosmos with the line,

import * as Cosmos from "@azure/cosmos";

Now make use of all the available functions in the SDK to add,delete,update items in the cosmosdb.

Step 8: To make application compatible with android/ios , run the following command,

Ionic cordova build ios/android

If you want to make the development faster, you could try building your ionic application with capacitor as well.

Now your hybrid application uses cosmosdb as a backend, with this demo you know how to use cosmosdb as a database for your hybrid application. I hope you should be able to create more applications in the future with cosmosdb and ionic.

You can check the demo application source code from here.

This is going to be a very small blog post, but indeed it will be helpful for lot of Angular developers out there. One of the recent question i answered on Stack overflow on how to reverse engineer an Angular application.

The question was " I built an Angular app using ng build. I have the built version but I accidently deleted my code. Is there any way I can get my code back from my build version? "

Even though the correct answer is NO, but you will be able to retrieve 80% of the code with the following steps.

Step 1: Deploy the app code in the Dist folder

Step 2: Use Google Chrome Developer tools (F12).

Step 3: Under debugger tab, look under Webpack -> Src you will see all typescript files. you can copy and past the code provided which would help you to at least build the structure of your application.

In any business, the value of your data is the cost to reproduce it.  For an instance if the data came from any form the cost is the amount of money in wages to recreate the data in the same form where it came from. While in the case of application running in production, some data can’t be reproduced. So if lost or destroyed, and there is no backup no amount of money can get it back. It is really important that having a data backup and recovery plan is important to the overall success of your business. If there is no data backup plan business can suffer from permanent data loss, massive downtime, and unnecessary expenses.

What is Cosmic Clone?​

If you are from a SQL background dealt with many relational databases, system like SQL server, this can be achieved by out of the box tools such as SQL backup and N number of tools. If you are from NOSQL background and a fan of Azure CosmosDB (Microsoft’s NoSQL Document\Multi model Database) there has not been really a good tool to perform these tasks up to now. Personally i have been waiting for long time to get this kind of tool out. Finally Cosmic clone has arrived to serve the purpose.

Features of Cosmic Clone​

• To clone collections for any environments
• Create collections with similar settings(indexes, partition, TTL etc)
• To Anonymize data through scrubbing or shuffling of sensitive data in documents.

How does it differ from Cosmosdb Migrator tool?​

As a Cosmosdb user, you might have already familiarized with the Cosmosdb migration tool. Data migration tool actually helps to copy documents but does not provide options either to create a similar collection (with partition keys or indexes) and it does not provide a way to copy the related code starting with stored procedures, UDFs, and triggers etc. Further, there are no options to anonymize data in a collection.
Cosmic Clone help to ease the above process and aid in the copy and anonymization of a cosmos collection.

How to use it?​

Prerequisites:​

• Microsoft .Net Framework 4.6.1 or higher
• Source Cosmos collection and read only keys to its account (Could be production environment)
• Destination Cosmos Account and its read write keys
• Make sure the IP address of the machine running the tool is allowed in the firewall settings.

Deployment:​

1. Navigate to https://github.com/Microsoft/CosmicClone and clone the repo (Switch to the demo branch if the master is not stable)
2. Just Compile and Run the Code.
3. Or Download a pre compiled binary from the releases section and run the “CosmicCloneUI.exe” file.
4. For Best performance you can run the compiled code in an Azure VM that is in the same region as the source and destination Cosmos Collection. (Source : https://github.com/Microsoft/CosmicClone)

Create backup of a collection​

Step 1: Provide source connection details.

Obtain the connection string and the keys from the source cosmosdb account and enter it as below. if you are not sure how to obtain them, read my earlier blog here. You should see a notification as Validation passed once you set up the connection details.

Step 2: Provide destination connection details

You can follow the above step to obtain the destination connection details and fill the same way as you did it in the above step.

Step 3: Provide the options you want to backup

In this step you could select the things you want to copy over to the destination collection. Starting with Documents, Stored Procedures, User defined functions, Triggers, Indexing policies and Partition Keys.
All the options are checked by default but allow you to configure to optout of any.

The next step is to Anonymous the data of a cosmos collection. But this is necessarily not needed if you are doing just a backup.

Step 4: See the data in the document explorer

That's all, once you click next, you will be able to see the documents being transferred to your new collection as per the settings you've provided. Within few clicks you can transfer the data from one collection to another collection residing in another account. Also you can see the status of each copied items in the logs window. Once it is done, explore the destination cosmos portal and one can observe the new collection created with the required settings.

How anonymization  helps with GDPR?​

As we have seen it in the 3rd step, you are provided with an option to anonymize the data with few settings. With GDPR now you should mandate data anonymization in all non-production environments. With the amazing cosmic clone tool it can save the manual effort of a developer, as we will no longer need to write, test, update, or maintain their own anonymization scripts. Is not it cool? You can read about the steps to anomyze the data from here. Based on my experiment with Cosmic tool, it is the best tool so far to backup\clone\restore a azure Cosmos database Collection. Try it out and see. If you want to contribute to the repo, you can do that as well. Cheers!